Fingerprint feature matching using graph neural networks

Abstract

The disclosed invention introduces a novel fingerprint feature matching method that employs a graph neural network to calculate a similarity score between two sets of fingerprint features. By learning distinctive feature representations and graph aggregations in an end-to-end manner, the method eliminates the need for manual feature descriptor construction and manual matching formulas. The method is not limited to fingerprints, as it can utilize various local fingerprint features such as feature points, pores, and singularities, and can also work with other friction ridge patterns such as palm prints and footprints. The method can be integrated into biometric systems to reliably identify individuals based on fingerprints or other friction ridge patterns.

Description

[Technical Field]

[0001] This invention relates to fingerprint recognition, and more specifically, to the application of graph neural networks to fingerprint feature matching. [Background technology]

[0002] Fingerprint matching is a challenging task, especially when dealing with low-quality, distorted, or hidden fingerprints, or when matching fingerprints with small overlapping areas. Numerous automated fingerprint matching algorithms exist. Well-known and widely used techniques are based on feature points (minutiaes), which are features in the fingerprint pattern where fingerprint ridges terminate or branch off. Since each finger has a unique feature point pattern, feature point-based matching primarily involves finding a match between two sets of feature points that yield the highest similarity according to a selected score definition.

[0003] The most widely used feature point matching algorithms include a local feature point matching stage. In this stage, feature points from two fingerprints are compared based on their associated feature attributes. A list of such attributes is typically called a feature point descriptor. Conventional techniques construct these descriptors based on the local geometric structure of the feature point neighborhood. For better matching, some methods incorporate additional texture information, such as local ridge frequencies and local ridge orientation fields. Typically, after determining corresponding feature point pairs, the matching algorithm proceeds to a global integration stage. In this stage, the consistency of the local matches at the global level is evaluated, and based on this, a final similarity score between the two fingerprints is calculated.

[0004] Such conventional feature point matching methods have several weaknesses. Firstly, there are many ways to construct feature point descriptors, and arbitrary manual methods and selected attributes may not be optimal for matching. Furthermore, comparing these descriptors is not straightforward and requires carefully designed formulas, which may also not be optimal. As a result, these techniques are sensitive to missing or incorrect feature points, inaccurate feature point locations or angles, and distortions. In the integration stage, manually designed heuristic rules are required to recalculate the similarity score. These rules may not work well in all situations and are known to have shortcomings in cases involving missing feature points or small overlapping regions.

[0005] Several fingerprint matching techniques utilize not only feature points, which are fingerprint features defined in ISO / IEC 19794-2 and ANSI INCITS 378 standards, but also other local fingerprint features such as sweat pores, dots, and ridge origins. When such types of features are reliably extracted, these methods are usually superior to those based solely on feature points. However, these local feature-based methods share the same weakness as feature-point-only methods: the feature descriptors and matching functions must be manually created.

[0006] To avoid designing complex rules for fingerprint matching, some techniques employ learning-based methods. For example, to improve feature descriptor matching, certain techniques train a trainable classifier to determine whether two feature points from different fingerprints should match. Several techniques also use machine learning in the integration stage, where heuristic parameters for combining local similarity scores into a single score are optimized to best distinguish the same finger from different fingers. Other techniques apply machine learning, specifically convolutional neural networks, to learn texture-based feature points or pore descriptors. However, the main drawback of these learning-based techniques is that machine learning is applied only to individual steps and the entire system cannot be trained end-to-end; that is, it cannot start with two raw feature sets and learn the similarity between them.

[0007] An end-to-end trainable system that operates on raw fingerprints rather than pre-extracted features is described in [1]. In the disclosed method, fingerprints are encoded into fixed-length numerical representations by a convolutional neural network, and two fingerprints are compared by calculating the vector similarity between their fixed-length representations. This method has many advantages over conventional techniques, such as significantly faster matching speeds and the elimination of the need to extract and match fingerprint features, but it has a significant drawback in that the two fingerprint representations are completely independent of each other, which can limit the accuracy of fingerprint matching.

[0008] When matching accuracy is prioritized, a more advanced and trainable matching function should be designed that learns to compare two fingerprints and generate a similarity score. This can be achieved by formulating fingerprint feature matching as a graph matching problem and solving it using a graph neural network. Multiple techniques have formulated fingerprint feature matching as a graph matching problem, but have solved it using heuristic graph matching methods. For example, graph matching has been applied to keypoint matching in [2] and [3], and to pore matching in [4]. On the other hand, graph neural networks have been applied to general graph matching problems unrelated to fingerprint matching in multiple studies such as [5] and [6].

[0009] One method of adopting a graph neural network in fingerprint matching has been proposed in [7]. The authors represent fingerprint keypoints as nodes of a graph along with their initial descriptors, and connect adjacent keypoints by edges. They propose a graph neural network for learning new keypoint descriptors and the similarity between keypoint descriptors from different fingerprints. In this way, the geometric shape of the graph is incorporated into the newly learned keypoint descriptors, making them more distinctive. However, the main limitation of this method is that the calculation of the final similarity score between two fingerprints at the integration stage is still implemented by a manual function that has not been learned and may not be optimal. Another drawback of this method is that since there are no edges between graphs, there is no way for the two graphs being matched to interact with each other, and this lack of interaction can also limit the accuracy of fingerprint matching.

[0010] The present invention is a method designed to address and overcome the above-mentioned drawbacks of both conventional and learning-based methods, and provides further advantages over existing techniques in the art.

Summary of the Invention

[0011] The disclosed invention introduces a novel fingerprint feature matching method (Fig. 3) that includes obtaining two initial sets of fingerprint features, constructing two graphs from these sets of features, and employing a graph neural network to calculate a similarity score between the two graphs that represents the similarity between two fingerprints.

[0012] The present invention also introduces a novel end-to-end trainable system for fingerprint feature matching that starts with two graphs constructed from two sets of fingerprint features and learns to generate a similarity score between the two sets using labeled data. This approach eliminates the need to manually construct structure-based feature point descriptors because the graph neural network can learn to compute discriminative feature point representations based on the graph structure and in a way that is directly useful for matching. The graph neural network can learn to aggregate the feature representations of two graphs into a single number that represents the similarity between two fingerprints, eliminating the need to design an equation for combining local matches into an overall similarity score.

[0013] This fingerprint feature matching method is not limited to matching based only on feature points. In addition to or instead of feature points, other local fingerprint features can be employed. Such features can include, for example, singularities, pores, breaks in ridges, etc. The type of features that can be used in this matching method depends on the selected feature extraction algorithm.

[0014] This method is not limited to fingerprints only. In fact, with this method, individuals can be identified using feature points (or other features) from various friction ridge patterns and impressions. These other friction ridge patterns include, for example, palm prints and foot prints.

[0015] This method can be integrated into a biometric authentication system to reliably identify an individual or verify the identity information of an individual based on the individual's fingerprint or other friction ridge pattern. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Novel aspects of the present invention are described in particular in the appended claims. However, the method itself can be best understood by referring to the detailed description in conjunction with the accompanying drawings.

[0017] [Figure 1] This shows an example of a fingerprint and several examples of local features that may be present within it. Different types of features are marked with separate symbols. Note that not all features present in the fingerprint shown are marked.

[0018] [Figure 2] Figure 1 shows an example of a graph constructed from local fingerprint features. The graph consists of nodes and edges. Different types of nodes are associated with the corresponding feature types.

[0019] [Figure 3] This document outlines the basic operating principle of this fingerprint feature matching method, which takes two sets of fingerprint features and generates a similarity score.

[0020] [Figure 4] This demonstrates the basic operating principle of a graph neural network, which involves obtaining two initial graphs and generating similarity scores.

[0021] [Figure 5] Figure 4 shows another variation of the graph neural network, where the network is split into two separate networks: one network that embeds the initial graph, and another network that takes the two embedding graphs and generates similarity scores.

[0022] [Figure 6]This diagram illustrates the basic operating principle of a fingerprint-based biometric authentication system that incorporates this matching method and utilizes the graph neural network shown in Figure 4. This diagram only shows the registration and verification processes; the identification process is not depicted.

[0023] [Figure 7] This diagram illustrates the basic operating principle of a fingerprint-based biometric authentication system that incorporates this matching method and utilizes the two graph neural networks shown in Figure 5. This diagram only shows the registration and verification processes; the identification process is not depicted.

[0024] The present invention will be described in detail below with reference to these figures. Throughout all figures, specific elements are labeled with consistent numbers, specific processes are outlined by dashed lines, and are indicated by the same names throughout all figures. [Modes for carrying out the invention]

[0025] In the following description, several specific details are provided to ensure a thorough and clear understanding of the present invention. However, the embodiments presented are not intended to limit applications of the present invention that can be implemented without these specific examples. Well-known methods, procedures, and components, particularly those involving general-purpose machine learning, graph neural networks, and biological systems, are not described in detail in order to maintain focus on the core principles of the present invention. This description should not be interpreted as limiting the present invention to the examples given, but rather as illustrating various possible embodiments of the present invention.

[0026] In the present invention for fingerprint feature matching, various types of local features extracted from a fingerprint (101) may be used (Figure 1). These types of features may include, for example, feature points (102), sweat pores (103), singularities (104), dots (105), borders (106), ridge bridges (107), ridge breaks (108), etc. In practice, any prominent local feature may be employed. In addition, these features may not even be explicitly visible; for example, they may be regularly sampled points (109) on the fingerprint indicating the direction of ridge flow. The types of features that may be employed in this matching method depend on the specifications and capabilities of the selected feature extraction algorithm that must be performed on the fingerprint to prepare an initial set of features. Processing steps may be required to process the extracted features, such as filtering features or deriving secondary features from initial features and assigning attributes to them.

[0027] In the context of the graph, local fingerprint features, such as those shown in Figure 1, constitute the graph (200) and are referred to as graph nodes (Figure 2). Each node may have multiple numerically encoded attributes. For example, in the case of feature points, these attributes may include x and y coordinates, angle, type, quality, curvature of the ridge, etc. These attributes may also include uninterpretable numerical representations, for example, computed by a convolutional neural network operating on the fingerprint image. Multiple types of nodes may exist corresponding to different types of fingerprint features. Different types of nodes may have distinct sets of attributes. In Figure 2, the type of each existing node is indicated by a unique symbol.

[0028] In addition to nodes, a graph can also have edges between nodes. Each edge (201) connects two nodes. Nodes can be connected in any chosen way; for example, each node can be connected to its k nearest neighbors, or it can be connected to all nodes up to a certain distance according to a chosen proximity metric. Edges can be either directed or undirected. A directed edge suggests that if node A has node B as a neighbor, node B may not have node A as a neighbor. Edges can have multiple numerically encoded attributes. These attributes can be derived from node attributes, such as the distance between two feature points or the relative angle between two feature points. In addition, edge attributes can also include independent fingerprint features, such as the number of ridges between two feature points. If a graph has more than one type of node (e.g., feature points and pores), there can be more than one type of edge connecting nodes (e.g., edges between feature points, edges between feature points and pores, and edges between pores). Different types of edges can have different sets of attributes.

[0029] A graph may also have additional nodes that are not associated with any fingerprint features. For example, a graph may have virtual nodes that can be connected to all nodes in the graph.

[0030] In addition to node and edge attributes, several graph-level attributes may exist that are associated with the entire graph. These attributes correspond to overall fingerprint features such as fingerprint quality, fingerprint orientation, and the probability that a finger belongs to a particular type. For example, any numerical fingerprint representation that can be computed by some convolutional neural network can also be given as a graph-level attribute.

[0031] In short, and more formally, a graph G=(V,E) is represented by a set of nodes V and a set of edges E. Optionally, each node i∈V has an attribute vector z i Each edge (i,j)∈E is associated with the attribute vector z ij Associated with, the entire graph G is the attribute vector z GIt can be associated with.

[0032] The basic operating principle of this fingerprint feature matching method is shown in Figure 3. Given two feature sets (301a, 301b), this method first constructs two initial graphs G1=(V1,E1)(303a) and G2=(V2,E2)(303b) (302). Next, it calculates a similarity score s between the two initial graphs using a graph neural network (304), which is referred to in this explanation as a matcher network (or simply a matcher): s = MATCHER(G1, G2)

[0033] Within the Matcher network (304), the attribute vector z of each of the two graphs i , z ij , and z G These are referred to as the initial node, edge, and graph-level representation, or hidden embedding, respectively.

[0034] Figure 4 illustrates the basic operating principle of a matcher network (304). The matcher transforms node representations through a message-passing process. The message-passing layer passes information, i.e., messages, between nodes to generate discriminative node representations. Message-passing layers can be intra-graph or inter-graph. Intra-graph layers pass messages only between nodes belonging to the same graph. Inter-graph layers pass messages between nodes belonging to different graphs. After message-passing, the node representations of both graphs are aggregated into a single vector, which is then reduced to a single value representing the similarity score between the two graphs.

[0035] A key part of the matcher network (304) is the in-graph message passing layer. For each node, the in-graph message passing layer collects messages from the node's neighbors through edges and aggregates these messages into a new node representation. Messages between two nodes can be computed based on the current representations of the two nodes and edges. More formally, the message passing layer, zi ' = UPD(z i , AGGR({m ij , ∀j∈N(i)})) m ij = MSG(z i , z ij , z G ) As shown below, the node representation z i is converted into a new representation z i '.

[0036] Here, MSG is a function that calculates the message m ij from the j-th node to the i-th node, and generally depends on the representations z i and z j of both nodes, as well as the edge representation z jj . The messages from the neighborhood N(i) of the i-th node, which is the set of nodes connected to node i, are aggregated into a single vector by the function AGGR (e.g., element-wise addition). Next, the function UPD combines the initial node representation with the aggregation result to generate an updated node representation. The functions MSG, AGGR, and UPD need to be differentiable to enable the training of neural networks.

[0037] The message function MSG that calculates the message m ij from the j-th node to the i-th node can have various forms. The following are some examples: m ij = Wz j , m ij = α ij Wz j , m ij = MLP([z i || z j || z ij )

[0038] In the first example, the message is simply calculated by multiplying the neighborhood node representation z j by a learnable weight matrix W. In the second example, each message Wz jThe intrinsic attention coefficient α ij It is scaled by a coefficient that may depend on the representations of both nodes and can be normalized across all nodes in N(i). In the third example, the message is computed by a multilayer perceptron MLP for the concatenated representation of both nodes and edges. Many other forms of message functions are possible.

[0039] The aggregate function AGGR, which takes a set of messages and returns a single aggregated message, can be any order-invariant operation, such as element-wise addition, averaging, or maximizing. In some cases, the function AGGR may have learnable parameters. In some cases, the function does not have to be order-invariant; for example, it may be a recurrent neural network cell for a set of messages.

[0040] The update function UPD is based on the initial node representation z i Combine this with the output of the function AGGR to obtain the updated node representation z i This function generates a '. This function can take various forms; for example, simple addition followed by activation, concatenation, multilayer perceptron, recurrent neural network cell, etc.

[0041] The functions MSG, AGGR, and UPD can differ for each message passing layer. In some cases, the message function MSG can differ for each edge type. The aggregation function AGGR can first aggregate incoming messages from each edge type individually and separately, and then combine these partial aggregates into a single aggregate. The update function UPD can differ for each node type.

[0042] It is important to note that some variations of the message-passing layer in a graph neural network can be described without the message-passing form described above, and without explicit MSG, AGGR, and UPD functions. For example, by stacking the node representations on a matrix Z and denoting the graph adjacency matrix by A, the layer can be described as a single matrix operation Z'=σ(AZW), where σ represents the nonlinear activation function. The message-passing layer can also be described as a spectral graph convolution. Other equivalent formulations for message-passing are also possible.

[0043] In addition to the intragraph message passing layer, the matcher network (304) also includes an intergraph message passing layer that passes messages between nodes belonging to separate graphs. These layers can update the representation of one graph based on the representation of the other graph. Nodes belonging to one graph may be connected to some or all nodes belonging to the other graph. Formally, the intergraph message passing layer for node V1 is: z i '=UPD(z i AGGR({m ij ,∀j∈U2})),i∈V1,U2⊆V2 It can be described as follows.

[0044] Node V2 is transformed in a similar symmetrical manner. Edges between two nodes belonging to two separate graphs may not have a representation, or their representation may be calculated based on the representations of the two nodes. For example, it could be a single number indicating the similarity between the two nodes.

[0045] In addition to message passing, a matcher network (304) may include any other layers and operations commonly found in graph neural networks and neural networks in general. For example, there may be fully connected layers before or between message passing layers, layers that transform edge representations, layers that modify graph connectivity, residual connections between layers, various normalization layers, and pooling layers that reduce the number of nodes. All of these layers and operations should be differentiable to ensure that gradients can be used effectively during the training process.

[0046] Within the matcher network (304), a discriminative graph-level representation can be computed for each of the two graphs. This can be done by collecting and aggregating the representations of all nodes in the graph, as is the case in a standard message-passing layer: z G =AGGR({z i ,∀i∈V}

[0047] In some cases, a graph-level representation can be computed by a series of graph pooling layers. If a graph has virtual nodes connected to all nodes in the graph, the representation of these nodes can be interpreted as a graph-level representation. Each output of an existing message-passing layer can be used to compute a separate graph-level representation, and all of these can be combined. A graph-level representation can be combined with an initial graph-level representation and with individual node representations at any point in the network.

[0048] To generate a similarity score s between two graphs, two graph-level representations z are used for each of the two graphs. G1 and z G2 However, it can be calculated and compared using several vector similarity metrics, such as cosine similarity:

number

[0049] Alternatively, a combined graph-level representation of two graphs can be computed by first concatenating or element-wise multiplying the graph-level representations of the two graphs, or by collecting and aggregating the representations of all nodes in the two graphs. This combined graph-level representation can then be reduced to a single value indicating similarity by a fully connected layer.

[0050] In certain cases, it may be beneficial to split the matcher network (304) into two separate networks. Such a variation of the matching method is shown in Figure 5. In this explanation, the first graph neural network (501), called the embedder network (or simply the embedder), takes a single initial graph G=(V,E)(303a), performs only graph operations, and generates the embedding graph G'=(V',E')(502a): G'=EMBEDDER(G)

[0051] The embedded graph (502a) is the node embedding z, which is the final node representation before inter-graph message passing. i , includes ∀i∈V'. In addition to node embeddings, the embedding graph (502a) includes edge embeddings z ij , may also include ∀(i,j)∈E', graph-level embedding z G This may also include the number of node embeddings in the embedding graph (502a). The number of node embeddings in the embedding graph (502a) does not need to correspond to the number of nodes in the initial graph (301a). For example, if there are two types of nodes, the embedder network may return embeddings associated with only one type of node. The same applies to edges, if their embeddings are returned. The embedding graph (502a) may be stored as a template in a template database and can be retrieved whenever it is necessary to match a given fingerprint with a template in the database.

[0052] The second neural network (503) is another variation of the matcher network, which here generates similarity scores s from two embedding graphs G1'(502a) and G2'(502b): s = MATCHER(G1', G2')

[0053] This variation of the matcher network primarily involves inter-graph operations, including an inter-graph message passing layer. For example, the matcher network (503) first calculates a score matrix indicating how well the nodes of one graph match the nodes of the other graph, then performs pooling on each of the two graphs to leave only the nodes that have matches, then creates cross edges between the most similar nodes, then performs inter-graph message passing, and finally aggregates the node representations of both graphs into a single representation, which can then be reduced to a final score value. In some cases, the matcher network (503) may also include an intra-graph message passing layer. In some of these cases, the two embedding graphs (502a, 502b) given to the matcher network (503) do not need to have edge information, and the edges and their representations may be created only within the matcher network (503).

[0054] To function properly and produce reasonable similarity scores, a graph neural network, i.e., a matcher network (303), or an embedded network (501) combined with a matcher network (503) that operates on an embedded graph, should be trained on a sufficiently large fingerprint database with known finger identification information. A selected fingerprint feature extraction algorithm is first run on the fingerprint database to generate an initial set of features, which are used to construct the initial graph. Since all operations within the graph neural network are differentiable, the training procedure may be end-to-end and involve sampling pairs of graphs with known pair labels that are either positive (same finger) or negative (different finger), inputting these pairs of graphs into the neural network to predict similarity scores, and minimizing a selected loss function that measures how well the predicted similarity scores correspond to the pair labels. The entire training procedure may be performed using a machine learning framework such as TensorFlow or PyTorch.

[0055] The trained neural network and the entire algorithm for fingerprint feature matching require implementation on a computer system equipped with suitable hardware and software components to effectively perform this method. The algorithm must be integrated into a specific software application, which may be developed using a suitable programming language such as Python®, C++, or Java®, and utilizing relevant machine learning libraries or frameworks to facilitate graph neural network computations. The computer system should be configured with sufficient processing power, memory, and storage capabilities to handle the complex computations required to perform this method.

[0056] After a graph neural network is trained and an algorithm is developed, the software application encompassing the algorithm can be integrated into a fingerprint-based biometric authentication system. Such a system involves a registration process and a verification or identification process. In the registration process, a subject's fingerprint template is generated and stored in a template database. In the verification process, a subject's fingerprint template is generated and compared with the template of declared identification information in the database, resulting in a similarity score used to determine match / non-match. In the identification process, the subject does not explicitly declare identification information, and the system compares the generated fingerprint template with the templates of all subjects in the database and outputs a list of candidate matching identification information, which may be empty.

[0057] A biometric authentication system using this fingerprint feature matching method can be designed in at least two ways. In one example (Figure 6), only a single neural network, i.e., a matcher that operates on an initial graph (304), is used. In this example, the registration process does not involve the use of a graph neural network and consists of the following steps: scanning the subject's finger (601) with an imaging device (602) such as a fingerprint scanner or a non-contact camera; running the acquired fingerprint (603) through a feature extractor (604) to generate a feature set (605); and storing the feature set (605) together with the subject's identification information (606) as a template in a template database (607). In the verification process, a matcher network (304) is used to compare two graphs (612, 613) constructed (302) from two feature sets (610, 611). The first feature set (610) is generated from the current subject's finger (608), and the second feature set (611) is retrieved from a template database (607) based on the declared identification information (614). In the identification process, the system uses the matcher network (304) to compare the feature set (610) extracted from the captured fingerprint (609) with the feature sets of all subjects in the database (607).

[0058] Another way to design a biometric authentication system incorporating this method is to use two graph neural networks: an embedder (501) and a matcher (503) that operates on the embedding graph. In this example (Figure 7), the registration process involves the additional steps of constructing a graph (701) from a feature set (605) and embedding it using the embedder (501); the embedding graph (702) is stored in a template database. The verification or identification process generates an embedding graph (703) for the current subject using the embedder (501) and compares this graph with an embedding graph (704) retrieved from the database using the matcher (503).

[0059] It is important to note that this method is not limited to fingerprints. In fact, this method, and biometric authentication systems incorporating this method, may operate based on other frictional ridge patterns of the subject. These other frictional ridge patterns or impressions include, for example, palm prints and footprints.

[0060] The above description outlines several features, details, and potential applications of the present invention. This description serves as an exemplary embodiment of the present invention. Modifications to the details may occur in accordance with the most widely understood meanings of the concepts and definitions used in the claims, without departing from the core principles of the present invention. (Other possible items) (Item 1) A method for fingerprint feature matching, The first step of receiving the first set of fingerprint features along with their initial descriptors, Here, each initial descriptor includes the attributes of the corresponding fingerprint feature, and The stage of receiving a second set of fingerprint features along with their initial descriptors, Here, each initial descriptor contains the attributes of the corresponding fingerprint feature. A step of constructing a first graph including nodes and edges, where nodes represent local fingerprint features from the first set, and edges connect adjacent nodes, and The next step is to construct a second graph including nodes and edges, where nodes represent local fingerprint features from the second set and edges connect adjacent nodes. The constructed graph G=(V,E) includes a set of nodes V and a set of edges E. Each node has a numerical representation z i , is associated with ∀i∈V, Each edge is represented numerically by z ij It is associated with ∀(i,j)∈E, The initial node representation is the initial descriptor of the corresponding local fingerprint feature, The initial edge representation is the initial descriptor of the corresponding fingerprint feature between two local fingerprint features, or The initial edge representation is derived from the initial node representation. The step of updating the node representation of the first graph and the node representation of the second graph using a graph neural network, Within the aforementioned graph neural network, the node representation is transformed and updated by the message passing layer. For each node, the message passing layer collects messages from the node's vicinity through edges and aggregates these messages into a new node representation. The first set and the second set of fingerprint features are matched by matching the first graph and the second graph with the graph neural network, and a similarity score is generated. Equipped with, The aforementioned message passing layer is located within and between graphs. Here, the inner layer of the graph passes messages only between nodes belonging to the same graph. Here, the inter-graph layer passes messages between nodes belonging to two separate, matched graphs. After message passing, the updated node representations of both graphs are aggregated into a single vector, and this vector is reduced to a single value representing the similarity score between the two graphs, thereby performing the matching of the two graphs. All operations within the graph neural network are differentiable so that the graph neural network can be trained end-to-end using a gradient-based optimization method. method. (Item 2) The graph itself has graph-level attributes associated with the overall fingerprint features. These attribute vectors are the initial graph-level representations. The method described in item 1. (Item 3) The message passing layer is z i '=UPD(z i AGGR({m ij ,∀j∈N(i)})) M ij=MSG(z i ,z ij ,z G ) As shown, node representation z i new expression z i Convert to ', where MSG is a message from the j-th node to the i-th node m ij A function that calculates AGGR is a function that aggregates messages from neighbors N(i) of the i-th node, which is a set of nodes connected to node i, into a single vector, and UPD is a function that combines the initial node representation with the result of the aggregation to generate the updated node representation. If the message passing layer is in a graph, then N(i) ⊆ V1 when i ∈ V1, and N(i) ⊆ V2 when i ∈ V2, where V1 is the set of nodes in the first graph and V2 is the set of nodes in the second graph. If a message passing layer exists between graphs, then if i ∈ V1, then N(i) ⊆ V2, and if i ∈ V2, then N(i) ⊆ V1, where V1 is the set of nodes in the first graph and V2 is the set of nodes in the second graph. The method described in item 1. (Item 4) Graph-level representation is performed for each of the two graphs in the following way: By collecting and aggregating the representations of all the nodes in the graph, and combining the aggregated results with the initial graph-level representation, By using a series of graph pooling layers and combining the pooling results with the initial graph-level representation, If the graph has virtual nodes connected to all of the nodes of the graph, the representation of these nodes is obtained as a graph-level representation and combined with the initial graph-level representation. The method described in any one of items 1 to 3, calculated by any one of the following: (Item 5) The method described in item 4, in which each output of an existing message-passing layer is used to compute a separate graph-level representation, and all of them are combined into a single representation. (Item 6) The graph-level representation is combined with the node representation at any point in the graph neural network, as described in item 4. (Item 7) The combined graph-level representation of both graphs can be done in the following way: By concatenating the graph-level representations of the two graphs mentioned above, By multiplying the graph-level representations of the two graphs element by element, By collecting and aggregating the representations of all nodes in the two graphs, The method described in any one of items 1 through 6, calculated by any one of the following: (Item 8) The similarity score between the two graphs is, From the graph-level representations of the two graphs mentioned above, Two graph-level representations are compared using a vector similarity metric such as cosine similarity or Euclidean distance. From the combined graph-level representation of the two graphs mentioned above, The connected graph-level representation is reduced to a single value by the fully connected layer. The method described in any one of items 1 through 7, which is generated. (Item 9) The aforementioned graph neural network is Layers that change the connectivity of the graph Fully connected layers that transform node, edge, or graph-level representations. Layer that transforms edge representation, Various normalization layers, A pooling layer that reduces the number of nodes. Interlayer residual connections The method according to any one of items 1 to 8, having any other differentiable layers and operations of a general-purpose graph neural network, including the above. (Item 10) The aforementioned graph neural network is divided into two separate networks. Here, the first graph neural network, the embedder network, takes a single initial graph G=(V,E) and generates an embedding graph G'=(V',E'), Here, the embedder network performs only graph-internal operations. Here, the embedded graph is, Node embedding z is the final node representation before inter-graph message passing. i ,∀i∈V'、 Edge embedding z ij ,∀(i,j)∈E', Graph level embedding z G Includes, Here, the second neural network, the matcher network, obtains two embedding graphs G1' and G2' and generates similarity scores. Here, the matcher network performs operations within and between graphs. The method described in any one of items 1 through 10. (Item 11) The number of node embeddings does not correspond to the number of nodes in the initial graph. The number of edge embeddings does not correspond to the number of edges in the initial graph. The method described in item 10. (Item 12) The method according to item 10 or 11, wherein the two embedding graphs provided to the matcher network do not contain edge information, and the edges and their representations are created only within the matcher network. (Item 13) The training procedure for the aforementioned graph neural network is as follows: The step of obtaining a fingerprint database containing known finger identification information, The step of generating an initial set of features by executing a selected fingerprint feature extraction algorithm on the aforementioned fingerprint database, The stage of constructing the initial graph from an initial set of features, The process involves sampling pairs of graphs that have known pair labels, either positive (same finger) or negative (different finger), inputting these pairs of graphs into the graph neural network to predict similarity scores, and minimizing a selected loss function that measures how well the predicted similarity scores correspond to the pair labels. The method according to any one of items 1 to 12, comprising:

[0061] References

[0062] [1] Engelsma, JJ, Cao, K., Jain, AK(2022).Fixed length fingerprint representation.United States Patent 11,373,438B2.

[0063] [2] Chikkerur, S., Cartwright, AN, Govindaraju, V. (2005). K-plet and Coupled BFS: A Graph Based Fingerprint Representation and Matching Algorithm. In: Zhang, D., Jain, AK (eds) Advances in Biometrics. ICB 2006. Lecture Notes in Computer Science, vol. 3832. Springer, Berlin, Heidelberg.

[0064] [3] Fu, X., Liu, C., Bian, J., Feng, J., Wang, H., Mao, Z. (2013). Extended clique models: A new matching strategy for fingerprint recognition. In: Proceedings of International Conference on Biometrics (ICB) (pp. 1-6), Madrid, Spain.

[0065] [4] Xu,Y.,Lu,G.,Lu,Y.,Liu,F.,Zhang,D.(2018).Fingerprint Pore Comparison Using Local Features and Spatial Relations.In:IEEE Transactions on Circuits and Systems for Video Technology,vol.29,no.10,pp.2927-2940.

[0066] [5] Li,Y.,Gu,C.,Dullien,T.,Vinyals,O.,Kohli,P.(2019).Graph matching networks for learning the similarity of graph structured objects.In:International Conference on Machine Learning,PMLR,pp.3835-3845.

[0067] [6] Fey,M.,Lenssen,J.E.,Morris,C.,Masci,J.,Kriege,N.M.(2020).Deep graph matching consensus. In: International Conference on Learning Representations.

[0068] [7] Shi,Y.,Zhang,Z.,Liu,S.,Liu,M.(2023).Towards More Accurate Matching of Contactless Fingerprints With a Deep Geometric Graph Convolutional Network.In:IEEE Transactions on Biometrics,Behavior,and Identity Science,vol.5,no.1,pp.29-38.

Claims

1. A method for fingerprint feature matching, The first set of fingerprint features is received along with their initial descriptors. Here, each initial descriptor includes the attributes of the corresponding fingerprint feature, and The step of receiving a second set of fingerprint features along with their initial descriptors, Here, each initial descriptor contains the attributes of the corresponding fingerprint feature. A step of constructing a first graph including nodes and edges, where nodes represent local fingerprint features from the first set, edges connect adjacent nodes, and The next step is to construct a second graph including nodes and edges, where nodes represent local fingerprint features from the second set and edges connect adjacent nodes. The constructed graph G = (V, E) includes a set of nodes V and a set of edges E. Each node has a numerical representation z ｉ , is associated with ∀i∈V, Each edge has a numerical representation z ｉｊ , is associated with ∀(i,j)∈E, The initial node representation is the initial descriptor of the corresponding local fingerprint feature, The initial edge representation is the initial descriptor of the corresponding fingerprint feature between two local fingerprint features, or The initial edge representation is derived from the initial node representation. The step of updating the node representation of the first graph and the node representation of the second graph using a graph neural network. Within the aforementioned graph neural network, the node representation is transformed and updated by the message passing layer. For each node, the message passing layer collects messages from the node's vicinity through edges and aggregates these messages into a new node representation. The first set and the second set of fingerprint features are matched by matching the first graph and the second graph with the graph neural network, and a similarity score is generated. Equipped with, The aforementioned message passing layer is located within and between graphs. Here, the inner layer of the graph passes messages only between nodes belonging to the same graph. Here, the inter-graph layer passes messages between nodes belonging to two separate, matched graphs. After message passing, the updated node representations of both graphs are aggregated into a single vector, and this vector is reduced to a single value representing the similarity score between the two graphs, thereby performing the matching of the two graphs. All operations within the graph neural network are differentiable so that the graph neural network can be trained end-to-end using a gradient-based optimization method. method.

2. The graph itself has graph-level attributes associated with the overall fingerprint features. These attribute vectors are the initial graph-level representations. The method according to claim 1.

3. The message passing layer is z ｉ '=UPD(z ｉ ,AGGR({m ｉｊ ,∀j∈N(i)})) M ｉｊ =MSG(z ｉ , z ｉｊ , z Ｇ ) As shown, node representation z ｉ new expression z ｉ Convert to ', where MSG is a message from the j-th node to the i-th node m ｉｊ A function that calculates AGGR is a function that aggregates messages from neighbors N(i) of the i-th node, which is a set of nodes connected to node i, into a single vector, and UPD is a function that combines the initial node representation with the result of the aggregation to generate the updated node representation. If the message passing layer is in the graph, then i∈V １ When N(i)⊆V １ i∈V ２ When N(i)⊆V ２ And here, V １ V is a set of nodes in the first graph mentioned above. ２ This is the set of nodes in the second graph, If the message passing layer is between graphs, then i∈V １ When N(i)⊆V ２ i∈V ２ When N(i)⊆V １ And here, V １ V is a set of nodes in the first graph mentioned above. ２ This is the set of nodes in the second graph mentioned above. The method according to claim 1.

4. Graph-level representation is performed for each of the two graphs in the following way: By collecting and aggregating the representations of all the nodes in the graph, and combining the aggregated results with the initial graph-level representation, By using a series of graph pooling layers and combining the pooling results with the initial graph-level representation, If the graph has virtual nodes connected to all of the nodes of the graph, the representation of these nodes is obtained as a graph-level representation and combined with the initial graph-level representation. The method according to claim 1, calculated by any one of the following:

5. The method according to claim 4, wherein each output of an existing message passing layer is used to compute a separate graph-level representation, and all of them are combined into a single representation.

6. The method according to claim 4, wherein the graph-level representation is combined with the node representation at any point in the graph neural network.

7. The combined graph-level representation of both graphs is as follows: By concatenating the graph-level representations of the two graphs mentioned above, By multiplying the graph-level representations of the two graphs element by element, By collecting and aggregating the representations of all nodes in the two graphs, The method according to claim 1, calculated by any one of the following:

8. The similarity score between the two graphs is, From the graph-level representations of the two graphs mentioned above, Two graph-level representations are compared using a vector similarity metric such as cosine similarity or Euclidean distance. From the combined graph-level representation of the two graphs mentioned above, The connected graph-level representation is reduced to a single value by the fully connected layer. The method according to claim 1, which is generated.

9. The aforementioned graph neural network is Layers that change the connectivity of the graph Fully connected layers that transform node, edge, or graph-level representations. Layer that transforms edge representation, Various normalization layers, A pooling layer that reduces the number of nodes. Interlayer residual connections The method according to claim 1, further comprising any other differentiable layers and operations of a general-purpose graph neural network, including the above.

10. The aforementioned graph neural network is divided into two separate networks. Here, the first graph neural network, the embedder network, takes a single initial graph G = (V, E) and generates an embedding graph G' = (V', E'), Here, the embedder network performs only graph-internal operations. Here, the embedded graph is, Node embedding z is the final node representation before inter-graph message passing. ｉ ,∀i∈V', Edge embedding z ｉｊ ,∀(i,j)∈E', Graph level embedding z Ｇ Includes, Here, the second neural network, the Matcher network, uses two embedding graphs G １ 'and G ２ Obtain ' and generate a similarity score, Here, the matcher network performs operations within and between graphs. The method according to claim 1.

11. The number of node embeddings does not correspond to the number of nodes in the initial graph. The number of edge embeddings does not correspond to the number of edges in the initial graph. The method according to claim 10.

12. The method according to claim 10, wherein the two embedding graphs provided to the matcher network do not contain edge information, and the edges and their representations are created only within the matcher network.

13. The training procedure for the aforementioned graph neural network is as follows: The step of obtaining a fingerprint database containing known finger identification information, The step of generating an initial set of features by executing a selected fingerprint feature extraction algorithm on the aforementioned fingerprint database, The stage of constructing the initial graph from an initial set of features, The process involves sampling pairs of graphs with known pair labels that are either positive (same finger) or negative (different finger), inputting these pairs of graphs into the graph neural network to predict similarity scores, and minimizing a selected loss function that measures how well the predicted similarity scores correspond to the pair labels. The method according to any one of claims 1 to 12, comprising:

Images