Fingerprint Image Matching Model Training Method, Fingerprint Matching Method and Related Media

Abstract

The invention discloses a fingerprint image matching model training method, a fingerprint matching method and related media, the method comprising: extracting features from a fingerprint image to obtain an original feature map; extracting branches based on sweat pore coordinate points, convolving the original feature map to obtain a sweat pore point heat map, and generating branches based on the feature map to convolve the original feature map to obtain an intermediate feature map; using a sliding window algorithm to extract the coordinates of sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point; extracting descriptive features from the intermediate feature map; connecting all sweat pore points using a graph neural network to obtain a structure map corresponding to the fingerprint image; calculating the similarity of the structure map corresponding to the fingerprint image to be matched, and determining the matching degree, and when the matching degree meets the preset training conditions, using the obtained model as the fingerprint image matching model. The invention is used to achieve both the accuracy of sweat pore extraction and the accuracy of fingerprint matching.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular, to a method for training a fingerprint image matching model, a fingerprint matching method, and related media. Background Art

[0002] Biometric recognition systems are widely used in various security fields, including personal identity authentication, mobile payment, criminal investigation, etc. Among them, the automatic fingerprint recognition system is the most widely used biometric recognition system. By collecting the imaging on the fingerprint surface, the features of the fingerprint image are extracted and the same fingerprints are matched. The existing fingerprint image matching methods mainly include fingerprint recognition based on sweat pores and fingerprint matching according to the extracted sweat pores. Among them, sweat pores are rich features in fingerprint images, with a high appearance rate and easy to extract. Using sweat pores as features for fingerprint recognition can improve the security and reliability of fingerprint recognition. This method mainly performs fingerprint matching through steps such as extracting the sweat pore features on the fingerprint image, calculating the descriptors of the sweat pore features, and performing feature matching based on the descriptors and the target fingerprint image. However, due to the various sizes and shapes of sweat pores, traditional methods can only extract some of the sweat pore features, and there are disadvantages such as low accuracy and weak robustness. Especially when facing a large-scale fingerprint data set, there are technical problems in difficultly taking into account both the accuracy of extracting sweat pores and the accuracy of fingerprint matching. Summary of the Invention

[0003] Embodiments of the present invention provide a method for training a fingerprint image matching model, a fingerprint matching method, and related media to achieve both the accuracy of extracting sweat pores and the accuracy of fingerprint matching.

[0004] To solve the above technical problems, an embodiment of the present application provides a method for training a fingerprint image matching model, including:

[0005] Based on the feature extraction network of the initial fingerprint image matching model, perform feature extraction on the fingerprint image to obtain an original feature map. Among them, the initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch;

[0006] Based on the sweat pore coordinate point extraction branch, perform convolution processing on the original feature map to obtain a sweat pore point heat map, and based on the feature map generation branch, perform convolution processing on the original feature map to obtain an intermediate feature map. Among them, the sweat pore point heat map is an image constructed by the features of sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image;

[0007] Adopt a sliding window algorithm to extract the coordinates of the sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point;

[0008] For each of the sweat pore points, description features are extracted from the intermediate feature map according to the coordinates of the sweat pore point, where the description features are used to characterize the features of the sweat pore point;

[0009] Based on the coordinates and description features of all the obtained sweat pore points, the graph neural network is used to connect all the sweat pore points to obtain a structure diagram corresponding to the fingerprint image, where the sweat pore point corresponds to a node of the structure diagram;

[0010] Calculate the similarity of the structure diagram corresponding to the fingerprint image to be matched, and determine the matching degree according to the calculated similarity result. When the matching degree meets the preset training condition, the obtained model is used as the fingerprint image matching model.

[0011] To solve the above technical problems, an embodiment of the present application provides a fingerprint matching method, including:

[0012] Obtain at least two fingerprint images;

[0013] Input all the fingerprint images into the fingerprint image matching model for matching to obtain a matching result, where the fingerprint image matching model is a model trained according to the above fingerprint image matching model training method.

[0014] To solve the above technical problems, an embodiment of the present application further provides a fingerprint image matching model training device, including:

[0015] An original feature map determination module, configured to perform feature extraction on a fingerprint image based on a feature extraction network of an initial fingerprint image matching model to obtain an original feature map, where the initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch;

[0016] A convolution module, configured to perform convolution processing on the original feature map based on the sweat pore coordinate point extraction branch to obtain a sweat pore point heat map, and perform convolution processing on the original feature map based on the feature map generation branch to obtain an intermediate feature map, where the sweat pore point heat map is an image constructed by the features of the sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image;

[0017] A coordinate determination module, configured to adopt a sliding window algorithm to extract the coordinates of the sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point;

[0018] A description feature extraction module, configured to extract description features from the intermediate feature map according to the coordinates of each sweat pore point, where the description features are used to characterize the features of the sweat pore point;

[0019] A structure diagram determination module, configured to connect all the sweat pore points by using the graph neural network based on the coordinates and description features of all the obtained sweat pore points, so as to obtain a structure diagram corresponding to the fingerprint image, where the sweat pore points correspond to the nodes of the structure diagram;

[0020] A fingerprint image matching model determination module, configured to calculate the similarity of the structure diagram corresponding to the fingerprint image to be matched, and determine the matching degree according to the calculated similarity result. When the matching degree meets the preset training condition, the obtained model is used as the fingerprint image matching model.

[0021] To solve the above technical problems, an embodiment of the present application further provides a fingerprint matching device, including:

[0022] A fingerprint image acquisition module, configured to acquire at least two fingerprint images;

[0023] A matching module, configured to input all the fingerprint images into the fingerprint image matching model for matching to obtain a matching result, where the fingerprint image matching model is a model trained according to the above fingerprint image matching model training method.

[0024] To solve the above technical problems, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above fingerprint image matching model training method are implemented, or when the processor executes the computer program, the steps of the above fingerprint matching method are implemented.

[0025] To solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above fingerprint image matching model training method are implemented, or when the computer program is executed by a processor, the steps of the above fingerprint matching method are implemented.

[0026] The fingerprint image matching model training method, fingerprint matching method, device, computer device and storage medium provided by the embodiments of the present invention extract features from fingerprint images through a feature extraction network based on an initial fingerprint image matching model to obtain an original feature map. The initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch; based on the sweat pore coordinate point extraction branch, perform convolution processing on the original feature map to obtain a sweat pore point heat map, and based on the feature map generation branch, perform convolution processing on the original feature map to obtain an intermediate feature map. The sweat pore point heat map is an image constructed from the features of sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image; adopt a sliding window algorithm to extract the coordinates of the sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point; for each sweat pore point, extract a description feature from the intermediate feature map according to the coordinates of the sweat pore point, where the description feature is used to characterize the features of the sweat pore point; based on the coordinates and description features of all the obtained sweat pore points, use the graph neural network to connect all the sweat pore points to obtain a structure diagram corresponding to the fingerprint image, where the sweat pore points correspond to the nodes of the structure diagram; calculate the similarity of the structure diagram corresponding to the fingerprint image to be matched, and determine the matching degree according to the calculated similarity result. When the matching degree meets the preset training conditions, use the obtained model as the fingerprint image matching model, improve the accuracy of sweat pore extraction in fingerprint images and enhance the matching accuracy, thereby solving the problem that the feature extraction and matching of high-precision fingerprint images cannot be integrated, and making the fingerprint recognition system more accurate and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts.

[0028] Figure 1 is an exemplary system architecture diagram to which the present application can be applied;

[0029] Figure 2 is a flowchart of an embodiment of the fingerprint image matching model training method of the present application;

[0030] Figure 3 is a flowchart of an embodiment of the fingerprint matching method of the present application;

[0031] Figure 4It is a schematic structural diagram of an embodiment of a fingerprint image matching model training device according to the present application;

[0032] Figure 5 It is a schematic structural diagram of an embodiment of a fingerprint matching device according to the present application;

[0033] Figure 6 It is a schematic structural diagram of an embodiment of a computer device according to the present application. Detailed implementation manners

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and not to describe a specific order.

[0035] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0037] Please refer to Figure 1 , as Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0038] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc.

[0039] The terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture EpertsGroup Audio Layer III), MP4 (Moving PictureEperts Group Audio Layer IV) players, laptop computers, desktop computers, and so on.

[0040] The server 105 can be a server that provides various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, and 103.

[0041] It should be noted that the fingerprint image matching model training method and the fingerprint matching method provided by the embodiments of the present application are executed by the server. Correspondingly, the fingerprint image matching model training device and the fingerprint matching device are set in the server.

[0042] It should be understood that Figure 1 the numbers of the terminal devices, the network, and the server in

[0043] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. The terminal devices 101, 102, and 103 in the embodiments of the present application can specifically correspond to the application systems in actual production.

[0043] Please refer to Figure 2 , Figure 2 which shows a fingerprint image matching model training method provided by an embodiment of the present invention. Taking the method applied to the Figure 1 server side in

[0044] S201. Based on the feature extraction network of the initial fingerprint image matching model, extract features from the fingerprint image to obtain an original feature map. Among them, the initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch.

[0045] Specifically, the above feature extraction network includes but is not limited to a VGG convolutional neural network and a residual neural network.

[0046] Preferably, the present application adopts a U-net structure feature extraction network, which includes a convolution module, a downsampling module, an upsampling module, and a feature map splicing module. Among them, the convolution module is used to perform convolution operations with a 3×3 convolution kernel, batch normalization operations, and ReLu activation processing. The downsampling module successively passes through 4 convolution modules and max pooling operations to ensure that the height and width of the feature map are respectively reduced to 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the height and width of the fingerprint image. At the same time, the dimensions of the feature map are respectively increased to 64, 128, 256, 512, and 1024. The upsampling module is used to double the height and width of the obtained feature map through bilinear interpolation. The feature map splicing module is used to splice the feature maps of the same size during the upsampling module and the downsampling module, and obtain the original feature map through convolution.

[0047] It should be noted here that the dimension of the fingerprint image is 512. The original feature map has the same height and width as the fingerprint image and a dimension of 256.

[0048] The above graph neural network is a neural network used to construct a structure diagram for the extracted feature map.

[0049] The above sweat pore coordinate point extraction branch is used to extract the coordinate points of sweat pores on the fingerprint image.

[0050] The above feature map generation branch is used to generate a feature map of the same dimension as the fingerprint image.

[0051] In this embodiment, through the feature extraction network of the initial fingerprint image matching model, the fingerprint image is feature-extracted to obtain the original feature map, so as to facilitate subsequent processing of the sweat pore coordinate points of the original feature map and generate the corresponding feature map, and a structure diagram is constructed through the graph neural network to improve the accuracy of sweat pore extraction in the fingerprint image and enhance the matching accuracy, thereby solving the problem that the feature extraction and matching of high-precision fingerprint images cannot be integrated, making the fingerprint recognition system more accurate and efficient.

[0052] S202. Based on the sweat pore coordinate point extraction branch, the original feature map is subjected to convolution processing to obtain a sweat pore point heat map, and based on the feature map generation branch, the original feature map is subjected to convolution processing to obtain an intermediate feature map. Among them, the sweat pore point heat map is an image constructed from the features of sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image.

[0053] Specifically, through the sweat pore coordinate point extraction branch, the original feature map is subjected to convolution operations to convert the original feature map into a 1D sweat pore point heat map, which is used to learn the coordinate information of sweat pores in the fingerprint image.

[0054] It should be noted here that by extracting branches through pore coordinate points, pore points are extracted from the original feature map, and the features of the extracted pore points are convolved to obtain 1D features. The pore points and the features of the pore points are constructed to form a pore point heat map.

[0055] Through the feature map generation branch, a convolution operation is performed on the original feature map to obtain an intermediate feature map with the same size as the fingerprint image and a dimension of 512. This intermediate feature map is used to obtain the description feature of the pore point from the intermediate feature map itself using the coordinate information of the pore points in the pore point heat map.

[0056] For example, when there is a pore point A with coordinates (10, 5), the description feature B is extracted from the 10th row and 5th column of the intermediate feature map, and this description feature B is the description feature of the pore point.

[0057] In this embodiment, by performing a convolution operation on the original feature map, a pore point heat map and an intermediate feature map are obtained, improving the accuracy of pore extraction in the fingerprint image.

[0058] S203. Adopt a sliding window algorithm to extract the coordinates of the pore points in the pore point heat map and determine the coordinates of each pore point.

[0059] Specifically, the above sliding window algorithm refers to a method of processing on a string or array of a specific size, and this string or array is the sliding window. That is, a sliding window of a specific size is used to extract the coordinates of the pore points in the pore point heat map, thereby determining the coordinates of each pore point.

[0060] S204. For each pore point, extract the description feature from the intermediate feature map according to the coordinates of the pore point, where the description feature is used to characterize the feature of the pore point.

[0061] The above description feature is a ridge feature.

[0062] The process of extracting the ridge feature includes but is not limited to operations such as image normalization, direction extraction, ridge frequency calculation, filtering, and binarization of the intermediate feature map.

[0063] In this embodiment, by extracting the description feature from the intermediate feature map according to the coordinates of the pore points, the accuracy of pore extraction in the fingerprint image can be improved.

[0064] S205. Based on the coordinates and description features of all the obtained pore points, use a graph neural network to connect all the pore points to obtain a structure diagram corresponding to the fingerprint image, where the pore points correspond to the nodes of the structure diagram.

[0065] Specifically, a multi-layer perceptron is used as a position encoder to fuse the position information with the intermediate feature map, obtaining a fused feature vector, which is the initial value of the graph node feature vector. The position information is the coordinates of each sweat pore point and the corresponding descriptive features of the sweat pore point.

[0066] It should be understood that a sweat pore point corresponds to a node in the structure diagram. The position information of a sweat pore point is fused with the intermediate feature map to obtain the fused feature vector corresponding to the sweat pore point.

[0067] When constructing each edge of the structure diagram, the feature vector corresponding to the edge is defined by the feature vectors of the two nodes connected by the edge.

[0068] When all nodes and all edges are constructed, an initial structure diagram is obtained. Graph convolution is performed on the initial structure diagram to obtain the structure diagram.

[0069] It should be noted here that graph convolution means that for each node, the information of its adjacent nodes is aggregated to the node through different convolutional kernels, so that the features of each node can combine the information of other nodes and the global information of the graph, and then achieve the fusion of local features and global features to generate a new graph.

[0070] In this embodiment, after the graph convolution operation, each fingerprint image can obtain a set of descriptors representing sweat pore points, improving the accuracy of sweat pore extraction in the fingerprint image and enhancing the matching accuracy, thereby solving the problem that the feature extraction and matching of high-precision fingerprint images cannot be integrated, making the fingerprint recognition system more accurate and efficient.

[0071] S206. Calculate the similarity of the structure diagrams corresponding to the fingerprint images to be matched, and determine the matching degree according to the calculated similarity result. When the matching degree meets the preset training condition, the obtained model is used as the fingerprint image matching model.

[0072] Specifically, each fingerprint image corresponds to a structure diagram. The fingerprint images to be matched refer to at least two fingerprint images.

[0073] By calculating the similarity of the structure diagrams of the fingerprint images to be matched, a similarity result is obtained. The similarity result is used to determine whether the fingerprint images to be matched are matched. The matching degree is calculated through the similarity result.

[0074] The above-mentioned preset training condition means that when the matching degree is greater than the preset matching degree, the obtained model is used as the fingerprint image matching model.

[0075] It should be noted here that in the sweat pore coordinate extraction branch, the sweat pore point heat map is used for supervision. By calculating the mean square error loss between the predicted label of the output sweat pore point heat map and the annotation result of the fingerprint image, this mean square error loss is denoted as Lossmse 。

[0076] Calculate the cross - entropy loss between the similarity matrix and the similarity matrix corresponding to the annotation results of the fingerprint images to be matched. This cross - entropy loss is denoted as Loss bce 。

[0077] During the network training process, use a siamese network with shared weights for two input fingerprint images. The two obtained pore - point heatmaps are used as predictions of feature coordinates, and the similarity matrix is used as the prediction of feature matching scores. Since the entire forward process is differentiable, both of these losses can be backpropagated. Use a loss - weight hyperparameter α to adjust the two losses, that is, calculate the loss function according to the following formula (1):

[0078] Loss all =Loss mse +α×Loss bce (1)

[0079] Among them, Loss mse is the mean - square error loss, Loss bce is the cross - entropy loss, a is the loss - weight hyperparameter, and Loss all is the loss function.

[0080] When the matching degree does not meet the preset training conditions, then calculate the loss function and adjust the model parameters to retrain the fingerprint images.

[0081] In this embodiment, through the above steps, improve the accuracy of pore extraction in fingerprint images and enhance the matching accuracy, thereby solving the problem that the feature extraction and matching of high - precision fingerprint images cannot be integrated, making the fingerprint recognition system more accurate and efficient.

[0082] In some alternative implementation manners of this embodiment, step S203 includes:

[0083] S2031. Traverse the pixel values corresponding to all pixel points of the pore - point heatmap, and set the pixel values of the pixel points that meet the preset conditions to 0 to obtain an intermediate heatmap. Among them, the preset condition is that the pixel value of the pixel point is less than the preset pixel threshold.

[0084] S2032. For each pixel point of the intermediate heatmap, construct a window for the pixel point and the pixel points within the preset - size window to obtain a window.

[0085] S2033. Traverse the pixel values in the window, and mark the pixel point with the largest pixel value as a pore.

[0086] S2034. Determine the coordinates of the pore points according to the positions of the pores in the intermediate heatmap.

[0087] For step S2032, the above-mentioned preset size window refers to a window composed of a string or an array of preset sizes.

[0088] For example, when the predicted size window is 4 and the window is a square window, the traversed pixel point can be used as the first value of the window, the first pixel point to the left of this pixel point as the second value, the first pixel point below this pixel point as the third value, and the second pixel point below this pixel point as the fourth value to construct a window.

[0089] In this embodiment, by determining the coordinates of the sweat pore points through the above steps, the extraction accuracy of the sweat pore points is improved.

[0090] In some optional implementation manners of this embodiment, step S205 includes:

[0091] S2051. Based on the connectivity algorithm, perform region division on all ridge line features on the intermediate feature map to obtain a region division map, where the sweat pore points are located on the ridge lines.

[0092] S2052. Based on the preset connection rules, use a graph neural network to connect all the sweat pore points in the region division map to obtain a structure map corresponding to the fingerprint image.

[0093] For step S2051, the above-mentioned connectivity algorithm refers to an algorithm for judging whether there is a connected path between two nodes.

[0094] The implementation manners of the above-mentioned connectivity algorithm include but are not limited to depth-first search manner, breadth-first search manner, and union-find set manner.

[0095] For step S2052, the above-mentioned preset connection rules include but are not limited to that the sweat pore points and the ridge lines are in the same region, the sweat pore points with similar ridge line directions, the sweat pore points with close distances between them, and each sweat pore point is connected to at most two points. The specific connection rules can be set according to the actual situation.

[0096] When the sweat pore points are connected, a graph structure can be constructed. In the structure map, each node corresponds to a sweat pore point on the fingerprint image, and the edges in the structure map are consistent with the trend of the ridge lines in the fingerprint image.

[0097] In this embodiment, using the graph neural network to combine the local features of the sweat pore points and the global features of the ridge lines in the fingerprint image improves the accuracy of extracting the sweat pore points in the fingerprint image, enhances the matching accuracy, and solves the problem that the feature extraction and matching of high-precision fingerprint images cannot be integrated, making the fingerprint recognition system more accurate and efficient.

[0098] In some optional implementation manners of this embodiment, step S206 includes:

[0099] S2061. Calculate the similarity of the structure diagrams corresponding to the fingerprint images to be matched to obtain a similarity matrix.

[0100] S2062. Determine the matching sweat pore points on the fingerprint images to be matched according to the similarity matrix.

[0101] S2063. Determine the matching degree according to the matching sweat pore points and the total number of sweat pore points.

[0102] S2064. When the matching degree meets the preset training conditions, use the obtained model as the fingerprint image matching model.

[0103] For step S2062, if an element in a certain item of the similarity matrix is the maximum value in the row and the maximum value in the column, it is considered that the sweat pore points in the corresponding two diagrams are matched.

[0104] The matching sweat pore points refer to the sweat pore points determined to be matched on the fingerprint images to be matched. For step S2063, the matching degree refers to the ratio of the number of matching sweat pore points to the total number of sweat pore points.

[0105] When the total number of sweat pore points of the fingerprint images to be matched is different, calculate the matching degree based on the total number of sweat pore points of the fingerprint image with fewer sweat pore points.

[0106] For step S2064, the above-mentioned preset training conditions mean that the matching degree is not less than the preset matching degree.

[0107] In this embodiment, through the above steps, the accuracy of extracting sweat pore points in fingerprint images is improved, the matching accuracy is enhanced, and the problem that the feature extraction and matching of high-precision fingerprint images cannot be integrated is solved, making the fingerprint recognition system more accurate and efficient.

[0108] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution is prior or subsequent. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0109] Please refer to Figure 3 , Figure 3 which shows a fingerprint matching method provided by an embodiment of the present invention. Taking the fingerprint image matching model applied in Figure 2 as an example, the following steps S301 to step S302 are described in detail:

[0110] S301. Obtain at least two fingerprint images.

[0111] S302. Input all fingerprint images into the fingerprint image matching model for matching to obtain a matching result, where the fingerprint image matching model is a model trained according to the above fingerprint image matching model training method.

[0112] For step S301, the above fingerprint image is a fingerprint image to be matched.

[0113] For step S302, input all fingerprint images into the fingerprint image matching model. The fingerprint image matching model extracts the sweat pore point heat map and the intermediate feature map of each fingerprint image, uses a graph neural network to construct a structure graph for the sweat pore point heat map and the intermediate feature map, obtains the structure graph of the fingerprint image, calculates the similarity of the structure graph of the fingerprint image to be matched, and determines the matching result according to the similarity.

[0114] In this embodiment, the accuracy of sweat pore extraction in fingerprint images is improved and the matching accuracy is enhanced, thereby solving the problem that the feature extraction and matching of high-precision fingerprint images cannot be integrated, making the fingerprint recognition system more accurate and efficient.

[0115] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0116] Figure 4 The principle block diagram of the fingerprint image matching model training device corresponding one by one to the fingerprint image matching model training method in the above embodiment is shown. As Figure 4 shown, the fingerprint image matching model training device includes an original feature map determination module 41, a convolution module 42, a coordinate determination module 43, a description feature extraction module 44, a structure graph determination module 45, and a fingerprint image matching model determination module 46. The detailed description of each functional module is as follows:

[0117] The original feature map determination module 41 is configured to perform feature extraction on a fingerprint image based on the feature extraction network of the initial fingerprint image matching model to obtain an original feature map, where the initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch.

[0118] The convolution module 42 is configured to perform convolution processing on the original feature map based on the sweat pore coordinate point extraction branch to obtain a sweat pore point heat map, and perform convolution processing on the original feature map based on the feature map generation branch to obtain an intermediate feature map, where the sweat pore point heat map is an image constructed from the features of sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image.

[0119] The coordinate determination module 43 is configured to extract the coordinates of the sweat pore points in the sweat pore point heat map by using a sliding window algorithm, and determine the coordinates of each sweat pore point.

[0120] The description feature extraction module 44 is configured to extract the description features from the intermediate feature map according to the coordinates of each sweat pore point for each sweat pore point, where the description features are used to characterize the features of the sweat pore points.

[0121] The structure diagram determination module 45 is configured to connect all the sweat pore points by using a graph neural network based on the coordinates and description features of all the obtained sweat pore points, and obtain the structure diagram corresponding to the fingerprint image, where the sweat pore points correspond to the nodes of the structure diagram.

[0122] The fingerprint image matching model determination module 46 is configured to calculate the similarity of the structure diagram corresponding to the fingerprint image to be matched, and determine the matching degree according to the calculated similarity result. When the matching degree meets the preset training conditions, the obtained model is used as the fingerprint image matching model.

[0123] In some optional implementation manners of this embodiment, the coordinate determination module 43 includes:

[0124] The intermediate heat map determination unit is configured to traverse the pixel values corresponding to all the pixel points of the sweat pore point heat map, and set the pixel values of the pixel points that meet the preset conditions to 0 to obtain an intermediate heat map, where the preset condition is that the pixel value of the pixel point is less than the preset pixel threshold.

[0125] The window determination unit is configured to construct a window for each pixel point of the intermediate heat map with the pixel points within a preset size window to obtain a window.

[0126] The sweat pore determination unit is configured to traverse the pixel values in the window, and mark the pixel point with the largest pixel value as a sweat pore.

[0127] The coordinate determination unit is configured to determine the coordinates of the sweat pore points according to the positions of the sweat pores in the intermediate heat map.

[0128] In some optional implementation manners of this embodiment, the description feature is a ridge feature.

[0129] In some optional implementation manners of this embodiment, the structure diagram determination module 45 includes:

[0130] The region division unit is configured to perform region division on all the ridge features on the intermediate feature map based on a connectivity algorithm to obtain a region division map, where the sweat pore points are located on the ridges.

[0131] The structure diagram determination unit is configured to connect all the sweat pore points of the region division map by using a graph neural network based on a preset connection rule to obtain the structure diagram corresponding to the fingerprint image.

[0132] In some alternative implementation manners of this embodiment, the fingerprint image matching model determination module 46 includes:

[0133] A similarity matrix calculation unit, configured to calculate the similarity of the structure diagrams corresponding to the fingerprint images to be matched, and obtain a similarity matrix.

[0134] A matching pore point determination unit, configured to determine the matching pore points on the fingerprint images to be matched according to the similarity matrix.

[0135] A matching degree determination unit, configured to determine the matching degree according to the matching pore points and the total number of pore points.

[0136] A fingerprint image matching model determination unit, configured to use the obtained model as the fingerprint image matching model when the matching degree meets the preset training conditions.

[0137] For the specific limitations on the fingerprint image matching model training device, reference may be made to the limitations on the fingerprint image matching model training method in the foregoing text, which will not be elaborated herein. Each module in the foregoing fingerprint image matching model training device may be implemented in whole or in part by software, hardware, and their combination. The foregoing modules may be embedded in the processor in the computer device in hardware form or independent of the processor, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the foregoing modules.

[0138] Figure 5 The principle block diagram of the fingerprint matching device corresponding one by one to the fingerprint matching method in the foregoing embodiment is shown. As Figure 5 shown, the fingerprint matching device includes a fingerprint image acquisition module 51 and a matching module 52. The detailed description of each functional module is as follows:

[0139] The fingerprint image acquisition module 51 is configured to acquire at least two fingerprint images.

[0140] The matching module 52 is configured to input all the fingerprint images into the fingerprint image matching model for matching, and obtain a matching result, where the fingerprint image matching model is a model trained according to the foregoing fingerprint image matching model training method.

[0141] For the specific limitations on the fingerprint matching device, reference may be made to the limitations on the fingerprint matching method in the foregoing text, which will not be elaborated herein. Each module in the foregoing fingerprint matching device may be implemented in whole or in part by software, hardware, and their combination. The foregoing modules may be embedded in the processor in the computer device in hardware form or independent of the processor, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the foregoing modules.

[0142] To solve the above technical problems, the embodiments of the present application also provide a computer device. Specifically, please refer to Figure 6 , Figure 6 , which is the basic structural block diagram of the computer device in this embodiment.

[0143] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are communicatively connected to each other through a system bus. It should be noted that only the computer device 6 with components connected to the memory 61, the processor 62, and the network interface 63 is shown in the figure. However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of the present technology can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0144] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can interact with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0145] The memory 61 includes at least one type of readable storage medium, which includes flash memory, hard disk, multimedia card, card-type memory (such as SD or D-interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as the hard disk or memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the computer device 6. Of course, the memory 61 may also include both the internal storage unit and the external storage device of the computer device 6. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as program codes for controlling electronic files. In addition, the memory 61 may also be used to temporarily store various types of data that have been output or will be output.

[0146] In some embodiments, the processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the program codes stored in the memory 61 or process data, such as running the program codes for controlling electronic files.

[0147] The network interface 63 may include a wireless network interface or a wired network interface, and this network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0148] This application also provides another implementation manner, that is, to provide a computer-readable storage medium storing an interface display program, and the interface display program can be executed by at least one processor, so that the at least one processor executes the steps of the fingerprint image matching model training method as described above, or so that the at least one processor executes the steps of the fingerprint matching method as described above.

[0149] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0150] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields is equally within the scope of the patent protection of the present application.

Claims

1. A method for training a fingerprint image matching model, characterized in that, the method for training the fingerprint image matching model includes: Based on the feature extraction network of the initial fingerprint image matching model, extracting features from the fingerprint image to obtain an original feature map. Among them, the initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch; Based on the sweat pore coordinate point extraction branch, performing convolution processing on the original feature map to obtain a sweat pore point heat map, and based on the feature map generation branch, performing convolution processing on the original feature map to obtain an intermediate feature map. Among them, the sweat pore point heat map is an image constructed by the features of sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image; Adopting a sliding window algorithm to extract the coordinates of the sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point; For each of the sweat pore points, extracting a description feature from the intermediate feature map according to the coordinates of the sweat pore point, where the description feature is used to characterize the features of the sweat pore point; Based on the coordinates and description features of all the obtained sweat pore points, using the graph neural network to connect all the sweat pore points to obtain a structure diagram corresponding to the fingerprint image, where the sweat pore points correspond to the nodes of the structure diagram; Calculating the similarity of the structure diagram corresponding to the fingerprint image to be matched, and determining the matching degree according to the calculated similarity result. When the matching degree meets the preset training conditions, the obtained model is used as the fingerprint image matching model.

2. The method for training a fingerprint image matching model according to claim 1, characterized in that, the step of adopting a sliding window algorithm to extract the coordinates of the sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point includes: Traversing the pixel values corresponding to all pixel points of the sweat pore point heat map, and setting the pixel values of the pixel points that meet the preset conditions to 0 to obtain an intermediate heat map, where the preset condition is that the pixel value of the pixel point is less than the preset pixel threshold; For each pixel point of the intermediate heat map, constructing a window for the pixel point and the pixel points within a preset size window to obtain a window; Traversing the pixel values in the window, and marking the pixel point with the largest pixel value as a sweat pore; Determining the coordinates of the sweat pore point according to the position of the sweat pore in the intermediate heat map.

3. The method for training a fingerprint image matching model according to claim 1, characterized in that, the description feature is a ridge feature.

4. The method for training a fingerprint image matching model according to claim 3, characterized in that, the step of based on the coordinates and description features of all the obtained sweat pore points, using the graph neural network to connect all the sweat pore points to obtain a structure diagram corresponding to the fingerprint image includes: Based on a connectivity algorithm, performing region division on all the ridge features on the intermediate feature map to obtain a region division map, where the sweat pore points are located on the ridges; Based on the preset connection rules, use a graph neural network to connect all the sweat pore points of the regional division graph to obtain the structure graph corresponding to the fingerprint image.

5. The fingerprint image matching model training method according to claim 1, wherein, the steps of calculating the similarity of the structure graph corresponding to the fingerprint image to be matched, and determining the matching degree according to the calculated similarity result, and when the matching degree meets the preset training conditions, using the obtained model as the fingerprint image matching model include: Calculating the similarity of the structure graph corresponding to the fingerprint image to be matched to obtain a similarity matrix; Determining the matching sweat pore points on the fingerprint image to be matched according to the similarity matrix; Determining the matching degree according to the matching sweat pore points and the total number of sweat pore points; When the matching degree meets the preset training conditions, using the obtained model as the fingerprint image matching model.

6. A fingerprint matching method, wherein, the fingerprint matching method includes: Obtaining at least two fingerprint images; Inputting all the fingerprint images into a fingerprint image matching model for matching to obtain a matching result, wherein the fingerprint image matching model is a model trained according to the fingerprint image matching model training method according to any one of claims 1 to 5.

7. A fingerprint image matching model training device, wherein, the fingerprint image matching model training device includes: An original feature map determination module, configured to perform feature extraction on a fingerprint image based on the feature extraction network of an initial fingerprint image matching model to obtain an original feature map, wherein the initial fingerprint image matching model further includes a graph neural network, and the feature extraction network further includes a sweat pore coordinate point extraction branch and a feature map generation branch; A convolution module, configured to perform convolution processing on the original feature map based on the sweat pore coordinate point extraction branch to obtain a sweat pore point heat map, and perform convolution processing on the original feature map based on the feature map generation branch to obtain an intermediate feature map, wherein the sweat pore point heat map is an image constructed from the features of sweat pore points, and the intermediate feature map has the same feature dimension as the fingerprint image; A coordinate determination module, configured to use a sliding window algorithm to extract the coordinates of the sweat pore points in the sweat pore point heat map to determine the coordinates of each sweat pore point; A description feature extraction module, configured to extract a description feature from the intermediate feature map according to the coordinates of each sweat pore point, wherein the description feature is used to characterize the features of the sweat pore point; A structure graph determination module, configured to connect all the sweat pore points using the graph neural network based on the obtained coordinates and description features of all the sweat pore points to obtain the structure graph corresponding to the fingerprint image, wherein the sweat pore points correspond to the nodes of the structure graph; A fingerprint image matching model determination module, configured to calculate the similarity of the structure graph corresponding to the fingerprint image to be matched, and determine the matching degree according to the calculated similarity result, and when the matching degree meets the preset training conditions, using the obtained model as the fingerprint image matching model.

8. A fingerprint matching device, wherein, the fingerprint matching device includes: A fingerprint image acquisition module, configured to acquire at least two fingerprint images; A matching module, configured to input all the fingerprint images into a fingerprint image matching model for matching to obtain a matching result, wherein the fingerprint image matching model is a model trained according to the fingerprint image matching model training method described in any one of claims 1 to 5.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, when the processor executes the computer program, it implements the fingerprint image matching model training method described in any one of claims 1 to 5, or when the processor executes the computer program, it implements the fingerprint matching method described in claim 6.

10. A computer-readable storage medium storing a computer program, wherein, when the computer program is executed by a processor, it implements the fingerprint image matching model training method described in any one of claims 1 to 5, or when the computer program is executed by a processor, it implements the fingerprint matching method described in claim 6.

Images