Fingerprint image feature enhancement method, device, equipment and storage medium

Abstract

The present application relates to the field of image processing technology, and discloses a fingerprint image feature enhancement method, device, equipment and storage medium. The method comprises: obtaining an original fingerprint image; dividing the original fingerprint image into blocks to obtain a plurality of fingerprint image blocks; extracting texture features, directional features and grain clarity features from the fingerprint image blocks to obtain texture feature information, directional feature information and grain clarity feature information; classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, directional feature information and grain clarity feature information; when the original fingerprint image is a wet image, inputting the original fingerprint image into a first feature enhancement model to obtain a corresponding enhanced fingerprint image; when the original fingerprint image is a dry image, inputting the original fingerprint image into a second feature enhancement model to obtain a corresponding enhanced fingerprint image. The embodiment of the present application can enhance the fingerprint features in the fingerprint image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a fingerprint image feature enhancement method, device, equipment and storage medium. Background Art

[0002] Fingerprint recognition, as a biometric-based identity authentication technology, is increasingly being used.

[0003] In the related art, fingerprint recognition requires first extracting fingerprint feature data, then performing preliminary processing on the fingerprint feature data to obtain clearer fingerprint feature data, and then matching feature points of the processed fingerprint feature data with a pre-stored fingerprint template.

[0004] However, current fingerprint recognition methods do not work well for fingerprint images with few features and low quality. For example, wet fingers can cause fingerprint images to have blurred fingerprint surface contour details and reduced contrast between ridges and valleys. If a small-area fingerprint sensor is used to collect fingerprint images, the fingerprint image quality will be greatly reduced based on the scarcity of fingerprint features, making fingerprint recognition very difficult. Summary of the invention

[0005] The purpose of this application is to provide a fingerprint image feature enhancement method, device, equipment and storage medium, which can enhance the fingerprint features in the fingerprint image to improve the accuracy of fingerprint image recognition.

[0006] The present application provides a fingerprint image feature enhancement method, comprising:

[0007] Obtaining the original fingerprint image;

[0008] Dividing the original fingerprint image into blocks to obtain a plurality of fingerprint image blocks;

[0009] Extracting texture features from the fingerprint image block to obtain texture feature information;

[0010] Extracting directional features from the fingerprint image block to obtain directional feature information;

[0011] Extracting texture clarity features from the fingerprint image block to obtain texture clarity feature information;

[0012] Classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information;

[0013] When the original fingerprint image is a wet image, the original fingerprint image is input into a first feature enhancement model, and ridge feature enhancement is performed on the original fingerprint image to obtain a corresponding enhanced fingerprint image;

[0014] When the original fingerprint image is a dry image, the original fingerprint image is input into a second feature enhancement model, and ridge feature enhancement is performed on the original fingerprint image to obtain a corresponding enhanced fingerprint image.

[0015] In some embodiments, extracting texture features from the fingerprint image block to obtain texture feature information includes:

[0016] Calculating a grayscale histogram of the fingerprint image block;

[0017] According to the grayscale histogram, generating a corresponding grayscale level statistical vector;

[0018] Generate a gray level co-occurrence matrix of the fingerprint image block, and calculate the normalized mean, variance, energy, contrast, inverse moment and entropy of the gray level co-occurrence matrix;

[0019] Output the gray level statistical vector and the normalized mean, variance, energy, contrast, inverse moment and entropy of the gray level co-occurrence matrix to obtain the texture feature information.

[0020] In some embodiments, extracting the directional feature of the fingerprint image block to obtain the directional feature information includes:

[0021] Calculating the gradient of each pixel in the fingerprint image block;

[0022] Calculating the directional saliency feature information and directional consistency feature information of the fingerprint image block according to the gradient of the pixel point;

[0023] The directional saliency feature information and the directional consistency feature information are output to obtain the directional feature information.

[0024] In some embodiments, extracting the texture clarity feature of the fingerprint image block to obtain texture clarity feature information includes:

[0025] Calculating the average gradient of the fingerprint image block;

[0026] Calculating main direction feature information of the fingerprint image block according to the average gradient;

[0027] According to the main direction feature information, grayscale projection is performed on the fingerprint image block, and pixel points of the fingerprint image block are classified as ridge pixel points or valley pixel points according to the grayscale projection result;

[0028] Generate grayscale histograms of the ridge line pixels and the valley line pixels to calculate the proportion of pixels that do not meet the clarity threshold condition, and obtain ridge line clarity feature information and valley line clarity feature information;

[0029] According to the grayscale projection result, the width ratio between the ridge line and the valley line is calculated;

[0030] The ridge line clarity feature information, the valley line clarity feature information and the width ratio between the ridge line and the valley line are output to obtain the texture clarity feature information.

[0031] In some embodiments, the classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information includes:

[0032] splicing the texture feature information, the direction feature information and the texture clarity feature information to obtain fused feature information;

[0033] The fused feature information is input into an image classification model to classify the fingerprint image block into a wet image block, a dry image block or a normal image block; the image classification model comprises three classification sub-models, and the classification sub-models respectively perform corresponding binary classification judgment operations, and the binary classification judgment operations include judging whether the fingerprint image block is a wet image block, judging whether the fingerprint image block is a dry image block, and judging whether the fingerprint image block is a normal image block;

[0034] According to the number of the wet image blocks, the dry image blocks and the normal image blocks, the original fingerprint image is classified as a wet image, a dry image or a normal image.

[0035] In some embodiments, the step of performing ridge feature enhancement on the original fingerprint image to obtain a corresponding enhanced fingerprint image includes:

[0036] Performing a depth-separable convolution on the original fingerprint image to obtain a first feature map;

[0037] Superimposing the first feature map and the original fingerprint image, and performing normalization, convolution and nonlinear activation in sequence after superposition to obtain a second feature map;

[0038] The third feature map and the fourth feature map are superimposed and convolved after superposition to obtain the enhanced fingerprint image; the third feature map is obtained by splicing two sets of identical original fingerprint images, and the fourth feature map is obtained by splicing a feature map obtained by deconvolving the original fingerprint image and a feature map obtained by multiplying the second feature map and the original fingerprint image.

[0039] In some embodiments, the fingerprint image feature enhancement method further includes:

[0040] Acquire multiple sets of training image sets; the training image sets include sample wet images, sample dry images and sample normal images obtained by collecting the same fingerprint in different humidity states;

[0041] Input the wet sample image into one neural network model, and input the dry sample image into another neural network model, and perform ridge feature enhancement respectively to obtain corresponding sample enhanced fingerprint images;

[0042] According to the sample enhanced fingerprint image and the sample normal image, model loss information is determined; the model loss information is obtained by fitting image pixel loss information, image gradient loss information and Laplace loss information, the image pixel loss information represents the pixel point deviation between the sample enhanced fingerprint image and the sample normal image, the image gradient loss information represents the gradient deviation between the sample enhanced fingerprint image and the sample normal image, and the Laplace loss information represents the edge information deviation between the sample enhanced fingerprint image and the sample normal image;

[0043] According to the model loss information, the two neural network models are trained respectively to obtain the first feature enhancement model and the second feature enhancement model.

[0044] The present application also provides a fingerprint image feature enhancement device, including:

[0045] The first module is used to obtain the original fingerprint image;

[0046] The second module is used to divide the original fingerprint image into blocks to obtain multiple fingerprint image blocks;

[0047] The third module is used to extract texture features from the fingerprint image block to obtain texture feature information;

[0048] The fourth module is used to extract the directional features of the fingerprint image block to obtain directional feature information;

[0049] The fifth module is used to extract the texture clarity feature of the fingerprint image block to obtain the texture clarity feature information;

[0050] A sixth module, configured to classify the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information;

[0051] The seventh module is used for inputting the original fingerprint image into the first feature enhancement model when the original fingerprint image is a wet image, performing ridge feature enhancement on the original fingerprint image, and obtaining a corresponding enhanced fingerprint image;

[0052] The eighth module is used for inputting the original fingerprint image into the second feature enhancement model when the original fingerprint image is a dry image, performing ridge feature enhancement on the original fingerprint image, and obtaining a corresponding enhanced fingerprint image.

[0053] An embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned fingerprint image feature enhancement method when executing the computer program.

[0054] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned fingerprint image feature enhancement method is implemented.

[0055] The beneficial effects of the present application are as follows: by extracting the texture features, directional features and grain clarity features of the fingerprint image blocks obtained by dividing the original fingerprint image into blocks, the texture feature information, directional feature information and grain clarity feature information of the fingerprint image blocks are obtained, and then the original fingerprint image is classified into a wet image, a dry image or a normal image according to the texture feature information, directional feature information and grain clarity feature information. When the original fingerprint image is a wet image, the original fingerprint image is input into the first feature enhancement model, and the ridge feature of the original fingerprint image is enhanced to obtain a corresponding enhanced fingerprint image. When the original fingerprint image is a dry image, the original fingerprint image is input into the second feature enhancement model, and the ridge feature of the original fingerprint image is enhanced to obtain a corresponding enhanced fingerprint image. The embodiment of the present application can enhance the fingerprint features in the fingerprint image. Since the original fingerprint image is first classified according to the texture feature information, directional feature information and grain clarity feature information, and then the ridge feature of the original fingerprint image is enhanced using the corresponding feature enhancement model according to the classification result of the original fingerprint image to obtain the corresponding enhanced fingerprint image, the fingerprint features in the fingerprint image can be enhanced in a targeted manner, and the accuracy of fingerprint image recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of the fingerprint image feature enhancement method provided in the embodiment of the present application.

[0057] Figure 2 It is a flowchart of the specific method of step S103 provided in an embodiment of the present application.

[0058] Figure 3 It is a flowchart of the specific method of step S104 provided in an embodiment of the present application.

[0059] Figure 4 It is a flowchart of the specific method of step S105 provided in an embodiment of the present application.

[0060] Figure 5 It is a flowchart of the specific method of step S106 provided in an embodiment of the present application.

[0061] Figure 6 It is a flowchart of the specific method of step S107 and step S108 provided in the embodiment of the present application.

[0062] Figure 7 It is a flowchart of the training method of both the first feature enhancement model and the second feature enhancement model provided in the embodiment of the present application.

[0063] Figure 8 It is a structural schematic diagram of the fingerprint image feature enhancement device provided in an embodiment of the present application.

[0064] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown can be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second" and the like in the specification, claims and drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0068] See also Figure 1 , Figure 1 is a flow chart of a fingerprint image feature enhancement method provided in an embodiment of the present application. In some embodiments of the present application, Figure 1 The method may specifically include but is not limited to steps S101 to S108. Figure 1 These eight steps are introduced in detail.

[0069] Step S101, obtaining an original fingerprint image.

[0070] The original fingerprint image can be obtained by collecting the fingerprint through a fingerprint collection device, and the collected original fingerprint image is stored in a fingerprint database.

[0071] In specific implementation, the original fingerprint image may be obtained by reading a fingerprint database.

[0072] It is understandable that the original fingerprint image can be collected by a fingerprint collection device with a larger collection size, including all fingerprint information of a finger, or it can be collected by a fingerprint collection device with a smaller collection size, including partial fingerprint information of a finger.

[0073] During the fingerprint collection process, if the surface of the finger to be collected is moist, the fingerprint contour details in the collected original fingerprint image are blurred and the contrast between the ridges and valleys is reduced. The original fingerprint image is classified as a wet image. If the surface of the finger to be collected is relatively dry, the ridges in the collected original fingerprint image are unclear and have poor continuity. The original fingerprint image is classified as a dry image. If the surface of the finger to be collected is moderately dry and wet, the fingerprint information in the collected original fingerprint image has no obvious defects, and the original fingerprint image is classified as a normal image.

[0074] Step S102, dividing the original fingerprint image into blocks to obtain a plurality of fingerprint image blocks.

[0075] In specific implementation, the original fingerprint image is divided into blocks to obtain multiple fingerprint image blocks of the same size, and the shape of the fingerprint image blocks is square. For example, the size of the original fingerprint image is W×H, and the size of each fingerprint image block obtained by block division is L×L.

[0076] Step S103, extracting texture features from the fingerprint image block to obtain texture feature information.

[0077] It should be noted that texture feature extraction refers to the process of extracting texture feature information of an image through certain image processing techniques to obtain a qualitative or quantitative description of the texture. Texture feature information is feature information formed by the repeated appearance of grayscale distribution in spatial positions. There will be a certain grayscale relationship between two pixels separated by a certain distance in the image space, that is, the spatial correlation characteristics of grayscale in the image.

[0078] See also Figure 2 In a specific embodiment, step S103 may specifically include but is not limited to steps S201 to S204. Figure 2 These four steps are introduced in detail.

[0079] Step S201, calculating the grayscale histogram of the fingerprint image block.

[0080] Step S202: Generate a corresponding gray level statistical vector according to the gray level histogram.

[0081] Step S203, generating a gray level co-occurrence matrix of the fingerprint image block, and calculating the normalized mean, variance, energy, contrast, inverse moment and entropy of the gray level co-occurrence matrix.

[0082] Step S204, outputting the gray level statistical vector and the normalized mean, variance, energy, contrast, inverse moment and entropy of the gray level co-occurrence matrix to obtain texture feature information.

[0083] In the specific implementation, for each fingerprint image block, its grayscale histogram is calculated, and the grayscale interval of 0~255 is divided into 10 segments to obtain a 10-dimensional statistical vector H∈R 10 , normalize the statistical vector H to obtain the gray level statistical vector H norm .

[0084] For each fingerprint image block, calculate its n1-level gray-level co-occurrence matrix, take the step size as 1, and select the union of the four directions of 0°, 45°, 90°, and 135°. The calculation formula of the gray-level co-occurrence matrix is:

[0085] ,

[0086] in, is the gray-level co-occurrence matrix, L is the length and width of the fingerprint image block, is the pixel point in the fingerprint image block The pixel value of For collection The number of elements in is the Gaussian rounding function, For logical AND, is the logical OR, i=0,1,2,...,n1-1, j=0,1,2,...,n1-1.

[0087] Normalize the gray-level co-occurrence matrix to obtain the normalized gray-level co-occurrence matrix, and calculate the mean, variance, energy, contrast, inverse moment and entropy of the normalized gray-level co-occurrence matrix. The calculation formula is:

[0088] ,

[0089] ,

[0090] ,

[0091] ,

[0092] ,

[0093] ,

[0094] in, is the normalized gray-level co-occurrence matrix, is the mean, is the variance, For energy, is the contrast, is the inverse moment, For entropy.

[0095] Step S104, extracting directional features from the fingerprint image block to obtain directional feature information.

[0096] It should be noted that directional feature extraction refers to the process of extracting the directional feature information of an image through certain image processing techniques to obtain a qualitative or quantitative description of the direction. The directional feature information includes main direction feature information, directional significance feature information and directional consistency feature information. The main direction feature information refers to the feature information of the main direction of the fingerprint lines, the directional significance feature information refers to the feature information of the clarity and consistency of the direction of the fingerprint lines, and the directional consistency feature information refers to the feature information that the directional features of the fingerprint remain consistent in different regions and scales.

[0097] See also Figure 3 In a specific embodiment, step S104 may specifically include but is not limited to steps S301 to S303. Figure 3 These three steps are introduced in detail.

[0098] Step S301, calculating the gradient of each pixel in the fingerprint image block.

[0099] Step S302: Calculate the directional saliency feature information and directional consistency feature information of the fingerprint image block according to the gradient of the pixel points.

[0100] Step S303: output directional saliency feature information and directional consistency feature information to obtain directional feature information.

[0101] In specific implementation, for each fingerprint image block, the gradient vector of each pixel is calculated, and the average gradient vector of each pixel is calculated to obtain the average gradient vector of the fingerprint image block. Then, the directional significance feature information and directional consistency feature information of the fingerprint image block can be calculated based on the average gradient vector of the fingerprint image block to obtain the directional feature information.

[0102] The calculation formula of the gradient vector of a pixel is:

[0103] ,

[0104] in, Pixel The x-axis component of the gradient vector, Pixel The y-axis component of the gradient vector, T is the transpose operation.

[0105] The calculation formula of the average gradient vector of the fingerprint image block is:

[0106] ,

[0107] ,

[0108] in, is the x-axis component of the average gradient vector of the fingerprint image block, is the y-axis component of the average gradient vector of the fingerprint image block.

[0109] The calculation formula of directional saliency feature information and directional consistency feature information is:

[0110] ,

[0111] ,

[0112] Among them, Sgf is the directional saliency feature information, and Cst is the directional consistency feature information.

[0113] Step S105, extracting texture clarity features from the fingerprint image block to obtain texture clarity feature information.

[0114] It should be noted that texture clarity feature extraction refers to the process of extracting the texture clarity feature information of an image through certain image processing techniques, so as to obtain a qualitative or quantitative description of the texture clarity. The texture clarity feature information includes ridge line clarity feature information, valley line clarity feature information, and the width ratio between the ridge line and the valley line. The ridge line clarity feature information characterizes the clarity of the ridge line, and the valley line clarity feature information characterizes the clarity of the valley line.

[0115] See also Figure 4 In a specific embodiment, step S105 may specifically include but is not limited to steps S401 to S406. Figure 4 These six steps are introduced in detail.

[0116] Step S401, calculating the average gradient of the fingerprint image block.

[0117] Step S402: Calculate the main direction feature information of the fingerprint image block according to the average gradient.

[0118] Step S403, grayscale projection is performed on the fingerprint image block according to the main direction feature information, and pixels of the fingerprint image block are classified as ridge pixels or valley pixels according to the grayscale projection result.

[0119] Step S404, generating grayscale histograms of both ridge line pixels and valley line pixels to calculate the proportion of pixels that do not meet the clarity threshold condition, and obtaining ridge line clarity feature information and valley line clarity feature information;

[0120] Step S405, calculating the width ratio between the ridge line and the valley line according to the grayscale projection result.

[0121] Step S406, outputting ridge line clarity feature information, valley line clarity feature information and the width ratio between the ridge line and the valley line to obtain texture clarity feature information.

[0122] In specific implementation, for each fingerprint image block, the gradient vector of each pixel point is calculated, and the average gradient vector of each pixel point is calculated to obtain the average gradient vector of the fingerprint image block. Then, the main direction feature information of the fingerprint image block can be calculated based on the average gradient vector of the fingerprint image block. By projecting the grayscale value of the fingerprint image block, the projection vector of the fingerprint image block, that is, the grayscale projection result, is obtained. The projection vector of the fingerprint image block is used to construct a least squares problem to calculate the analytical expression of the ridge-valley separation judgment line. According to the positional relationship between the pixel point and the ridge-valley separation judgment line, the pixel point of the fingerprint image block is classified as a ridge line pixel point or a valley line pixel point. After classifying the pixels of the fingerprint image block as ridge pixels or valley pixels, the grayscale histograms of the ridge pixels and the grayscale histograms of the valley pixels are counted respectively, and then the proportions of the pixels that do not meet the clarity threshold conditions in their respective histograms are calculated to obtain the ridge clarity feature information and the valley clarity feature information. At the same time, by counting the components belonging to continuous ridges and valleys in the projection vector of the fingerprint image block, the average ridge width and the average valley width can be obtained, and then the width ratio between the ridge and the valley can be obtained.

[0123] The calculation formula of the main direction feature information of the fingerprint image block is:

[0124] ,

[0125] in, It is the main direction feature information of the fingerprint image block.

[0126] The calculation formula of the projection vector V of the fingerprint image block is:

[0127] ,

[0128] in, is the k-th dimension component of the projection vector of the fingerprint image block, k=0,1,2,...,L-1, s=0,1,2,...,L-1.

[0129] The analytical expression of the ridge-valley separation judgment line is set to ax+b, and the least squares problem is constructed using the projection vector of the fingerprint image block to calculate the analytical expression of the ridge-valley separation judgment line:

[0130] ,

[0131] ,

[0132] ,

[0133] in, is the component mean of the projection vector of the fingerprint image block.

[0134] After obtaining the ridge-valley separation judgment line, each pixel in the fingerprint image block can be judged as belonging to a ridge or valley, that is, the pixel of the fingerprint image block is classified as a ridge pixel or a valley pixel if:

[0135] ,

[0136] Then the pixel is a valley pixel, otherwise it is a ridge pixel.

[0137] The calculation formula of ridge clarity feature information and valley clarity feature information is:

[0138] ,

[0139] ,

[0140] Among them, LCR is the ridge clarity feature information, LCV is the valley clarity feature information, is the value of the i1th gray level in the gray histogram of the ridge pixel, is the value of the i2th gray level in the gray level histogram of the ridge pixel, is the value of the i1th gray level in the gray histogram of the valley pixel, is the value of the i2th gray level in the gray histogram of the valley pixel.

[0141] The width ratio between ridges and valleys is calculated as:

[0142] ,

[0143] Among them, RVR is the width ratio between ridges and valleys, RL is the average width of ridges, and VL is the average width of valleys.

[0144] Step S106, classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information.

[0145] In specific implementation, the texture feature information, directional feature information and grain clarity feature information are spliced ​​and input into a pre-trained image classification model. The input texture feature information, directional feature information and grain clarity feature information are processed by the image classification model, and the original fingerprint image is classified into a wet image, a dry image or a normal image according to the processing result.

[0146] See also Figure 5 In a specific embodiment, step S106 may specifically include but is not limited to steps S501 to S503. Figure 5 These three steps are introduced in detail.

[0147] Step S501, concatenating texture feature information, direction feature information and texture definition feature information to obtain fused feature information.

[0148] Step S502: input the fused feature information into an image classification model to classify the fingerprint image block into a wet image block, a dry image block or a normal image block.

[0149] Among them, the image classification model includes three classification sub-models, and the classification sub-models perform corresponding binary classification judgment operations respectively. The binary classification judgment operations include judging whether the fingerprint image block is a wet image block, judging whether the fingerprint image block is a dry image block, and judging whether the fingerprint image block is a normal image block.

[0150] Preferably, the three classification sub-models in the image classification model are linear support vector machine structures.

[0151] The three classification sub-models in the image classification model obtain corresponding classification hyperplanes through learning. For the first classification sub-model, it is used to judge whether the fingerprint image block is a wet image block. Through learning, the corresponding fusion feature space is made to be the clustering area of ​​the wet image blocks. For the second classification sub-model, it is used to judge whether the fingerprint image block is a dry image block. Through learning, the corresponding fusion feature space is made to be the clustering area of ​​the dry image blocks. For the third classification sub-model, it is used to judge whether the fingerprint image block is a normal image block. Through learning, the corresponding fusion feature space is made to be the clustering area of ​​the normal image blocks.

[0152] For example, the classification hyperplane of the three classification sub-models can be expressed as:

[0153] ,

[0154] ,

[0155] ,

[0156] in, , and are the output results of the classification sub-model, F is the fusion feature information, T is the transposition operation, , , , , and are the weight parameters of the classification sub-model respectively.

[0157] For the first classification sub-model, by learning The fusion feature space is the clustering area of ​​the wet fingerprint image block. For the second classification sub-model, through learning, The fusion feature space is the clustering area of ​​the wet fingerprint image block. For the third classification sub-model, through learning, The fusion feature space is the clustering area of ​​the wet fingerprint image blocks. When , the fingerprint image block is judged to be a wet image block. When , the fingerprint image block is judged to be a dry image block. , the fingerprint image block is judged to be a normal image block.

[0158] Step S503: classify the original fingerprint image into a wet image, a dry image or a normal image according to the number of wet image blocks, dry image blocks and normal image blocks.

[0159] In a specific implementation, the fused feature information obtained by splicing the texture feature information, the direction feature information and the grain clarity feature information is input into a pre-trained image classification model, and the three classification sub-models in the image classification model process the input fused feature information, and classify each fingerprint image block in the original fingerprint image into a wet image block, a dry image block or a normal image block according to the processing result. If the number of wet image blocks exceeds a preset number threshold (for example, reaching 50% of the number of fingerprint image blocks), the original fingerprint image is determined to be a wet image; if the number of dry image blocks exceeds the preset number threshold, the original fingerprint image is determined to be a dry image; if the number of wet image blocks and the number of dry image blocks do not exceed the preset number threshold, the original fingerprint image is determined to be a normal image.

[0160] Step S107: when the original fingerprint image is a wet image, the original fingerprint image is input into a first feature enhancement model, and ridge feature enhancement is performed on the original fingerprint image to obtain a corresponding enhanced fingerprint image.

[0161] Step S108, when the original fingerprint image is a dry image, the original fingerprint image is input into the second feature enhancement model, and the ridge feature of the original fingerprint image is enhanced to obtain a corresponding enhanced fingerprint image.

[0162] In specific implementation, if the original fingerprint image is classified as a normal image, it is directly output; if the original fingerprint image is classified as a wet image, the original fingerprint image is input into the first feature enhancement model, and the first feature enhancement model performs ridge feature enhancement on the original fingerprint image to obtain an enhanced fingerprint image obtained by ridge feature enhancement for the wet image; if the original fingerprint image is classified as a dry image, the original fingerprint image is input into the second feature enhancement model, and the second feature enhancement model performs ridge feature enhancement on the original fingerprint image to obtain an enhanced fingerprint image obtained by ridge feature enhancement for the dry image.

[0163] In this embodiment, the first feature enhancement model and the second feature enhancement model are trained by a neural network model with the same structure. The first feature enhancement model is trained by performing ridge feature enhancement on a wet image, and the second feature enhancement model is trained by performing ridge feature enhancement on a dry image.

[0164] See also Figure 6 In a specific embodiment, step S107 and step S108 may include but are not limited to step S601 to step S603. Figure 6 These three steps are introduced in detail.

[0165] Step S601, performing depth-wise separable convolution on the original fingerprint image to obtain a first feature map.

[0166] Step S602, superimposing the first feature map and the original fingerprint image, and performing normalization, convolution and nonlinear activation in sequence after superposition to obtain a second feature map.

[0167] Step S603, superimposing the third feature map and the fourth feature map, and performing convolution after superposition to obtain an enhanced fingerprint image.

[0168] The third feature map is obtained by splicing two sets of identical original fingerprint images, and the fourth feature map is obtained by splicing a feature map obtained by deconvolving the original fingerprint image and a feature map obtained by multiplying the second feature map and the original fingerprint image.

[0169] In this embodiment, the first feature enhancement model and the second feature enhancement model both include a depth separation convolution layer, a convolution layer, a deconvolution layer, a normalization layer, a maximum pooling layer and a nonlinear activation function layer.

[0170] In the specific implementation, a feature transformation operation is first performed, and the original fingerprint image is subjected to a depth-separable convolution using two convolution kernels to obtain two sets of first feature maps. The first feature map and the original fingerprint image are then superimposed. After superposition, normalization, convolution and nonlinear activation are performed in sequence to obtain a second feature map. A feature dimensionality reduction operation is then performed, and the two sets of identical original fingerprint images are spliced ​​to obtain a third feature map. The feature map obtained by deconvolving the original fingerprint image and the feature map obtained by multiplying the second feature map and the original fingerprint image are spliced ​​to obtain a fourth feature map. The third feature map and the fourth feature map are superimposed, and after superposition, convolution dimensionality reduction is performed to obtain an enhanced fingerprint image.

[0171] The expression of the feature transformation operation is:

[0172] ,

[0173] ,

[0174] ,

[0175] The expression of feature dimension reduction operation is:

[0176] ,

[0177] Among them, X is the original fingerprint image, and is the first feature map, is the second feature map, is the third characteristic graph, is the fourth characteristic graph, To enhance the fingerprint image, is the depth-wise separation convolution operation, is the convolution operation, is the deconvolution operation, BN is the normalization operation, is a nonlinear activation operation, is the convolution from c1 dimension to c2 dimension, For splicing operation, Represents the Hadamard product.

[0178] See also Figure 7 In a specific embodiment, the training methods of the first feature enhancement model and the second feature enhancement model may include but are not limited to steps S701 to S704. Figure 7 These four steps are introduced in detail.

[0179] Step S701, obtaining multiple sets of training image sets.

[0180] The training image set includes sample wet images, sample dry images and sample normal images obtained by collecting the same fingerprint in different humidity states.

[0181] In the specific implementation, the actual simulation method is adopted for collection. First, the fingerprint provider is asked to clean his finger, dry it naturally, and collect fingerprints at room temperature and pressure. The collected fingerprint image is marked as the sample normal image. Then, the finger surface is dipped in water, and fingerprints are collected in different temperature environments. The collected mark is the sample wet image. Then, the finger surface is dried and fingerprints are collected in a low temperature environment. The collected mark is the sample dry image. The collected sample wet image, sample dry image and sample normal image are marked with detail points, and then the matching algorithm based on detail points is used to obtain the correspondence between the sample wet image, sample dry image and sample normal image, and the detail point position correction is performed on the sample wet image, sample dry image and sample normal image to obtain multiple sets of training image sets.

[0182] Step S702, input the sample wet image into one neural network model, and input the sample dry image into another neural network model, and perform ridge feature enhancement respectively to obtain corresponding sample enhanced fingerprint images.

[0183] Step S703, determining model loss information according to the sample enhanced fingerprint image and the sample normal image.

[0184] Among them, the model loss information is obtained by fitting the image pixel loss information, image gradient loss information and Laplace loss information. The image pixel loss information represents the pixel point deviation between the sample enhanced fingerprint image and the sample normal image. The image gradient loss information represents the gradient deviation between the sample enhanced fingerprint image and the sample normal image. The Laplace loss information represents the edge information deviation between the sample enhanced fingerprint image and the sample normal image.

[0185] The calculation formula of image pixel loss information is:

[0186] ,

[0187] ,

[0188] The calculation formula of image gradient loss information is:

[0189] ,

[0190] The calculation formula of Laplace loss information is:

[0191] ,

[0192] The calculation formula of model loss information is:

[0193] ,

[0194] in, is the image pixel loss information, is the pixel loss evaluation value with pixel deviation t, Enhance the pixels in the fingerprint image for the sample , is the pixel point in the sample normal image , W and H are the number of pixels in the width direction and the number of pixels in the height direction of the sample enhanced fingerprint image, respectively. is the image gradient loss information, is the Laplace loss information, is the model loss information, N is the total number of sample enhanced fingerprint images, , and are loss weight coefficients.

[0195] Step S704: According to the model loss information, the two neural network models are trained respectively to obtain a first feature enhancement model and a second feature enhancement model.

[0196] In specific implementation, a loss threshold interval and a reset threshold are preset as training end conditions. When the model loss information is within the loss threshold interval and the number of sample parameter resets reaches the reset threshold, the training is terminated to obtain the neural network model of the last iteration. When the number of sample parameter resets does not reach the reset threshold, the weight parameters of the neural network model are adjusted according to the degree of deviation of the model loss information from the loss threshold interval, so that the model loss information gradually approaches the loss threshold interval during the iteration process and is finally within the loss threshold interval. Then, the sample wet image or the sample dry image is replaced, and the replaced sample wet image or sample dry image is used to iteratively train the neural network model until the model loss information is within the loss threshold interval. The above steps are repeated until the number of sample parameter resets reaches the reset threshold, and the training is terminated to obtain the first feature enhancement model and the second feature enhancement model.

[0197] See also Figure 8 The present application also provides a fingerprint image feature enhancement device, which can implement the above-mentioned fingerprint image feature enhancement method, and the device includes:

[0198] The first module 801 is used to obtain an original fingerprint image;

[0199] The second module 802 is used to divide the original fingerprint image into blocks to obtain multiple fingerprint image blocks;

[0200] The third module 803 is used to extract texture features from the fingerprint image block to obtain texture feature information;

[0201] The fourth module 804 is used to extract the directional feature of the fingerprint image block to obtain the directional feature information;

[0202] The fifth module 805 is used to extract the texture clarity feature of the fingerprint image block to obtain the texture clarity feature information;

[0203] The sixth module 806 is used to classify the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information;

[0204] The seventh module 807 is used to input the original fingerprint image into the first feature enhancement model when the original fingerprint image is a wet image, perform ridge feature enhancement on the original fingerprint image, and obtain a corresponding enhanced fingerprint image;

[0205] The eighth module 808 is used to input the original fingerprint image into the second feature enhancement model when the original fingerprint image is a dry image, perform ridge feature enhancement on the original fingerprint image, and obtain a corresponding enhanced fingerprint image.

[0206] The specific implementation of the fingerprint image feature enhancement device is basically the same as the specific implementation of the above-mentioned fingerprint image feature enhancement method, and will not be repeated here.

[0207] Fig. 9 It is a block diagram of an electronic device according to an exemplary embodiment.

[0208] Refer to the following Fig. 9 The electronic device 900 according to this embodiment of the present disclosure is described. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0209] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), a display unit 940, etc.

[0210] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 executes the steps according to various exemplary embodiments of the present disclosure described in the fingerprint image feature enhancement method section above.

[0211] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0212] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0213] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0214] The electronic device 900 may also communicate with one or more external devices 900' (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 960. The network adapter 960 may communicate with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0215] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned fingerprint image feature enhancement method is implemented.

[0216] The fingerprint image feature enhancement method, device, equipment and storage medium provided in the embodiment of the present application extract texture features, directional features and grain clarity features of the fingerprint image blocks obtained by dividing the original fingerprint image into blocks, thereby obtaining texture feature information, directional feature information and grain clarity feature information of the fingerprint image blocks, and then classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, directional feature information and grain clarity feature information. When the original fingerprint image is a wet image, the original fingerprint image is input into a first feature enhancement model, and the ridge feature of the original fingerprint image is enhanced to obtain a corresponding enhanced fingerprint image. When the original fingerprint image is a dry image, the original fingerprint image is input into a second feature enhancement model, and the ridge feature of the original fingerprint image is enhanced to obtain a corresponding enhanced fingerprint image. The embodiment of the present application can enhance the fingerprint features in the fingerprint image. Since the original fingerprint image is first classified according to texture feature information, direction feature information and line clarity feature information, and then the ridge feature of the original fingerprint image is enhanced using the corresponding feature enhancement model according to the classification result of the original fingerprint image to obtain the corresponding enhanced fingerprint image, the fingerprint features in the fingerprint image can be enhanced in a targeted manner, thereby improving the accuracy of fingerprint image recognition.

[0217] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the implementation of the present disclosure.

[0218] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0219] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0220] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0221] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0222] The exemplary embodiments of the present disclosure are specifically shown and described above. It should be understood that the present disclosure is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present disclosure is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A fingerprint image feature enhancement method, characterized in that: include: Obtaining the original fingerprint image; Dividing the original fingerprint image into blocks to obtain a plurality of fingerprint image blocks; Extracting texture features from the fingerprint image block to obtain texture feature information; Extracting directional features from the fingerprint image block to obtain directional feature information; Extracting texture clarity features from the fingerprint image block to obtain texture clarity feature information; Classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information; When the original fingerprint image is a wet image, the original fingerprint image is input into a first feature enhancement model, and ridge feature enhancement is performed on the original fingerprint image to obtain a corresponding enhanced fingerprint image; When the original fingerprint image is a dry image, the original fingerprint image is input into a second feature enhancement model, and ridge feature enhancement is performed on the original fingerprint image to obtain a corresponding enhanced fingerprint image; The classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information comprises: splicing the texture feature information, the direction feature information and the texture clarity feature information to obtain fused feature information; The fused feature information is input into an image classification model to classify the fingerprint image block into a wet image block, a dry image block or a normal image block; the image classification model comprises three classification sub-models, and the classification sub-models respectively perform corresponding binary classification judgment operations, and the binary classification judgment operations include judging whether the fingerprint image block is a wet image block, judging whether the fingerprint image block is a dry image block, and judging whether the fingerprint image block is a normal image block; Classifying the original fingerprint image into a wet image, a dry image or a normal image according to the number of the wet image blocks, the dry image blocks and the normal image blocks; The step of performing ridge feature enhancement on the original fingerprint image to obtain a corresponding enhanced fingerprint image includes: Performing a depth-separable convolution on the original fingerprint image to obtain a first feature map; Superimposing the first feature map and the original fingerprint image, and performing normalization, convolution and nonlinear activation in sequence after superposition to obtain a second feature map; The third feature map and the fourth feature map are superimposed and convolved after superposition to obtain the enhanced fingerprint image; the third feature map is obtained by splicing two sets of identical original fingerprint images, and the fourth feature map is obtained by splicing a feature map obtained by deconvolving the original fingerprint image and a feature map obtained by multiplying the second feature map and the original fingerprint image.

2. The fingerprint image feature enhancement method according to claim 1, characterized in that: The step of extracting texture features from the fingerprint image block to obtain texture feature information includes: Calculating a grayscale histogram of the fingerprint image block; According to the grayscale histogram, generating a corresponding grayscale level statistical vector; Generate a gray level co-occurrence matrix of the fingerprint image block, and calculate the normalized mean, variance, energy, contrast, inverse moment and entropy of the gray level co-occurrence matrix; Output the gray level statistical vector and the normalized mean, variance, energy, contrast, inverse moment and entropy of the gray level co-occurrence matrix to obtain the texture feature information.

3. The fingerprint image feature enhancement method according to claim 1, characterized in that: The step of extracting directional features from the fingerprint image block to obtain directional feature information includes: Calculating the gradient of each pixel in the fingerprint image block; Calculating the directional saliency feature information and directional consistency feature information of the fingerprint image block according to the gradient of the pixel point; The directional saliency feature information and the directional consistency feature information are output to obtain the directional feature information.

4. The fingerprint image feature enhancement method according to claim 1, characterized in that: The extracting of the texture clarity feature of the fingerprint image block to obtain the texture clarity feature information includes: Calculating the average gradient of the fingerprint image block; Calculating main direction feature information of the fingerprint image block according to the average gradient; According to the main direction feature information, grayscale projection is performed on the fingerprint image block, and pixel points of the fingerprint image block are classified as ridge pixel points or valley pixel points according to the grayscale projection result; Generate grayscale histograms of the ridge line pixels and the valley line pixels to calculate the proportion of pixels that do not meet the clarity threshold condition, and obtain ridge line clarity feature information and valley line clarity feature information; According to the grayscale projection result, the width ratio between the ridge line and the valley line is calculated; The ridge line clarity feature information, the valley line clarity feature information and the width ratio between the ridge line and the valley line are output to obtain the texture clarity feature information.

5. The fingerprint image feature enhancement method according to claim 1, characterized in that: Also includes: Obtain multiple sets of training image sets; The training image set includes a sample wet image, a sample dry image and a sample normal image obtained by collecting the same fingerprint in different humidity states; Input the wet sample image into one neural network model, and input the dry sample image into another neural network model, and perform ridge feature enhancement respectively to obtain corresponding sample enhanced fingerprint images; According to the sample enhanced fingerprint image and the sample normal image, model loss information is determined; the model loss information is obtained by fitting image pixel loss information, image gradient loss information and Laplace loss information, the image pixel loss information represents the pixel point deviation between the sample enhanced fingerprint image and the sample normal image, the image gradient loss information represents the gradient deviation between the sample enhanced fingerprint image and the sample normal image, and the Laplace loss information represents the edge information deviation between the sample enhanced fingerprint image and the sample normal image; According to the model loss information, the two neural network models are trained respectively to obtain the first feature enhancement model and the second feature enhancement model.

6. A fingerprint image feature enhancement device, characterized in that: include: The first module is used to obtain the original fingerprint image; The second module is used to divide the original fingerprint image into blocks to obtain multiple fingerprint image blocks; The third module is used to extract texture features from the fingerprint image block to obtain texture feature information; The fourth module is used to extract the directional features of the fingerprint image block to obtain directional feature information; The fifth module is used to extract the texture clarity feature of the fingerprint image block to obtain the texture clarity feature information; A sixth module, configured to classify the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information; The seventh module is used for inputting the original fingerprint image into the first feature enhancement model when the original fingerprint image is a wet image, performing ridge feature enhancement on the original fingerprint image, and obtaining a corresponding enhanced fingerprint image; An eighth module is used for inputting the original fingerprint image into a second feature enhancement model when the original fingerprint image is a dry image, performing ridge feature enhancement on the original fingerprint image, and obtaining a corresponding enhanced fingerprint image; The classifying the original fingerprint image into a wet image, a dry image or a normal image according to the texture feature information, the direction feature information and the grain clarity feature information comprises: splicing the texture feature information, the direction feature information and the texture clarity feature information to obtain fused feature information; The fused feature information is input into an image classification model to classify the fingerprint image block into a wet image block, a dry image block or a normal image block; the image classification model comprises three classification sub-models, and the classification sub-models respectively perform corresponding binary classification judgment operations, and the binary classification judgment operations include judging whether the fingerprint image block is a wet image block, judging whether the fingerprint image block is a dry image block, and judging whether the fingerprint image block is a normal image block; Classifying the original fingerprint image into a wet image, a dry image or a normal image according to the number of the wet image blocks, the dry image blocks and the normal image blocks; The step of performing ridge feature enhancement on the original fingerprint image to obtain a corresponding enhanced fingerprint image includes: Performing a depth-separable convolution on the original fingerprint image to obtain a first feature map; Superimposing the first feature map and the original fingerprint image, and performing normalization, convolution and nonlinear activation in sequence after superposition to obtain a second feature map; The third feature map and the fourth feature map are superimposed and convolved after superposition to obtain the enhanced fingerprint image; the third feature map is obtained by splicing two sets of identical original fingerprint images, and the fourth feature map is obtained by splicing a feature map obtained by deconvolving the original fingerprint image and a feature map obtained by multiplying the second feature map and the original fingerprint image.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the fingerprint image feature enhancement method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the fingerprint image feature enhancement method according to any one of claims 1 to 5 is implemented.

Images