A Complex Background Fingerprint Identification Method for Mobile Intelligent Terminals

Abstract

The present invention discloses a complex background fingerprint identification method for a mobile intelligent terminal, comprising the following steps: 1) performing downsampling and image contrast enhancement on a complex fingerprint snapshot obtained by a mobile intelligent device; 2) intercepting a fingerprint local block sequence based on the preprocessed original image; 3) using a feature extraction network based on a multi-scale residual structure optimized by pruning to perform confidence prediction and obtaining a corresponding local block confidence sequence; 4) calculating a local block quality sequence based on the fingerprint local block sequence, and using a score fusion strategy based on image quality to jointly calculate the final prediction probability of the complex background fingerprint of the mobile intelligent terminal. The method of the present invention runs automatically based on the mobile intelligent terminal, achieving the purpose of identifying complex background fingerprints anytime and anywhere, not relying on high-precision professional equipment, breaking the limitations of time and space, and reducing equipment costs and labor costs; the multi-scale deep neural network ensures that the method has high performance.

Description

Technical Field

[0001] The present invention belongs to the technical fields of deep learning, digital image processing, fingerprint identification, etc., and particularly relates to a method for identifying fingerprints with complex backgrounds on a mobile intelligent terminal. Background Art

[0002] In terms of people's property safety, for documents signed in various economic activities, such as business contracts, labor contracts, etc., signatures and fingerprints of the signatories can be taken as the basis for identity authentication; in terms of personal safety, the police can identify the identity information of criminals by extracting the fingerprints left by criminals at the crime scene. In order to ensure the authenticity and validity of this information, it is necessary to confirm the authenticity of the fingerprints themselves. Fingerprint prostheses made of various chemical materials by lawbreakers can imitate fingerprints that are almost indistinguishable from the real ones.

[0003] Such fingerprints with complex backgrounds have serious noise interference: some contract documents, such as handwritten IOU, may be on backgrounds such as stripes and grids, and are accompanied by complex background lines; on-site fingerprints can appear on any background. These complex backgrounds will cause great damage to the integrity of the ridge texture on the fingerprints, and there will be intractable problems such as the ridge texture being covered, truncated, and blurred. At present, there are mainly three problems in the identification of fingerprints with complex backgrounds. First, traditional methods are difficult to apply to fingerprints with complex backgrounds. Second, these fingerprints are usually manually identified by experts arranged by professional appraisal institutions. However, the number of appraisal institutions is small and unevenly distributed, and the original fingerprints need to be used as the basis during the appraisal, which has limitations in time and space. Third, image data is usually obtained by high-precision scanning equipment, and there are also certain limitations in the algorithm operation platform. Summary of the Invention

[0004] In order to solve the influence of complex backgrounds on fingerprint identification algorithms and the problem that fingerprints with complex backgrounds cannot be identified at any time, the purpose of the present invention is to propose a method for identifying fingerprints with complex backgrounds on a mobile intelligent terminal. By making full use of the characteristics of mobile intelligent terminals, it realizes the integration of obtaining, processing, and detecting fingerprints with complex backgrounds, aiming to solve the limitations of time, space, and equipment, and can independently complete the task of identifying fingerprints with complex backgrounds offline at any time and place using mobile devices with a relatively high popularity.

[0005] To achieve the above object, the following technical solutions are proposed:

[0006] A method for identifying fingerprints with complex backgrounds on a mobile intelligent terminal, the method comprising the following steps:

[0007] 1) Use a mobile intelligent terminal device to obtain a snapshot of a fingerprint with a complex background, and preprocess it to obtain a fingerprint original image with a size of 600 * 600 pixels;

[0008] 2) intercepting a series of local block image sequences of 300*300 pixels from the original fingerprint image obtained in step 1);

[0009] 3) Using the representation ability of the residual network, a multi-scale deep residual network is constructed and deployed on mobile smart devices to extract features from the local block image sequence and make confidence predictions of the real fingerprint to obtain the local block confidence sequence;

[0010] 4) The local block image sequence is calculated to obtain the quality weight sequence, and the final prediction confidence is calculated using a score fusion strategy based on the local block quality weight.

[0011] Furthermore, the step 1) specifically includes the following steps:

[0012] 1.1) Use a mobile smart terminal device with a camera function to take a picture of the fingerprint original with a fixed size acquisition window of 3000*3000 pixels to obtain a snapshot of the fingerprint in a complex background;

[0013] 1.2) Use average pooling technology to downsample the complex background fingerprint snapshot and adjust the snapshot resolution to one fifth of the original image. The adjusted image is 600*600 pixels.

[0014] 1.3) Use adaptive gamma correction to enhance the contrast of fingerprint snapshots, improve image color difference, increase the contrast between the foreground area and the complex background, and highlight the fingerprint texture;

[0015] Furthermore, the step 2) specifically includes the following steps:

[0016] 2.1) Use the fingerprint center positioning algorithm to locate its center point and perform coordinate correction;

[0017] 2.2) Taking the vertical coordinate of the center point as the central axis, set up a sliding window of fixed size of 300*300 pixels, randomly crop with a step size of 50 pixels and a horizontal jitter of 10 pixels to establish a local block image sequence;

[0018] 2.3) Further screening is performed to remove low-quality local blocks with a small effective area ratio or severely affected by noise interference;

[0019] Furthermore, the step 3) specifically includes the following steps:

[0020] 3.1) Construct a deep residual network with a multi-scale structure, and use L1 regularization of reused network parameters to prune low-weight channels in the network, reducing the number of network parameters and shortening the computation time;

[0021] 3.2) Use the local block image sequence obtained in step 2.3) as the network input, pass it through a deep residual network with a multi-scale structure, and calculate the confidence sequence that the local block is a true fingerprint;

[0022] Furthermore, step 4) specifically includes the following steps:

[0023] 4.1) Statistically calculate the histogram of pixel points in each similar color value interval in the local block;

[0024] 4.2) Calculate the proportion Q of the pixel points in the color value interval where the effective area is located within the local block ROI , and the proportion Q of the color value with the highest proportion in the complex background within the local block Background , and calculate the local block quality score based on this. The calculation formula is as follows:

[0025]

[0026] where ε(x 0 ∈ X) is the step function, and its value is 1 when x 0 ∈ X, and its value is 0 when, w and h are the width and height of the local block respectively, p(i, j) is the pixel value of the current coordinate, P ROI is the threshold interval for distinguishing the effective area of the fingerprint, P Background is the threshold interval for the color value with the highest proportion in the complex background, and σ and τ are the correction coefficients of the effective area weight and the signature line weight respectively;

[0027] 4.3) Calculate the local block quality weight W i , and construct the local block quality weight sequence. The calculation formula is as follows:

[0028]

[0029] where Qi represents the quality score calculated from the current local block i;

[0030] 4.4) Jointly calculate the final prediction score of the complex background fingerprint from the local block confidence sequence and the local block quality weight sequence. The calculation formula is as follows:

[0031]

[0032] where, P(y i | x i ) represents the prediction score of the current local block, x i represents the current local block input into the network for prediction, y i is the prediction label corresponding to the local block output by the network, and threshlod represents the true / false probability judgment threshold.

[0033] The beneficial effects of the present invention are as follows:

[0034] This method does not require manual identification and can run automatically based on a mobile intelligent terminal, achieving the purpose of identifying fingerprints with complex backgrounds anytime and anywhere. It does not rely on high-precision professional equipment, breaking the limitations of time and space and reducing equipment costs and labor costs. Moreover, the fingerprint identification algorithm for complex backgrounds based on a multi-scale deep neural network ensures that this method has high performance. Brief Description of the Drawings

[0035] Figure 1 is the implementation flowchart of the algorithm proposed by the present invention;

[0036] Figure 2 is the flowchart of the standardized preprocessing of the complex background fingerprint snapshot of the mobile intelligent terminal;

[0037] Figure 3 is the flowchart for obtaining the local block image sequence;

[0038] Figure 4 is the network model diagram of the complex background fingerprint identification;

[0039] Figure 5 is the flowchart for obtaining the local block confidence sequence;

[0040] Figure 6 is the confidence decision flowchart. Detailed Embodiments

[0041] The present invention will be further described below in conjunction with the accompanying drawings of the specification, but the protection scope of the present invention is not limited thereto.

[0042] Refer to Figures 1 - 6 , a method for identifying complex background fingerprints of a mobile intelligent terminal, the method comprising the following steps:

[0043] 1) As Figure 2 shown, a flowchart for the standardized preprocessing of the complex background fingerprint snapshot of the mobile intelligent terminal is designed. Based on common devices such as mobile phones and tablets, the standardized processed complex background fingerprint can be obtained, getting rid of the limitation of the previous need for high-precision scanning equipment. The implementation steps of this part are as follows:

[0044] 1.1) Use a program with a set fixed-size acquisition window of 3000 * 3000 pixels on the mobile intelligent terminal device. In a well-lit environment, parallel the camera to the plane where the complex background fingerprint is located for shooting. The height of the fingerprint in the fingerprint snapshot should be consistent with the height of the acquisition window, thereby obtaining the complex background fingerprint snapshot;

[0045] 1.2) Downsample the snapshot of the fingerprint with complex background using the average pooling method. The snapshot resolution is adjusted to one-fifth of the original image, and the size of the adjusted fingerprint should be uniformly 600 * 600 pixels;

[0046] 1.3) Use adaptive gamma correction to enhance the image contrast, increase the color difference between the foreground area and the complex background to highlight the fingerprint texture, and reduce the differences caused by shooting angles and lighting factors.

[0047] 2) As Figure 3 shown, intercept a series of local block image sequences of 300 * 300 pixels in size from the original fingerprint image with complex background obtained by preprocessing in step 1). The implementation steps of this part are as follows:

[0048] 2.1) Use the fingerprint center positioning algorithm to locate its center point. First, use the edge detection method to frame the fingerprint range, and then use the fingerprint centroid calculation algorithm to correct the coordinates to obtain more accurate center point coordinates;

[0049] 2.2) Set a sliding window with a fixed size of 300 * 300 pixels centered on the vertical coordinate where the center point is located, and randomly crop at a step size of 50 pixels to establish a local block image sequence. Add a random horizontal offset within 10 pixels to the sliding window to reduce the center point prediction error caused by the complex background;

[0050] 2.3) Further screen and remove low-quality local blocks with a small proportion of valid areas or particularly severe noise interference. To reduce manual consumption, the screening is carried out according to the following steps;

[0051] 2.3.1) Use the method of color space statistical histogram to remove fingerprint local blocks containing large areas of blank background or large areas of complex background;

[0052] 2.3.2) Refine the fingerprint local blocks and remove fingerprint local blocks with obvious interference, which are specifically manifested as a large number of cross lines in the refined image;

[0053] 3) As Figure 5 shown, construct a deep residual network with a multi-scale structure for fingerprint feature identification in complex backgrounds. To meet the device requirements of mobile intelligent terminals, use the pruning optimization method to reduce the model parameters and lower its space occupancy rate. The implementation steps of this part are as follows:

[0054] 3.1) Improve on the basis of the classic ResNet residual structure, and add an intra-block multi-scale structure inside the residual block, that is, the feature channels are decomposed into a feature sequence, and the sub-features in the sequence are connected by sequential class-residual convolution and then recombined into a single feature to complete the construction of the multi-scale residual structure;

[0055] 3.2) Connect the multi-scale residual structure using convolutional layers and max pooling layers. Add batch normalization layers and activation units after all convolutional layers. The convolutional layers with different numbers of channels correspond to two multi-scale residual structures with the same number of channels to complete the construction of the multi-scale deep neural network;

[0056] 3.3) Use the method of L1 regularization for network pruning optimization. Introduce a scaling factor α to multiply with the output of the channels, jointly train the weights of the network and the scaling factor and perform sparse regularization processing; replace the parameter γ in the widely existing BN layer of the network with α as the scaling factor for network optimization, prune the channels with low weights in the network, reduce the number of network parameters, and at the same time do not increase the additional computational burden and the parameter γ is the theoretically optimal solution that the network can learn;

[0057]

[0058]

[0059]

[0060] Among them, L 0 is the original loss function, the latter term is the L1 regularization term, α is the regularization coefficient, z in and z out are the input and output of the BN layer, B represents the current mini-batch, where μ B and σ B are the mean and standard deviation values of the input activation on B, and γ and β are the affine transformation parameters obtained by training

[0061] 3.4) Input the local block sequence into the network to obtain a local block confidence sequence with one-to-one corresponding indexes;

[0062] 4) As Figure 6 shown, in the confidence decision link, calculate the local block image sequence to obtain a quality weight sequence, and use a score fusion strategy based on the local block quality weight to calculate the final prediction confidence. The implementation steps of this part are as follows:

[0063] 4.1) Statistically calculate the histogram of the total number of pixels in each similar color value interval in the local block;

[0064] 4.2) Calculate the proportion Q ROI of the pixels in the color value interval where the effective area is located in the local block, and the proportion Q Background of the color value with the highest proportion in the complex background in the local block, and obtain the local block quality score through calculation;

[0065]

[0066] Among them, ε(x 0∈X) is a step function, x 0 When it ∈ X, its value is 1, When it is, its value is 0, w and h are the width and height of the local block respectively, p(i,j) is the pixel value of the current coordinate, P ROI is the threshold interval for distinguishing the effective area of the fingerprint, P Background is the threshold interval for the color value with the highest proportion in the complex background, σ and τ are the correction coefficients of the effective area weight and the signature line weight respectively;

[0067] 4.3) Calculate the local block quality weight W from the local block quality score i to construct the local block quality weight sequence corresponding to the fingerprint local block sequence;

[0068]

[0069] where Qi represents the quality score calculated from the current local block i;

[0070] 4.4) Jointly calculate the final prediction score of the fingerprint in the complex background from the local block confidence sequence and the local block quality weight sequence,

[0071]

[0072] where, P(y i |x i ) represents the prediction score of the current local block, x i represents the current local block input to the network for prediction, y i is the prediction label corresponding to the output of the network for the local block, and threshlod represents the true / false probability judgment threshold.

Claims

1. A complex background fingerprint identification method for mobile intelligent terminals, Features The steps include: 1) Use a mobile smart terminal device to obtain a snapshot of the fingerprint in a complex background, and pre-process it to obtain a 600*600 pixel original fingerprint image; 2) intercepting a series of local block image sequences of 300*300 pixels from the original fingerprint image obtained in step 1); 3) Using the representation ability of the residual network, a multi-scale deep residual network is constructed and deployed on mobile smart devices to extract features from the local block image sequence and make confidence predictions of the real fingerprint to obtain the local block confidence sequence; 4) Calculate the local block image sequence and obtain the quality weight sequence, and use the score fusion strategy based on the local block quality weight to calculate the final prediction confidence; The specific implementation process of step 3) is: 3.1) Construct a deep residual network with a multi-scale structure, and use L1 regularization of reused network parameters to prune low-weight channels in the network, reducing the number of network parameters and shortening the computation time; 3.2) The local block image sequence obtained in step 2.3) is used as the network input, and the confidence sequence of the local block being a true fingerprint is calculated through a deep residual network with a multi-scale structure; The specific implementation process of step 4) is: 4.1) Counting the statistical histogram of the pixels in each similar color value interval in the local block in step 2.3); 4.2) Calculate the proportion Q of the pixels in the color value interval where the effective area is located within the local block ROI , and the proportion Q of the color value with the highest proportion in the complex background within the local block Background , and calculate the local block quality score accordingly. The calculation formula is as follows: where ε(x 0 ∈ X) is a step function, whose value is 1 when x 0 ∈ X, 0 when , w and h are the width and height of the local block respectively, p(i, j) is the pixel value at the current coordinate, P ROI is the threshold interval for distinguishing the effective area of fingerprints, P Background is the threshold interval for the color value with the highest proportion in the complex background, and σ and τ are the correction coefficients of the effective area weight and the signature line weight respectively; 4.3) Calculate the local block quality weight W i , and construct the local block quality weight sequence. The calculation formula is as follows: where Q i represents the mass fraction calculated by the current local block i; 4.4) Combine the local block confidence sequence of step 3.2) and the local block quality weight sequence of step 4.3) to calculate the final prediction score of the complex background fingerprint. The calculation formula is as follows: Among them, P(y i |x i ) represents the prediction score of the current local block, x i represents the current local block for which the input network makes a prediction, and y i is the predicted label of the corresponding local block output by the network, and threshlod represents the true / false probability judgment threshold.

2. A complex background fingerprint identification method for a mobile intelligent terminal as claimed in claim 1, Features The specific implementation process of step 1) is: 1.1) Use a mobile smart terminal device with a camera function to take a picture of the fingerprint original with a fixed size acquisition window of 3000*3000 pixels to obtain a snapshot of the fingerprint in a complex background; 1.2) Use average pooling technology to downsample the complex background fingerprint snapshot and adjust the snapshot resolution to one fifth of the original image. The adjusted image is 600*600 pixels. 1.3) Use adaptive gamma correction to enhance the contrast of the fingerprint snapshot in step 1.2), improve the image color difference, increase the contrast between the foreground area and the complex background, and highlight the fingerprint texture.

3. A complex background fingerprint identification method for a mobile intelligent terminal as claimed in claim 2, Features The specific implementation process of step 2) is: 2.1) Using the fingerprint center positioning algorithm to locate the center point of the image processed in step 1.3) and perform coordinate correction; 2.2) Taking the vertical coordinate of the center point of step 2.1) as the central axis, set up a sliding window of fixed size of 300*300 pixels, randomly crop with a step size of 50 pixels and a horizontal jitter of 10 pixels to establish a local block image sequence; 2.3) Further screening is performed to remove low-quality local blocks with a small effective area ratio or that are particularly severely affected by noise.

Images