A fingerprint texture feature extraction method based on spectrum

Abstract

The present invention discloses a fingerprint texture feature extraction method based on spectrum attention. The present invention utilizes the characteristics of texture images in the frequency domain to perform deep learning training, combines the time domain characteristics and spectrum characteristics of the image, and better extracts the texture features of the fingerprint image, thereby improving the recognition accuracy of deformable fingerprints. The present invention is based on the experiments conducted by InceptionV3 and can also be applied to other deep learning networks.

Description

Technical Field

[0001] The present invention is a neural network improvement method, specifically a fingerprint texture feature extraction method based on spectrum attention, which introduces the spectrum features of the image in the feature extraction process to improve the effect of feature extraction. Background Art

[0002] Fingerprint recognition has been widely used in many areas of daily life. In order to avoid being recognized, some criminals change the ridge structure of fingerprints by cutting, transplanting, burning, etc. The resulting fingerprints are called deformed fingerprints. They use deformed fingerprints to interfere with the recognition system and evade inspection.

[0003] Fingerprint images have good texture, and deep learning methods can be used to extract the features of fingerprint images and identify them.

[0004] The spectrum of fingerprint images also has obvious characteristics, with high and low frequencies being more obvious, and appearing as a ring on the spectrum. Ordinary deep learning networks do not learn this kind of spectrum texture feature of fingerprints. Summary of the invention

[0005] In order to solve the problems of the prior art, the purpose of the present invention is to overcome the shortcomings of the existing technology and provide a fingerprint texture feature extraction method based on spectrum attention. The added spectrum attention module helps the convolutional neural network to better extract the texture features of the fingerprint and improve the recognition accuracy of the deformed fingerprint. In order to utilize the features on the spectrum, the present invention adds a spectrum attention module to the deep learning method to improve the effect of feature extraction.

[0006] In order to achieve the above invention purpose, the present invention adopts the following technical solution:

[0007] A spectrum-based fingerprint texture feature extraction method adds a spectrum attention module to the convolutional neural network, including the following steps:

[0008] Step 1: Perform Fourier transform on the input feature map to convert it from the time domain map to the spectrum map. The formula used is shown in formula (1):

[0009]

[0010] In formula (1), u and v are frequency variables, x and y are spatial domain image variables; j is an imaginary unit, f(x, y) is the time domain image representation, M1, N1 are the sizes of the time domain image, and F(u, v) is the frequency domain image representation;

[0011] Step 2, the real part and imaginary part of the Fourier transformed spectrum are separated into two feature maps;

[0012] Step 3: The two feature maps are respectively passed through three convolution layers with convolution kernels of 1*1, 3*3, and 1*1 for network learning to extract spectral features. The convolution formula is shown in formula (2):

[0013]

[0014] In formula (2), H(m,n) is the image after convolution; f is the time domain image before convolution, ω is a rectangular convolution kernel, ω(s,t) is the value of the sth row and tth column in the convolution kernel ω; f(ms,nt) is the distance s along the horizontal axis and the distance t along the vertical axis of the pixel f(m,n) in the mth row and nth column in the image f;

[0015] Step 4: Re-merge the feature map that has passed the convolutional layer into a spectrum map, and return it to the time domain feature map through inverse Fourier transform. The formula used is as follows:

[0016]

[0017] In formula (3), M2, N2 are the sizes of the frequency domain image;

[0018] Step 5: Softmax the obtained time domain image to obtain the mapping value corresponding to the range of 0-1;

[0019] Step 6: Multiply the mapping value by the original image to perform attention calculation;

[0020] Step 7: Embed the spectrum attention module into different network layers for calculation.

[0021] Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

[0022] 1. The present invention not only focuses on the time domain features of the image, but also focuses on the spectrum features of the fingerprint image. By combining the time domain features and spectrum features of the fingerprint image, the texture features of the fingerprint image are better preserved, and the recognition accuracy of the deformed fingerprint is improved;

[0023] 2. The present invention utilizes the characteristics of texture images in the frequency domain to perform deep learning training, combines the time domain characteristics and spectrum characteristics of the image, and better extracts the texture features of the fingerprint image, thereby improving the recognition accuracy of deformable fingerprints; the present invention is based on the experiments conducted on InceptionV3 and can also be applied to other deep learning networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1This is the deformed fingerprint image extracted by the method of the present invention.

[0025] Figure 2 This is the spectrum diagram of the fingerprint image extracted by the method of the present invention.

[0026] Figure 3 This is a spectrum attention structure diagram of the method of the present invention.

[0027] Figure 4 The convolutional network structure in the spectrum attention of the method of the present invention

[0028] Figure 5 This is the training loss graph extracted by the method of the present invention. DETAILED DESCRIPTION

[0029] The above scheme is further described below in conjunction with specific implementation examples. The preferred embodiments of the present invention are described in detail as follows:

[0030] Embodiment 1:

[0031] In this embodiment, a spectrum-based fingerprint texture feature extraction method adds a spectrum attention module to a convolutional neural network, including the following steps:

[0032] Step 1: Perform Fourier transform on the input feature map to convert it from the time domain map to the spectrum map. The formula used is shown in formula (1):

[0033]

[0034] In formula (1), u and v are frequency variables, x and y are spatial domain image variables; j is an imaginary unit, f(x, y) is the time domain image representation, M1, N1 are the sizes of the time domain image, and F(u, v) is the frequency domain image representation;

[0035] Step 2, the real part and imaginary part of the Fourier transformed spectrum are separated into two feature maps;

[0036] Step 3: The two feature maps are respectively passed through three convolution layers with convolution kernels of 1*1, 3*3, and 1*1 for network learning to extract spectral features. The convolution formula is shown in formula (2):

[0037]

[0038] In formula (2), H(m,n) is the image after convolution; f is the time domain image before convolution, ω is a rectangular convolution kernel, ω(s,t) is the value of the sth row and tth column in the convolution kernel ω; f(ms,nt) is the distance s along the horizontal axis and the distance t along the vertical axis of the pixel f(m,n) in the mth row and nth column in the image f;

[0039] Step 4: Re-merge the feature map that has passed the convolutional layer into a spectrum map, and return it to the time domain feature map through inverse Fourier transform. The formula used is as follows:

[0040]

[0041] In formula (3), M2, N2 are the sizes of the frequency domain image;

[0042] Step 5: Softmax the obtained time domain image to obtain the mapping value corresponding to the range of 0-1;

[0043] Step 6: Multiply the mapping value by the original image to perform attention calculation;

[0044] Step 7: Embed the spectrum attention module into different network layers for calculation.

[0045] The fingerprint texture feature extraction method based on spectrum attention in this embodiment utilizes the features of texture images in the frequency domain to perform deep learning training, combines the time domain features and spectrum features of the image, and better extracts the texture features of the fingerprint image, thereby improving the recognition accuracy of deformable fingerprints. The method in this embodiment is based on the experiment conducted by InceptionV3 and can also be applied to other deep learning networks.

[0046] Embodiment 2:

[0047] This embodiment is basically the same as the first embodiment, except that:

[0048] In this embodiment, the feature map is input into the InceptionV3 deep learning network, and the network structure is shown in Table 1 below.

[0049] Table 1. InceptionV3 network structure data table of the second embodiment of the present invention

[0050] Network layer type Network layer structure Input size Convolutional Layer 3*3 convolution, stride 2 3*299*299 Convolutional Layer 3*3 convolution, stride 1 32*149*149 Convolutional Layer 3*3 convolution, stride 1 32*147*147 Pooling Layer 3*3 pooling, step size is 2 64*147*147 Convolutional Layer 3*3 convolution, stride 1 64*73*73 Convolutional Layer 3*3 convolution, stride 2 80*73*73 Pooling Layer 3*3 pooling, step size is 1 192*71*71 3*Inception Multiple convolutional layers 192*35*35 5*Inception Multiple convolutional layers 288*35*35 2*Inception Multiple convolutional layers 768*17*17 Global Pooling Layer 8*8 convolution, stride 1 2048*8*8 Fully connected layer 2048*1*1

[0051] The spectral attention module is added to the Inception layer of the Inception network to assist in extracting texture features.

[0052] The steps of the spectral attention module are as follows:

[0053] Step 1: Perform Fourier transform on the input original image to convert it from the time domain image to the spectrum image. The formula used is shown in formula (1):

[0054]

[0055] In formula (1), u and v are frequency variables, x and y are spatial domain image variables; j is an imaginary unit, f(x, y) is the time domain image representation, M1, N1 are the sizes of the time domain image, and F(u, v) is the frequency domain image representation;

[0056] Step 2: Separate the real and imaginary parts of the Fourier transformed spectrum into two feature maps.

[0057] Step 3: The two feature maps are respectively passed through three convolution layers with convolution kernels of 1*1, 3*3, and 1*1 for network learning to extract spectral features. The convolution formula is shown in formula (2):

[0058]

[0059] In formula (2), H(m,n) is the image after convolution; f is the time domain image before convolution, ω is a rectangular convolution kernel, ω(s,t) is the value of the sth row and tth column in the convolution kernel ω; f(ms,nt) is the distance s along the horizontal axis and the distance t along the vertical axis of the pixel f(m,n) in the mth row and nth column in the image f;

[0060] Step 4: Re-merge the feature map that has passed the convolutional layer into a spectrum map, and transform it back to a time domain map through an inverse Fourier transform. The formula used is as follows:

[0061]

[0062] In formula (3), M2, N2 are the sizes of the frequency domain image;

[0063] Step 5: The obtained time domain image is subjected to softmax to obtain the mapping value corresponding to the 0-1 range.

[0064] Step 6: Multiply the mapping value by the original image to perform attention calculation.

[0065] Step 7. Repeat steps 1-6, add the spectral attention module to all Inception layers, and perform network training.

[0066] The purpose of the spectrum attention module used in the present invention is to select the frequency band with more prominent texture features in the feature extraction process of the spectrum graph, and to strengthen the features of the image of this frequency band in the time domain graph by using attention, so as to better extract the texture features of the fingerprint.

[0067] The experimental results are summarized as follows:

[0068] The dataset is the deformation fingerprint data collected and processed by myself. See Table 2.

[0069] Table 2. Comparison of recognition accuracy under the same number of training rounds (epoch)

[0070] Epoch10 Epoch20 Epoch30 Inception Network 96.82 97.38 97.36 The present invention 96.93 97.51 97.52

[0071] This embodiment is an experiment based on the InceptionV3 network, but can be applied to feature extraction of other networks.

[0072] This embodiment not only focuses on the time domain features of the image, but also focuses on the spectral features of the fingerprint image, combines the time domain features and spectral features of the fingerprint image, better retains the texture features of the fingerprint image, and improves the recognition accuracy of the deformable fingerprint; this embodiment utilizes the features of the texture image in the frequency domain, performs deep learning training, combines the time domain features and spectral features of the image, and better extracts the texture features of the fingerprint image, thereby improving the recognition accuracy of the deformable fingerprint; the present invention is based on the experiments conducted on InceptionV3 and can also be applied to other deep learning networks.

[0073] The above describes the embodiments of the present invention in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made according to the purpose of the invention. Any changes, modifications, substitutions, combinations or simplifications made according to the spirit and principle of the technical solution of the present invention should be equivalent replacement methods. As long as they meet the purpose of the invention and do not deviate from the technical principles and inventive concepts of the present invention, they belong to the protection scope of the present invention.

Claims

1. A fingerprint texture feature extraction method based on spectrum, characterized in that: A spectral attention module is added to the convolutional neural network, including the following steps: Step 1: Perform Fourier transform on the input feature map to convert it from the time domain map to the spectrum map. The formula used is shown in formula (1): In formula (1), u and v are frequency variables, x and y are spatial domain image variables; j is an imaginary unit, f(x, y) is the time domain image representation, M1, N1 are the sizes of the time domain image, and F(u, v) is the frequency domain image representation; Step 2: The real part and imaginary part of the Fourier transformed spectrum are separated into two feature maps; Step 3: The two feature maps are respectively subjected to three convolution layers with convolution kernels of 1*1, 3*3, and 1*1 for network learning to extract spectral features. The convolution formula is shown in formula (2): In formula (2), H(m,n) is the image after convolution; f is the time domain image before convolution, ω is a rectangular convolution kernel, ω(s,t) is the value of the sth row and tth column in the convolution kernel ω; f(ms,nt) is the pixel f(m,n) in the mth row and nth column of the image f, which moves s distance along the horizontal axis and t distance along the vertical axis; Step 4: Re-merge the feature map that has passed the convolutional layer into a spectrum map, and return it to the time domain feature map through inverse Fourier transform. The formula used is as follows: In formula (3), M2, N2 are the sizes of the frequency domain image; Step 5: The obtained time domain image is subjected to softmax to obtain the mapping value corresponding to the range of 0-1; Step 6: Multiply the mapping value by the original image to perform attention calculation; Step 7: Embed the spectrum attention module into different network layers for calculation.

Images