Ultrasonic guided wave inversion method based on prior learning restoration for fingerprint recognition

Abstract

The present invention proposes an ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition, and the steps are as follows: first, define the material parameters of the plate waveguide structure and the fingerprint stratum corneum, and establish a finite difference model; second, establish a fingerprint image database and a velocity model database; then, obtain the reconstruction result of the rapid inversion tomography by rapid inversion tomography forward modeling, assign training parameters and learn the descent direction, and reconstruct the optimal model; then, repair the grayscale image of the reconstruction result by establishing a convolutional neural network model database, building a convolutional neural network model, and repairing the fingerprint reconstructed image; finally, perform fingerprint recognition based on morphological refinement operation. The present invention combines ultrasonic guided wave technology with image deep learning technology and applies it to fingerprint recognition, realizes rapid fingerprint reconstruction in a large physical domain, improves the quality of array guided wave reconstruction imaging, corrects fingerprint details, enhances fingerprint feature extraction, and improves fingerprint matching confidence.

Description

Technical Field

[0001] The invention relates to the field of biometrics and ultrasonic guided wave technology, and in particular to an ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition. Background Art

[0002] Human features, such as fingerprints, facial features, irises, etc., have inherent uniqueness that cannot be replicated. Biometrics technology controls personal identity authentication in areas, data, and objects by measuring the physical and biological characteristics of these human features. Among biometric technologies, the research on fingerprint recognition technology is the earliest and most mature. Its main advantages are: fingerprint scanning almost does not require the storage of biometric templates, which can reduce the size of the database; fingerprint recognition has relatively high accuracy and reliability, and the cost of use is lower.

[0003] With the combination of fingerprint recognition and optical, acoustic, and biological sensing technologies, innovative identity authentication for compact devices such as mobile phones and computers has become the latest research hotspot. However, it is difficult for capacitive-based fingerprint sensors to distinguish the dielectric constants of fingers and water or oil, so the detection sensitivity of wet fingers is poor; the size of optical sensors using charge-coupled devices is difficult to miniaturize, and it is easy to misjudge the identification of finger images and fingers themselves. In recent years, ultrasonic fingerprint sensing technology has been deeply studied, but because the current ultrasonic fingerprint scanning uses pulse echo detection methods, the physical domain for fingerprint recognition can only be limited to the fixed range that the ultrasonic sensor can scan.

[0004] Ultrasonic guided waves combined with tomography can provide a fast and reliable method for monitoring thickness changes of structures within an array without detecting all points on the surface of the structure. This method is expected to achieve fingerprint reconstruction at any location in a large physical domain. However, it is quite challenging to clearly reconstruct submillimeter fingerprint details using high-frequency ultrasonic guided wave tomography. In general, ray tomography assumes that waves propagate in a straight line, ignores refraction and diffraction, and the minimum resolution is usually described by the first Fresnel zone. Therefore, it is only suitable for reconstructing large and smoothly changing defects; diffraction tomography is usually based on a linearized scattering field approximation model and is generally suitable for reconstructing defects with low phase distortion or low contrast. Full waveform inversion imaging uses a gradient optimization method to minimize the mismatch energy between modeling data and field data, thereby achieving elastic material parameter estimation. This method uses high-order diffraction and scattering in the numerical solution, so it can resolve structures below the wavelength and is more suitable for clear reconstruction of submillimeter fingerprint details.

[0005] However, the calculation of the Hessian matrix in full waveform inversion imaging is extremely time-consuming and is not suitable for fast fingerprint reconstruction at the field level. Recently, some scholars have proposed a fast inversion tomography method in the field of structural health monitoring. The theory of this algorithm is based on full waveform inversion imaging. It adopts an acoustic inversion framework based on supervised descent learning method to obtain the pre-trained descent direction, replacing the time-consuming calculation of the steepest descent method of traditional full waveform inversion imaging. However, this fast inversion tomography method has a reconstruction error of 2%, especially when high-frequency fluctuations with a wavelength comparable to the size of fingerprint submillimeter details are used as excitation signals, the multimodal interference and Gaussian noise of the guided wave will pose greater challenges to fingerprint reconstruction and detail extraction.

[0006] Therefore, it is necessary to establish a convolutional neural network model suitable for fast inversion tomography reconstruction by prior learning and repair. This model is different from the traditional manual parameter debugging denoising method. Instead, it achieves discriminative, efficient and stable fast repair of reconstructed images by learning the prior of image artifacts and noise, thereby achieving significant improvement in fingerprint inversion imaging effect and rapid improvement in fingerprint recognition rate. Summary of the invention

[0007] In view of the shortcomings of the above-mentioned background technology, the present invention proposes an ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition. Based on fast inversion tomography for high-frequency field parameter optimization for submillimeter details of fingerprints, a convolutional neural network model of prior learning and repair suitable for fast inversion tomography reconstruction is established, the prior is learned from the fast inversion tomography training image with Gaussian noise added, and the model is used to separate artifacts and noise from the reconstructed image, thereby correcting fingerprint details, enhancing fingerprint feature extraction, and improving fingerprint matching confidence, providing exponential acceleration for fingerprint reconstruction.

[0008] The technical solution of the present invention is achieved in this way:

[0009] An ultrasonic guided wave inversion method based on prior learning restoration suitable for fingerprint recognition, the steps are as follows:

[0010] Step 1: Define the material parameters of the plate waveguide structure and fingerprint stratum corneum used for fingerprint detection, and establish a finite difference model based on the material parameters;

[0011] Step 2: Import the fingerprint grayscale image from the FVC2002 database, perform image processing on the fingerprint grayscale image to obtain a fingerprint image database; and convert the fingerprint image database into a velocity model database;

[0012] Step 3: Perform morphological refinement operation on the fingerprint grayscale image in step 2 to obtain the registered fingerprint;

[0013] Step 4: define the sensor array, set the sensor array in the finite difference model, perform fast inversion tomography forward modeling on the training samples in the velocity model database, and collect the forward modeling frequency domain signal matrix through the sensor array;

[0014] Step 5: Assign training parameters and learn the descent direction, reconstruct the optimal model using fast inversion tomography according to the learned descent direction, and obtain the fingerprint reconstructed image of the forward frequency domain signal matrix using the reconstructed optimal model;

[0015] Step 6: Use the fingerprint grayscale image in step 2 and the forward modeling frequency domain signal matrix in step 4 to establish a convolutional neural network model database for prior learning and repair; wherein, the convolutional neural network model database for prior learning and repair includes two parts, the first part includes the grayscale image of the ImageNet database and the fingerprint grayscale image of the FVC2002 database after adding random additive Gaussian noise, and the second part is the forward modeling frequency domain signal matrix after adding Gaussian noise;

[0016] Step 7: Use the convolutional neural network model database of prior learning and repair to build a convolutional neural network model for fingerprint reconstruction and repair;

[0017] Step 8: Use the convolutional neural network model of fingerprint reconstruction and repair to repair the fingerprint reconstructed image to obtain the repaired fingerprint reconstructed image;

[0018] Step 9: Perform morphological refinement on the repaired fingerprint reconstructed image to obtain the verification fingerprint; and perform fingerprint matching on the verification fingerprint and the registered fingerprint to complete fingerprint recognition.

[0019] Preferably, the method for establishing a finite difference model according to material parameters is: inputting the material parameters of the plate-like waveguide structure and the fingerprint stratum corneum into the ultrasonic guided wave Rayleigh-Lamb dispersion equation to obtain a phase velocity dispersion curve, which gives the frequency-thickness product fd and the phase velocity c of the guided wave propagating in the plate. p The corresponding relationship between the frequency and the phase velocity c is: p The corresponding relationship defines the finite difference model.

[0020] Preferably, the method for obtaining the fingerprint image database and the velocity model database is as follows: applying different degrees of beveling, rotation, flipping, stretching and liquefaction processing to each fingerprint grayscale image to generate a fingerprint database in PNG format; according to the phase velocity dispersion curve, the intermediate frequency thickness product fd and the phase velocity c p The fingerprint database in PNG format is converted into a fingerprint velocity model database according to the corresponding relationship.

[0021] Preferably, the finite difference model is expressed as:

[0022]

[0023] in, are the Fourier components of p(t), where p(t) represents the field vector, ω is the angular frequency, L represents the load vector, M represents the mass matrix, D represents the damping matrix, S represents the stiffness matrix, and i is the imaginary unit.

[0024] Preferably, the method of collecting the forward modeling frequency domain signal matrix through the sensor array is: forward modeling is performed on all training samples in the fingerprint velocity model database, and the forward modeling prediction wave field collected by the sensor array is expressed as:

[0025]

[0026] Where j = 1 / c p 2 is the phase velocity c p Calculated squared slowness, F r is the signal reception projection operator, α is the forward modeling operator, d pred is a frequency domain wave field with dimension W×W.

[0027] Preferably, the method of assigning training parameters and learning the descent direction is:

[0028] Fast inversion tomography uses an acoustic inversion framework based on a supervised descent learning method. The gradient descent matrix R is obtained through offline training, and its minimization cost function is:

[0029]

[0030] Among them, ||·|| F represents the Frobenius norm; △U n =U obs -U n is the model difference between the acquired slowness model and the slowness model after the nth iteration, △Q n =Q obs -Q n is the wave field difference between the collected frequency domain wave field and the frequency domain wave field after the nth iteration, △U n and △Q n The matrix dimensions of R are all W×W; n is the drop matrix after the nth iteration, and λ represents the normalized weight.

[0031] Preferably, the reconstructed optimal model is:

[0032]

[0033] Among them, u k+1represents the slowness model of the test sample updated after the kth iteration, u k represents the slowness model of the test sample updated after the k-1th iteration, R k is the descent matrix after the kth iteration, represents the predicted frequency domain wave field of the test sample after the kth iteration, q obs It represents the frequency domain wave field collected by the sensor of the test sample after the kth iteration.

[0034] Preferably, the convolutional neural network model for fingerprint reconstruction and restoration is to learn the image restoration prior through a convolutional neural network, specifically to learn the nonlinear mapping relationship between the noisy image and the denoised image through iterative training using a semi-quadratic splitting method, and the minimization cost function is:

[0035]

[0036] Among them, T cle is the potential denoised image, t cle It is T cle Auxiliary variables of is the fidelity term, G represents the attenuation matrix of the image, T obs represents the noise image; ξΦ(t cle ) is the regularization term, ξ represents the weight, and Φ(·) represents the regularization function; is a penalty term, ε is a penalty parameter;

[0037] The convolutional neural network model for fingerprint reconstruction and restoration includes an input layer, an intermediate layer, and an output layer; the input of the input layer is a grayscale image of the reconstruction result of the rapid inversion tomography;

[0038] The network depth of the middle layer contains 23 layers. The architecture of the first layer is convolution layer I-linear rectification unit I; the architecture of the second to the eighteenth layers is convolution layer II-batch normalization layer I-linear rectification unit II; the linear rectification unit II of the second layer is connected to the convolution layer II of the third layer, and the linear rectification unit II of the third layer is connected to the convolution layer II of the fourth layer. Similarly, the linear rectification unit II of the seventeenth layer is connected to the convolution layer II of the eighteenth layer; the architecture of the nineteenth layer is convolution layer III-batch normalization layer II-linear rectification unit III; the architecture of the twentieth layer is convolution layer IV-batch normalization layer III-linear rectification unit IV; the architecture of the twenty-first layer is convolution layer V-batch normalization layer I V-linear rectifier unit V; the architecture of the twenty-second layer is convolution layer VI-batch normalization layer V-linear rectifier unit VI; the architecture of the twenty-third layer is convolution layer VII; the linear rectifier unit I is connected to the convolution layer II of the second layer, the linear rectifier unit II of the eighteenth layer is connected to the convolution layer III of the nineteenth layer, the linear rectifier unit III of the nineteenth layer is connected to the convolution layer IV of the twentieth layer, the batch normalization II of the nineteenth layer is respectively connected to the convolution layer V of the twenty-first layer and the convolution layer VI of the twenty-second layer, the linear rectifier unit IV of the twentieth layer, the linear rectifier unit V of the twenty-first layer and the linear rectifier unit VI of the twenty-second layer are all connected to the adder, and the adder is connected to the convolution layer VII;

[0039] The convolution kernel sizes of convolution layer I, convolution layer II, convolution layer III, convolution layer IV, and convolution layer VII are all 3×3, the convolution kernel size of convolution layer V is 5×5, and the convolution kernel size of convolution layer VI is 7×7. The number of convolution kernels of convolution layer I and convolution layer VII is 3, and the number of convolution kernels of convolution layer II, convolution layer III, convolution layer IV, convolution layer V, and convolution layer VI is 64.

[0040] Preferably, the morphological thinning operation includes ridge extraction, ridge thinning and minutiae extraction, wherein the method of ridge extraction and ridge thinning is to convert the fingerprint grayscale image into a binary image to extract the skeleton of the fingerprint through the mathematical morphological thinning operation of Gonzalez; the method of minutiae extraction is to calculate the Rutovitz crossing number N of its eight neighborhoods in a counterclockwise manner with each pixel point s as the center in the binary image, and the formula is as follows:

[0041]

[0042] Among them, κ∈{0,1} represents the pixel value of the binary image, and mod is the modulus operator.

[0043] Preferably, the method for performing fingerprint matching on the verification fingerprint and the registration fingerprint is: after a given Euclidean distance threshold and an angle difference threshold, the fingerprint matching confidence C is calculated, and the formula is:

[0044]

[0045] Among them, reg represents the minutiae data of the registered fingerprint, ver represents the minutiae data of the verified fingerprint, and K reg Indicates the number of minutiae points extracted from the registered fingerprint, K ver Represents the number of minutiae points extracted from the verified fingerprint, K mat Indicates the number of minutiae points of a successful fingerprint match.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1) The present invention utilizes ultrasonic guided waves to monitor the wave field changes within the sensor array, achieving rapid fingerprint reconstruction at any position within a large physical domain, breaking through the limitations of the fixed physical domain of traditional pulse-echo fingerprint scanning.

[0048] 2) The ultrasonic guided wave inversion method based on prior learning repair can significantly improve the imaging quality of fingerprint reconstruction, improve the resolution, correct fingerprint details, enhance fingerprint feature extraction, improve fingerprint matching confidence, and support reducing the order of magnitude of sensors in the array, which exponentially improves the computational efficiency of the reconstruction algorithm.

[0049] 3) The present invention provides an alternative solution for fingerprint scanning in compact devices, achieving efficient, reliable, fast and convenient personal identity authentication at any location within a large physical domain under the screen. It is of great significance for steadily promoting scientific and technological development and improving people's livelihood, and establishing advanced demonstrations of multi-technology integration in the fields of biometrics, ultrasonic guided waves, machine learning and computer vision. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 It is a flow chart of the present invention.

[0052] Figure 2 The phase velocity dispersion curve of the plate glass before and after being covered with fingerprints and the phase velocity change of the A0 mode at fd = 2.25 MHz mm after the plate glass is covered with fingerprints. The thick black line represents the phase velocity dispersion curve of the plate glass, and the thin gray line represents the phase velocity dispersion curve after the plate glass is covered with fingerprints.

[0053] Figure 3Schematic diagram of a fingerprint reconstruction test example used in an embodiment of the present invention and its corresponding finite difference velocity model. A uniformly distributed sensor array is arranged on the plate-shaped glass, and the array contains 128 sensors.

[0054] Figure 4 This is a frequency domain wave field of a training sample in an array of 128 sensors in an embodiment of the present invention, in which the signal intensity and the displacement intensity of the excitation are normalized. In order to more intuitively show the difference in the intensity of each signal in the wave field, the pulse echo data on the diagonal of the wave field is ignored.

[0055] Figure 5 The present invention proposes a convolutional neural network model for prior learning and repair suitable for fast inversion tomography reconstruction.

[0056] Figure 6 These are a grayscale image of a fast inversion tomography reconstruction result of a test sample in an array of 128 sensors in an embodiment of the present invention, a grayscale image after fingerprint restoration and denoising, and an original grayscale image of the fingerprint of the test sample.

[0057] Figure 7 for Figure 6 The grayscale image of the fast inversion tomography reconstruction result, the grayscale image after fingerprint restoration and denoising, and the fingerprint matching result corresponding to the original grayscale image of the fingerprint of the test sample, where C* represents the confidence after normalization processing of the fingerprint matching with the original grayscale image of the fingerprint.

[0058] Figure 8 Grayscale images of the fast inversion tomography reconstruction results of the test samples used in the embodiments of the present invention in 64 and 32 sensor arrays and their corresponding fingerprint matching results, and grayscale images after fingerprint restoration and denoising and their corresponding fingerprint matching results. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] like Figure 1As shown, the embodiment of the present invention provides an ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition. Based on fast inversion tomography with high-frequency field parameter optimization for sub-millimeter details of fingerprints, a convolutional neural network model for prior learning and repair suitable for fast inversion tomography reconstruction is established, and the prior is learned from the fast inversion tomography training image with Gaussian noise added, and the model is used to separate artifacts and noise from the reconstructed image, thereby correcting fingerprint details, enhancing fingerprint feature extraction, and improving fingerprint matching confidence, providing exponential acceleration for fingerprint reconstruction. The specific steps are as follows:

[0061] Step 1: Define the material parameters of the plate waveguide structure and fingerprint stratum corneum for fingerprint detection, and establish a finite difference model based on the material parameters; both materials are considered to be isotropic materials. In this embodiment, the plate waveguide structure material uses Corning Gorilla 6, which is commonly used as mobile phone screen glass, and its Young's modulus E and Poisson's ratio ν are 77GPa and 0.21 respectively. The material of the fingerprint is simulated as a stratum corneum that is similar to water, and its E and ν are 2.19GPa and 0.4 respectively.

[0062] Substituting the material parameters into the Rayleigh-Lamb dispersion equation of ultrasonic guided waves, the frequency-thickness product fd and phase velocity c of the guided wave propagating in the plate can be obtained: p The corresponding relationship of phase velocity dispersion curve is as follows: Figure 2 As shown. The thick black line represents the phase velocity dispersion curve of the plate glass, where A0 and S0 represent the first antisymmetric mode and symmetric mode in the curve, respectively. The thin gray line represents the phase velocity dispersion curve of the plate glass after being covered with a 1.5 mm thick fingerprint, which shows that the coverage of the fingerprint causes a huge change in the surface wave of the plate glass. In order to minimize the multimodal interference in the high-frequency field while ensuring a short wavelength to provide sufficient reconstruction resolution, this embodiment selects the A0 mode at fd = 2.25 MHz·mm as the excitation, where the glass thickness d = 0.5 mm and the excitation frequency f = 4.5 MHz. The solid-liquid coupling theory of waveguides shows that the smaller the △c after the mode conversion, the smaller the p It will transfer more energy flow, so at f = 4.5MHz, the corresponding point in the gray dispersion curve is selected as the c after fingerprint coverage. p Therefore, c p =2.57km / s and c p = 2.37 km / s as the background speed and the speed of the fingerprint coverage in the finite difference model of this embodiment, respectively. The 7.78% c caused by the fingerprint coverage p The descent will be the key to fingerprint detection, because the shape of the fingerprint will eventually be characterized by reconstructing the velocity field.

[0063] The finite element method is used to discretize a plate glass with a size of 50×50×0.5mm covered with fingerprints. To ensure the effective resolution of the submillimeter details of the fingerprint, the finite element grid spacing is set to 0.1mm, so that in the x and y directions, the 50×50mm area of ​​the plate glass can be approximated by a 500×500 grid. In the z direction, according to Figure 2 The frequency-thickness product fd and phase velocity c p The corresponding relationship between the c of each point on the 500×500 grid p The value can be mapped to a thickness value, so the plate glass with a size of 50×50×0.5mm covered with fingerprints can be discretized into a finite element grid of 500×500, such as Figure 3 To control the effective physical domain, a circular array with a radius of r = 20 mm is used in this embodiment, where the number of sensors is W = 128.

[0064] Step 2: Import the fingerprint grayscale image from the FVC2002 database, perform image processing on the fingerprint grayscale image to obtain a fingerprint image database; and convert the fingerprint image database into a velocity model database; because the fingerprint structure is not easy to generate randomly, this embodiment imports 200 PNG format fingerprint grayscale images from the FVC2002 database and edits them into a 500×500 pixel distribution with the same dimensions as the finite element grid. Apply different degrees of bevel, rotation, flip, stretch and liquefaction to each image to generate a PNG format fingerprint database with a large number of detailed diversity. By establishing a 500×500 pixel grayscale value and a 500×500 grid c p The corresponding relationship between the values, the grayscale value of each pixel in the grayscale image ranges from 0 to 255, which can be mapped one by one to the c of 2.37 to 2.57 km / s p Therefore, the fingerprint database in PNG format will be converted into a fingerprint velocity model database for the descent direction training of the rapid inversion tomography of this embodiment.

[0065] Step 3: Perform morphological refinement on the fingerprint grayscale image in step 2 to obtain the registered fingerprint for the final fingerprint matching.

[0066] Step 4: define the sensor array, set the sensor array in the finite difference model, perform fast inversion tomography forward modeling on the training samples in the velocity model database, and collect the forward modeling frequency domain signal matrix through the sensor array;

[0067] In isotropic materials, the general finite difference expression based on the partial differential operator of the elastic equation is:

[0068]

[0069] Wherein the vectors M, D, S, and L represent the mass matrix, the damping matrix, the stiffness matrix, and the load vector, respectively, and p(t) represents the field vector. In order to improve the computational efficiency, the present invention solves the above equation in the frequency domain by Fourier transform, and the rewritten expression is:

[0070]

[0071] in, is the Fourier component of p(t), p(t) represents the field vector, and ω is the angular frequency; in this embodiment, L is an excitation signal of a point source type, which is the A0 mode with a frequency of 4.5 MHz.

[0072] Forward modeling is performed on all training samples in the fingerprint velocity model database, and the forward predicted wave field collected by the sensor array is expressed as:

[0073]

[0074] Where j = 1 / c p 2 is the phase velocity c p Calculated squared slowness, F r is the signal receiving projection operator, α is the forward simulation operator, which represents a nonlinear relationship between j and the field vector; d pred is a frequency domain wave field with a dimension of W×W, and in this embodiment, it is a frequency domain wave field of 128×128. Each value in the wave field matrix records the displacement signal collected by the sensor in the array, and the signal amplitude is normalized with reference to the displacement amplitude of the excitation signal. Figure 4 As shown, it is a frequency domain wave field of a training sample. In order to more intuitively show the difference of each signal strength in the wave field, the pulse echo data on the diagonal of the wave field is ignored.

[0075] Step 5: Assign training parameters and learn the descent direction, reconstruct the optimal model using fast inversion tomography according to the learned descent direction, and obtain the fingerprint reconstructed image of the forward frequency domain signal matrix using the reconstructed optimal model;

[0076] Fast inversion tomography uses an acoustic inversion framework based on a supervised descent learning method. The gradient descent matrix R is obtained through offline training, and its minimization cost function is:

[0077]

[0078] Among them, ||·|| F represents the Frobenius norm; △U n =U obs -U nis the model difference between the acquired slowness model and the slowness model after the nth iteration, △Q n =Q obs -Q n is the wave field difference between the collected frequency domain wave field and the frequency domain wave field after the nth iteration, △U n and △Q n The matrix dimensions of R are all W×W; n is the descent matrix after the nth iteration, and λ represents the normalized weight. In this embodiment, the minimization threshold of the first term of the above formula is set to 1×10 -12 , the value of the second term λ is set to 0.01. In view of the superior learning performance of the supervised descent method, 512 training samples are randomly selected from the database and 10 iterations are assigned to train the descent direction R.

[0079] In the training phase, the algorithm generates a descent matrix R after each iteration, which can be used to recursively update the slowness model of the test sample. The test sample is an unknown model, and its slowness model u after the kth recursive update is k+1 It can be expressed as:

[0080]

[0081] Among them, u k+1 represents the slowness model of the test sample updated after the kth iteration, u k represents the slowness model of the test sample updated after the k-1th iteration, R k is the descent matrix after the kth iteration, represents the predicted frequency domain wave field of the test sample after the kth iteration, q obs represents the frequency domain wave field collected by the sensor after the kth iteration of the test sample. As the number of recursive updates k of the slowness model increases, the slowness model and frequency domain wave field reconstructed by the test sample will produce nonlinear errors with the true model of the test sample. Therefore, for arrays with different numbers of sensors and different test samples, in order to reconstruct the optimal model, the number of recursive updates k in the descending direction R used by the fast inversion tomography reconstruction will be different.

[0082] Step six: Use the fingerprint grayscale image in step two and the forward modeling frequency domain signal matrix in step four to establish a convolutional neural network model database for prior learning and repair; wherein, the convolutional neural network model database for prior learning and repair includes two parts, the first part includes the grayscale image of the ImageNet database after adding random additive Gaussian noise and the fingerprint grayscale image of the FVC2002 database, and the second part is the forward modeling frequency domain signal matrix after adding Gaussian noise.

[0083] The database of the convolutional neural network model for prior learning and repair of fingerprint reconstruction in this embodiment contains 1550 samples, and is randomly divided into training set and test set according to the ratio of 8:2. These samples are divided into two parts. The samples in the first part are 150 grayscale images from the ImageNet database and 200 original grayscale images from the FVC2002 database in step 2. Random additive Gaussian noise is added in the training to train the image denoising performance. The samples in the second part are grayscale images of the reconstruction results of 300 training samples in step 3. The difference is that Gaussian noise is added to the frequency domain wave field of these samples, so that artifacts of varying degrees are produced in the reconstruction, which is used to train the image restoration performance of the model. In addition, a one-to-one correspondence between the noise image and the real image of these 1550 samples should be established so that the convolutional neural network model can learn the image restoration prior in training.

[0084] Step 7: Use the convolutional neural network model database of prior learning and restoration to build a convolutional neural network model for fingerprint reconstruction and restoration; the convolutional neural network model for fingerprint reconstruction and restoration proposed in this embodiment is to learn the image restoration prior through a convolutional neural network, specifically, to learn the nonlinear mapping relationship between the noisy image and the denoised image through iterative training by using the semi-quadratic splitting method, and its minimization cost function is:

[0085]

[0086] Among them, T obs represents a noise image; T cle is the potential denoised image, t cle It is T cle Auxiliary variables of is the fidelity term, G represents the attenuation matrix of the image; ξ represents the weight, and Φ(·) is the regularization function; is a penalty term; ε is a penalty parameter, which is also a priori parameter that this model needs to iteratively train through model optimization to solve the nonlinear mapping relationship between the noisy image and the denoised image.

[0087] like Figure 5As shown, the convolutional neural network model for fingerprint reconstruction and repair is a denoising model based on the Visual Geometry Group (VGG) network. The input of the model is a grayscale image of the reconstruction result of the rapid inversion tomography. In order to enhance the network learning ability and reduce the training error, the grayscale image is extracted as input and cropped into multiple small-sized images for training. In this embodiment, the image input is a grayscale image of the reconstruction result of the rapid inversion tomography in the previous step, and is cropped to a size of 265×265 according to the feature parts. When entering the convolutional neural network model, each image will be cut out of 4 small images of size 180×180 in four directions of the Cartesian coordinate system. When entering the model input layer, each small image of size 180×180 will be divided again into 4 grayscale images of size 90×90 and input into the middle layer of the convolutional neural network model.

[0088] The network depth of the middle layer contains 23 layers. The architecture of the first layer is convolution layer I-linear rectification unit I; the architecture of the second to the eighteenth layers is convolution layer II-batch normalization layer I-linear rectification unit II; the linear rectification unit II of the second layer is connected to the convolution layer II of the third layer, and the linear rectification unit II of the third layer is connected to the convolution layer II of the fourth layer. Similarly, the linear rectification unit II of the seventeenth layer is connected to the convolution layer II of the eighteenth layer; the architecture of the nineteenth layer is convolution layer III-batch normalization layer II-linear rectification unit III; the architecture of the twentieth layer is convolution layer IV-batch normalization layer III-linear rectification unit IV; the architecture of the twenty-first layer is convolution layer V-batch normalization layer IV-linear rectification unit V; the architecture of the twenty-second layer is convolution layer VI-batch normalization layer V-linear rectification unit VI; the architecture of the twenty-third layer is convolution layer VII; the linear rectification unit I is connected to The convolution layer II of the second layer is connected, the linear rectifier unit II of the eighteenth layer is connected to the convolution layer III of the nineteenth layer, the linear rectifier unit III of the nineteenth layer is connected to the convolution layer IV of the twentieth layer, the batch normalization II of the nineteenth layer is connected to the convolution layer V of the twenty-first layer and the convolution layer VI of the twenty-second layer respectively, the linear rectifier unit IV of the twentieth layer, the linear rectifier unit V of the twenty-first layer and the linear rectifier unit VI of the twenty-second layer are all connected to the adder, and the adder is connected to the convolution layer VII; the convolution kernel size of convolution layer I, convolution layer II, convolution layer III, convolution layer IV, and convolution layer VII is 3×3, the convolution kernel size of convolution layer V is 5×5, and the convolution kernel size of convolution layer VI is 7×7. The number of convolution kernels of convolution layer I and convolution layer VII is 3, and the number of convolution kernels of convolution layer II, convolution layer III, convolution layer IV, convolution layer V, and convolution layer VI is 64.

[0089] In the middle layer, batch normalization is added to the second to twenty-second layers, aiming to make the model training faster and more stable by reducing the nonlinear changes in the covariate shift in each layer. Since small-sized convolution models with deep network layers are more suitable for image processing, the size of most convolutions in this convolutional neural network model is set to 3×3, and the pooling layer is deleted in all layers. The number of convolution kernels in the first and twenty-third layers is 3, which is easy to process the RGB channels of the image. The number of convolution kernels in other layers is set to 64 to generate the feature map of the image. In addition, the convolution sizes of the 21st and 22nd layers in the convolutional neural network model are 5×5 and 7×7 respectively, and are placed in parallel with the 20th layer to form a multi-scale layer to establish a wider range of receptive fields. In this embodiment, Stochastic gradient descent momentum (SGDM) is used as the optimizer of the convolutional neural network model, the momentum is set to 0.9, the gradient threshold is set to 0.005, the mini-batch size, the maximum number of training times, the initialization learning rate and the weight of the L2 regularization term are set to 32, 30, 0.1 and 0.0001 respectively, and the parameters of the step size, padding, input and output channels are set to 1. This step will finally generate a convolutional neural network model suitable for prior learning and repair of fingerprint reconstruction, which can discriminate the input fast inversion tomography reconstruction on the image, and remove artifacts and denoise the grayscale image.

[0090] Step 8: Use the convolutional neural network model of fingerprint reconstruction and repair to repair the fingerprint reconstructed image to obtain the repaired fingerprint reconstructed image; call the convolutional neural network model of prior learning repair obtained in step 7 to repair the grayscale image of the fast inversion tomography reconstruction result obtained in step 5, such as Figure 6 This embodiment proves that the convolutional neural network model of prior learning restoration suitable for fingerprint reconstruction can effectively repair the grayscale image of the rapid inversion tomography reconstruction result. After the image restoration, except for a few details that were once submerged in artifacts, most of the artifacts and noise in the image have been significantly removed, and the fingerprint morphology has been basically restored.

[0091] Step nine: Perform morphological refinement on the repaired fingerprint reconstructed image to obtain the verification fingerprint; and perform fingerprint matching on the verification fingerprint and the registered fingerprint to complete fingerprint recognition. The fingerprint recognition method adopted by the present invention is a method for characterizing fingerprints by extracting detail points, and the method specifically includes effective area estimation, direction field estimation, ridge extraction, ridge refinement and detail point extraction technology. Among them, the detail points used for fingerprint recognition are mainly the endpoints and bifurcation points of the fingerprint ridges; the method of ridge extraction and ridge refinement is to convert the fingerprint grayscale image into a binary image through the mathematical morphology refinement operation of Gonzalez to extract the skeleton of the fingerprint; the method of detail point extraction is to calculate the Rutovitz crossing number N of its eight neighborhoods in a counterclockwise manner with each pixel point s as the center in the binary image, and the formula is as follows:

[0092]

[0093] Among them, κ∈{0,1} represents the pixel value of the binary image, and mod is the modulus operator. Therefore, in the skeleton fingerprint, the ridge endpoints and bifurcations can be searched pixel by pixel, and their coordinates and direction information can be stored in the database or used for fingerprint matching. In this embodiment, in step 3, Figure 6 The fingerprint grayscale original image in the image was used for detail extraction and fingerprint registration.

[0094] For fingerprint matching between the verification fingerprint and the registered fingerprint, after the given Euclidean distance threshold and angle difference threshold, the fingerprint matching confidence C is calculated, and the formula is:

[0095]

[0096] Among them, reg represents the minutiae data of the registered fingerprint, ver represents the minutiae data of the verified fingerprint, and K reg Indicates the number of minutiae points extracted from the registered fingerprint, K ver Represents the number of minutiae points extracted from the verified fingerprint, K mat Indicates the number of minutiae points that the fingerprint successfully matches. Figure 6 The three images are fingerprinted separately, and the results and matching confidence are as follows Figure 7 As shown in the figure, the small square marks represent the extracted fingerprint detail points, and the superscript * of the matching confidence C represents the normalization processing based on the matching confidence level of the fingerprint grayscale original image itself. Figure 7 It shows that after the convolutional neural network model is repaired, the artifacts in the fingerprint reconstructed image of this embodiment are effectively removed, thereby deleting most of the pseudo details, correcting the morphology of the fingerprint detail points, and increasing the fingerprint matching confidence from 0.34 to 0.8, an increase of 135.29%. Therefore, it is proved that the convolutional neural network model repaired by prior learning has good fingerprint reconstructed image repair capabilities.

[0097] On the other hand, in order to further prove the repair capability of the prior learning repair convolutional neural network model suitable for fast inversion tomography reconstruction, the test results of two other embodiments are also attached at the end of this embodiment, such as Figure 8 The figures show the fingerprint reconstruction results, fingerprint repair results and fingerprint recognition results when the number of sensors in the fast inversion tomography array is reduced to 64 and 32. When the sensors in the array are too sparse, a large number of artifacts will exist in the fingerprint reconstruction due to the serious lack of wave field information, resulting in a large number of pseudo details in the fingerprint detail extraction, so that the fingerprint matching confidence will be higher than the matching confidence of 128 sensors. This phenomenon is more obvious in the embodiment when the number of sensors is reduced to 32.

[0098] After the restoration of the fast inversion tomography reconstruction image of the 64 sensor array, the fingerprint pseudo-details were basically deleted, the morphology of most detail points was corrected, the image quality was greatly improved, and the fingerprint matching confidence increased from 0.43 to 0.61, an increase of 41.86%. This matching confidence can be applied to personal identity authentication of the device to a certain extent, which shows that the prior learning restoration convolutional neural network model of the present invention can provide support for fast inversion tomography reconstruction in terms of image restoration and imaging quality, allowing fewer sensors in the array to be used for reconstruction, bringing an exponential improvement in computational efficiency.

[0099] After the restoration of the fast inversion tomography image reconstructed by the 32 sensor array, the fingerprint pseudo details can also be deleted, but the low-resolution image reconstructed by the overly sparse array limits the restoration effect of the prior learning restoration convolutional neural network model to a certain extent, so the fingerprint matching confidence only increases from 0.47 to 0.61, an increase of 14.89%. This embodiment proves that the convolutional neural network model of prior learning restoration has a certain restoration potential for the reconstruction of overly sparse arrays, and its good artifact removal and detail correction capabilities can greatly improve the accuracy of personal identity authentication.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An ultrasonic guided wave inversion method based on prior learning repair suitable for fingerprint recognition, characterized in that: The steps are as follows: Step 1: Define the material parameters of the plate waveguide structure and fingerprint stratum corneum used for fingerprint detection, and establish a finite difference model based on the material parameters; Step 2: Import the fingerprint grayscale image from the FVC2002 database, perform image processing on the fingerprint grayscale image to obtain a fingerprint image database; and convert the fingerprint image database into a velocity model database; Step 3: Perform morphological refinement operation on the fingerprint grayscale image in step 2 to obtain the registered fingerprint; Step 4: define the sensor array, set the sensor array in the finite difference model, perform fast inversion tomography forward modeling on the training samples in the velocity model database, and collect the forward modeling frequency domain signal matrix through the sensor array; Step 5: Assign training parameters and learn the descent direction, reconstruct the optimal model using fast inversion tomography according to the learned descent direction, and obtain the fingerprint reconstructed image of the forward frequency domain signal matrix using the reconstructed optimal model; Step 6: Use the fingerprint grayscale image in step 2 and the forward modeling frequency domain signal matrix in step 4 to establish a convolutional neural network model database for prior learning and repair; wherein, the convolutional neural network model database for prior learning and repair includes two parts, the first part includes the grayscale image of the ImageNet database and the fingerprint grayscale image of the FVC2002 database after adding random additive Gaussian noise, and the second part is the forward modeling frequency domain signal matrix after adding Gaussian noise; Step 7: Use the convolutional neural network model database of prior learning and repair to build a convolutional neural network model for fingerprint reconstruction and repair; Step 8: Use the convolutional neural network model of fingerprint reconstruction and repair to repair the fingerprint reconstructed image to obtain the repaired fingerprint reconstructed image; Step 9: Perform morphological refinement on the repaired fingerprint reconstructed image to obtain the verification fingerprint; and perform fingerprint matching on the verification fingerprint and the registered fingerprint to complete fingerprint recognition.

2. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 1 is characterized in that: The method for establishing a finite difference model based on material parameters is as follows: inputting the material parameters of the plate-like waveguide structure and the fingerprint stratum corneum into the ultrasonic guided wave Rayleigh-Lamb dispersion equation to obtain a phase velocity dispersion curve, which gives the frequency-thickness product fd and the phase velocity c of the guided wave propagating in the plate. p The corresponding relationship between the frequency and the phase velocity c is: p The corresponding relationship defines the finite difference model.

3. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 1 is characterized in that: The method for obtaining the fingerprint image database and the velocity model database is as follows: each fingerprint grayscale image is subjected to different degrees of beveling, rotation, flipping, stretching and liquefaction processing to generate a fingerprint database in PNG format; according to the phase velocity dispersion curve, the intermediate frequency thickness product fd and the phase velocity c p The fingerprint database in PNG format is converted into a fingerprint velocity model database according to the corresponding relationship.

4. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 1, characterized in that: The finite difference model is expressed as: in, are the Fourier components of p(t), where p(t) represents the field vector, ω is the angular frequency, L represents the load vector, M represents the mass matrix, D represents the damping matrix, S represents the stiffness matrix, and i is the imaginary unit.

5. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 4 is characterized in that: The method for collecting the forward modeling frequency domain signal matrix through the sensor array is: forward modeling is performed on all training samples in the fingerprint velocity model database, and the forward modeling prediction wave field collected by the sensor array is expressed as: Where j = 1 / c p 2 is the phase velocity c p Calculated squared slowness, F r is the signal reception projection operator, α is the forward modeling operator, d pred is a frequency domain wave field with dimension W×W.

6. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 1, characterized in that: The method for assigning training parameters and learning the descent direction is: Fast inversion tomography uses an acoustic inversion framework based on a supervised descent learning method. The gradient descent matrix R is obtained through offline training, and its minimization cost function is: Among them, ||·|| F represents the Frobenius norm; △U n =U obs -U n is the model difference between the acquired slowness model and the slowness model after the nth iteration, △Q n =Q obs -Q n is the wave field difference between the collected frequency domain wave field and the frequency domain wave field after the nth iteration, △U n and △Q n The matrix dimensions of R are all W×W; n is the drop matrix after the nth iteration, and λ represents the normalized weight.

7. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 6, characterized in that: The optimal reconstruction model is: Among them, u k+1 represents the slowness model of the test sample updated after the kth iteration, u k represents the slowness model of the test sample updated after the k-1th iteration, R k is the descent matrix after the kth iteration, represents the predicted frequency domain wave field of the test sample after the kth iteration, q obs It represents the frequency domain wave field collected by the sensor after the kth iteration of the test sample.

8. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 1, characterized in that: The convolutional neural network model for fingerprint reconstruction and restoration is to learn the image restoration prior through the convolutional neural network, specifically, to learn the nonlinear mapping relationship between the noisy image and the denoised image through iterative training by using the semi-quadratic splitting method, and its minimization cost function is: s.t·t cle =T cle Among them, T cle is the potential denoised image, t cle It is T cle Auxiliary variables of is the fidelity term, G represents the attenuation matrix of the image, T obs represents the noise image; ξΦ(t cle ) is the regularization term, ξ represents the weight, and Φ(·) represents the regularization function; is a penalty term, e is a penalty parameter; The convolutional neural network model for fingerprint reconstruction and restoration includes an input layer, an intermediate layer, and an output layer; the input of the input layer is a grayscale image of the reconstruction result of the rapid inversion tomography; The network depth of the middle layer contains 23 layers. The architecture of the first layer is convolution layer I-linear rectification unit I; the architecture of the second to the eighteenth layers is convolution layer II-batch normalization layer I-linear rectification unit II; the linear rectification unit II of the second layer is connected to the convolution layer II of the third layer, and the linear rectification unit II of the third layer is connected to the convolution layer II of the fourth layer. Similarly, the linear rectification unit II of the seventeenth layer is connected to the convolution layer II of the eighteenth layer; the architecture of the nineteenth layer is convolution layer III-batch normalization layer II-linear rectification unit III; the architecture of the twentieth layer is convolution layer IV-batch normalization layer III-linear rectification unit IV; the architecture of the twenty-first layer is convolution layer V-batch normalization layer I V-linear rectifier unit V; the architecture of the twenty-second layer is convolution layer VI-batch normalization layer V-linear rectifier unit VI; the architecture of the twenty-third layer is convolution layer VII; the linear rectifier unit I is connected to the convolution layer II of the second layer, the linear rectifier unit II of the eighteenth layer is connected to the convolution layer III of the nineteenth layer, the linear rectifier unit III of the nineteenth layer is connected to the convolution layer IV of the twentieth layer, the batch normalization II of the nineteenth layer is respectively connected to the convolution layer V of the twenty-first layer and the convolution layer VI of the twenty-second layer, the linear rectifier unit IV of the twentieth layer, the linear rectifier unit V of the twenty-first layer and the linear rectifier unit VI of the twenty-second layer are all connected to the adder, and the adder is connected to the convolution layer VII; The convolution kernel sizes of convolution layer I, convolution layer II, convolution layer III, convolution layer IV, and convolution layer VII are all 3×3, the convolution kernel size of convolution layer V is 5×5, and the convolution kernel size of convolution layer VI is 7×7. The number of convolution kernels of convolution layer I and convolution layer VII is 3, and the number of convolution kernels of convolution layer II, convolution layer III, convolution layer IV, convolution layer V, and convolution layer VI is 64.

9. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 1, characterized in that: The morphological thinning operation includes ridge extraction, ridge thinning and minutiae extraction, wherein the ridge extraction and ridge thinning methods are to convert the fingerprint grayscale image into a binary image to extract the fingerprint skeleton through the mathematical morphological thinning operation of Gonzalez; the minutiae extraction method is to calculate the Rutovitz crossing number N of its eight neighborhoods in a counterclockwise manner with each pixel point s as the center in the binary image, and the formula is as follows: Among them, κ∈{0,1} represents the pixel value of the binary image, and mod is the modulus operator.

10. The ultrasonic guided wave inversion method based on prior learning and repair suitable for fingerprint recognition according to claim 9, characterized in that: The method for matching the verification fingerprint with the registered fingerprint is: after a given Euclidean distance threshold and an angle difference threshold, the fingerprint matching confidence C is calculated, and the formula is: Among them, reg represents the minutiae data of the registered fingerprint, ver represents the minutiae data of the verified fingerprint, and K reg Indicates the number of minutiae points extracted from the registered fingerprint, K ver Represents the number of minutiae points extracted from the verified fingerprint, K mat Indicates the number of minutiae points of a successful fingerprint match.

Images