A 3D fingerprinting method based on unsupervised super-resolution

Abstract

A 3D fingerprinting method based on unsupervised super-resolution includes the following steps: 1) designing an unsupervised super-resolution network model; 2) using a fringe projection fingerprint image acquired by the system, improving the resolution of the fringe projection fingerprint image through the unsupervised super-resolution neural network model, and performing 3D reconstruction using the super-resolved image to obtain the 3D structure of the fingerprint. The present invention designs an unsupervised neural network that can achieve super-resolution of a single low-resolution fringe projection fingerprint while also improving the edge blur of fringe projection fingerprints acquired using structured light, resulting in a clearer and more accurate 3D reconstructed fingerprint image.

Description

Technical Field

[0001] The present invention relates to the application field of fingerprint image super-resolution technology, and in particular to a three-dimensional fingerprint method based on unsupervised super-resolution. Background Art

[0002] With the continuous progress and development of the times, information security has become increasingly important. How to effectively protect information security is an urgent issue that needs to be addressed. Driven by this issue, biometric collection technology has become a breakthrough in the field of information security.

[0003] Fingerprints are unique, universal, and durable identifiers, and are widely used in many fields. Currently, fingerprint data collection primarily relies on contact and pressure, which presents drawbacks such as susceptibility to fingerprint deformation and safety concerns. To overcome these limitations, contactless three-dimensional fingerprint systems have emerged. Compared to traditional contact-based two-dimensional fingerprints, three-dimensional fingerprints contain more finger information and preserve the finger's geometric details, while offering enhanced security and safety.

[0004] Structured light technology is a non-contact 3D measurement technique with advantages such as high measurement accuracy, fast speed, and high measurement point density. Using this technology to collect non-contact 3D fingerprints has been widely used in academia and industry. However, fringe projection images captured by limited-resolution cameras currently suffer from image blur, particularly around the edges of fingerprint images. Using higher-resolution cameras is a viable solution. Unfortunately, sometimes replacing cameras is not economically feasible due to cost, equipment limitations, and implementation difficulties. In such cases, super-resolution technology offers a more economical and practical solution. Super-resolution technology has a wide range of applications, including fingerprint imaging and medical imaging. Super-resolution technology can convert low-resolution images into high-resolution ones.

[0005] Fingerprint super-resolution technologies are categorized into interpolation-based and learning-based methods. Interpolation-based methods improve resolution by interpolating pixels. While faster than deep learning methods, they can also amplify image noise and cause image blurring. Learning-based methods utilize deep learning and other technologies to train neural networks to achieve super-resolution, resulting in more accurate high-resolution images. Current methods rely on supervised learning based on paired datasets (LR-HR) or known high-resolution images (HR). However, real-world images captured with cameras are noisy and impure. These images cannot be obtained using HR downsampling techniques like bicubic interpolation. Therefore, supervised learning cannot solve this problem in these situations. Summary of the Invention

[0006] In order to overcome the technical deficiency of the existing system in collecting blurred edges of fringe projection images, the present invention provides an unsupervised super-resolution method (without using a high-resolution dataset). This method can improve the blurred edges of fringe projection images, thereby making the three-dimensional fingerprint after three-dimensional reconstruction clearer and more accurate.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A three-dimensional fingerprint method based on unsupervised super-resolution includes the following steps:

[0009] 1) Design an unsupervised super-resolution network model; 2) Use the stripe projection fingerprint image collected by the system, improve the resolution of the stripe projection fingerprint image through the super-resolution network model, and use the super-resolution image to perform three-dimensional reconstruction to obtain the three-dimensional structure of the fingerprint.

[0010] Furthermore, the step 1) includes the following steps:

[0011] (11) Design an unsupervised super-resolution network architecture. In the forward loop, use the m×n (e.g. 480×640 resolution) stripe projection fingerprint image x as the input of the network, and pass the upsampling generator G U (Upsampling generator for fringe projection fingerprint image) obtains 2m×2n (upsampling can also be 3x or 4x, just need to change the pixel_shuffle method) fringe projection fingerprint image G U (x). Next, the downsampling generator G D (The downsampling generator is the process of learning image degradation) to obtain the m×n stripe projection fingerprint image G D (G U (x)). In the backward loop, the m×n stripe projection fingerprint image x is still used as the input of the network, and the downsampling generator G D Get the 0.5m×0.5n stripe projection fingerprint image G D (x), the distribution of the downsampled image is as close as possible to the input fringe projection fingerprint image. Then, the discriminator D D Will learn the real image x and the fake image G D (x). Next, the upsampling generator G U Get the m×n stripe projection fingerprint image G U (G D (x)).

[0012] Design an upsampling generator. In the upsampling generator G UIn the convolution operation, a pixel is added to each end of the image to avoid border effects during upsampling. A predefined bilinear kernel is then used for convolution. This is a 4D tensor, where each element is a 2×2 tensor, representing how the weight of a pixel is distributed during upsampling. Upsampling is then performed using the pixel_shuffle method, scaling the image's height and width by a factor of 2 and removing edge pixels from the image to restore the original size. Eight convolutional layers are then applied, each with a 3×3 kernel, a stride of 1, and padding of 1, using the Reinforced Luminance (ReLU) activation function. Finally, skip connections are added between the output of the pixel_shuffle upsampling and the final output to facilitate learning. Kaiming initialization is used for initializing the weights and biases of the upsampling generator. This method accounts for the nonlinearity of the Reinforced Luminance (ReLU) activation function and the effect of network depth on weight distribution. The biases are also initialized to 0.

[0013] Design a downsampling generator. In the downsampling generator G D In

[15] , we use six convolutional layers. The first convolution layer uses a 7×7 kernel with a stride of 1; the second convolution layer uses a 5×5 kernel; the third convolution layer uses a 3×3 kernel; the fourth and fifth convolution layers use a 1×1 kernel with a stride of 2; and the final convolution layer uses a 1×1 kernel. When initializing the weights and biases of the downsampling generator, for the convolutional layers, a normal distribution is used to initialize the weights. The standard deviation of this distribution is 1 / sqrt(n), where n is the number of input features. This initialization method ensures that the input distribution of each neuron has the same variance, thereby improving the stability of the network. At the same time, the bias is initialized to 0.

[0014] Design the discriminator. In the discriminator D D In , we use seven convolutional layers. The convolution kernel of the first layer is 7×7 and the stride is 1, and then the spectral normalization technique is used to stabilize the training; the convolution kernel of the second to fifth layers is 1×1 and the stride is 1, and then the spectral normalization is also used to stabilize the training, followed by batch normalization, and the activation function uses the Relu function; the convolution kernel of the last layer is 1×1 and the stride is 1, and spectral normalization is used to stabilize the training. In initializing the weights and bias of the discriminator, for the convolution layer, the Xavier initialization method is used to initialize the weights and the Sigmoid activation function is used to output a value between 0-1, indicating the classification of real images / synthetic images. In initialization, this method takes into account the number and size of input and output features to ensure that the weights of the network are widely distributed. At the same time, the bias is initialized to 0. For the batch normalization layer, the weights are initialized using normal distribution and the bias is set to 0;

[0015] (12) Design the loss function. The total loss function for training upsampler and downsampler is:

[0016] L total =L GAN +λ cycle L cycle +λ inter L inter (1)

[0017] Where λ represents the weight of these different loss functions. The details are as follows:

[0018] (121) Adversarial loss: It uses a generative adversarial network (GAN) to accurately estimate the degradation kernel of the image. This network ensures that the distribution of each scale of the downsampled LR image remains unchanged from the original image before downsampling. Downsampling generator G D It is a model of the degradation process, which downsamples the input image so that the discriminator cannot output G D (x) and the patches cropped from the input image. We define the downsampling generator adversarial loss as:

[0019] L GAN =E x [D D (G D (x))-1] 2 +R (2)

[0020] Where R is the regularization term that applies the estimated blur kernel to the real prior knowledge. The loss function of the discriminator is defined as:

[0021] L D =E x [(D D (x)-1) 2 ]+E x [D D (G D (x)) 2 ] (3)

[0022] (122) Cycle consistency loss: In the training of unsupervised super-resolution stripe projection networks, both forward cycle consistency and backward cycle consistency play a key role. First, in the forward cycle, it is beneficial to downsample the generator G. D The convergence of training also helps the upsampling generator G U Generate high-resolution images. Secondly, in the backward loop, the up-sampling generator learns to obtain the high-resolution images from the down-sampling generator G. D The generated low-resolution image is reconstructed. The final cycle consistency loss function is the sum of the forward cycle loss and the backward cycle loss:

[0023] L cycle =E x ||G D (G U (x))-x||1+E x ||G U (G D (x))-x||1 (4)

[0024] (123) Binarization interpolation loss: upsampling generator G U Since there is no true HR image, the output image will be blurred and the valley ridge texture is unclear. In addition, some artifacts will appear in the low-frequency area of the output image. D This problem becomes more serious when the blur kernel cannot be accurately estimated. In order to eliminate these effects, an interpolation loss function is introduced, which minimizes the loss between the bicubic upsampled image and the upsampled generator G while ensuring that the contrast of the fingerprint valley ridge remains unchanged. U The difference between the outputs. It is a priori knowledge that bicubic interpolation can correctly sample low-frequency areas, but cannot reconstruct high-frequency details (especially on the valley ridge details of the fingerprint). Therefore, applying a uniform cost to each pixel will generate a blurred streak fingerprint image. In order to avoid blurring, binary interpolation is only applied to the low-frequency part of the image. To this end, a frequency mask f is generated by applying fingerprint binarization and bicubic to the input image. mask :

[0025] f mask =1-Bicubic(Thershold(x)) (5)

[0026] Among them, since the fringe projection fingerprint collected by structured light is a non-contact fingerprint, this non-contact fingerprint image usually has low contrast, and directly binarizing it is not ideal. Therefore, a special method is needed to improve the contrast of the fringe projection image, and then use the traditional contact-based fingerprint enhancement algorithm (such as using Gabor filter) on these fringe projection images. Define the binarization interpolation loss function:

[0027] L inter =E x ||[G U (x)-Bicubic(x) ]×f mask ||1 (6)

[0028] It encourages the upsampling generator G U Bicubic interpolation is performed on the low frequency of the fingerprint image to reduce edge blur and maintain valley-ridge contrast.

[0029] Furthermore, the step 2) includes the following steps:

[0030] (21) Using the fringe projection fingerprint image collected by the system, the resolution of the fringe projection fingerprint image collected by the system is improved through a super-resolution network model;

[0031] (22) Using the super-resolution image, perform three-dimensional reconstruction to obtain the three-dimensional structure of the fingerprint. The three-dimensional reconstruction steps are as follows:

[0032] (221) The super-resolution fringe projection fingerprint is subjected to phase extraction and phase unwrapping using a four-step phase shift to obtain a continuous phase;

[0033] (222) In addition to continuous phase, three-dimensional reconstruction also requires a transformation relationship from the pixel coordinate system to the world coordinate system. The fringe projection image belongs to the pixel coordinate system, while the reconstructed three-dimensional fingerprint belongs to the world coordinate system. The pixel coordinate system can be transformed into the image coordinate system, the image coordinate system can be transformed into the camera coordinate system, and the camera coordinate system can be transformed into the world coordinate system. In order to determine the intrinsic and extrinsic parameters of the camera, the Zhang Zhengyou calibration method is used. Since the image resolution of the super-resolution reconstruction is the input image resolution multiplied by the scaling factor, it is necessary to divide the pixel coordinates of the super-resolution reconstruction image by the scaling factor to bring the physical position back to the same size as the original input image. Finally, the three-dimensional fingerprint is obtained using the continuous phase and transformation relationship.

[0034] The working principle of the present invention is as follows: a structured light system is used to collect fringe projection images, the quality of the fringe projection images is improved through a designed unsupervised super-resolution network, and finally the super-reconstructed fringe projection images are three-dimensionally reconstructed to obtain the three-dimensional structure of the fingerprint.

[0035] The advantages of the present invention are: under the premise of limited camera resolution, only low-resolution data sets are used for super-reconstruction; the three-dimensional fingerprint edge details captured by structured light are lost, and super-resolution technology is used to enhance the fingerprint edge details. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the unsupervised super-resolution and 3D reconstruction steps in the present invention.

[0037] Figure 2 This is an unsupervised super-resolution network architecture provided by an embodiment of the present invention.

[0038] Figure 3 In one embodiment of the present invention, a three-dimensional fingerprint is obtained by performing three-dimensional reconstruction on the super-resolution fringe projection fingerprint.

[0039] Figure 4a-4b This is an embodiment of the present invention to test the super-resolution of two-dimensional fringe projection fingerprints, wherein Figure 4a It is a low-resolution stripe projection fingerprint collected by structured light. Figure 4b It is the fringe projection fingerprint after super resolution. DETAILED DESCRIPTION

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0041] Reference Figure 1 ,The flowchart of unsupervised super-resolution and 3D reconstruction steps includes the following steps:

[0042] 1) Design an unsupervised super-resolution network model. The processing steps are as follows:

[0043] (11)Reference Figure 2 , in the forward loop, the m×n (e.g. 480×640 resolution) stripe projection fingerprint image x is used as the input of the network, and the upsampling generator G is used. U (Upsampling generator for fringe projection fingerprint image) obtains 2m×2n (upsampling can also be 3x or 4x, just need to change the pixel_shuffle method) fringe projection fingerprint image G U (x). Next, the downsampling generator G D (The downsampling generator is the process of learning image degradation) to obtain the m×n stripe projection fingerprint image G D (G U (x)). In the backward loop, the m×n stripe projection fingerprint image x is still used as the input of the network, and the downsampling generator G D Get the 0.5m×0.5n stripe projection fingerprint image G D (x), the distribution of the downsampled image is as close as possible to the input fringe projection fingerprint image. Then, the discriminator D D The real image (input image x) is learned to compare with the fake image (generator G D The generated image G D (x)). Next, the upsampling generator G U Get the m×n stripe projection fingerprint image G U (G D (x)).

[0044] Design an upsampling generator. In the upsampling generator G UIn the convolution operation, a pixel is added to each end of the image to avoid border effects during upsampling. A predefined bilinear kernel is then used for convolution. This is a 4D tensor, where each element is a 2×2 tensor, representing how the weight of a pixel is distributed during upsampling. Upsampling is then performed using the pixel_shuffle method, scaling the image's height and width by a factor of 2 and removing edge pixels from the image to restore the original size. Eight convolutional layers are then applied, each with a 3×3 kernel, a stride of 1, and padding of 1, using the Reinforced Luminance (ReLU) activation function. Finally, skip connections are added between the output of the pixel_shuffle upsampling and the final output to facilitate learning. Kaiming initialization is used for initializing the weights and biases of the upsampling generator. This method accounts for the nonlinearity of the Reinforced Luminance (ReLU) activation function and the effect of network depth on weight distribution. The biases are also initialized to 0.

[0045] Design a downsampling generator. In the downsampling generator G D In

[15] , we use six convolutional layers. The first convolution layer uses a 7×7 kernel with a stride of 1; the second convolution layer uses a 5×5 kernel; the third convolution layer uses a 3×3 kernel; the fourth and fifth convolution layers use a 1×1 kernel with a stride of 2; and the final convolution layer uses a 1×1 kernel. When initializing the weights and biases of the downsampling generator, for the convolutional layers, a normal distribution is used to initialize the weights. The standard deviation of this distribution is 1 / sqrt(n), where n is the number of input features. This initialization method ensures that the input distribution of each neuron has the same variance, thereby improving the stability of the network. At the same time, the bias is initialized to 0.

[0046] Design the discriminator. In the discriminator D D In , we use seven convolutional layers. The convolution kernel of the first layer is 7×7 and the stride is 1, and then the spectral normalization technique is used to stabilize the training; the convolution kernel of the second to fifth layers is 1×1 and the stride is 1, and then the spectral normalization is also used to stabilize the training, followed by batch normalization, and the activation function uses the Relu function; the convolution kernel of the last layer is 1×1 and the stride is 1, and spectral normalization is used to stabilize the training. In initializing the weights and bias of the discriminator, for the convolution layer, the Xavier initialization method is used to initialize the weights and the Sigmoid activation function is used to output a value between 0-1, indicating the classification of real images / synthetic images. In initialization, this method takes into account the number and size of input and output features to ensure that the weights of the network are widely distributed. At the same time, the bias is initialized to 0. For the batch normalization layer, the weights are initialized using normal distribution and the bias is set to 0;

[0047] (12) Design the loss function. The total loss function for training upsampler and downsampler is:

[0048] L total =L GAN +λ cycle L cycle +λ inter L inter (1)

[0049] Where λ represents the weight of these different loss functions. The details are as follows:

[0050] (121) Adversarial loss: It uses a generative adversarial network (GAN) to accurately estimate the degradation kernel of the image. This network ensures that the distribution of each scale of the downsampled LR image remains unchanged from the original image before downsampling. Downsampling generator G D It is a model of the degradation process, which downsamples the input image so that the discriminator cannot output G D (x) and the patches cropped from the input image. We define the downsampling generator adversarial loss as:

[0051] L GAN =E x [D D (G D (x))-1] 2 +R (2)

[0052] Where R is the regularization term that applies the estimated blur kernel to the real prior knowledge. The loss function of the discriminator is defined as:

[0053] L D =E x [(D D (x)-1) 2 ]+E x [D D (G D (x)) 2 ] (3)

[0054] (122) Cycle consistency loss: In the training of unsupervised super-resolution stripe projection networks, both forward cycle consistency and backward cycle consistency play a key role. First, in the forward cycle, it is beneficial to downsample the generator G. D The convergence of training also helps the upsampling generator G U Generate high-resolution images. Secondly, in the backward loop, the up-sampling generator learns to obtain the high-resolution images from the down-sampling generator G. D The generated low-resolution image is reconstructed. The final cycle consistency loss function is the sum of the forward cycle loss and the backward cycle loss:

[0055] L cycle =E x ||G D (G U (x))-x||1+E x ||G U (G D (x))-x||1 (4)

[0056] (123) Binarization interpolation loss: upsampling generator G U Since there is no true HR image, the output image will be blurred and the valley ridge texture is unclear. In addition, some artifacts will appear in the low-frequency area of the output image. D This problem becomes more serious when the blur kernel cannot be accurately estimated. In order to eliminate these effects, an interpolation loss function is introduced, which minimizes the loss between the bicubic upsampled image and the upsampled generator G while ensuring that the contrast of the fingerprint valley ridge remains unchanged. U The difference between the outputs. It is a priori knowledge that bicubic interpolation can correctly sample low-frequency areas, but cannot reconstruct high-frequency details (especially on the valley ridge details of the fingerprint). Therefore, applying a uniform cost to each pixel will generate a blurred streak fingerprint image. In order to avoid blurring, binary interpolation is only applied to the low-frequency part of the image. To this end, a frequency mask f is generated by applying fingerprint binarization and bicubic to the input image. mask :

[0057] f mask =1-Bicubic(Thershold(x)) (5)

[0058] Among them, since the fringe projection fingerprint collected by structured light is a non-contact fingerprint, this non-contact fingerprint image usually has low contrast, and directly binarizing it is not ideal. Therefore, a special method is needed to improve the contrast of the fringe projection image, and then use the traditional contact-based fingerprint enhancement algorithm (such as using Gabor filter) on these fringe projection images. Define the binarization interpolation loss function:

[0059] L inter =E x ||[G U (x)-Bicubic(x) ]×f mask ||1 (6)

[0060] It encourages the upsampling generator G U Perform bicubic interpolation at low frequencies in fingerprint images to reduce edge blur and maintain valley-ridge contrast;

[0061] 2) According to step 1), the fringe projection fingerprint image collected by the system is used to improve the resolution of the fringe projection fingerprint image collected by the system through a super-resolution network model, and the super-resolution image is used to perform three-dimensional reconstruction to obtain the three-dimensional structure of the fingerprint, wherein the three-dimensional fingerprint is as follows Figure 3 shown.

[0062] (21) Using the fringe projection fingerprint image collected by the system, the resolution of the fringe projection fingerprint image collected by the system is improved through the super-resolution network model. Among them, the low-resolution fringe projection fingerprint collected by the structured light, such as Figure 4a As shown; the stripe projection fingerprint reconstructed by the super-resolution network is as follows Figure 4b As shown;

[0063] (22) Using the super-resolution image, perform three-dimensional reconstruction to obtain the three-dimensional structure of the fingerprint. The three-dimensional reconstruction steps are as follows:

[0064] (221) The super-resolution fringe projection fingerprint is subjected to phase extraction and phase unwrapping using a four-step phase shift to obtain a continuous phase;

[0065] (222) In addition to continuous phase, three-dimensional reconstruction also requires a transformation relationship from the pixel coordinate system to the world coordinate system. The fringe projection image belongs to the pixel coordinate system, while the reconstructed three-dimensional fingerprint belongs to the world coordinate system. The pixel coordinate system can be transformed into the image coordinate system, the image coordinate system can be transformed into the camera coordinate system, and the camera coordinate system can be transformed into the world coordinate system. In order to determine the intrinsic and extrinsic parameters of the camera, the Zhang Zhengyou calibration method is used. Since the calibrated image is also m×n, the super-resolution reconstructed image is 2m×2n, the pixel coordinates of the super-resolution reconstructed image need to be divided by 2 (if the scaling factor is 3 or 4, it is divided by 3 or 4 accordingly) to pull the physical position back to the same size as m×n. Finally, the three-dimensional fingerprint is obtained using the continuous phase and transformation relationship.

Claims

1. A three-dimensional fingerprint method based on unsupervised super-resolution, characterized in that: The steps include: 1) Construct an unsupervised super-resolution network model; 2) Using the fringe projection fingerprint image collected by the system, the resolution of the fringe projection fingerprint image is improved through a super-resolution network model, and the super-resolution image is used for 3D reconstruction to obtain the 3D structure of the fingerprint; The step 1) comprises the following steps: (11) Construct an unsupervised super-resolution network architecture; use the m×n stripe projection fingerprint image x as the input of the network in the forward loop, and pass the upsampling generator G U Get the 2m×2n stripe projection fingerprint image G U (x); Next, pass the downsampling generator G D Get the m×n stripe projection fingerprint image G D (G U (x)); In the backward loop, the m×n stripe projection fingerprint image x is still used as the input of the network, and the downsampling generator G D Get the 0.5m×0.5n stripe projection fingerprint image G D (x), the distribution of the downsampled image is as close as possible to the input stripe projection fingerprint image; then, the discriminator D D Will learn the real image x and the fake image G D (x); Next, pass the upsampling generator G U Get the m×n stripe projection fingerprint image G U (G D (x)); Construct an upsampling generator; in the upsampling generator G U In the convolution operation, a pixel is added to each end of the image to avoid boundary effects during upsampling; then a bilinear kernel is predefined for convolution operation; it is a 4D tensor, in which each element is a 2×2 tensor, indicating how a pixel is weighted during upsampling; then the pixel_shuffle method is used for upsampling, the height and width of the image are enlarged by 2 times respectively, and the edge pixels at both ends of the image are deleted to restore the original size; then eight convolution layers with 8 convolution kernels of 3×3, stride of 1, and padding of 1 are passed through, and Relu is uniformly used as the activation function; finally, the output result of upsampling using the pixel_shuffle method is added to the final output result with a jump connection, so that the network can learn more information; in initializing the weights and bias of the upsampling generator, for the convolution layer, the Kaiming initialization method is used to initialize the weights. This method can take into account the nonlinear characteristics of the Relu activation function and the influence of the network depth on the weight distribution; at the same time, the bias is initialized to 0; Construct a downsampling generator; in the downsampling generator G D In , six convolutional layers are used; The convolution kernel of the first layer is 7×7 and the stride is 1; the convolution kernel of the second layer is 5×5; the convolution kernel of the third layer is 3×3; the convolution kernel of the fourth and fifth layers is 1×1 and the stride is 2; the convolution kernel of the last layer is 1×1; In initializing the weights and biases of the downsampling generator, for the convolutional layer, a normal distribution is used to initialize the weights, and the standard deviation of this distribution is 1 / sqrt(n), where n is the number of input features; This initialization method ensures that the input distribution of each neuron has the same variance, thereby improving the stability of the network; at the same time, the bias is initialized to 0; Construct the discriminator; in the discriminator D D In the model, seven convolutional layers are used; the convolution kernel of the first layer is 7×7 and the stride is 1, and then the spectral normalization technique is used to stabilize the training; the convolution kernel of the second to fifth layers is 1×1 and the stride is 1, and then the spectral normalization is also used to stabilize the training, followed by batch normalization, and the activation function uses the Relu function; the convolution kernel of the last layer is 1×1 and the stride is 1, and spectral normalization is used to stabilize the training; in initializing the weights and bias of the discriminator, for the convolution layer, the Xavier initialization method is used to initialize the weights and the Sigmoid activation function is used to output a value between 0 and 1, indicating the classification of real images / synthetic images; in initialization, this method takes into account the number and size of input and output features to ensure that the weight distribution of the network is wide; at the same time, the bias is initialized to 0; for the batch normalization layer, the normal distribution is used to initialize the weights and the bias is set to 0; (12) Construct the loss function. The total loss function for training upsampler and downsampler is: L total =L GAN +λ cycle L cycle +λ inter L inter (1) Where λ represents the weight of different loss functions; L GAN is the downsampling generator adversarial loss function, L cycle is the cycle consistency loss function, L inter is the binarization interpolation loss function.

2. The three-dimensional fingerprint method based on unsupervised super-resolution according to claim 1, It is characterized by: The step (12) comprises the following steps: (121) Constructing adversarial loss: Use the generative adversarial network GAN to accurately estimate the degradation kernel of the image; this network ensures that the distribution of each scale of the downsampled LR image remains unchanged from the original image before downsampling; the downsampling generator G D It is a model of the degradation process, which downsamples the input image so that the discriminator cannot output G D (x) and the patches cropped from the input image; the downsampling generator adversarial loss is defined as: Where R is the regularization term that applies the estimated blur kernel to the real prior knowledge; at the same time, the discriminator tries to downsample the generator G D The generated fake images are distinguishable from real patches of the input image; the loss function of the discriminator is defined as: No discriminator D is added in the upsampling of the forward loop D ; Because there is no HR image for comparison in the forward loop, it does not help; simply adding the discriminator will cause unnecessary artifacts in the output image; (122) Constructing cycle consistency loss: In the training of unsupervised super-resolution stripe projection networks, both forward cycle consistency and backward cycle consistency play a key role; first, in the forward cycle, it is beneficial to downsample the generator G. D The convergence of training also helps the upsampling generator G U Generate high-resolution images; secondly, in the backward cycle, the up-sampled generator learns from the down-sampled generator G D The generated low-resolution image is reconstructed; the final cycle consistency loss function is the sum of the forward cycle loss and the backward cycle loss: (123) Constructing Binarized Interpolation Loss: Upsampling Generator G U Since there is no true HR image, the output image will be blurred and the valley ridge texture is unclear. In addition, some artifacts will appear in the low-frequency area of the output image. This problem will be more serious when the downsampling generator cannot accurately estimate the blur kernel. In order to eliminate these effects, an interpolation loss function is introduced, which minimizes the loss between the bicubic upsampled image and the upsampled generator G while ensuring that the fingerprint valley ridge contrast remains unchanged. U The difference between the outputs; it is a priori knowledge that bicubic interpolation can correctly sample the low-frequency area, but cannot reconstruct the high-frequency details; therefore, applying a uniform cost to each pixel will generate a blurred stripe projection fingerprint image; in order to avoid blurring, only the low-frequency part of the image is applied with binary interpolation; for this purpose, a frequency mask f is generated by applying fingerprint binarization and bicubic to the input image mask : f mask =1-Bicubic(Thershold(x)) (5) Among them, since the fringe projection fingerprint collected by structured light is a non-contact fingerprint, this non-contact fingerprint image usually has low contrast, and directly binarizing it is not ideal. Therefore, a special method is needed to improve the contrast of the fringe projection image, and then use the traditional contact-based fingerprint enhancement algorithm on these fringe projection images; define the binarization interpolation loss function: It encourages the upsampling generator G U Bicubic interpolation is performed on the low frequency of the fingerprint image to reduce edge blur and maintain valley-ridge contrast.

3. The three-dimensional fingerprint method based on unsupervised super-resolution according to claim 1, characterized in that: The step 2) comprises the following steps: (21) Using the system to collect fringe projection fingerprint images, and improving the resolution of the fringe projection fingerprint images collected by the system through a super-resolution network model; (22) Using the super-resolution image to perform three-dimensional reconstruction to obtain the three-dimensional structure of the fingerprint; wherein the three-dimensional reconstruction steps are as follows: (221) The super-resolution fringe projection fingerprint is subjected to phase extraction and phase unwrapping using a four-step phase shift to obtain a continuous phase; (222) In addition to continuous phase, three-dimensional reconstruction also requires a transformation relationship from the pixel coordinate system to the world coordinate system; the fringe projection image belongs to the pixel coordinate system, while the reconstructed three-dimensional fingerprint belongs to the world coordinate system; the pixel coordinate system can be transformed into the image coordinate system, the image coordinate system can be transformed into the camera coordinate system, and the camera coordinate system can be transformed into the world coordinate system; in order to determine the intrinsic and extrinsic parameters of the camera, the Zhang Zhengyou calibration method is used; since the resolution of the super-resolution reconstructed image is the input image resolution multiplied by the scaling factor, it is necessary to divide the pixel coordinates of the super-resolution reconstructed image by the scaling factor to bring the physical position back to the same size as the original input image; finally, the three-dimensional fingerprint is obtained using the continuous phase and transformation relationship.

Images