Fingerprint image restoration method and system based on generative adversarial network

Abstract

The invention discloses a fingerprint image restoration method and system based on a generative adversarial network, and relates to the technical field of fingerprint restoration, and the method comprises the steps: obtaining a complete fingerprint image, carrying out the preprocessing of the complete fingerprint image, and generating a plurality of fingerprint missing images with missing regions in combination with masks; inputting the fingerprint missing image and the corresponding mask into a generator comprising an AOT block, and capturing and integrating remote fingerprint context information to obtain a generated fingerprint image; inputting the generated fingerprint image and the complete fingerprint image into a discriminator to obtain a repair judgment result; optimizing the generator based on the repair judgment result and the total loss function to obtain an optimized generator; and obtaining a to-be-repaired fingerprint image and the corresponding mask, and inputting the to-be-repaired fingerprint image and the corresponding mask into the optimization generator to obtain a repaired fingerprint image. And the quality and the efficiency of a repairing result are improved while the authenticity and the continuity of the fingerprint streakline structure are ensured.

Description

Technical Field

[0001] This invention relates to the field of fingerprint restoration technology, and more specifically to a fingerprint image restoration method and system based on generative adversarial networks. Background Technology

[0002] Fingerprints are a unique biometric feature; even identical twins have different fingerprint characteristics. Currently, fingerprints are widely used in criminal investigations and offender identification. Although fingerprints possess high stability and uniqueness, fingerprint images can be damaged or incomplete due to environmental factors, physical damage, equipment problems, human error, and the passage of time, such as scratches, breaks, and abrasions. This damage compromises the integrity of the fingerprint image, affecting the accuracy and reliability of fingerprint recognition. To improve fingerprint image quality, existing restoration techniques are mainly divided into two categories: traditional filtering methods and deep learning-based methods. Filtering techniques, such as Gabor filtering, wavelet transform, gradient filtering, and Gaussian / median filtering, effectively improve image quality in the preprocessing stage by enhancing contrast, suppressing noise, and highlighting ridge structures. However, over-filtering can easily lead to blurred edge details, thus damaging key features. Deep learning methods have become a research hotspot in recent years, including Convolutional Neural Networks (CNN), U-Net, ResNet, Transformers, and Generative Adversarial Networks (GAN). These models can automatically learn the complex texture and topology of fingerprints, demonstrating excellent performance in repairing breaks and restoring connectivity. For example, dual-branch U-Net with an attention mechanism, multi-task CNNs incorporating orientation field information, STUA Net based on the Swin Transformer, and FPN-ResUNet with residual feature point fusion have all significantly improved restoration accuracy and structural consistency. In particular, GANs and their variants (such as D-GAN, Pix2Pix, and SGAN), with their powerful generative capabilities, can effectively reconstruct nonlinear textures and missing regions, generating natural and coherent fingerprint images.

[0003] Despite significant progress, current fingerprint restoration still faces challenges: strong data dependence: model performance is highly dependent on the quality and diversity of training data, and it is difficult to obtain high-quality and diverse damaged fingerprint samples; limited generalization ability: when faced with unseen complex damage types (such as damage caused by multiple factors), existing single-image restoration methods often show reduced effectiveness; high requirements for detail fidelity: fingerprint recognition is extremely sensitive to ridge topology, and any slight distortion may lead to misidentification.

[0004] Therefore, how to improve the quality and efficiency of restoration results while ensuring the authenticity and continuity of fingerprint ridge structure is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a fingerprint image restoration method and system based on generative adversarial networks to overcome or at least partially solve the above problems, thereby improving the quality and efficiency of restoration results while ensuring the authenticity and coherence of fingerprint ridge structure.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a fingerprint image restoration method based on generative adversarial networks, comprising: Acquire a complete fingerprint image and preprocess it, then combine it with a mask to generate multiple fingerprint missing images with missing regions; Based on the missing fingerprint image and the corresponding mask, the generator including the AOT block is input to capture and integrate long-range fingerprint context information to obtain a generated fingerprint image; The generated fingerprint image and the complete fingerprint image are input into the discriminator to obtain a repair determination result; The generator is optimized based on the repair determination result and the total loss function to obtain an optimized generator; The fingerprint image to be repaired and the corresponding mask are obtained and input into the optimization generator to obtain the repaired fingerprint image.

[0008] In another embodiment, the method for obtaining the missing fingerprint image is as follows: Based on the complete fingerprint image, cropping, normalization, and grayscale processing are performed to obtain a preprocessed fingerprint image; Randomly generate multiple different masks; The mask indicates the location of the missing region; A fingerprint missing image with the missing region is generated based on the mask and the complete fingerprint image.

[0009] In another embodiment, the generator includes: a first convolutional layer, a second convolutional layer, a first AOT block, a second AOT block, a third convolutional layer, and a fourth convolutional layer; The missing fingerprint image and the corresponding mask are multiplied and fused pixel by pixel to obtain the identifying fingerprint image; Based on the identified fingerprint image, it is sequentially input into the first convolutional layer and the second convolutional layer to obtain the first extracted feature; Based on the first extracted features, they are sequentially input into the first AOT block and the second AOT block to capture and integrate long-range fingerprint context information and feature patterns, update and fill features in the missing regions, and obtain spatial aggregated features. Based on the spatial aggregation features, the data are sequentially input into the third and fourth convolutional layers to obtain the generated fingerprint image.

[0010] In another embodiment, the first AOT block and the second AOT block have the same structure, both including: First extraction branch, second extraction branch, third extraction branch, fourth extraction branch, first standard convolutional layer, second standard convolutional layer, and gated branch; The first extracted features are respectively input to the first extraction branch, the second extraction branch, the third extraction branch, the fourth extraction branch and the first standard convolutional layer to obtain the first scale features, the second scale features, the third scale features, the fourth scale features and the fifth scale features; Multi-scale fused features are obtained by fusing the first scale feature, the second scale feature, the third scale feature and the fourth scale feature; The multi-scale fusion features are input into the second standard convolutional layer and then fused with the fifth-scale features to obtain the learned residual features. The current gate value is obtained by inputting the first extracted feature into the gated branch; The learned residual features and the first extracted features are fused based on the current threshold to obtain the initial aggregated features; The spatial aggregation features are obtained by inputting the initial aggregation features into the second AOT block.

[0011] In another embodiment, the first extraction branch includes a first 3×3 convolution and a first activation layer, wherein the first extracted features are sequentially input into the first 3×3 convolution and the first activation layer to obtain the first scale features; The second extraction branch includes: a second 3×3 convolution and a second activation layer. The first extracted features are sequentially input into the second 3×3 convolution and the second activation layer to obtain the second scale features. The third extraction branch includes a third 3×3 convolution and a third activation layer. The first extracted features are sequentially input into the third 3×3 convolution and the third activation layer to obtain the third scale features. The fourth extraction branch includes a fourth 3×3 convolution and a fourth activation layer. The first extracted features are sequentially input into the fourth 3×3 convolution and the fourth activation layer to obtain the fourth scale features.

[0012] In another embodiment, the gated branch includes: a third standard convolution and a Sigmoid activation layer; The first extracted feature is sequentially input into the third standard convolution and the Sigmoid activation layer to obtain the current gate value.

[0013] In another embodiment, the discriminator includes five stacked 4×4 convolutional layers. The first four 4×4 convolutional layers are followed by activation layers and instance normalization layers to stabilize training and enhance feature representation capabilities. The last 4×4 convolutional layer outputs the confidence level of a local patch on the input image corresponding to each position as the repair determination result.

[0014] In another embodiment, the total loss function includes: L1 loss, style loss, perceptual loss, and SMPatchGAN adversarial loss; The L1 loss, style loss, perceptual loss, and SMPatchGAN adversarial loss are each weighted and summed based on their respective weights to form the total loss function.

[0015] In another embodiment, it also includes: Edge enhancement is performed on the repaired fingerprint image based on the Laplacian convolution kernel to obtain an enhanced fingerprint image; Based on the enhanced fingerprint image, histogram equalization is used to adjust the grayscale distribution of the image, thereby enhancing its overall contrast and obtaining a clear and complete restored fingerprint image.

[0016] In a second aspect, embodiments of the present invention provide a fingerprint image restoration system based on generative adversarial networks, comprising: a missing fingerprint generation module, a fingerprint initial restoration module, a restoration result judgment module, a generator optimization module, and a fingerprint restoration output module; The missing fingerprint generation module is used to acquire a complete fingerprint image and perform preprocessing, and combine it with a mask to generate multiple missing fingerprint images with missing regions. The fingerprint initial repair module is used to input the missing fingerprint image and the corresponding mask into a generator including an AOT block to capture and integrate long-distance fingerprint context information to obtain a generated fingerprint image. The repair result judgment module is used to input the generated fingerprint image and the complete fingerprint image into the discriminator to obtain a repair judgment result; The generator optimization module is used to optimize the generator based on the repair judgment result and the total loss function to obtain an optimized generator; The fingerprint repair output module is used to acquire the fingerprint image to be repaired and the corresponding mask and input them into the optimization generator to obtain the repaired fingerprint image.

[0017] As can be seen from the above technical solutions, compared with existing technologies, this invention discloses a fingerprint image restoration method and system based on generative adversarial networks (GANs). It introduces a context switching mechanism through the IAOT-GAN model to modify the traditional GAN ​​architecture, thereby enhancing the model's understanding of the context and texture generation of damaged areas. IAOT-GAN not only improves the effectiveness of fingerprint restoration but also enhances the model's generalization ability and practical application performance through synthetic data augmentation techniques using limited training samples. Furthermore, this invention combines grayscale, sharpening, and histogram equalization with multiple traditional image processing techniques, providing new ideas and methods for the development and application of fingerprint recognition technology. The method proposed in this invention not only ensures real-time performance but also maximizes the clarity and contrast of the restored image, which has significant application value for low-quality or damaged fingerprint images in practical criminal investigation scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart of a fingerprint image restoration method based on generative adversarial networks provided in an embodiment of the present invention.

[0020] Figure 2 The user-selected domain provided in the embodiments of the present invention Schematic diagram.

[0021] Figure 3 This is a schematic diagram of the discrete domain selected by the user in an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the damaged fingerprint repair process provided in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram illustrating the results of various incomplete fingerprint repair methods provided in the embodiments of the present invention.

[0024] Figure 6 This is a schematic diagram illustrating fingerprint repair of damaged areas of various sizes provided in an embodiment of the present invention.

[0025] Figure 7 This is a schematic diagram illustrating the evolution of the quantitative measurement capability for repairing incomplete fingerprint images provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1 like Figure 1 As shown, this embodiment of the invention discloses a fingerprint image restoration method based on generative adversarial networks, including the following steps. For ease of description, these steps are numbered S1 to S5, and these numbers are not used to limit the sequential relationship between the various steps of this invention: S1 acquires a complete fingerprint image and performs preprocessing, then combines it with a mask to generate multiple fingerprint missing images with missing regions.

[0028] Furthermore, a camera is used to capture a complete fingerprint image of the target fingerprint to obtain the original complete fingerprint image.

[0029] Furthermore, the method for obtaining fingerprint missing images is as follows: The preprocessed fingerprint image is obtained by cropping, normalizing, and grayscale conversion based on the complete fingerprint image; Randomly generate multiple different masks; The location of the missing area is indicated by the mask; Generate a missing fingerprint image with missing regions based on the mask and the complete fingerprint image.

[0030] S2 inputs the missing fingerprint image and the corresponding mask into a generator including an AOT block to capture and integrate long-range fingerprint context information to obtain a generated fingerprint image.

[0031] Furthermore, the generator includes: a first convolutional layer, a second convolutional layer, a first AOT block, a second AOT block, a third convolutional layer, and a fourth convolutional layer; The missing fingerprint image and its corresponding mask are multiplied and fused pixel by pixel to obtain the identifying fingerprint image; The fingerprint image is sequentially input into the first convolutional layer and the second convolutional layer to obtain the first extracted feature; Based on the first extracted features, they are sequentially input into the first AOT block and the second AOT block to capture and integrate long-range fingerprint context information and feature patterns, update and fill the features in the missing regions, and obtain spatial aggregated features. Based on spatial aggregation features, the data is sequentially input into the third and fourth convolutional layers to generate fingerprint images.

[0032] Furthermore, the first AOT block and the second AOT block have the same structure, both including: First extraction branch, second extraction branch, third extraction branch, fourth extraction branch, first standard convolutional layer, second standard convolutional layer, and gated branch; The first extracted features are input into the first extraction branch, the second extraction branch, the third extraction branch, the fourth extraction branch and the first standard convolutional layer, respectively, to obtain the first scale feature, the second scale feature, the third scale feature, the fourth scale feature and the fifth scale feature. Multi-scale fused features are obtained by fusing features at the first, second, third, and fourth scales. The multi-scale fusion features are input into the second standard convolutional layer and then fused with the fifth-scale features to obtain the learned residual features. The current gate value is obtained by inputting the first extracted feature into the gated branch; The residual features and the first extracted features are fused based on the current threshold to obtain the initial aggregated features; The spatial aggregate features are obtained by inputting the initial aggregate features into the second AOT block.

[0033] Furthermore, the first extraction branch includes a first 3×3 convolution and a first activation layer. The first extracted features are sequentially input into the first 3×3 convolution and the first activation layer to obtain the first scale features. The second extraction branch includes: a second 3×3 convolution and a second activation layer. The first extracted features are sequentially input into the second 3×3 convolution and the second activation layer to obtain the second scale features. The third extraction branch includes: a third 3×3 convolution and a third activation layer. The first extracted features are sequentially input into the third 3×3 convolution and the third activation layer to obtain the third scale features. The fourth extraction branch includes a fourth 3×3 convolution and a fourth activation layer. The first extracted features are sequentially input into the fourth 3×3 convolution and the fourth activation layer to obtain the fourth scale features.

[0034] Furthermore, the first, second, third, and fourth 3×3 convolutions use different dilation rates, corresponding to 1, 2, 4, and 8, respectively. Each sub-kernel applies a different transformation to the input features. By inserting zero values ​​between consecutive sampling points of the dilated convolution kernels, the ability to capture multi-scale features in semantic segmentation tasks is significantly improved. The sub-kernels focus on fewer local regions with lower dilation rates or cover more regions of the original image with higher dilation rates.

[0035] Furthermore, AOT blocks can predict each output pixel from multiple perspectives, effectively combining various types of contextual data to enhance their contextual reasoning capabilities. The generator integrates multiple routes more richly by stacking multiple AOT blocks, which helps the generator capture as many important feature patterns as possible for fingerprint image inpainting.

[0036] Furthermore, the gated branches include: a third standard convolution and a sigmoid activation layer; The first extracted feature x1 is sequentially input into the third standard convolutional layer and the Sigmoid activation layer to obtain the current gate value g; By introducing a gated residual connection through gated branches, the network adaptively decides whether to retain the original input or use the residual for updates. When g is close to 1, the output mainly comes from the first extracted feature x1, that is, the current pixel is trusted; When g approaches 0, the output mainly comes from the learned residual feature x2, i.e., it depends on the network prediction; It achieves adaptive fusion of different regions.

[0037] Furthermore, the initial aggregated feature x3 is: ; This spatial feature aggregation method can effectively update features in missing regions of fingerprint images while maintaining the stability of features outside the missing regions.

[0038] S3 inputs the generated fingerprint image and the complete fingerprint image into the discriminator to obtain the repair judgment result.

[0039] Furthermore, the discriminator includes five stacked 4×4 convolutional layers. The first four 4×4 convolutional layers are followed by activation layers and instance normalization layers to stabilize training and enhance feature representation capabilities. The last 4×4 convolutional layer outputs the confidence level of a local patch on the input image at each location as the repair determination result.

[0040] Furthermore, this invention uses G as the generator, an appropriate binary repair mask of m, a known pixel value of 0, a missing region value of 1, and the actual image as x. Therefore, the repair result is as follows: ; in, This indicates pixel-by-pixel multiplication.

[0041] Furthermore, in another embodiment, this invention employs a PatchGAN network as the discriminator. The PatchGAN discriminator divides the image into multiple local patches, independently judging the authenticity of each patch, and ultimately outputting a true / false probability map instead of a single value. For training, this invention overlays a mask around the missing regions using Gaussian filtering to create a soft mask. The adversarial loss of the discriminator is as follows: ; in, This represents the composite function of Gaussian filtering and downsampling. middle This represents the expected value (average over the sample). express From the distribution Mid-sampling, This refers to the distribution of the generated / restored image samples, which is the restoration result produced by the generator. This indicates the discriminator's assessment of the repaired image. z The output prediction map, where m represents the binary repair mask. Represents a real image Expected value Representing real images, from data distribution The image obtained by sampling in the middle, It represents the probability distribution of real images, indicating how samples were drawn from the training dataset. This indicates that the discriminator recognizes the real image. x The output prediction graph.

[0042] Furthermore, σ represents the combined operation of Gaussian filtering and downsampling. The effect of this step is that at the boundary of the missing region: the original 0 / 1 abrupt change is smoothed into continuously changing real values ​​(0~1). The label corresponding to each patch is no longer: all true / all false, but: part true + part generated, so that the soft mask can characterize the missing regions and their boundary transition characteristics formed by the mask in the complete fingerprint image.

[0043] Furthermore, the adversarial loss of the generator is: .

[0044] Furthermore, the generator is optimized by predicting synthetic patches for missing regions. This method allows the segmented image to more accurately distinguish synthetic patches for missing regions from the real context outside the missing regions. An enhanced texture generator helps synthesize more realistic textures. Hard Mask Guided PatchGAN (HM PatchGAN) uses hard masks during training, improving PatchGAN training by using binary hard masks instead of Gaussian filtering. However, this method ignores the issue that patches may contain a mixture of real and synthetic pixels, which can affect training performance. To address this, SM PatchGAN employs Gaussian-filtered soft labels, thus avoiding these problems.

[0045] Further, generate fingerprint image I out With complete fingerprint image I gt The image is obtained by combining pixels one by one according to the mask M. I comp : ; The missing regions are filled by the generator, while the non-missing regions retain their original pixels. Next, I comp After concatenating the fingerprint with the corresponding mask M along the channel dimension, the image is fed into a discriminator to determine whether the texture of the repaired area is realistic. Simultaneously, the complete fingerprint image I is... gt When concatenated with the same mask M, it serves as the positive sample input discriminator, forming an adversarial game between the generator and discriminator around the missing region.

[0046] S4 optimizes the generator based on the repair decision result and the total loss function to obtain an optimized generator.

[0047] Furthermore, the total loss function includes: L1 loss, style loss, perceptual loss, and SMPatchGAN adversarial loss; L1 loss, style loss, perceptual loss, and SMPatchGAN adversarial loss are each weighted and summed based on their respective weights to form the total loss function.

[0048] Furthermore, L1 loss To ensure pixel-level reconstruction accuracy: .

[0049] Furthermore, perceived loss This is achieved by reducing the distance between the activation maps of the restored image and the real image, specifically: ; in, and All represent activation maps from pre-trained network layers, where, It is a real image Input a pre-trained feature extraction network with fixed parameters, and take the first... The activation feature map output by the layer. N i express Various elements in This involves repairing the image. Input the same network, and take the first... Layer output, Ni represents The various elements in; i represents the first... Layer feature maps, || ||1 represents the L1 distance, which is the sum of the absolute values ​​of the element-wise differences between the two feature maps.

[0050] Furthermore, style loss The Gram matrix used to define the depth features of the restored image and the original image. distance: ; in, This indicates averaging across different layers, over several selected layers. i Calculate style differences, then perform an average or weighted summary. , This represents the feature correlation matrix.

[0051] Furthermore, an adversarial loss for the generator SM PatchGAN is introduced. This improves the visual accuracy of fingerprint image restoration; These four objectives are jointly optimized to train the complete generator, forming the total loss function L: ; In the experiment, we selected , and Conduct training.

[0052] S5 acquires the fingerprint image to be repaired and the corresponding mask, inputs them into the optimization generator, and obtains the repaired fingerprint image.

[0053] Furthermore, it also includes: Edge enhancement is performed on the repaired fingerprint image based on the Laplacian convolution kernel to obtain an enhanced fingerprint image; Based on the enhanced fingerprint image, histogram equalization is used to adjust the grayscale distribution of the image, thereby enhancing its overall contrast and obtaining a clear and complete restored fingerprint image.

[0054] Furthermore, the generator and discriminator of this invention together form an Aggregated Context Transition Generative Adversarial Network (AOT-GAN). This adversarial network not only possesses the deep learning capabilities of modern neural networks but also integrates various traditional image processing techniques, such as grayscale processing, sharpening, and histogram equalization. Grayscale processing reduces color information, simplifies the image, and minimizes interference from color channels during fingerprint texture restoration. Sharpening enhances the detailed features of the fingerprint and improves the contrast between ridges and valleys. Histogram equalization adjusts the grayscale distribution of the image, further enhancing its overall contrast and making the fingerprint image clearer. Combining these traditional techniques with modern neural networks improves the fingerprint image restoration capabilities of AOT-GAN.

[0055] Further, fingerprint restoration experiments: In the process of acquiring the fingerprint image to be repaired and converting it to a grayscale image, grayscale processing is used to simplify the image. This step reduces the color complexity of the image, making subsequent processing simpler and more efficient. The grayscale processing of the image uses the maximum value method, where the highest value among the R, G, and B components is directly selected (0 is considered the minimum value, and 255 is considered the maximum value). The formula is: R = G = B = max(R, G, B); IAOT-GAN employs a generative adversarial network approach to learn image features for high-quality fingerprint image inpainting, thereby eliminating defects and damage in the image. The inpainted image may still have some degree of blurriness. To improve image sharpness, sharpening is performed. By enhancing the image's edge information, fingerprint details become clearer and more accurate. We use a custom sharpening method based on local chromatic aberration. This method first uses OpenCV's Laplacian function to compute the Laplacian image of the image, i.e., the second derivative, to evaluate the image's details. Then, the method iterates over each pixel in the original image, determining the color difference between each pixel and its eight surrounding pixels. This pixel is considered a region of detail, and if the color difference exceeds a predefined threshold, it is enhanced.

[0056] The enhancement method increases the current pixel value by 50, but limits the maximum value to 255. This method distinguishes between detailed and smooth areas of the image, thus achieving a sharpening effect. Unlike common sharpening techniques such as high-pass filtering or Laplacian sharpening, this invention provides a custom sharpening strategy based on local features. For each pixel in [the image / database]... Calculate the absolute color difference D(r,c): ; Where I represents the input grayscale image, and is the sharpened output image. Represents the rows and columns of pixels; Define threshold To determine whether to sharpen pixels, if so This increases the intensity of the pixel: ; The min function ensures that the pixel value does not exceed 255, which is the upper limit of the image pixel intensity. If it is Then the pixels remain unchanged: ; Finally, histogram equalization is performed to improve the overall contrast of the image. Histogram equalization adjusts the distribution of gray values, making the dark and bright parts of the image more evenly distributed, thereby improving contrast and making details stand out more. The formula for histogram equalization is as follows: ; in This indicates that the gray levels after histogram equalization are... The corresponding pixel; Indicates grayscale value Pixel values; Grayscale; This represents the total number of pixels.

[0057] Furthermore, this invention selected a total of 1878 fingerprint images from the FVC2002 and FVC2004 datasets. These images cover a wide quality range, from high-quality, clear images to low-quality images with blurriness and partial defects. This diverse sample effectively tests the model's repair capabilities under different conditions. To further enhance the model's robustness, this invention generates an equal number of masks for each image to simulate potential defects in real-world applications. To improve the model's ability to repair various missing regions, partial fingerprint images are generated in this manner.

[0058] During the experiment, this invention collected fingerprint data from 100 individuals, selecting 229 fingerprint images as the validation set to accurately evaluate the model. Furthermore, to comprehensively test the model's actual performance, we also selected 228 images as the test set. The entire dataset was divided as follows: 80.5% training set, 9.8% validation set, and 9.7% test set. This partitioning method ensures sufficient independence between the training and test sets, while also guaranteeing the representativeness of the validation and test sets, providing a sufficient guarantee for the model's performance evaluation.

[0059] Example 2 Robustness verification of the method of this invention: Fingerprint patterns can be categorized into spiral, ring, arch, bundled arc, and radial ring patterns. These shapes are the result of the combined effects of pressure, environmental influences, and genetic factors on the skin growth and development of the fetus during gestation, leading to specific fine line patterns on the skin surface, arranged in linear, arched, and ring shapes.

[0060] This embodiment discloses a simple technique for repairing fingerprints on any surface by targeting a selected area. The user selects an area with damaged portions in the fingerprint image. The damaged portions of the fingerprint image and any area selected by the user are displayed as shown below. Figure 2 As shown. Assume the spatial step size is h, which is a discrete region with discrete boundaries, such as... Figure 3 As shown.

[0061] This invention demonstrates the simultaneous restoration of fingerprint images and several damaged areas to illustrate the robustness of the proposed method. For example... Figure 4 As shown, multi-point, partial fingerprint images are displayed, where (a) represents the original fingerprint, (b) represents the incomplete fingerprint, and (c) represents the repaired result.

[0062] The method of this invention is applicable to various fingerprint patterns, and experimental results show that it can simultaneously repair many areas. For example... Figure 5 As shown, the results of various incomplete fingerprint restorations are displayed. The first row (a) contains five original fingerprint images; the second row (b) contains the corresponding incomplete fingerprint images; and the third row (c) contains the restored fingerprint images. It can be seen that the damaged ridges were restored in multiple areas, and the restored portions are aligned with the direction of adjacent ridges.

[0063] Damaged fingerprint repair quality evaluation: like Figure 6 As shown in the figures, (a) shows the original fingerprint image, (b) is a schematic diagram of an incomplete fingerprint image, and (c) shows the final result after applying the fingerprint restoration algorithm. By comparing the restoration result (c) with the original fingerprint image (a), it can be seen that the algorithm of this invention can effectively restore damaged fingerprint images of various structures. It can be concluded that the restoration of incomplete fingerprints is affected by the surrounding ridge information, and the restoration effect is better when the area is smaller.

[0064] This embodiment uses visual quality assessment, peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) to measure the performance of the IAOT-GAN model. If a clean fingerprint image and a noisy image are provided, the mean squared error (MSE) is defined as follows: ; PSNR (dB) is defined as: ; in, This represents the maximum pixel value of the image.

[0065] If a pixel is represented by binary B, then SSIM's formula is based on three comparison measures between samples X and Y. Brightness, contrast, and structure describe the similarity between two images.

[0066] ; in , It is the average of X and Y; and It is the variance of X and Y; It is the variance of XY; , They are two constants. , It is the default value; the range of values ​​is determined by... express.

[0067] like Figure 7 As shown, the PSNR and SSIM data during the restoration process are displayed. These small-scale figures represent the specific trends in the restoration process in the computational domain as the restoration area increases from 30 square millimeters to 350 square millimeters. In the post-processing stage of the fingerprint image, this invention applies sharpening technology to enhance the edges and details in the image, making the fingerprint texture clearer and easier to distinguish. Simultaneously, this invention also implements histogram equalization, adjusting the pixel intensity distribution of the image and significantly enhancing the overall contrast, thereby more effectively highlighting key features in the fingerprint. To verify the effectiveness of the proposed method, the PSNR and SSIM values ​​were calculated to be 19.828 and 0.954, respectively. A comprehensive comparison was made between the proposed method and advanced technologies. The comparison results are shown in Table 1. Table 1 Comparison Results

[0068] Comprehensive experimental tests show that this method exhibits excellent performance and effectively restores the details and quality of fingerprint images.

[0069] Comprehensive experimental tests show that the method of this invention achieves the highest values ​​in both PSNR and SSIM, at 24.018 and 0.974 respectively, significantly outperforming existing comparative methods. This indicates that it can more effectively suppress noise, restore fingerprint ridge details, and maintain structural coherence during the restoration process. Compared with existing methods, the method of this application achieves higher structural similarity while ensuring restoration quality, demonstrating comprehensive advantages in fingerprint image restoration.

[0070] Example 3 Based on the same inventive concept, this invention also provides a fingerprint image restoration system based on generative adversarial networks, including: a missing fingerprint generation module, a fingerprint initial restoration module, a restoration result judgment module, a generator optimization module, and a fingerprint restoration output module; The missing fingerprint generation module is used to acquire a complete fingerprint image and preprocess it, and combine it with a mask to generate multiple missing fingerprint images with missing regions; The fingerprint initial repair module is used to capture and integrate long-range fingerprint context information by inputting the missing fingerprint image and the corresponding mask into a generator including an AOT block to obtain a generated fingerprint image. The repair result judgment module is used to obtain the repair judgment result based on the input of the generated fingerprint image and the complete fingerprint image into the discriminator; The generator optimization module is used to optimize the generator based on the repair judgment result and the total loss function to obtain an optimized generator; The fingerprint repair output module is used to acquire the fingerprint image to be repaired and the corresponding mask, input them into the optimization generator, and obtain the repaired fingerprint image.

[0071] Furthermore, in this embodiment, the functional implementation methods of each functional module correspond one-to-one with the methods described above, and will not be repeated here.

[0072] Example 4 Based on the same inventive concept, the present invention also provides an electronic device, which includes a processor and a memory, wherein the memory stores instructions, characterized in that the instructions are loaded and executed by the processor to implement a fingerprint image restoration method based on a generative adversarial network as in Embodiment 1.

[0073] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When the processor executes a program stored in memory, it can implement a fingerprint image restoration method based on a generative adversarial network as shown in Example 1.

[0074] The electronic device may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions in the memory to execute a fingerprint image restoration method based on a generative adversarial network as described in Embodiment 1.

[0075] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0077] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fingerprint image restoration method based on generative adversarial networks, characterized in that, include: Acquire a complete fingerprint image and preprocess it, then combine it with a mask to generate multiple fingerprint missing images with missing regions; Based on the missing fingerprint image and the corresponding mask, the generator including the AOT block is input to capture and integrate long-range fingerprint context information to obtain a generated fingerprint image; The generated fingerprint image and the complete fingerprint image are input into the discriminator to obtain a repair determination result; The generator is optimized based on the repair determination result and the total loss function to obtain an optimized generator; The fingerprint image to be repaired and the corresponding mask are obtained and input into the optimization generator to obtain the repaired fingerprint image.

2. The fingerprint image restoration method based on generative adversarial networks according to claim 1, characterized in that, The method for obtaining the missing fingerprint image is as follows: Based on the complete fingerprint image, cropping, normalization, and grayscale processing are performed to obtain a preprocessed fingerprint image; Randomly generate multiple different masks; The mask indicates the location of the missing region; A fingerprint missing image with the missing region is generated based on the mask and the complete fingerprint image.

3. The fingerprint image restoration method based on generative adversarial networks according to claim 1, characterized in that, The generator includes: a first convolutional layer, a second convolutional layer, a first AOT block, a second AOT block, a third convolutional layer, and a fourth convolutional layer; The missing fingerprint image and the corresponding mask are multiplied and fused pixel by pixel to obtain the identifying fingerprint image; Based on the identified fingerprint image, it is sequentially input into the first convolutional layer and the second convolutional layer to obtain the first extracted feature; Based on the first extracted features, they are sequentially input into the first AOT block and the second AOT block to capture and integrate long-range fingerprint context information and feature patterns, update and fill features in the missing regions, and obtain spatial aggregated features. Based on the spatial aggregation features, the data are sequentially input into the third and fourth convolutional layers to obtain the generated fingerprint image.

4. The fingerprint image restoration method based on generative adversarial networks according to claim 3, characterized in that, The first AOT block and the second AOT block have the same structure, both including: First extraction branch, second extraction branch, third extraction branch, fourth extraction branch, first standard convolutional layer, second standard convolutional layer, and gated branch; The first extracted features are respectively input to the first extraction branch, the second extraction branch, the third extraction branch, the fourth extraction branch and the first standard convolutional layer to obtain the first scale features, the second scale features, the third scale features, the fourth scale features and the fifth scale features; Multi-scale fused features are obtained by fusing the first scale feature, the second scale feature, the third scale feature and the fourth scale feature; The multi-scale fusion features are input into the second standard convolutional layer and then fused with the fifth-scale features to obtain the learned residual features. The current gate value is obtained by inputting the first extracted feature into the gated branch; The learned residual features and the first extracted features are fused based on the current threshold to obtain the initial aggregated features; The spatial aggregation features are obtained by inputting the initial aggregation features into the second AOT block.

5. The fingerprint image restoration method based on generative adversarial networks according to claim 4, characterized in that, The first extraction branch includes: a first 3×3 convolution and a first activation layer. The first extracted features are sequentially input into the first 3×3 convolution and the first activation layer to obtain the first scale features. The second extraction branch includes: a second 3×3 convolution and a second activation layer. The first extracted features are sequentially input into the second 3×3 convolution and the second activation layer to obtain the second scale features. The third extraction branch includes a third 3×3 convolution and a third activation layer. The first extracted features are sequentially input into the third 3×3 convolution and the third activation layer to obtain the third scale features. The fourth extraction branch includes a fourth 3×3 convolution and a fourth activation layer. The first extracted features are sequentially input into the fourth 3×3 convolution and the fourth activation layer to obtain the fourth scale features.

6. The fingerprint image restoration method based on generative adversarial networks according to claim 4, characterized in that, The gated branch includes: a third standard convolution and a Sigmoid activation layer; The first extracted feature is sequentially input into the third standard convolution and the Sigmoid activation layer to obtain the current gate value.

7. The fingerprint image restoration method based on generative adversarial networks according to claim 1, characterized in that, The discriminator comprises five stacked 4×4 convolutional layers. The first four 4×4 convolutional layers are followed by activation layers and instance normalization layers to stabilize training and enhance feature representation capabilities. The last 4×4 convolutional layer outputs the confidence level of a local patch on the input image at each location as the repair determination result.

8. The fingerprint image restoration method based on generative adversarial networks according to claim 1, characterized in that, The total loss function includes: L1 loss, style loss, perceptual loss, and SMPatchGAN adversarial loss; The L1 loss, style loss, perceptual loss, and SMPatchGAN adversarial loss are each weighted and summed based on their respective weights to form the total loss function.

9. The fingerprint image restoration method based on generative adversarial networks according to claim 1, characterized in that, Also includes: Edge enhancement is performed on the repaired fingerprint image based on the Laplacian convolution kernel to obtain an enhanced fingerprint image; Based on the enhanced fingerprint image, histogram equalization is used to adjust the grayscale distribution of the image, thereby enhancing its overall contrast and obtaining a clear and complete restored fingerprint image.

10. A fingerprint image restoration system based on generative adversarial networks, used to execute a fingerprint image restoration method based on generative adversarial networks as described in any one of claims 1-9, characterized in that, include: Missing fingerprint generation module, fingerprint initial repair module, repair result judgment module, generator optimization module, and fingerprint repair output module; The missing fingerprint generation module is used to acquire a complete fingerprint image and perform preprocessing, and combine it with a mask to generate multiple missing fingerprint images with missing regions. The fingerprint initial repair module is used to input the missing fingerprint image and the corresponding mask into a generator including an AOT block to capture and integrate long-distance fingerprint context information to obtain a generated fingerprint image. The repair result judgment module is used to input the generated fingerprint image and the complete fingerprint image into the discriminator to obtain a repair judgment result; The generator optimization module is used to optimize the generator based on the repair judgment result and the total loss function to obtain an optimized generator; The fingerprint repair output module is used to acquire the fingerprint image to be repaired and the corresponding mask and input them into the optimization generator to obtain the repaired fingerprint image.

Images