Fingerprint direction field repairing method and device, nonvolatile storage medium and electronic equipment

Abstract

The application discloses a fingerprint direction field repairing method and device, a nonvolatile storage medium and an electronic device. The method comprises the following steps: determining an initial direction field and a quality sketch of a to-be-repaired fingerprint image, wherein the quality sketch is used for indicating a high-quality area and a low-quality area in the to-be-repaired fingerprint image; determining a low-quality fingerprint area and a high-quality fingerprint area of the to-be-repaired fingerprint image according to the quality sketch; and repairing a to-be-repaired area in the initial direction field according to a high-quality direction field area corresponding to the high-quality fingerprint area in the initial direction field, to obtain a target direction field, wherein the to-be-repaired area is a direction field area corresponding to the low-quality fingerprint area in the initial direction field. The application solves the technical problem that more information cannot be obtained from a field fingerprint collected on site due to the fact that the direction field of the fingerprint image cannot be repaired in the related art.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a fingerprint orientation field restoration method, apparatus, non-volatile storage medium, and electronic device. Background Technology

[0002] Currently, in forensic and other fields, accurate identification of fingerprints collected at crime scenes is essential. Among the various information carried by fingerprints, fingerprint orientation field information is a particularly important type. However, crime scene fingerprints often suffer from complex noise interference or fingerprint overlap, making it impossible to obtain the orientation field information and thus limiting the amount of information that can be gleaned from them.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a fingerprint orientation field repair method, apparatus, non-volatile storage medium, and electronic device to at least solve the technical problem that more information cannot be obtained from on-site fingerprints due to the inability to repair the orientation field of fingerprint images in related technologies.

[0005] According to one aspect of the embodiments of this application, a fingerprint orientation field repair method is provided, comprising: determining an initial orientation field and a quality diagram of a fingerprint image to be repaired, wherein the quality diagram is used to indicate high-quality regions and low-quality regions in the fingerprint image to be repaired; determining low-quality fingerprint regions and high-quality fingerprint regions in the fingerprint image to be repaired based on the quality diagram; repairing the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain a target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0006] Optionally, the step of determining the initial orientation field of the fingerprint image to be repaired includes: determining the probability distribution map of the fingerprint image to be repaired, wherein the probability distribution map includes orientation probability information of each image block; determining the orientation of the image block with the highest probability based on the probability distribution map; and encoding the orientation of the image block into a two-dimensional vector to obtain the initial orientation field.

[0007] Optionally, the step of determining the quality schematic diagram of the fingerprint image to be repaired includes: determining a preset probability threshold; comparing the target probability of an image block with the preset probability threshold, wherein the target probability is the direction with the highest probability in the directional probability information of the image block; determining a first type of image block belonging to a high-quality region and a second type of image block belonging to a low-quality region based on the comparison result, wherein the target probability of the first type of image block is not less than the preset probability threshold, and the target probability of the second type of image block is less than the preset probability threshold; and determining the quality schematic diagram based on the first type of image block and the second type of image block.

[0008] Optionally, the steps of assigning a first label value to the first type of image block and a second label value to the second type of image block to obtain a quality schematic diagram include: determining the label value corresponding to the first type of image block as the first label value and the label value corresponding to the second type of image block as the second label value to obtain an initial quality schematic diagram; determining a segmentation map of the fingerprint image to be repaired, wherein the segmentation map is used to determine the background image region and the fingerprint image region in the fingerprint image to be repaired; segmenting the initial quality schematic diagram using the segmentation map to obtain a quality schematic diagram, wherein the quality schematic diagram only includes the label values ​​corresponding to the image blocks belonging to the fingerprint region.

[0009] Optionally, the step of repairing the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain the target orientation field includes: inputting the quality schematic diagram and the high-quality orientation field region into the orientation field repair network to obtain the repaired orientation field output by the orientation field repair network; and repairing the region to be repaired in the initial orientation field based on the repaired orientation field to obtain the target orientation field.

[0010] Optionally, the orientation field restoration network is trained by: acquiring multiple complete fingerprint images and the complete fingerprint orientation field corresponding to each complete fingerprint image; masking a portion of the complete fingerprint orientation field to obtain a fragmented fingerprint orientation field; and using the fragmented fingerprint orientation field as input to the orientation field restoration network and the complete fingerprint orientation field as a control to train the orientation field restoration network.

[0011] Optionally, the steps of determining the initial orientation field, segmentation map, and quality schematic of the fingerprint image to be repaired include: inputting the fingerprint image to be repaired into an orientation field prediction network to obtain the initial orientation field, quality schematic, and segmentation map output by the orientation field prediction network, wherein the segmentation map is used to determine the fingerprint image region and background image region in the fingerprint image to be repaired.

[0012] Optionally, the orientation field prediction network is trained as follows: the orientation of the image patch in the fingerprint image to be repaired is discretized to obtain multiple orientations of different categories, where each category uniquely corresponds to a class number; the target cross-entropy loss function is determined based on the class number corresponding to each category; the target coherence loss function of the orientation field prediction network is determined, and the target loss function of the orientation field prediction network is obtained by combining the target cross-entropy loss function and the target coherence loss function; the orientation field prediction network is trained using the target loss function.

[0013] Optionally, the low-quality image region in the fingerprint image to be repaired includes at least one of the following: fingerprint overlap region, noise region, and blurred fingerprint ridge region.

[0014] According to another aspect of the embodiments of this application, a fingerprint orientation field repair device is also provided, comprising: a first processing module, configured to determine an initial orientation field and a quality diagram of a fingerprint image to be repaired, wherein the quality diagram is used to indicate high-quality regions and low-quality regions in the fingerprint image to be repaired; a second processing module, configured to determine low-quality fingerprint regions and high-quality fingerprint regions of the fingerprint image to be repaired based on the quality diagram; and a third processing module, configured to repair the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field, to obtain a target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, and the program controls the device where the non-volatile storage medium is located to execute a fingerprint orientation field repair method when it runs.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a fingerprint orientation field repair method during runtime.

[0017] In this embodiment, an initial orientation field and quality diagram of the fingerprint image to be repaired are determined. The quality diagram indicates high-quality and low-quality regions in the fingerprint image. Based on the quality diagram, low-quality and high-quality fingerprint regions are determined. The region to be repaired in the initial orientation field is repaired based on the high-quality orientation field region corresponding to the high-quality fingerprint region, resulting in a target orientation field. The region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field. By determining the low-quality and high-quality regions in the orientation field and using the high-quality region to repair the low-quality region, the orientation field of the fingerprint image is repaired. This achieves the technical effect of obtaining orientation field information from the fingerprint image, thus solving the technical problem that related technologies cannot obtain more information from on-site fingerprints due to the inability to repair the orientation field of the fingerprint image. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a schematic diagram of the structure of a computer terminal (mobile terminal) according to an embodiment of this application;

[0020] Figure 2 This is a schematic flowchart of a fingerprint orientation field repair method provided according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a fingerprint image to be repaired and its corresponding orientation probability information provided according to an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of a fingerprint image to be repaired according to an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of a low-quality region in a fingerprint image to be repaired, according to an embodiment of this application.

[0024] Figure 6 This is a schematic diagram of the network structure of a direction field prediction network according to an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of a repaired fingerprint image provided according to an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of a process for obtaining orientation field labels for training data according to an embodiment of this application;

[0027] Figure 9 This is a schematic diagram of the input image in the ablation experiment provided according to the embodiments of this application;

[0028] Figure 10 This is a schematic diagram illustrating the identification of low-quality regions in the input image during an ablation experiment according to an embodiment of this application;

[0029] Figure 11 This is a schematic diagram of the initial orientation field of the input image in the ablation experiment provided according to the embodiments of this application;

[0030] Figure 12 This is a schematic diagram of the orientation field after repair in the ablation experiment provided according to the embodiments of this application;

[0031] Figure 13 This is a schematic diagram of the fingerprint orientation field repair device provided according to an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application are explained below:

[0035] Fingerprint orientation field: The fingerprint orientation field represents the orientation information of each image patch in a fingerprint image. It describes the ridges and valleys in a fingerprint image, as well as their variations. The fingerprint orientation field can be used to extract fingerprint features and has important applications in fingerprint recognition and matching.

[0036] Latent fingerprints refer to fingerprints found at the scene that are likely left by the suspect, are highly noisy, and may be incomplete.

[0037] Aligned rolled fingerprints refer to fingerprints acquired by rolling on a plane. In this application embodiment, it mainly refers to fingerprints acquired under controlled conditions.

[0038] In the field of forensic medicine, identifying fingerprints collected at crime scenes is an important means of obtaining crucial information. However, crime scene fingerprint images often suffer from severe noise interference and fingerprint overlap, resulting in low-quality areas in the acquired images and making it impossible to obtain the complete orientation field of the crime scene fingerprint image.

[0039] In related technologies, dictionary-based orientation field restoration methods are commonly used to repair the orientation field of on-site fingerprint images and obtain a complete orientation field. However, the performance of this restoration method is heavily dependent on the quality of the initial orientation field, and serious errors often occur in the singularity regions of the fingerprint.

[0040] Related technologies also provide a fingerprint orientation field restoration method based on a mathematical model and a fingerprint orientation field restoration method based on a deep neural network. However, the fingerprint orientation field restoration method based on a mathematical model requires manual design of the mathematical model according to the environment when restoring fingerprints, resulting in poor adaptability, especially in open and noisy environments. Meanwhile, the fingerprint orientation field restoration method based on a deep neural network still performs poorly in areas with unclear ridges and overlapping fingerprint areas.

[0041] To address the aforementioned issues, this application provides relevant solutions, which are detailed below.

[0042] According to an embodiment of this application, a method embodiment for fingerprint orientation field restoration is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] The method embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a fingerprint orientation field restoration method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0044] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0045] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the fingerprint orientation field repair method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the fingerprint orientation field repair method described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0047] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0048] Under the above operating environment, this application provides a fingerprint orientation field restoration method, such as... Figure 2 As shown, the method includes the following steps:

[0049] Step S202: Determine the initial orientation field and quality diagram of the fingerprint image to be repaired, wherein the quality diagram is used to indicate the high-quality and low-quality regions in the fingerprint image to be repaired;

[0050] It should be noted that the low-quality image regions in the fingerprint images to be repaired include at least one of the following: fingerprint overlap regions, noise regions, and blurred fingerprint ridge regions.

[0051] The fingerprint orientation field restoration method provided in this application can accurately identify low-quality image areas in the fingerprint image to be restored. In particular, it has a higher recognition accuracy compared with related technologies when identifying fingerprint overlapping areas and blurred fingerprint ridge areas.

[0052] In the technical solution provided in step S202, the step of determining the initial orientation field of the fingerprint image to be repaired includes: determining the probability distribution map of the fingerprint image to be repaired, wherein the probability distribution map includes the orientation probability information of each image block; determining the orientation of the image block with the highest probability based on the probability distribution map; and encoding the orientation of the image block into a two-dimensional vector to obtain the initial orientation field.

[0053] Specifically, an image patch is an image region in the fingerprint image to be repaired, and different image patches do not overlap. The direction with the highest probability mentioned above refers to the peak probability in the direction probability corresponding to the image patch. Assuming this direction can be represented by an angle θ, the above two-dimensional vector can be represented as [sin2θ, cos2θ]. It should be noted that from... Figure 3 As can be seen, the probability distribution map can also be used to determine whether an image patch belongs to a low-quality image region. Figure 3 In the fingerprint image on the left, the area within the circle is a high-quality region, the area within the triangle is a low-quality region, and the area within the square is an overlapping area within the low-quality region. Understandably, the orientation probability information is essentially the probability that an image patch's orientation is a certain direction. For high-quality regions that are unaffected by noise, have no overlap, and have clear ridges, the specific orientation of the image patch is relatively easier to predict due to less interference. This is reflected in... Figure 3 The probability distribution plot on the right shows that the probability peak for this type of image patch is large, indicating that the orientation information of the image patch is relatively clear. However, for low-quality image patches, due to noise interference, fingerprint overlap, or unclear ridges, it is difficult to predict the orientation of the image patch, which is reflected in... Figure 3 The probability distribution plot on the right shows that the probability peak of this type of image patch is relatively small, indicating that the directional information of the image patch is unclear. As for overlapping fingerprint regions, from... Figure 3 As can be seen, the orientation probabilities of image blocks in the overlapping fingerprint region are closer to a uniform distribution, with two peaks and a large variance. Therefore, a threshold for the orientation probability peaks can be set to distinguish whether each image block belongs to a high-quality region.

[0054] In summary, the quality diagram of the fingerprint image to be repaired can be determined as follows: A preset probability threshold is determined; the target probability of an image block is compared with the preset probability threshold, where the target probability is the direction with the highest probability in the directional probability information of the image block, and the image block is the image region in the fingerprint image to be repaired; based on the comparison result, first-class pixels belonging to high-quality regions and second-class pixels belonging to low-quality regions are determined, where the target probability of the first-class pixels is not less than the preset probability threshold, and the target probability of the second-class pixels is less than the preset probability threshold; the quality diagram is determined based on the first-class pixels and the second-class pixels.

[0055] As an optional implementation, the orientation probability information of the image patch can be determined from the feature points. These feature points are the feature points in the feature map of the last layer after downsampling in the neural network, before the softmax layer. Each feature point corresponds to a size of n. n image patches, where n is the downsampling factor. Feature points are vector points extracted from features by a neural network. Their physical meaning is usually information about the orientation field in a specific image patch of a field or fingerprint. The specific size of the image patch represented by the feature point can be set by the user through the stride parameter of the convolution downsampling, for example, it can be set to 8×8, in which case the convolution will downsample the image by a factor of 1 / 8, and different image patches will not overlap. As an optional implementation, by using the maximum probability direction of the 8×8 image patch as the feature value corresponding to the image patch, the resolution of the final quality schematic diagram can be smaller than the resolution of the fingerprint image to be repaired, thus reducing the consumption of computing resources without affecting the final result. In addition, the aforementioned maximum probability direction refers to the probability corresponding to the direction with the highest probability among the possible directions of the image patch.

[0056] Specifically, the steps of assigning a first label value to the first type of image block and assigning a second label value to the second type of image block to obtain a quality schematic diagram include: determining the label value corresponding to the first type of image block as the first label value and the label value corresponding to the second type of image block as the second label value to obtain an initial quality schematic diagram; determining a segmentation map of the fingerprint image to be repaired, wherein the segmentation map is used to determine the background image region and the fingerprint image region in the fingerprint image to be repaired; segmenting the initial quality schematic diagram using the segmentation map to obtain a quality schematic diagram, wherein the quality schematic diagram only includes the label values ​​corresponding to the image blocks belonging to the fingerprint region.

[0057] As an optional implementation, the steps of determining the initial orientation field, segmentation map, and quality diagram of the fingerprint image to be repaired include: inputting the fingerprint image to be repaired into an orientation field prediction network to obtain the initial orientation field, quality diagram, and segmentation map output by the orientation field prediction network, wherein the segmentation map is used to determine the fingerprint image region and background image region in the fingerprint image to be repaired.

[0058] Specifically, the unprocessed image to be repaired and its orientation field, such as... Figure 4 As shown, after processing by the orientation field prediction network, the low-quality region of the image to be repaired and its orientation field are predicted. Figure 5 As shown.

[0059] The structure of the orientation field prediction network provided in this application embodiment is as follows: Figure 6 As shown, the network architecture of the orientation field prediction network is similar to ResNet, but the max pooling layer has been removed.

[0060] In addition, convolutional layers with a stride greater than 1 were used for downsampling in the orientation field prediction network. Specifically, each of the first three layers of the orientation field prediction network ( Figure 6The region indicated by "×3" includes a convolutional layer, followed by a ReLU activation layer and an instance normalization layer. These convolutional, activation, and normalization layers reduce the resolution of the input sample image to 1 / 8 of its original value. Following these layers are 12 residual modules (…). Figure 6 The regions are indicated by "×12" in the diagram. Each residual module contains a convolutional layer with ReLU activation and instance normalization layers, and is linked to another residual module. The order of the convolutional layer, ReLU activation layer, and instance normalization layer in each residual module is convolutional layer, instance normalization layer, and ReLU activation layer. After the 12 residual blocks is an ASPP module, followed by several convolutional layers with ReLU activation and instance normalization layers. Figure 6 As can be seen, the orientation field prediction network can output two results: one is the segmentation result S, and the other is a vector with 90 channels, representing the orientation distribution at a 1 / 8 scale. Then, through max and threshold operations, the orientation distribution result can be mapped to a quality map M, and through the argmax operation, the orientation distribution can be mapped to an initial orientation field Φ, encoded as [sin2θ, cos2θ]. It should be noted that... Figure 6 In this context, tanh refers to the tanh activation function. Figure 6 Each layer in the middle follows Figure 6 The distribution pattern shown in the upper left corner indicates that each neural network layer contains three layers, in the following order: convolutional layer, instance normalization layer, and ReLU activation layer.

[0061] In some embodiments of this application, the orientation field prediction network is trained in the following manner: the orientation of the image patch in the fingerprint image to be repaired is discretized to obtain multiple orientations of different categories, wherein each category uniquely corresponds to a class number; the target cross-entropy loss function is determined based on the class number corresponding to each category; the target coherence loss function of the orientation field prediction network is determined, and the target loss function of the orientation field prediction network is obtained by combining the target cross-entropy loss function and the target coherence loss function; the orientation field prediction network is trained using the target loss function.

[0062] Specifically, there can be 90 categories, and the categories can be encoded starting from 1. The specific expression for the cross-entropy loss function is as follows:

[0063]

[0064] In the above formula, ROI represents the region of interest in the image. Representing feature points The peak directional probability of the image patch represented in the label map. Representing feature points The peak directional probability of the represented image patch in the quality diagram output by the network.

[0065] The specific expression for the coherence loss function is as follows:

[0066]

[0067] The ratio of the cross-entropy loss function to the coherence loss function can be 1:1.

[0068] Step S204: Based on the quality diagram, determine the low-quality fingerprint region and the high-quality fingerprint region of the fingerprint image to be repaired;

[0069] Step S206: Repair the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain the target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0070] In the technical solution provided in step S206, the step of repairing the area to be repaired in the initial orientation field according to the high-quality orientation field area corresponding to the high-quality fingerprint area in the initial orientation field to obtain the target orientation field includes: inputting the quality schematic diagram and the high-quality orientation field area into the orientation field repair network to obtain the repair orientation field output by the orientation field repair network; repairing the area to be repaired in the initial orientation field according to the repair orientation field to obtain the target orientation field.

[0071] The final repaired fingerprint image and orientation field are as follows: Figure 7 As shown.

[0072] Specifically, the purpose of the orientation field restoration network is to refine and correct the localization field estimated for low-quality regions. The orientation field restoration network has two inputs: a binarized quality schematic and a high-quality orientation field region. The binarized quality schematic is M′ = M·S, where peak values ​​below a threshold are set to 1, and peak values ​​above the threshold or in the background are set to 0. The second input is the high-quality initial orientation field within the segmented region. , can be represented as A high-quality initial orientation field can serve as a basis for repairing a low-quality orientation field. Specifically, the orientation field repair network can output a corrected orientation field Φ of M', from which the final orientation field can be obtained as Φ' = M'·Φ + .

[0073] It should be noted that since the image input to the orientation field inpainting network is 1 / 8 the size of the original image, the orientation field inpainting network has a larger receptive field compared to the orientation field prediction network, and can correct the orientation field in a larger background.

[0074] As an optional implementation, the orientation field restoration network is trained by: acquiring multiple complete fingerprint images and the complete fingerprint orientation field corresponding to the complete fingerprint images; masking a portion of the complete fingerprint orientation field to obtain a fragmented fingerprint orientation field; and using the fragmented fingerprint orientation field as the input to the orientation field restoration network and the complete fingerprint orientation field as a control to train the orientation field restoration network.

[0075] When correcting the orientation field repair network, the MES loss function can be used, and its specific expression is as follows:

[0076]

[0077] In the above formula, Mask represents the mask area. Representing feature points The true orientation field of the represented image patch, The feature points represent the output of the orientation field repair network. The orientation field predicted for the image patch it represents.

[0078] Specifically, when training the orientation field repair network, ground truth labels from real data can be used (instead of the output of the orientation field prediction network). 20% to 50% of the orientation field can be randomly masked from the fingerprint image segmented from real data, and the remaining orientation field can be used as input to the network. The orientation field repair network then predicts and repairs the missing parts of the orientation field. This training strategy helps the orientation field repair network learn the contextual relationships of the fingerprint, enabling it to effectively correct orientation errors in the orientation field of low-quality regions.

[0079] In some embodiments of this application, a data annotation method similar to FingerNet can be used to generate realistic latent noise data when training the orientation field prediction network and the orientation field repair network. Specifically, this can be achieved through methods such as... Figure 8 The steps shown are as follows to obtain the orientation field label: First, the in-situ fingerprint is aligned with the corresponding rolling fingerprint using matching details and least squares alignment; next, the orientation field of the aligned rolling fingerprint is predicted using FingerNet, and the in-situ fingerprint is segmented; finally, the predicted orientation field of the aligned rolling fingerprint is multiplied with the predicted segment of the in-situ fingerprint to obtain the final orientation field label.

[0080] Specifically, the above-mentioned segmentation of the on-site fingerprint refers to binarizing the on-site fingerprint image, where pixels belonging to the fingerprint region are assigned a value of 1 and background regions are assigned a value of 0, thus obtaining the segmented image shown in image (d) in the figure. Then, as shown in figure (e), the predicted orientation field and the predicted segmentation of the on-site fingerprint are multiplied to obtain the final orientation field label shown in image (f) in the figure.

[0081] By employing a method that determines the initial orientation field and quality diagram of the fingerprint image to be repaired, where the quality diagram indicates high-quality and low-quality regions in the fingerprint image; based on the quality diagram, the low-quality and high-quality fingerprint regions in the fingerprint image to be repaired are determined; and the regions to be repaired in the initial orientation field are repaired based on the high-quality orientation field regions corresponding to the high-quality fingerprint regions, resulting in the target orientation field, where the regions to be repaired are the orientation field regions corresponding to the low-quality fingerprint regions in the initial orientation field. This method, by identifying the low-quality and high-quality regions in the orientation field and using the high-quality regions to repair the low-quality regions, achieves the goal of repairing the orientation field of the fingerprint image, thus realizing the technical effect of obtaining the orientation field information of the fingerprint image. This solves the technical problem of not being able to obtain more information from field-collected fingerprints due to the inability to repair the orientation field of fingerprint images in related technologies.

[0082] To demonstrate the specific effectiveness of the fingerprint orientation field restoration method provided in this application embodiment, a verification experiment process for validating the fingerprint orientation field restoration method is also provided in this application embodiment. Specifically, before conducting the verification experiment, the two neural networks provided in this application embodiment are first trained. The training dataset used includes training data collected from real crime scenes, including 28,000 pairs of matched rolling fingerprints and crime scene fingerprints. Crime scene fingerprints can be labeled with minutiae using expert labeling, each fingerprint image has a high resolution of 500 pixels per inch (ppi), and each fingerprint is 512 × 512 pixels in size. To make the model more robust and accurate, all crime scene fingerprints were used in the verification process, and 8,000 rolling fingerprints were added to the training data. Labels were obtained using the method described above, and the orientation field was estimated using a method based on local details.

[0083] The fingerprint orientation field restoration method provided in this application embodiment was then evaluated using the widely used field fingerprint database NIST SD27. It should be noted that this database contains 258 pairs of field fingerprints and rolling fingerprints, and these fingerprint pairs can be divided into three subsets based on fingerprint quality: good (88 pairs), bad (85 pairs), and ugly (85 pairs).

[0084] Specifically, the accuracy of orientation field estimation in the fingerprint orientation field restoration method provided in this application was tested, and the accuracy of the estimation was used to identify the target. The final identification results show that, compared with existing technologies, the fingerprint orientation field restoration method provided in this application effectively improves the prediction quality of the orientation field in all quality category subsets, especially for bad and ugly subsets, where the improvement is more significant.

[0085] To further demonstrate the effectiveness of the fingerprint orientation field restoration method provided in this application embodiment in restoring fingerprints, especially low-quality fingerprints, an ablation experiment is also provided in this application embodiment for verification.

[0086] Specifically, the images input into the above model during the ablation experiment are as follows: Figure 9 As shown, the orientation field prediction network outputs the recognition result based on the input image, as follows: Figure 10 As shown, where Figure 10 The white areas represent the background, the black areas represent high-quality areas, and the gray areas represent low-quality areas. The identified initial orientation field is as follows: Figure 11 As shown, where Figure 11 Different gray levels in different regions represent different degrees of error. Specifically, the larger the gray value, the greater the error, and the smaller the gray value, the smaller the error, until the error is zero. Figure 12 For the repaired fingerprint orientation field, and Figure 11 In comparison, it can be seen that the number of low-quality areas has been significantly reduced.

[0087] In summary, the ablation experiments show that the fingerprint orientation field recognition method provided in this application can achieve better results than existing technologies in both low-quality region recognition and repair.

[0088] This application provides a fingerprint orientation field restoration device. Figure 13 This is a schematic diagram of the fingerprint orientation field restoration device, as shown below. Figure 13 As shown, the device includes: a first processing module 130, used to determine the initial orientation field and quality diagram of the fingerprint image to be repaired, wherein the quality diagram is used to indicate high-quality regions and low-quality regions in the fingerprint image to be repaired; a second processing module 132, used to determine the low-quality fingerprint region and high-quality fingerprint region of the fingerprint image to be repaired based on the quality diagram; and a third processing module 134, used to repair the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field, to obtain the target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0089] In some embodiments of this application, the step of the first processing module 130 in determining the initial orientation field of the fingerprint image to be repaired includes: determining a probability distribution map of the fingerprint image to be repaired, wherein the probability distribution map includes orientation probability information of each image block; determining the orientation of the image block with the highest probability based on the probability distribution map; and encoding the orientation of the image block into a two-dimensional vector to obtain the initial orientation field.

[0090] In some embodiments of this application, the step of the first processing module 130 in determining the quality schematic diagram of the fingerprint image to be repaired includes: determining a preset probability threshold; comparing the target probability of an image block with the preset probability threshold, wherein the target probability is the direction with the highest probability in the directional probability information of the image block, the image block is an image region in the fingerprint image, and the feature point is represented by the point in the feature map of the last layer after downsampling of the neural network and the layer before softmax; determining a first type of image block belonging to a high-quality region and a second type of image block belonging to a low-quality region based on the comparison result, wherein the target probability of the first type of image block is not less than the preset probability threshold, and the target probability of the second type of image block is less than the preset probability threshold; and determining the quality schematic diagram based on the first type of image block and the second type of image block.

[0091] In some embodiments of this application, the first processing module 130 assigns a first label value to a first type of feature point and a second label value to a second type of image block to obtain a quality schematic diagram. The steps include: determining the label value corresponding to the first type of feature point as the first label value and the label value corresponding to the second type of feature point as the second label value to obtain an initial quality schematic diagram; determining a segmentation map of the fingerprint image to be repaired, wherein the segmentation map is used to determine the background image region and the fingerprint image region in the fingerprint image to be repaired; segmenting the initial quality schematic diagram using the segmentation map to obtain a quality schematic diagram, wherein the quality schematic diagram only includes the label values ​​corresponding to feature points belonging to the fingerprint region.

[0092] In some embodiments of this application, the first processing module 130 determines the initial orientation field, segmentation map, and quality diagram of the fingerprint image to be repaired, including: inputting the fingerprint image to be repaired into an orientation field prediction network to obtain the initial orientation field, quality diagram, and segmentation map output by the orientation field prediction network, wherein the segmentation map is used to determine the fingerprint image region and background image region in the fingerprint image to be repaired.

[0093] In some embodiments of this application, the orientation field prediction network is trained in the following manner: the orientation of the image patch in the fingerprint image to be repaired is discretized to obtain multiple orientations of different categories, wherein each category uniquely corresponds to a class number; the target cross-entropy loss function is determined based on the class number corresponding to each category; the target coherence loss function of the orientation field prediction network is determined, and the target loss function of the orientation field prediction network is obtained by combining the target cross-entropy loss function and the target coherence loss function; the orientation field prediction network is trained using the target loss function.

[0094] In some embodiments of this application, the third processing module 134 repairs the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain the target orientation field. The steps include: inputting the quality schematic diagram and the high-quality orientation field region into the orientation field repair network to obtain the repair orientation field output by the orientation field repair network; and repairing the region to be repaired in the initial orientation field based on the repair orientation field to obtain the target orientation field.

[0095] In some embodiments of this application, the orientation field restoration network is trained in the following manner: acquiring multiple complete fingerprint images and the complete fingerprint orientation field corresponding to the complete fingerprint images; masking a portion of the complete fingerprint orientation field to obtain a fragmented fingerprint orientation field; using the fragmented fingerprint orientation field as the input to the orientation field restoration network and the complete fingerprint orientation field as a control to train the orientation field restoration network.

[0096] In some embodiments of this application, the low-quality image region in the fingerprint image to be repaired includes at least one of the following: fingerprint overlap region, noise region, and blurred fingerprint ridge region.

[0097] It should be noted that each module in the fingerprint orientation field repair device can be a program module (e.g., a set of program instructions to implement a specific function) or a hardware module. For the latter, it can take the following forms, but is not limited to them: each of the above modules is represented by a processor, or the functions of each of the above modules are implemented by a processor.

[0098] According to an embodiment of this application, a non-volatile storage medium is provided, which stores a program. When the program runs, it controls the device containing the non-volatile storage medium to execute the following fingerprint orientation field repair method: determining an initial orientation field and a quality diagram of the fingerprint image to be repaired, wherein the quality diagram indicates high-quality and low-quality regions in the fingerprint image to be repaired; determining low-quality and high-quality fingerprint regions in the fingerprint image to be repaired based on the quality diagram; repairing the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain a target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0099] According to an embodiment of this application, an electronic device is provided, including: a memory and a processor. The processor is configured to run a program stored in the memory, wherein the program executes the following fingerprint orientation field repair method: determining an initial orientation field and a quality diagram of a fingerprint image to be repaired, wherein the quality diagram is used to indicate high-quality regions and low-quality regions in the fingerprint image to be repaired; determining low-quality fingerprint regions and high-quality fingerprint regions of the fingerprint image to be repaired based on the quality diagram; repairing the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain a target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0100] According to an embodiment of this application, a computer program product is provided, which can be run by a processor in an electronic device and, during runtime, controls the electronic device to perform the following fingerprint orientation field repair method: determining an initial orientation field and a quality diagram of the fingerprint image to be repaired, wherein the quality diagram is used to indicate high-quality regions and low-quality regions in the fingerprint image to be repaired; determining low-quality fingerprint regions and high-quality fingerprint regions in the fingerprint image to be repaired based on the quality diagram; repairing the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain a target orientation field, wherein the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field.

[0101] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0103] The units described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0106] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A fingerprint orientation field restoration method, characterized in that, include: An initial orientation field, segmentation map, and quality map of the fingerprint image to be repaired are determined. The fingerprint image to be repaired is input into an orientation field prediction network to obtain the initial orientation field, the quality map, and the segmentation map output by the network. The segmentation map is used to identify the fingerprint image region and the background image region in the fingerprint image to be repaired. The quality map is used to indicate the high-quality and low-quality regions in the fingerprint image to be repaired. The orientation field prediction network is trained as follows: the orientation of image blocks in the fingerprint image to be repaired is discretized to obtain multiple orientation categories, each uniquely corresponding to a class number; a target cross-entropy loss function is determined based on the class number corresponding to each category; a target coherence loss function is determined for the orientation field prediction network, and the target cross-entropy loss function and the target coherence loss function are combined to obtain the target loss function of the orientation field prediction network; the orientation field prediction network is trained using the target loss function. Based on the quality diagram, the low-quality fingerprint regions and high-quality fingerprint regions of the fingerprint image to be repaired are determined; The region to be repaired in the initial orientation field is repaired based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field to obtain the target orientation field. The quality schematic diagram and the high-quality orientation field region are input into the orientation field repair network to obtain the repaired orientation field output by the orientation field repair network. The region to be repaired in the initial orientation field is repaired based on the repaired orientation field to obtain the target orientation field. The region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field. The orientation field repair network is trained as follows: multiple complete fingerprint images and complete fingerprint orientation fields corresponding to the complete fingerprint images are acquired; a portion of the complete fingerprint orientation field is masked to obtain a fragmented fingerprint orientation field; the fragmented fingerprint orientation field is used as the input to the orientation field repair network, and the complete fingerprint orientation field is used as a control to train the orientation field repair network.

2. The fingerprint orientation field restoration method according to claim 1, characterized in that, The step of determining the initial orientation field of the fingerprint image to be repaired includes: Determine the probability distribution map of the fingerprint image to be repaired, wherein the probability distribution map includes the orientation probability information of each image block in the fingerprint image to be repaired; Based on the probability distribution map, the direction with the highest probability is determined as the direction of the image patch; The orientation of the image patch is encoded into a two-dimensional vector to obtain the initial orientation field.

3. The fingerprint orientation field restoration method according to claim 2, characterized in that, The step of determining the quality schematic diagram of the fingerprint image to be repaired includes: Determine the preset probability threshold; The target probability of the image patch is compared with the preset probability threshold, wherein the target probability is the direction with the highest probability in the direction probability information of the image patch; Based on the comparison results, a first type of image patch belonging to a high-quality region and a second type of image patch belonging to a low-quality region are determined, wherein the target probability of the first type of image patch is not less than the preset probability threshold, and the target probability of the second type of image patch is less than the preset probability threshold. The quality schematic diagram is determined based on the first type of image block and the second type of image block.

4. The fingerprint orientation field restoration method according to claim 3, characterized in that, The steps of assigning a first label value to the first type of image patch and assigning a second label value to the second type of image patch to obtain the quality schematic diagram include: The label value corresponding to the first type of image block is determined to be the first label value, and the label value corresponding to the second type of image block is determined to be the second label value, thus obtaining an initial quality schematic diagram; The initial quality diagram is segmented using the segmentation map to obtain the quality diagram, wherein the quality diagram includes only the label values ​​corresponding to image blocks belonging to the fingerprint region.

5. The fingerprint orientation field restoration method according to claim 1, characterized in that, The low-quality image regions in the fingerprint image to be repaired include at least one of the following: fingerprint overlap region, noise region, and blurred fingerprint ridge region.

6. A fingerprint orientation field restoration device, characterized in that, include: The first processing module is used to determine the initial orientation field, segmentation map, and quality diagram of the fingerprint image to be repaired. The fingerprint image to be repaired is input into an orientation field prediction network to obtain the initial orientation field, the quality diagram, and the segmentation map output by the orientation field prediction network. The segmentation map is used to determine the fingerprint image region and the background image region in the fingerprint image to be repaired. The quality diagram is used to indicate the high-quality region and the low-quality region in the fingerprint image to be repaired. The orientation field prediction network is trained as follows: the orientation of the image blocks in the fingerprint image to be repaired is discretized to obtain multiple orientation categories, each category uniquely corresponding to a class number; a target cross-entropy loss function is determined based on the class number corresponding to each category; a target coherence loss function is determined for the orientation field prediction network, and the target cross-entropy loss function and the target coherence loss function are combined to obtain the target loss function of the orientation field prediction network; the orientation field prediction network is trained using the target loss function. The second processing module is used to determine the low-quality fingerprint region and the high-quality fingerprint region of the fingerprint image to be repaired based on the quality diagram. The third processing module is used to repair the region to be repaired in the initial orientation field based on the high-quality orientation field region corresponding to the high-quality fingerprint region in the initial orientation field, to obtain a target orientation field. Specifically, the quality schematic diagram and the high-quality orientation field region are input into an orientation field repair network to obtain a repaired orientation field output by the orientation field repair network; the region to be repaired in the initial orientation field is repaired based on the repaired orientation field to obtain the target orientation field; the region to be repaired is the orientation field region corresponding to the low-quality fingerprint region in the initial orientation field. The orientation field repair network is trained by: acquiring multiple complete fingerprint images and the complete fingerprint orientation fields corresponding to the complete fingerprint images; masking a portion of the complete fingerprint orientation field to obtain a fragmented fingerprint orientation field; and training the orientation field repair network using the fragmented fingerprint orientation field as input and the complete fingerprint orientation field as a control.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device containing the non-volatile storage medium to perform the fingerprint orientation field repair method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, performs the fingerprint orientation field repair method according to any one of claims 1 to 5.

Images