Monocular non-contact fingerprint perspective distortion correction method and device

Abstract

The present application proposes a monocular non-contact fingerprint perspective distortion correction method, which includes: acquiring a non-contact fingerprint image, preprocessing the non-contact fingerprint image, obtaining a preprocessed image and a mask; inputting the preprocessed image and mask into a gradient estimation network, and outputting a predicted surface gradient value; restoring the gradient value to its original size by bilinear interpolation to obtain a normal gradient value; reconstructing the depth map using the normal gradient value to obtain a three-dimensional finger model; and unfolding the non-contact fingerprint image according to the three-dimensional finger model for distortion correction. The present application solves the technical problem that the perspective distortion caused by projection imaging reduces the non-contact fingerprint recognition rate. By performing three-dimensional reconstruction and perspective distortion correction on a single non-contact fingerprint image, no complex acquisition equipment or multiple acquisition results are required, which greatly reduces the requirements for device hardware, improves the recognition performance of non-contact fingerprints, and is more suitable for actual production and life.

Description

Technical Field

[0001] The present application relates to the field of fingerprint recognition technology, and in particular to a method and device for correcting monocular non-contact fingerprint perspective distortion. Background Art

[0002] At present, fingerprint recognition technology is one of the most widely used biometric recognition technologies, and is widely used in ID cards, passports, entry and exit management, attendance systems, fingerprint unlocking, case detection, etc. However, with the continuous development of fingerprint recognition technology, a new fingerprint form, non-contact fingerprint, has emerged. Compared with traditional contact fingerprints, non-contact fingerprints do not require the finger to contact the collection device, so they can be collected without pressing, which is more hygienic and convenient. But at the same time, this new fingerprint form of non-contact fingerprints also brings new problems to fingerprint recognition algorithms: non-contact fingerprints are in principle a projection of a three-dimensional finger onto a two-dimensional image. Due to the curved surface structure of the finger surface, the closer the image is to the edge, the more serious the distortion is. The direct impact is that the ridge direction field and frequency change, and the position of the detail points are misaligned, which will affect the recognition performance. Therefore, if the collected non-contact fingerprints can be corrected for perspective distortion to reduce the degree of distortion, it will be very beneficial to improve the recognition performance of non-contact fingerprints. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first purpose of this application is to propose a monocular non-contact fingerprint perspective distortion correction method, which solves the perspective distortion problem caused by projection imaging in the existing method, causing changes in fingerprint features such as detail points, direction fields, and periodic diagrams, and reducing the recognition rate of non-contact fingerprints. The method realizes three-dimensional reconstruction and perspective distortion correction through a single non-contact fingerprint image, reduces the degree of distortion, greatly improves the recognition performance of non-contact fingerprints, does not require complex acquisition equipment or multiple acquisition results, greatly reduces the requirements for equipment hardware, and is more suitable for actual production and life.

[0005] The second objective of the present application is to provide a monocular non-contact fingerprint perspective distortion correction device.

[0006] A third object of the present application is to provide a non-transitory computer-readable storage medium.

[0007] To achieve the above-mentioned purpose, the first aspect of the present application proposes a monocular non-contact fingerprint perspective distortion correction method, including: acquiring a non-contact fingerprint image, preprocessing the non-contact fingerprint image, and obtaining a preprocessed image and a mask; inputting the preprocessed image and mask into a gradient estimation network, and outputting corresponding gradient values, wherein the gradient values ​​are predicted surface gradient values; restoring the gradient values ​​to their original sizes through bilinear interpolation to obtain normal gradient values; using the normal gradient values ​​to reconstruct a depth map to obtain a three-dimensional finger model; and expanding the non-contact fingerprint image according to the three-dimensional finger model to perform distortion correction.

[0008] Optionally, in one embodiment of the present application, preprocessing the contactless fingerprint image includes the following steps:

[0009] Perform threshold segmentation according to the pixel value of the contactless fingerprint image and output a mask;

[0010] In grayscale, adaptive histogram equalization is used to adjust the contrast of ridges and valleys of the non-contact image to unity;

[0011] In terms of size, the ridge period of the non-contact image is normalized, and the ridge period of the central area of ​​all non-contact fingerprints is adjusted to a uniform value;

[0012] In terms of posture, the posture of the contactless fingerprint that deviates from the vertical direction is corrected. First, the angle between the contactless fingerprint image and the vertical direction is estimated, and then the contactless fingerprint is straightened.

[0013] Optionally, in one embodiment of the present application, the preprocessed image and mask are input into a gradient estimation network, and the corresponding gradient value is output. The specific process is:

[0014] Normalize the preprocessed image and adjust the image brightness;

[0015] Perform feature extraction on the adjusted image to obtain the direction field and periodogram;

[0016] The direction field and the periodogram are processed respectively using masks to generate the final direction field and periodogram;

[0017] The mask, direction field, and periodogram are processed to generate gradient values.

[0018] Optionally, in one embodiment of the present application, training a gradient estimation network using a loss function comprises the following steps:

[0019] Using technology to simultaneously obtain a first set of finger three-dimensional models and a corresponding first set of non-contact fingerprint images;

[0020] Extracting a first set of direction fields, periodic maps, and masks from a first set of contactless fingerprint images using a fingerprint feature extraction technique, differentiating a first set of depth maps of a three-dimensional finger model to obtain a first set of gradient values, and sampling the first set of direction fields, periodic maps, masks, and the first set of gradient values ​​to generate a first set of training data;

[0021] Acquire a second set of non-contact fingerprint images by multi-angle acquisition, and use a shape recovery from contour method to calculate the second set of non-contact fingerprint images to obtain a second set of finger three-dimensional models;

[0022] Extracting a second set of direction fields, periodic maps, and masks from a second set of contactless fingerprint images using fingerprint feature extraction technology, differentiating a second set of depth maps of a three-dimensional finger model to obtain a second set of gradient values, and sampling the second set of direction fields, periodic maps, masks, and the second set of gradient values ​​to generate a second set of training data;

[0023] First, the first set of training data is input into the gradient estimation network for training, and then the second set of training data is input into the gradient estimation network for fine-tuning to complete the training.

[0024] Optionally, in one embodiment of the present application, the loss function includes direction field loss, period loss and gradient loss, and the final loss function is the sum of the three loss functions, where:

[0025] The direction field loss includes the cross entropy loss between the estimated value and the true value and the consistency constraint of the direction field;

[0026] The period loss is the cross entropy error between the estimated value and the true value and the smoothness constraint of the direction field;

[0027] Gradient loss is the weighted error between the estimated value and the true value;

[0028] Add weight constraints to the loss function to ensure that the relative error of the gradient is consistent everywhere.

[0029] Optionally, in one embodiment of the present application, a depth map is reconstructed by integrating from the center point to the edge area. The specific process is to select the point with the smallest gradient value as the integration origin, and adopt two integration paths along the x-axis and then along the y-axis and along the y-axis and then along the x-axis to obtain two results, and then average the two results to obtain the final result, which is expressed as:

[0030]

[0031]

[0032]

[0033] Among them, the integral path L 1First along the x-axis and then along the y-axis: origin (0,0)→(x,0)→(x,y), the integral path L 2 First along the y-axis and then along the x-axis: origin (0,0) → (0,y) → (x,y), g x (x,y) is the x-direction component of the gradient value, g y (x,y) is the y-direction component of the gradient value.

[0034] Optionally, in one embodiment of the present application, the non-contact fingerprint image is expanded, and the specific process is as follows:

[0035] The arc length is used as the new coordinate after expansion, and the non-contact fingerprint image is transformed according to the new coordinate to obtain the expanded non-contact fingerprint image.

[0036] The length of the arc in the horizontal direction is:

[0037]

[0038] The vertical arc length is:

[0039]

[0040] Among them, g x (x,y) is the x-direction component of the gradient value, g y (x,y) is the y-direction component of the gradient value,

[0041] The contactless fingerprint image is transformed according to the coordinates (u, v) to obtain the expanded contactless fingerprint image.

[0042] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a monocular non-contact fingerprint perspective distortion correction device, comprising:

[0043] An acquisition module is used to acquire a non-contact fingerprint image, pre-process the non-contact fingerprint image, and obtain a pre-processed image and a mask;

[0044] A calculation module, used for inputting the preprocessed image and mask into a gradient estimation network, and outputting a corresponding gradient value, wherein the gradient value is a predicted surface gradient value;

[0045] The recovery module is used to restore the gradient value to its original size through bilinear interpolation to obtain a normal gradient value;

[0046] A reconstruction module is used to reconstruct the depth map using normal gradient values ​​to obtain a three-dimensional finger model;

[0047] The correction module is used to unfold the non-contact fingerprint image according to the three-dimensional finger model to perform distortion correction.

[0048] In order to achieve the above-mentioned purpose, the third aspect of the present invention proposes a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor, a monocular non-contact fingerprint perspective distortion correction method and device can be executed.

[0049] The monocular non-contact fingerprint perspective distortion correction method, monocular non-contact fingerprint perspective distortion correction device and non-temporary computer-readable storage medium of the embodiments of the present application solve the technical problem that in the existing method, in the projection of a three-dimensional finger to a two-dimensional image, due to the curved surface structure of the finger surface, the image distortion becomes more serious as it is closer to the edge, resulting in changes in the ridge direction field and frequency, misalignment of the detail point position, and reduced recognition performance of the non-contact fingerprint. A deep learning method is used to learn the three-dimensional shape of the surface from the texture features of the non-contact fingerprint image, and then expand it from three dimensions to two dimensions, thereby correcting the perspective distortion of the non-contact fingerprint, greatly improving the recognition performance of the non-contact fingerprint, and does not require complex acquisition equipment or multiple acquisition results, greatly reducing the requirements for equipment hardware, making it more suitable for actual production and life.

[0050] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0052] Figure 1 A flow chart of a monocular non-contact fingerprint perspective distortion correction method provided in Example 1 of the present application;

[0053] Figure 2 Another flow chart of the monocular non-contact fingerprint perspective distortion correction method according to an embodiment of the present application;

[0054] Figure 3 A flowchart of a three-dimensional model for estimating a multi-angle non-contact fingerprint image according to a monocular non-contact fingerprint perspective distortion correction method according to an embodiment of the present application;

[0055] Figure 4 This is a gradient estimation network structure diagram of the monocular non-contact fingerprint perspective distortion correction method according to an embodiment of the present application;

[0056] Figure 5 The three-dimensional finger shape and unfolded image reconstructed by the monocular non-contact fingerprint perspective distortion correction method of the embodiment of the present application. DETAILED DESCRIPTION

[0057] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0058] The following describes a monocular non-contact fingerprint perspective distortion correction method and device according to an embodiment of the present application with reference to the accompanying drawings.

[0059] Figure 1 A flowchart of a monocular non-contact fingerprint perspective distortion correction method provided in Example 1 of the present application.

[0060] like Figure 1 As shown, the monocular non-contact fingerprint perspective distortion correction method includes the following steps:

[0061] Step 101, acquiring a non-contact fingerprint image, pre-processing the non-contact fingerprint image, and obtaining a pre-processed image and a mask;

[0062] Step 102, inputting the preprocessed image and mask into a gradient estimation network, and outputting a corresponding gradient value, wherein the gradient value is a predicted surface gradient value;

[0063] Step 103, restore the gradient value to its original value by bilinear interpolation to obtain a normal gradient value;

[0064] Step 104, reconstructing the depth map using normal gradient values ​​to obtain a three-dimensional finger model;

[0065] Step 105: Expand the contactless fingerprint image according to the three-dimensional finger model to perform distortion correction.

[0066] The monocular non-contact fingerprint perspective distortion correction method of the embodiment of the present application obtains a non-contact fingerprint image, pre-processes the non-contact fingerprint image, and obtains a pre-processed image and a mask; inputs the pre-processed image and mask into a gradient estimation network, and outputs the corresponding gradient value, wherein the gradient value is a predicted surface gradient value; restores the gradient value to its original size by bilinear interpolation to obtain a normal gradient value; uses the normal gradient value to reconstruct the depth map to obtain a three-dimensional finger model; and expands the non-contact fingerprint image according to the three-dimensional finger model to perform distortion correction. Thus, the perspective distortion problem caused by projection imaging in the existing method can be solved, which causes changes in fingerprint features such as detail points, direction fields, and periodic diagrams, and reduces the recognition rate of non-contact fingerprints. The three-dimensional reconstruction and perspective distortion correction are realized through a single non-contact fingerprint image, so that the degree of distortion is reduced, and the recognition performance of non-contact fingerprints is greatly improved. Complex acquisition equipment or multiple acquisition results are not required, which greatly reduces the requirements for equipment hardware and is more suitable for actual production and life.

[0067] Furthermore, in the embodiment of the present application, in order to learn the three-dimensional shape of the finger, all the non-contact fingerprint images need to be unified in grayscale, scale and posture to reduce uncertainty, and the non-contact fingerprint images are pre-processed, including the following steps:

[0068] Perform threshold segmentation according to the pixel value of the contactless fingerprint image and output a mask;

[0069] In grayscale, adaptive histogram equalization is used to adjust the contrast of ridges and valleys of the non-contact image to unity;

[0070] In terms of size, the ridge period of the non-contact image is normalized, and the ridge period of the central area of ​​all non-contact fingerprints is adjusted to a uniform value;

[0071] In terms of posture, the posture of the contactless fingerprint that deviates from the vertical direction is corrected. First, the angle between the contactless fingerprint image and the vertical direction is estimated, and then the contactless fingerprint is straightened.

[0072] In grayscale, the brightness of the non-contact fingerprint is normalized. In scale, the ridge period of the central area of ​​all non-contact fingerprints is adjusted to a uniform value to avoid the difficulty of depth prediction caused by scale uncertainty.

[0073] Further, in an embodiment of the present application, the preprocessed image and mask are input into a gradient estimation network, and the corresponding gradient value is output. The specific process is:

[0074] Normalize the preprocessed image and adjust the image brightness;

[0075] Perform feature extraction on the adjusted image to obtain the direction field and periodogram;

[0076] The direction field and the periodogram are processed respectively using masks to generate the final direction field and periodogram;

[0077] The mask, direction field, and periodogram are processed to generate gradient values.

[0078] Further, in an embodiment of the present application, using a loss function to train a gradient estimation network comprises the following steps:

[0079] Using technology to simultaneously obtain a first set of finger three-dimensional models and a corresponding first set of non-contact fingerprint images;

[0080] Extracting a first set of direction fields, periodic maps, and masks from a first set of contactless fingerprint images using a fingerprint feature extraction technique, differentiating a first set of depth maps of a three-dimensional finger model to obtain a first set of gradient values, and sampling the first set of direction fields, periodic maps, masks, and the first set of gradient values ​​to generate a first set of training data;

[0081] Acquire a second set of non-contact fingerprint images by multi-angle acquisition, and use a shape recovery from contour method to calculate the second set of non-contact fingerprint images to obtain a second set of finger three-dimensional models;

[0082] Extracting a second set of direction fields, periodic maps, and masks from a second set of contactless fingerprint images using fingerprint feature extraction technology, differentiating a second set of depth maps of a three-dimensional finger model to obtain a second set of gradient values, and sampling the second set of direction fields, periodic maps, masks, and the second set of gradient values ​​to generate a second set of training data;

[0083] First, the first set of training data is input into the gradient estimation network for training, and then the second set of training data is input into the gradient estimation network for fine-tuning to complete the training.

[0084] In order to learn the neural network, a large amount of training data is required. Two methods, real data processing and artificial synthesis, are used to obtain the non-contact fingerprint image and the corresponding finger 3D model. In terms of training data acquisition, any technology that can simultaneously obtain the finger 3D model and the corresponding non-contact fingerprint image can be used, such as structured light technology.

[0085] If the fingerprint 3D data has large noise, including ridge fluctuations, making the original data unstable, a smoothing method (such as moving least squares) can be used to obtain a stable depth map and gradient map. Specifically, for each point (x, y, z), use its K nearest neighbor points to minimize the weighted error and fit a quadratic surface f(x, y) to replace the original data point for smoothing.

[0086] f(x,y)=a 1 x 2 +a2 xy+a 3 y 2 +a 4 x+a 5 y+a 6

[0087]

[0088] Another method for obtaining a three-dimensional finger model and a corresponding non-contact fingerprint image is to calculate the three-dimensional finger model based on the non-contact fingerprint image collected at multiple angles according to the shape recovery method from the contour. Contour lines are extracted from the non-contact fingerprint images at three angles, and the main parameters of the finger shape model are calculated according to the contour lines projected at different angles, and finally the complete three-dimensional finger shape is obtained. The present invention adopts a longitudinally changing ellipse model, that is, it is assumed that each row is an ellipse. The main parameters of the ellipse are calculated according to the projection length of the ellipse at different angles based on the geometric relationship, thereby obtaining the three-dimensional shape of the finger.

[0089] A deep learning method is used to extract the direction field, detail points and surface gradients from the contactless fingerprint image. The network inputs the contactless fingerprint image and mask, and outputs the direction field, periodogram and gradient value. The output of the network is 1 / 8 of the input size, and then restored to the original size through bilinear interpolation. The output direction field is an estimated positive integer angle value of 0-179 degrees, the periodogram is the estimated ridge period of how many pixels, and the gradient is the estimated surface gradient value (including horizontal and vertical components).

[0090] Furthermore, in the embodiment of the present application, the loss function includes direction field loss, period loss and gradient loss, and the final loss function is the sum of the three loss functions, where:

[0091] The direction field loss includes the cross entropy loss between the estimated value and the true value and the consistency constraint of the direction field;

[0092] The period loss is the cross entropy error between the estimated value and the true value and the smoothness constraint of the direction field;

[0093] Gradient loss is the weighted error between the estimated value and the true value;

[0094] Add weight constraints to the loss function to ensure that the relative error of the gradient is consistent everywhere.

[0095] The loss function of the gradient estimation network includes direction field loss, periodogram loss and gradient loss. The three loss functions correspond to the three outputs of the network respectively, and the final loss function is the sum of the three loss functions. The present invention adds weight constraints to the loss function to balance the central area and the edge area of ​​the fingerprint, so that the larger gradient has a smaller weight, ensuring that the relative error of the gradient is consistent everywhere.

[0096] Further, in the embodiment of the present application, the depth map is reconstructed by integrating from the center point to the edge area. The specific process is to select the point with the smallest gradient value as the integration origin, and adopt two integration paths along the x-axis and then along the y-axis and along the y-axis and then along the x-axis to obtain two results, and then average the two results to obtain the final result, which is expressed as:

[0097]

[0098]

[0099]

[0100] Among them, the integral path L 1 First along the x-axis and then along the y-axis: origin (0,0)→(x,0)→(x,y), the integral path L 2 First along the y-axis and then along the x-axis: origin (0,0) → (0,y) → (x,y), g x (x,y) is the x-direction component of the gradient value, g y (x,y) is the y-direction component of the gradient value.

[0101] The depth map is reconstructed using the numerical integration method, which integrates from the center point to the edge area to reduce the numerical error.

[0102] Furthermore, in the embodiment of the present application, the contactless fingerprint image is expanded, and the specific process is as follows:

[0103] The arc length is used as the new coordinate after expansion, and the non-contact fingerprint image is transformed according to the new coordinate to obtain the expanded non-contact fingerprint image.

[0104] The length of the arc in the horizontal direction is:

[0105]

[0106] The vertical arc length is:

[0107]

[0108] Among them, g x (x,y) is the x-direction component of the gradient value, g y (x,y) is the y-direction component of the gradient value,

[0109] The contactless fingerprint image is transformed according to the coordinates (u, v) to obtain the expanded contactless fingerprint image.

[0110] The contactless fingerprint image is unfolded based on the obtained three-dimensional finger model to correct the distortion. The rolling of the finger on the plane is simulated, and the arc length is used as the new coordinate after unfolding, so as to ensure that the distance between two points on the surface can remain unchanged after unfolding.

[0111] Figure 2 This is another flow chart of the monocular non-contact fingerprint perspective distortion correction method according to an embodiment of the present application.

[0112] like Figure 2 As shown, the monocular non-contact fingerprint perspective distortion correction method includes: non-contact fingerprint image; image preprocessing; training data processing; surface gradient estimation; three-dimensional shape reconstruction; non-contact fingerprint expansion. The finger surface gradient is estimated from the non-contact fingerprint image by using a deep learning method, and the three-dimensional shape is reconstructed, and then the non-contact fingerprint is expanded according to the three-dimensional model to perform distortion correction.

[0113] Figure 3 This is a flow chart of a three-dimensional model for estimating multi-angle non-contact fingerprint images according to a monocular non-contact fingerprint perspective distortion correction method according to an embodiment of the present application.

[0114] like Figure 3 As shown, the multi-angle non-contact fingerprint image estimation 3D model method of the monocular non-contact fingerprint perspective distortion correction method includes: multi-angle non-contact fingerprint image; contour extraction; surface contour 3D reconstruction; surface 3D reconstruction. According to the shape recovery from contour method, the 3D model of the finger is calculated. Contour lines are extracted from the non-contact fingerprint images at three angles, and then the main parameters of the finger shape model are calculated according to the contour lines projected at different angles, and finally the complete 3D shape of the finger is obtained.

[0115] Figure 4 This is a gradient estimation network structure diagram of the monocular non-contact fingerprint perspective distortion correction method according to an embodiment of the present application.

[0116] like Figure 4 As shown in the figure, the gradient estimation network of the monocular non-contact fingerprint perspective distortion correction method inputs the non-contact fingerprint image and mask, and outputs the direction field, periodogram and gradient value. The output of the network is 1 / 8 of the input size, and then restored to the original size through bilinear interpolation. The output direction field is the estimated positive integer angle value of 0-179 degrees, the periodogram is the estimated ridge period of how many pixels, and the gradient is the estimated surface gradient value (including horizontal and vertical components).

[0117] Figure 5 The three-dimensional finger shape and unfolded image reconstructed by the monocular non-contact fingerprint perspective distortion correction method of the embodiment of the present application.

[0118] like Figure 5As shown in the figure, the monocular non-contact fingerprint perspective distortion correction method uses a deep learning method to estimate the finger surface gradient from the non-contact fingerprint image, reconstruct the three-dimensional shape, and then unfold the non-contact fingerprint based on the three-dimensional model to perform distortion correction. The depth map is reconstructed using a numerical integration method. In order to reconstruct more accurately, integration is performed from the center point to the edge area to reduce numerical errors. The rolling of the finger on the plane is simulated, and the arc length is used as the new coordinate after unfolding, so as to ensure that the distance between two points on the surface can remain unchanged after unfolding.

[0119] Embodiment 2 of the present application proposes a monocular non-contact fingerprint perspective distortion correction device, comprising:

[0120] An acquisition module is used to acquire a non-contact fingerprint image, pre-process the non-contact fingerprint image, and obtain a pre-processed image and a mask;

[0121] A calculation module, used for inputting the preprocessed image and mask into a gradient estimation network, and outputting a corresponding gradient value, wherein the gradient value is a predicted surface gradient value;

[0122] The recovery module is used to restore the gradient value to its original size through bilinear interpolation to obtain a normal gradient value;

[0123] A reconstruction module is used to reconstruct the depth map using normal gradient values ​​to obtain a three-dimensional finger model;

[0124] The correction module is used to unfold the non-contact fingerprint image according to the three-dimensional finger model to perform distortion correction.

[0125] The monocular non-contact fingerprint perspective distortion correction device of the embodiment of the present application solves the technical problem that in the existing method, in the projection of a three-dimensional finger to a two-dimensional image, due to the curved surface structure of the finger surface, the image distortion becomes more serious as it is closer to the edge, resulting in changes in the ridge direction field and frequency, misalignment of the position of detail points, and reduced recognition performance of the non-contact fingerprint. A deep learning method is used to learn the three-dimensional shape of the surface from the texture features of the non-contact fingerprint image, and then expand it from three dimensions to two dimensions, thereby correcting the perspective distortion of the non-contact fingerprint, greatly improving the recognition performance of the non-contact fingerprint, and does not require complex acquisition equipment or multiple acquisition results, greatly reducing the requirements for equipment hardware, making it more suitable for actual production and life.

[0126] In order to implement the above embodiments, the present invention also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the monocular non-contact fingerprint perspective distortion correction method and device of the above embodiments are implemented.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0128] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0129] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0130] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0131] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0132] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0133] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0134] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A monocular non-contact fingerprint perspective distortion correction method, characterized in that: The following steps are involved: Acquire a non-contact fingerprint image, and pre-process the non-contact fingerprint image to obtain a pre-processed image and a mask; Inputting the preprocessed image and mask into a gradient estimation network, and outputting corresponding gradient values, wherein the gradient values ​​are predicted surface gradient values; Restoring the gradient value to its original value by bilinear interpolation to obtain a normal gradient value; Reconstructing the depth map using the normal gradient value to obtain a three-dimensional finger model; The non-contact fingerprint image is expanded according to the three-dimensional finger model to perform distortion correction; wherein, The preprocessed image and mask are input into the gradient estimation network, and the corresponding gradient value is output. The specific process is as follows: Normalizing the preprocessed image and adjusting the image brightness; Perform feature extraction on the adjusted image to obtain the direction field and periodogram; Using the mask to process the direction field and the periodogram respectively, to generate a final direction field and periodogram; Processing the mask, the direction field and the periodogram to generate the gradient value; Training the gradient estimation network using the loss function includes the following steps: Using technology to simultaneously obtain a first set of finger three-dimensional models and a corresponding first set of non-contact fingerprint images; Extracting a first set of direction fields, periodograms, and masks from the first set of contactless fingerprint images using a fingerprint feature extraction technique, differentiating the first set of depth maps of the three-dimensional finger models to obtain a first set of gradient values, and sampling the first set of direction fields, periodograms, masks, and the first set of gradient values ​​to generate a first set of training data; Acquire a second set of non-contact fingerprint images from multiple angles, and calculate the second set of non-contact fingerprint images using a shape recovery from contour method to obtain a second set of finger three-dimensional models; Extracting a second set of direction fields, periodograms, and masks from the second set of contactless fingerprint images using fingerprint feature extraction technology, differentiating the depth map of the second set of finger three-dimensional models to obtain a second set of gradient values, and sampling the second set of direction fields, periodograms, masks, and the second set of gradient values ​​to generate a second set of training data; First, the first set of training data is input into the gradient estimation network for training, and then the second set of training data is input into the gradient estimation network for fine-tuning to complete the training.

2. The method according to claim 1, characterized in that Preprocessing the contactless fingerprint image includes the following steps: Performing threshold segmentation according to pixel values ​​of the contactless fingerprint image and outputting a mask; In grayscale, using adaptive histogram equalization to adjust the contrast of ridges and valleys of the non-contact fingerprint image to unity; In terms of size, the ridge period of the contactless fingerprint image is normalized, and the ridge period of the central area of ​​all contactless fingerprints is adjusted to a uniform value; In terms of posture, the posture of the contactless fingerprint deviating from the vertical direction is corrected by first estimating the angle between the contactless fingerprint image and the vertical direction, and then straightening the contactless fingerprint.

3. The method according to claim 1, characterized in that The loss function includes direction field loss, cycle loss and gradient loss. The final loss function is the sum of the three loss functions, where: The direction field loss includes a cross entropy loss between the estimated value and the true value and a consistency constraint of the direction field; The period loss is the cross entropy error between the estimated value and the true value and the smoothness constraint of the direction field; The gradient loss is the weighted error between the estimated value and the true value; A weight constraint is added to the loss function to ensure that the relative error of the gradient is consistent everywhere.

4. The method according to claim 1, characterized in that The depth map is reconstructed by integrating from the center point to the edge area. The specific process is to select the point with the smallest gradient value as the integration origin, and obtain two results by using two integration paths: first along the x-axis and then along the y-axis and first along the y-axis and then along the x-axis. The two results are then averaged to obtain the final result, which is expressed as: The integral path L1 is first along the x-axis and then along the y-axis: origin (0,0)→(x,0)→(x,y), and the integral path L2 is first along the y-axis and then along the x-axis: origin (0,0)→(0,y)→(x,y), g x (x,y) is the x-direction component of the gradient value, g y (x,y) is the y-direction component of the gradient value.

5. The method according to claim 1, characterized in that The contactless fingerprint image is expanded, and the specific process is as follows: The arc length is used as the new coordinate after expansion, and the non-contact fingerprint image is transformed according to the new coordinate to obtain the expanded non-contact fingerprint image. The length of the arc in the horizontal direction is: The vertical arc length is: Among them, g x (x,y) is the x-direction component of the gradient value, g y (x,y) is the y-direction component of the gradient value, The non-contact fingerprint image is transformed according to the coordinates (u, v) to obtain an expanded non-contact fingerprint image.

6. A monocular non-contact fingerprint perspective distortion correction device, characterized in that: include: An acquisition module, used for acquiring a non-contact fingerprint image, preprocessing the non-contact fingerprint image, and obtaining a preprocessed image and a mask; A calculation module, used for inputting the preprocessed image and mask into a gradient estimation network, and outputting a corresponding gradient value, wherein the gradient value is a predicted surface gradient value; A recovery module, used for restoring the gradient value to its original size through bilinear interpolation to obtain a normal gradient value; A reconstruction module, used to reconstruct the depth map using the normal gradient value to obtain a three-dimensional finger model; A correction module, used for expanding the non-contact fingerprint image according to the three-dimensional finger model to perform distortion correction; The calculation module is also used for: Normalizing the preprocessed image and adjusting the image brightness; Perform feature extraction on the adjusted image to obtain the direction field and periodogram; Using the mask to process the direction field and the periodogram respectively, to generate a final direction field and periodogram; Processing the mask, the direction field and the periodogram to generate the gradient value; Training the gradient estimation network using the loss function includes the following steps: Using technology to simultaneously obtain a first set of finger three-dimensional models and a corresponding first set of non-contact fingerprint images; Extracting a first set of direction fields, periodograms, and masks from the first set of contactless fingerprint images using a fingerprint feature extraction technique, differentiating the first set of depth maps of the three-dimensional finger models to obtain a first set of gradient values, and sampling the first set of direction fields, periodograms, masks, and the first set of gradient values ​​to generate a first set of training data; Acquire a second set of non-contact fingerprint images from multiple angles, and calculate the second set of non-contact fingerprint images using a shape recovery from contour method to obtain a second set of finger three-dimensional models; Extracting a second set of direction fields, periodograms, and masks from the second set of contactless fingerprint images using fingerprint feature extraction technology, differentiating the depth map of the second set of finger three-dimensional models to obtain a second set of gradient values, and sampling the second set of direction fields, periodograms, masks, and the second set of gradient values ​​to generate a second set of training data; First, the first set of training data is input into the gradient estimation network for training, and then the second set of training data is input into the gradient estimation network for fine-tuning to complete the training.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images