Three-dimensional fingerprint microstructure reconstruction method and system based on illumination self-calibration and sparse integration

Abstract

The invention relates to the technical field of optical three-dimensional reconstruction, in particular to a three-dimensional fingerprint microstructure reconstruction method and system based on illumination self-calibration and sparse integration, and the method comprises the following steps: constructing a multi-light-source imaging system, and arranging a standard spherical reflector in an imaging field of view to perform illumination direction self-calibration; performing multi-angle illumination acquisition on the surface of the finger to obtain a finger image sequence under multi-frame illumination variation; a reconstruction area of the finger image sequence is limited; carrying out vectorization solution on the pixel brightness change of the reconstruction region under the multi-illumination condition, and obtaining a global continuous normal gradient field based on the surface normal vector and reflectivity of each pixel; the normal gradient field is converted into a depth constraint equation, sparse matrix optimization and least square integral solving are adopted, and global consistent reconstruction of the depth map is achieved. According to the method, high-fidelity three-dimensional reconstruction of the finger surface microstructure is realized through an integrated technical framework of physical calibration, photometric solution and sparse integration.

Description

Technical Field

[0001] This application relates to the field of optical three-dimensional reconstruction technology, and in particular to a method and system for reconstructing three-dimensional fingerprint microstructures using illumination self-calibration and sparse integration. Background Technology

[0002] In visual recognition scenarios such as biometrics, security verification, and human-computer interaction, high-precision modeling of finger surface texture is a crucial step in achieving reliable identity recognition. However, due to the complex optical properties of the skin surface (such as diffuse reflection, specular reflection, and subsurface scattering) and its microscale undulations, imaging under various lighting conditions often results in uneven illumination, excessive reflection, local shadows, and image noise. These factors make it difficult for traditional two-dimensional fingerprint images to accurately describe the true geometric undulations of ridges, thus limiting their application in high-security identification and microstructure reconstruction. Meanwhile, the two-dimensional information from single-view imaging is inherently an ill-posed problem, meaning that there are multiple possible solutions to recovering the complete three-dimensional morphology from a finite brightness distribution. How to introduce reasonable physical constraints under these ill-posed conditions to achieve a solvable and high-precision reconstruction of the micromorphology of the real finger surface has become a key scientific and engineering challenge in the field of visual measurement.

[0003] To address this issue, existing 3D reconstruction technologies mainly include: stereo matching based on geometric parallax, structured light method based on optical coding, and photometric stereo method based on photometric constraints. Each of these three methods has its own characteristics in terms of measurement principles and applicable scope, but all have certain limitations in modeling flexible surfaces such as skin.

[0004] First, structured light-based reconstruction methods are extremely sensitive to lighting conditions and skin reflectivity, easily leading to local saturation, shadow occlusion, and phase calculation errors, resulting in a loss of ridge detail. Second, stereo vision-based geometric methods rely on texture feature matching, which can easily cause matching ambiguities in repetitive texture areas on the skin surface, resulting in discontinuities in the depth map or increased noise. Third, while traditional photometric stereo methods can resolve surface normals at the pixel level, the light source direction is difficult to accurately calibrate, the reflection model is too idealized, and the calculated normal field has systematic biases; at the same time, the depth integration process is prone to accumulating errors, leading to surface unevenness or ripple artifacts. Summary of the Invention

[0005] This application provides a method and system for three-dimensional fingerprint microstructure reconstruction based on illumination self-calibration and sparse integration. It can effectively suppress uneven illumination, specular reflection and local noise interference, improve the three-dimensional reconstruction accuracy and surface continuity of the fingerprint ridge region, thereby achieving high-fidelity digital reconstruction of skin microstructure and providing a new technical approach for high-precision biometrics and microscopic surface measurement.

[0006] To address the aforementioned technical problems, in a first aspect, embodiments of this application provide a method for reconstructing three-dimensional fingerprint microstructures using illumination self-calibration and sparse integration, comprising the following steps: First, a multi-source imaging system is constructed, and standard spherical reflectors are arranged in the imaging field of view for illumination direction self-calibration; then, under the calibrated illumination conditions, multi-angle illumination acquisition is performed on the finger surface to obtain a sequence of finger images under multiple illumination variations; next, a mask image is used to define the reconstruction region of the finger image sequence; then, based on the photometric stereo imaging model, the pixel brightness variation of the reconstruction region under multiple illumination conditions is vectorized and solved, and a globally continuous normal gradient field is obtained based on the surface normal vector and reflectivity of each pixel; finally, the normal gradient field is transformed into a depth constraint equation, and sparse matrix optimization and least squares integration are used to solve it, achieving globally consistent reconstruction of the depth map and obtaining the three-dimensional reconstruction result.

[0007] In some exemplary embodiments, constructing a multi-source imaging system includes: selecting multiple sets of LED point light sources with constant brightness, arranging them around the finger sample to be tested at a fixed angle, and ensuring that the camera optical axis is approximately perpendicular to the sample surface.

[0008] In some exemplary embodiments, the reflectance at any point on the surface of the finger sample to be tested is assumed to be... ,in Indicates the different light source numbers, Here are the pixel coordinates; according to the photometric stereo imaging model, the reflectance at any point on the surface of the finger sample to be tested is expressed by Lambert's law of reflection as:

[0009] in, For pixel coordinates The reflectivity of the point, The surface unit normal vector, For the first i Each light source direction vector.

[0010] In some exemplary embodiments, arranging a standard spherical reflector in the imaging field of view for illumination direction self-calibration includes: setting a standard spherical reflector in the imaging field of view, calculating the spatial incident direction of each light source by analyzing the positional distribution of specular reflection points on the standard sphere, and establishing an illumination direction matrix; assuming the radius of the sphere is... r The center coordinates are The location of the highlight detected in the image is Then the surface normal vector corresponding to that point is:

[0011] According to the law of reflection, the direction of light illumination... relative to the camera's incident direction The relationship is:

[0012] By detecting the highlight region corresponding to each light source and calculating its average coordinates, the spatial direction vectors of all light sources are calculated, thus forming a complete illumination matrix:

[0013] in, l nx , l ny , l nz These are the components of the light source direction vector along the x, y, and z axes, respectively.

[0014] In some exemplary embodiments, the reconstructed region of the finger image sequence is defined using a mask image, including: filtering the reconstructed region through threshold segmentation and binary masking operations; assuming the input image is... Its grayscale range is Define the maximum grayscale value as Then the mask function It can be represented as:

[0015] Among them, the threshold coefficient Between 0.05 and 0.1.

[0016] In some exemplary embodiments, the surface normal vector and reflectivity of each pixel are represented as follows:

[0017] in, L is the illumination matrix, and I is the pixel intensity.

[0018] In some exemplary embodiments, the normal gradient field is transformed into a depth constraint equation, which is solved using sparse matrix optimization and least squares integration, including: employing an integral optimization model based on sparse matrices; assuming that there are a total of m Given _{x} effective pixels, each forming two gradient constraints in the x and y directions, the overall linear equation is:

[0019] in, for The sparse difference matrix, The gradient observation vector contains information; The depth vector is obtained by least squares optimization: .

[0020] In some exemplary embodiments, the three-dimensional reconstruction results include a normal map, a reflectance map, a depth map, and a height map.

[0021] Secondly, this application also provides a three-dimensional fingerprint microstructure reconstruction system based on illumination self-calibration and sparse integration. This system is used to implement the three-dimensional fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integration described in the above embodiments. The system includes: an illumination self-calibration module, an acquisition module, an image preprocessing module, a normal vector solving module, and a depth reconstruction module connected in sequence. The illumination self-calibration module is used to construct a multi-source imaging system and arrange standard spherical reflectors in the imaging field of view for illumination direction self-calibration. The acquisition module is used to perform multi-source imaging on the finger surface under calibrated illumination conditions. The system acquires finger images under varying illumination conditions across multiple frames. An image preprocessing module defines the reconstructed region of the finger image sequence using a mask. A normal vector solving module vectorizes the pixel brightness variations in the reconstructed region under multiple illumination conditions based on the photometric stereo imaging model, and obtains a globally continuous normal gradient field based on the surface normal vector and reflectivity of each pixel. A depth reconstruction module transforms the normal gradient field into a depth constraint equation, employing sparse matrix optimization and least squares integration to achieve globally consistent depth map reconstruction, resulting in a 3D reconstruction.

[0022] In some exemplary embodiments, the image preprocessing module includes a mask extraction unit, a threshold segmentation unit, and a noise removal unit; the mask extraction unit is used to extract the reconstructed region using a mask image; the threshold segmentation unit is used to segment the reconstructed region by threshold segmentation to reduce noise interference caused by uneven illumination; the noise removal unit is used to effectively remove low-brightness, shadow, and high-reflection areas to reduce the interference of abnormal pixels on the solution results and make the reconstruction process more robust. The technical solution provided in this application has at least the following advantages: This application provides a method and system for reconstructing three-dimensional fingerprint microstructures using illumination self-calibration and sparse integration, comprising the following steps: First, a multi-source imaging system is constructed, and a standard spherical reflector is arranged in the imaging field of view for illumination direction self-calibration; then, under the calibrated illumination conditions, multi-angle illumination acquisition is performed on the finger surface to obtain a sequence of finger images under multiple illumination variations; next, a mask image is used to define the reconstruction region of the finger image sequence; then, based on a photometric stereo imaging model, the pixel brightness variation of the reconstruction region under multiple illumination conditions is vectorized and solved, and a globally continuous normal gradient field is obtained based on the surface normal vector and reflectivity of each pixel; finally, the normal gradient field is transformed into a depth constraint equation, and sparse matrix optimization and least squares integration are used to solve it, achieving globally consistent reconstruction of the depth map and obtaining the three-dimensional reconstruction result. This application achieves high-fidelity three-dimensional reconstruction of finger surface microstructures through an integrated technical framework of "physical calibration - photometric solution - sparse integration". This scheme not only theoretically optimizes the stability of the photometric stereo model, but also improves reconstruction accuracy and computational efficiency in engineering, and has significant practical and promotional value. Attached Figure Description

[0023] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0024] Figure 1 This is a flowchart illustrating a three-dimensional fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integration, provided as an embodiment of this application.

[0025] Figure 2 This is a schematic diagram of a three-dimensional fingerprint microstructure reconstruction system architecture based on illumination self-calibration and sparse integration, provided as an embodiment of this application.

[0026] Figure 3 This is a schematic diagram of a hardware system for high-fidelity 3D finger and other local reconstruction provided in an embodiment of this application.

[0027] Figure 4 This is a schematic diagram of a finger collection experiment provided in one embodiment of this application.

[0028] Figure 5 The high-fidelity 3D finger reconstruction effect provided in one embodiment of this application.

[0029] Figure 6 The high-fidelity 3D hand reconstruction effect provided in one embodiment of this application. Detailed Implementation

[0030] As can be seen from the background technology, existing 3D reconstruction technologies all have certain limitations in modeling flexible skin-like surfaces.

[0031] High-precision 3D reconstruction of fingerprints is one of the core issues in the fields of biometrics, security and anti-counterfeiting, human-computer interaction, and micro-surface measurement. Fingerprint surface microstructures (such as ridges, sweat pores, micro-wrinkles, and skin reflectivity) are not only important biometric features for individual identification but also have significant research and application value in areas such as dermatological pathology, rehabilitation medicine assessment, and bionic tactile modeling. However, existing 2D fingerprint imaging systems can only record grayscale or color information, lacking the ability to accurately reproduce the microscale geometric morphology of the skin surface and failing to reflect the three-dimensional depth characteristics and subtle undulations of fingerprint ridges, resulting in significant deficiencies in authenticity, anti-counterfeiting capabilities, and spatial information representation. To overcome the limitations of 2D acquisition, 3D fingerprint reconstruction technology has emerged, aiming to obtain fingerprint surface morphology with high geometric accuracy and high signal-to-noise ratio at the micrometer-level spatial scale.

[0032] In actual imaging, the surface of finger skin exhibits complex light reflection characteristics, containing both diffuse reflection and strong specular reflection components. Furthermore, due to the translucency of skin tissue, it also experiences a certain degree of subsurface scattering. These factors significantly limit the application of traditional 3D reconstruction methods such as structured light, laser scanning, or stereo matching when processing fingerprint surfaces. Specifically: 1) Multi-view geometry-based 3D reconstruction methods rely on stereo matching and depth recovery from images captured from multiple angles, with the core being the extraction of stable texture feature points. However, the texture of the fingerprint region is highly repetitive and periodic, easily causing matching ambiguities. Coupled with inconsistent illumination caused by microscopic skin undulations, this leads to a significant decrease in matching accuracy, making it impossible to obtain a continuous and smooth depth surface; 2) Structured light scanning calculates 3D topography by projecting coded gratings or phase-shifted fringes and acquiring deformed images by a camera. While this method demonstrates high measurement accuracy on rigid surfaces, for biological tissues with reflective or microtransparent properties (such as skin), projected fringes may exhibit overexposure, diffuse scattering, and local occlusion, leading to fringe decoding failure or unstable depth calculation. Furthermore, structured light systems typically require rigorous optical geometric calibration and are sensitive to ambient light interference, making it difficult to achieve rapid and low-cost finger surface reconstruction. 3) Traditional photometric stereo is a technique that recovers the surface normal vector of an object by analyzing grayscale changes under multiple light source illumination. It does not rely on geometric parallax but achieves pixel-level surface morphology recovery by solving the illumination equation, theoretically making it very suitable for measuring microstructured surfaces. However, traditional photometric stereo relies on precise light source orientation calibration and the ideal Lambertian reflection assumption. In reality, due to factors such as non-ideal light sources, non-uniform skin reflection, and image noise interference, normal vector estimation often has significant deviations. Further depth reconstruction is also susceptible to error accumulation, leading to surface fluctuations or distortion.

[0033] The following examples illustrate the limitations of existing 3D reconstruction technologies by listing several related technical solutions.

[0034] Related technology 1 discloses a method and device for three-dimensional finger vein recognition based on binocular vision. The device mainly includes an infrared light source tactile switch, two cameras (left and right), an infrared filter, a power supply, an ARM interface board, and a DSP processing board. The infrared light source, the finger, and the two cameras are on three horizontal planes. The two cameras are located on either side of the finger's projection line, and the line connecting the two cameras is perpendicular to the finger's projection line. If the tactile switch is detected to be pressed, the left and right cameras are operated to acquire binocular images of the finger. Then, based on the principle of binocular vision and the SIFT operator, the three-dimensional features of the finger veins are extracted and matched to achieve the function of finger registration or recognition.

[0035] Related technology 2 discloses a non-contact fingerprint acquisition device and method. The non-contact fingerprint acquisition device may include: a housing comprising a finger scanning area for at least one finger; at least two image capturing devices located within the housing and positioned at a predetermined baseline distance, each image capturing device having an optical axis at a predetermined angle to the vertical direction; and an illumination unit located within the housing for illuminating the at least one finger. The at least two image capturing devices are operable to acquire multiple partial fingerprint images of the at least one finger, and the multiple partial fingerprint images correspond to different parts of the at least one finger. This allows for the acquisition of fingerprint images with higher image quality and larger area.

[0036] Related technology 3 discloses an optical fingerprint recognition device, its fingerprint recognition method, and a display device, relating to the field of fingerprint recognition technology. This technology can reduce the thickness and cost of the optical fingerprint recognition device. The optical fingerprint recognition device includes: a cover plate; a pressure-emitting layer disposed below the cover plate, wherein the state of light emitted by the pressure-emitting layer changes when pressure is applied, emitting light of a certain wavelength; an optical sensing layer for detecting light emitted by the pressure-emitting layer and reflected by a finger; and a light-shielding pattern layer disposed between the optical sensing layer and the pressure-emitting layer for blocking light emitted by the pressure-emitting layer from directly hitting the optical sensing layer. Related technology 4 discloses a rapid three-dimensional fingerprint acquisition method and system, belonging to the field of image acquisition technology. The hardware system includes an image projection end, an image acquisition end, and a computer for image processing and three-dimensional calculation. The image projection end includes a projector, and the image acquisition end includes a left camera and a right camera, located to the left and right of the projector, respectively. The method involves the computer designing and encoding sinusoidal phase-shifted fringes and Gray code patterns and sending them to the projector. The projector projects the encoded patterns onto the finger surface. The left and right cameras capture images of the finger from their respective angles, acquiring the encoded patterns deformed by depth modulation of the finger surface and transmitting them to the computer. The computer then completes the three-dimensional reconstruction of the fingerprint through an algorithm module. This application can accurately reflect the true three-dimensional shape of the fingerprint, and the fingerprint texture is continuous and clear, with distinct ridges and valleys.

[0037] Related technology 5 discloses a method, system, and apparatus for leather defect detection based on photometric stereo vision. The system includes a first acquisition module, a first processing module, a second processing module, a third processing module, a fourth processing module, and a fifth processing module. The method involves acquiring multiple first images captured under different light sources; synthesizing these first images into a depth image; sequentially performing curvature filtering, grayscale stretching, and binarization to obtain a binary image of the leather to be tested; processing the binary image using SVM; and outputting the leather defect detection result. The apparatus includes a memory and a processor for executing the above detection method. Using this scheme, the enhancement effect of leather defect areas can be rapidly improved, increasing the accuracy and processing efficiency of leather defect detection. This scheme, as a method, system, and apparatus for leather defect detection based on photometric stereo vision, can be widely applied in the field of leather inspection.

[0038] In summary, existing fingerprint 3D reconstruction methods have shortcomings in terms of accuracy, robustness, and illumination adaptability. In particular, in the high-fidelity reconstruction of flexible objects such as skin, accurately characterizing the illumination distribution, estimating the normal field, and achieving stable integration have become key challenges.

[0039] To address the aforementioned technical challenges, this application proposes a method and system for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integral. Compared to existing methods for three-dimensional fingerprint reconstruction based on structured light, binocular vision, and traditional photometric stereo techniques, this application does not employ geometric parallax or coded projection, but instead constructs: 1. Based on a multi-source illumination direction self-calibration mechanism using a standard spherical reflector, it avoids manual angle measurement or preset parameters and does not rely on structured light encoding. Existing 3D fingerprint patents do not use this physical self-calibration strategy of "chrome ball illumination inversion".

[0040] 2. Modeling and compensating for phenomena such as non-Lambertian reflection characteristics of skin and subsurface scattering, through... The vectorized joint solution method recovers both normal and reflectivity at the pixel level. Existing photometric stereoscopic patents mainly target materials such as leather and steel, and do not address the complex reflection and texture periodicity issues of skin tissue.

[0041] 3. A sparse matrix global integral model is adopted to uniformly model the normal gradient field as follows: This method uses a sparse system of equations to achieve a robust deep solution, rather than relying on local integration or Poisson approximation. Existing fingerprint 3D reconstruction methods lack such a global sparse matrix solution mechanism.

[0042] 4. The entire system does not rely on structured light projection, laser scanning, Gray code, or phase-shifting fringes, nor on binocular geometric matching; it can achieve micron-level 3D reconstruction of fingerprint ridges solely using multi-source images. This is a technical approach not found in existing patent systems.

[0043] This application provides a 3D fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integral. Starting from a physical imaging model, it constructs a complete process including illumination calibration, normal estimation, and depth calculation. Through multi-channel image sequences and high-precision mathematical modeling, it achieves high-precision 3D reconstruction of skin microstructures under complex illumination and reflection conditions. The key problems to be solved in this application are mainly reflected in the following four aspects: (1) High-precision calibration and self-calibration of illumination direction.

[0044] Traditional photometric stereo methods rely on manual measurement of light source angles or empirical geometric assumptions, which can easily lead to inaccurate illumination vectors and thus systematic biases in normal estimation. This application utilizes a light source calibration module based on the imaging principle of a standard spherical reflector to infer the incident light direction from the position of the reflected bright spot on the spherical surface, achieving accurate self-calibration of multiple light source directions. This step effectively solves the problem of illumination vector deviation caused by assembly errors and environmental reflections in multi-light source systems, providing accurate prior information for subsequent normal recovery.

[0045] (2) The problem of joint estimation of pixel-level normal and reflectivity under complex surface reflection.

[0046] The light response of skin surfaces is affected by angle, roughness, and local humidity, which traditional algorithms, based on the Lambertian model, cannot accurately describe. This application establishes a linear relationship between pixel grayscale vectors and the illumination matrix through a normal vector estimation module, uses least squares to obtain the surface gradient vector, and then normalizes it to obtain the unit normal vector, while simultaneously estimating reflectivity. This method is physically equivalent to solving a system of local illumination and reflection equations, effectively separating illumination from material properties and improving the stability and accuracy of the normal field.

[0047] (3) The problem of stable integral from normal to depth and topography restoration.

[0048] Traditional integral algorithms are susceptible to boundary noise amplification and path dependence, leading to discontinuities or distorted surfaces in the reconstructed surface. This application constructs a global integral equation in sparse matrix form through a depth map estimation module, transforming pixel gradient constraints into a linear algebra system and using a sparse solver to obtain the global optimal solution. This process is equivalent to performing a Poisson integral on the normal field, ensuring global consistency while eliminating local noise interference, thus achieving high-precision depth reconstruction.

[0049] (4) Feature preservation and noise suppression issues in high-resolution fingerprint regions.

[0050] At the fingerprint ridge scale, minute noise can lead to the accumulation of normal errors and cause surface distortion. This application introduces a mask image to effectively define the reconstructed region and ensures data validity through pixel intensity thresholding. Furthermore, multi-channel accumulation and normalization operations can further improve the signal-to-noise ratio, enabling the output depth map to maintain realistic texture while avoiding excessive smoothing.

[0051] In summary, the 3D fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integral proposed in this application achieves an integrated processing flow at the system level, encompassing "automatic illumination direction calibration - multi-illumination image acquisition - normal vector and reflectivity estimation - depth reconstruction." It not only solves the core problems of traditional photometric stereo reconstruction, such as illumination uncertainty, unstable normal vectors, and easy integral drift, but also takes into account the modeling requirements of the complex reflective properties of the skin surface in response to illumination. Using this method, high spatial resolution and high geometric fidelity 3D fingerprint morphology data can be obtained solely from image sequences under multi-light source conditions without the need for complex coded projections or multi-view geometric matching.

[0052] The ultimate goal of this application is to achieve true three-dimensional morphological reconstruction of fingerprint surfaces at the micrometer scale, so that the reconstruction results can reflect both the geometric undulations of the ridges and the differences in skin reflectivity, providing reliable three-dimensional topographic information for high-security biometric systems. Simultaneously, this technical solution can also be extended to multiple related fields such as palm skin detection, subcutaneous blood flow distribution estimation, and surface calibration of bionic tactile sensors, providing a high-precision visual measurement foundation for biomedical engineering and intelligent sensing.

[0053] The embodiments of this application will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0054] See Figure 1This application provides a method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration, including the following steps: Step S1: Construct a multi-source imaging system and arrange standard spherical reflectors in the imaging field of view for self-calibration of illumination direction.

[0055] Step S2: Under the calibrated lighting conditions, perform multi-angle illumination acquisition on the finger surface to obtain a sequence of finger images under multiple lighting changes.

[0056] Step S3: Use a mask to define the reconstruction area of ​​the finger image sequence.

[0057] Step S4: Based on the photometric stereo imaging model, the pixel brightness changes of the reconstructed region under multiple illumination conditions are vectorized and solved, and a globally continuous normal gradient field is obtained based on the surface normal vector and reflectivity of each pixel.

[0058] Step S5: Transform the normal gradient field into a depth constraint equation, and solve it using sparse matrix optimization and least squares integration to achieve globally consistent reconstruction of the depth map and obtain the 3D reconstruction result.

[0059] The overall process of the 3D fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integral proposed in this application includes: multi-source image acquisition, illumination direction self-calibration, image mask extraction and preprocessing, normal vector solution, depth integral reconstruction, and 3D result output. Based on the principle of photometric stereo imaging, the entire process achieves high-precision inversion from intensity distribution to surface geometry through joint constraints of illumination and reflection models. Its technical route is as follows: Figure 2 As shown, the process includes: multi-source image acquisition → illumination direction self-calibration → image mask extraction and preprocessing → photometric equation normal solution → depth integral reconstruction → 3D result output. Specifically, the basic implementation content of the technical solution of this application includes: First, a multi-source imaging system is constructed, and a standard spherical reflector (chrome ball) is placed in the shooting scene for self-calibration of the illumination direction. By analyzing the positional distribution of the specular reflection points on the standard sphere, the spatial incident direction of each light source is calculated, and an accurate illumination direction matrix is ​​established, providing a physical basis for subsequent photometric calculations.

[0060] Secondly, under the calibrated lighting conditions, multi-angle illumination data was collected from the finger surface to obtain a sequence of color images under varying lighting conditions. A mask was then used to segment the target region, removing background and low-brightness pixels to ensure the effectiveness and stability of the subsequent reconstruction data.

[0061] Then, based on the imaging model of photometric stereo method, the pixel brightness changes under multiple illumination conditions are vectorized and solved, and the surface normal vector and reflectivity information of each pixel are jointly recovered, thereby obtaining a continuous and physically consistent normal field distribution.

[0062] Finally, the normal gradient field is transformed into a depth constraint equation, which is solved using sparse matrix optimization and least squares integration to achieve globally consistent reconstruction of the depth map. This method can accurately restore finger ridges and microstructural details, significantly improving surface continuity and signal-to-noise ratio.

[0063] In some embodiments, the construction of a multi-source imaging system in step S1 includes: selecting multiple sets of LED point light sources with constant brightness, arranging them around the finger sample to be tested at a fixed angle, and ensuring that the camera optical axis is approximately perpendicular to the sample surface.

[0064] In some embodiments, after constructing a multi-source imaging system, the reflectance at any point on the surface of the finger sample to be tested is set to be... ,in Indicates the different light source numbers, Here are the pixel coordinates; according to the photometric stereo imaging model, the reflectance at any point on the surface of the finger sample to be tested is expressed by Lambert's law of reflection as:

[0065] in, For pixel coordinates The reflectivity of the point, The surface unit normal vector, For the first i Each light source has a direction vector. By obtaining the intensity distribution under multiple light sources, a set of illumination constraint equations can be established, thereby inversely calculating the surface normal vector and reflectivity.

[0066] The self-calibration stage of illumination direction is the key to achieving high-precision reconstruction in this application. In order to eliminate light source installation errors and geometric deviations, this application sets a standard spherical reflector (chrome ball) in the imaging field of view and uses the position of the bright spot reflected by the specular surface to invert the illumination direction.

[0067] In some embodiments, step S1, arranging a standard spherical reflector in the imaging field of view for illumination direction self-calibration, includes: setting a standard spherical reflector in the imaging field of view, calculating the spatial incident direction of each light source by analyzing the positional distribution of specular reflection points on the standard sphere, and establishing an illumination direction matrix; assuming the radius of the sphere is... r The center coordinates are The location of the highlight detected in the image is Then the surface normal vector corresponding to that point is:

[0068] According to the law of reflection, the direction of light illumination... relative to the camera's incident direction The relationship is:

[0069] By detecting the highlight region corresponding to each light source and calculating its average coordinates, the spatial direction vectors of all light sources are calculated, thus forming a complete illumination matrix:

[0070] in, l nx , l ny , l nz These represent the components of the light source direction vector along the x, y, and z axes, respectively. This illumination matrix serves as the basis for subsequent normal vector calculation.

[0071] The image mask extraction and preprocessing stage is then performed. Since the finger surface contains shadows, background, and localized reflective areas, this application uses threshold segmentation and binary masking to filter the effective regions.

[0072] In some embodiments, the reconstructed region of the finger image sequence is defined using a mask image, including: filtering the reconstructed region through threshold segmentation and binary masking operations; assuming the input image is... Its grayscale range is Define the maximum grayscale value as Then the mask function It can be represented as:

[0073] Among them, the threshold coefficient The value is between 0.05 and 0.1. After masking, only the target area is retained for subsequent calculations to reduce noise interference caused by uneven illumination.

[0074] The normal vector calculation stage is the core computational step. This involves solving the illumination direction matrix. Given the known information, the pixel intensity under multiple lighting conditions can be determined. Represented in matrix form:

[0075] in The following can be obtained using the least squares method:

[0076] In some embodiments, the surface normal vector and reflectivity of each pixel are represented as follows:

[0077] Where L is the illumination matrix and I is the pixel intensity.

[0078] The above calculations are performed independently at the pixel level, resulting in a globally continuous normal distribution. Because this application introduces illumination normalization and mask constraints before the calculations, it effectively suppresses the impact of local illumination anomalies on the results.

[0079] After obtaining the complete normal field, the normal information needs to be integrated into a depth distribution to form a three-dimensional topography. According to differential geometry, the surface gradient and the normal vector satisfy the following:

[0080] To obtain depth This application adopts an integral optimization model based on sparse matrix construction.

[0081] In some embodiments, the normal gradient field is transformed into a depth constraint equation, which is solved using sparse matrix optimization and least squares integration, including: employing an integral optimization model based on sparse matrices; assuming that there are a total of m Given _{x} effective pixels, each forming two gradient constraints in the x and y directions, the overall linear equation is:

[0082] in, for The sparse difference matrix, The gradient observation vector contains information; The depth vector is obtained by least squares optimization: .

[0083] In its implementation, this application utilizes sparse matrix solvers (such as LU decomposition or conjugate gradient method) to improve computational efficiency, and introduces boundary constraints during the integration process to ensure the continuity and overall stability of the depth field.

[0084] To further improve the visual consistency and detail reproduction of the results, this application performs normalization and noise smoothing after depth integration. Let the obtained depth range be... The normalized depth mapping is then:

[0085] This generates a visual grayscale depth map and a normal map. The final output includes a normal map, an albedo map, a depth map, and a height map, thus forming a complete 3D reconstruction result.

[0086] In the entire process, illumination self-calibration and normal integration constitute the key components of the algorithm. Self-calibration ensures the accuracy of the light source direction matrix, reducing systematic errors at the source; the integration stage achieves a balanced distribution of errors through global optimization constraints, thereby avoiding the drift phenomenon of traditional line-by-line integration methods. The computational chain of this method strictly follows the physical imaging laws, avoids the uncertainties of black-box models, and has good physical interpretability.

[0087] Experimental verification shows that this application can accurately recover the three-dimensional undulation structure of fingerprint ridges under multi-source illumination conditions, such as... Figure 5 and 6 Compared with traditional structured light methods, it exhibits significant advantages in ridge detail, normal smoothness, and depth continuity. The reconstruction results can be used for subsequent feature extraction, ridge curvature analysis, and 3D identity recognition, providing new technical support for biometrics and surface microstructure analysis.

[0088] In summary, this application achieves high-fidelity 3D reconstruction of finger surface microstructures through an integrated technical framework of "physical calibration - photometric solution - sparse integration". This scheme not only theoretically optimizes the stability of the photometric stereo model, but also improves reconstruction accuracy and computational efficiency in engineering, demonstrating significant practical application value.

[0089] Furthermore, this application also provides a three-dimensional fingerprint microstructure reconstruction system based on illumination self-calibration and sparse integration. This system is used to implement the three-dimensional fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integration described in the above embodiments. The system includes: an illumination self-calibration module, an acquisition module, an image preprocessing module, a normal vector solving module, and a depth reconstruction module connected in sequence. The illumination self-calibration module is used to construct a multi-source imaging system and arrange standard spherical reflectors in the imaging field of view for illumination direction self-calibration. The acquisition module is used to perform multi-angle imaging of the finger surface under calibrated illumination conditions. The system acquires a sequence of finger images under varying illumination conditions. An image preprocessing module defines the reconstructed region of the finger image sequence using a mask. A normal vector solving module vectorizes the pixel brightness variations in the reconstructed region under multiple illumination conditions based on the photometric stereo imaging model, and obtains a globally continuous normal gradient field based on the surface normal vector and reflectivity of each pixel. A depth reconstruction module transforms the normal gradient field into a depth constraint equation, employs sparse matrix optimization and least squares integration to achieve globally consistent depth map reconstruction, resulting in a 3D reconstruction.

[0090] In some embodiments, the image preprocessing module includes a mask extraction unit, a threshold segmentation unit, and a noise removal unit; the mask extraction unit is used to extract the reconstructed region using a mask image; the threshold segmentation unit is used to segment the reconstructed region by threshold segmentation to reduce noise interference caused by uneven illumination; the noise removal unit is used to effectively remove low-brightness, shadow, and high-reflection areas to reduce the interference of abnormal pixels on the solution results and make the reconstruction process more robust. Compared with existing structured light or stereo vision methods, this application does not require complex optical projection or parallax matching, and can achieve 3D reconstruction of fine ridge structures under ordinary multi-light source conditions. This method has the advantages of strong model physicality, high reconstruction accuracy, good robustness, and simple system implementation, and can be widely used in fields such as high-security fingerprint recognition, skin tissue characterization, and micro-surface detection.

[0091] The core innovation of this application lies in establishing a three-dimensional fingerprint microstructure reconstruction method and system based on illumination self-calibration and sparse integral. Its key technical points are mainly reflected in the following aspects: (1) Self-calibration mechanism of illumination direction.

[0092] Traditional photometric stereo methods generally rely on manual measurement or theoretical assumptions to obtain the illumination direction, which is prone to systematic errors. This application introduces a standard spherical reflector into the imaging system. By geometrically inverting the positions of reflected bright spots, the spatial incident directions of multiple light sources are calculated, automatically generating the illumination direction matrix. This method utilizes the law of reflection. This enables adaptive calibration of light source parameters, significantly improving the physical consistency and accuracy of the photometric model.

[0093] (2) Joint solution model of normal and reflectivity under pixel-level photometric constraints.

[0094] Given the illumination matrix, this application establishes an intensity vector by taking the brightness change of each pixel under multiple illumination conditions as the observation. A system of linear equations was solved jointly using least squares optimization. This allows for the recovery of the pixel-level normal vector n and reflectivity. This joint solution model avoids the accumulation of errors in the separation of illumination and reflectivity in the traditional step-by-step method, making the normal recovery more stable and continuous.

[0095] (3) Deep recovery algorithm driven by sparse integral model.

[0096] This application transforms the normal gradient field into a linear integral constraint equation. The model employs sparse matrix optimization to achieve globally consistent reconstruction of normal and depth information. Compared to traditional local integration methods, this model suppresses noise accumulation, maintains surface smoothness and geometric consistency, and provides a mathematical guarantee for the complete recovery of subtle ridge morphology.

[0097] Compared with existing technologies, this application provides a method and system for three-dimensional fingerprint microstructure reconstruction based on illumination self-calibration and sparse integration, which has the following advantages: Compared with existing fingerprint 3D reconstruction technologies based on structured light, stereo vision and traditional photometric stereo, this application has significant advantages in system structure, algorithm principle and reconstruction accuracy.

[0098] Firstly, regarding illumination calibration methods, traditional photometric stereo methods typically rely on external angle measuring devices or manually preset illumination directions. Errors in these methods tend to accumulate with assembly deviations, affecting the accuracy of normal direction calculation. This application introduces a standard spherical reflector into the field of view and automatically calculates the directions of multiple light sources using specular reflection geometry, achieving self-calibration of the illumination direction matrix. This method avoids complex manual measurements, achieves illumination direction estimation errors of less than 1°, and effectively improves the system's stability and repeatability.

[0099] Secondly, existing methods for jointly estimating normal and reflectance often employ single-channel or simplified brightness models, neglecting spatial variations in reflectance, leading to discontinuities or texture blurring in local reconstructions. This application addresses this by constructing a set of photometric vector equations. and jointly solve It simultaneously recovers normal and reflectivity information at the pixel level, enabling the solution results to have physical consistency and spatial smoothness, thereby enhancing the ability to restore details.

[0100] Third, in the deep integration stage, traditional methods typically employ line-by-line integration or the Poisson equation, which is prone to error accumulation and surface ripple artifacts. This application establishes a sparse linear integral model. A least-squares global optimization algorithm is employed to achieve a stable mapping of the normal field to the depth field. This sparse solution mechanism maintains surface continuity and geometric consistency even in noisy environments, significantly improving the smoothness and accuracy of depth recovery.

[0101] Fourth, in terms of noise resistance and adaptability, this application introduces a mask constraint mechanism to effectively remove low-brightness, shadow, and high-reflection areas, reducing the interference of abnormal pixels on the solution results and making the reconstruction process more robust. Combined with multi-source illumination conditions, this application can still obtain normal and depth map results with high signal-to-noise ratio and high contrast under complex lighting environments.

[0102] Finally, regarding system implementation and application expansion, this application does not require projected stripes or multi-view geometric matching; it can be directly implemented based on conventional cameras and multiple LED light sources, such as... Figure 3 and Figure 4 As shown, the hardware structure is simple and inexpensive, and it possesses good real-time performance and portability. This method is not only applicable to high-precision three-dimensional fingerprint recognition, but can also be extended to fields such as skin tissue assessment, rehabilitation-aided detection, and bionic tactile representation.

[0103] Moreover, existing patents have not solved the problem of solving the photometric stereo problem of non-ideal skin reflection. This application proposes for the first time a "mask constraint + multi-light vectorization joint solution" to handle the complexity of skin reflection.

[0104] In summary, this application significantly outperforms existing state-of-the-art technologies in terms of automated illumination calibration, accuracy of normal estimation, stability of depth integration, and system versatility. It can achieve high-fidelity reconstruction of the microscale ridge structure of the finger, demonstrating clear innovation and application value. Moreover, this application has been experimentally and simulated to prove its feasibility.

[0105] Based on this, the core idea of ​​this application is based on the principle of photometric stereo imaging and a self-calibrated lighting model. Its key lies in recovering the surface normal and depth information of an object from pixel brightness variations under multi-source conditions. This principle has good versatility and scalability; therefore, various design modifications and alternative implementation schemes can be made without departing from the technical concept of this application.

[0106] In terms of lighting configuration, in addition to using fixed array LED light sources, programmable light source arrays, ring lighting modules, or multi-directional laser dot matrix lighting systems can also be used to achieve higher lighting coverage and angular resolution. The lighting self-calibration module can be implemented using different standard reflectors such as reflective spheres, mirrored plates, or calibration gratings. Its core objective is to establish a true lighting direction matrix to ensure the physical accuracy of the normal calculation.

[0107] In the joint solution stage of normal and reflectivity, the mathematical framework of this application can be combined with the non-Lambertian reflection model to handle heterogeneous surface materials such as skin, metal, and ceramics. For objects with complex reflection components, a polarization imaging module can be further introduced to separate diffuse reflection and specular reflection components through light intensity observation at multiple polarization angles, thereby improving the stability and spectral consistency of normal estimation.

[0108] In the deep reconstruction stage, the sparse matrix integral model can be extended into a multi-scale hierarchical solution framework to adapt to the reconstruction needs of surfaces with different resolutions or different hierarchical structures. For highly dynamic surfaces or flexible targets, temporal constraints (such as Kalman filtering or dynamic Bayesian networks) can be combined to achieve continuous reconstruction of time-varying surface morphology, thereby constructing a "4D spatiotemporal surface reconstruction model".

[0109] At the application level, in addition to 3D fingerprint recognition, this application can also be extended to the following fields: 1) Medical and Rehabilitation Testing: Used for skin tissue elasticity assessment, wound recovery tracking, and surface strain monitoring in hand rehabilitation training.

[0110] 2) Bionics and human-computer interaction: used to build high-precision bionic hand skin models and tactile sensing simulations to improve robot tactile recognition and grasping stability.

[0111] 3) Industrial and cultural relic testing: Used for high-precision detection of surface morphology changes such as metal micro-scratches, surface wear, and micro-cracks in cultural relics.

[0112] 4) Security and anti-counterfeiting identification: A multi-dimensional biometric model is established by combining three-dimensional ridge features to improve the anti-counterfeiting capability of the identity verification system.

[0113] Based on the above technical solutions, this application provides a method and system for three-dimensional fingerprint microstructure reconstruction using illumination self-calibration and sparse integration, comprising the following steps: First, a multi-source imaging system is constructed, and a standard spherical reflector is arranged in the imaging field of view for illumination direction self-calibration; then, under the calibrated illumination conditions, multi-angle illumination acquisition is performed on the finger surface to obtain a sequence of finger images under multiple illumination variations; next, a mask image is used to define the reconstruction area of ​​the finger image sequence; then, based on the photometric stereo imaging model, the pixel brightness variation of the reconstruction area under multiple illumination conditions is vectorized and solved, and a globally continuous normal gradient field is obtained based on the surface normal vector and reflectivity of each pixel; finally, the normal gradient field is transformed into a depth constraint equation, and sparse matrix optimization and least squares integration are used to solve it to achieve globally consistent reconstruction of the depth map, resulting in a three-dimensional reconstruction result. This application achieves high-fidelity three-dimensional reconstruction of finger surface microstructures through an integrated technical framework of "physical calibration - photometric solution - sparse integration". This scheme not only theoretically optimizes the stability of the photometric stereo model, but also improves reconstruction accuracy and computational efficiency in engineering, and has significant practical and promotional value.

[0114] Those skilled in the art will understand that the above-described embodiments are specific examples of implementing this application, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of this application. Any person skilled in the art can make their own modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. A method for reconstructing three-dimensional fingerprint microstructures using illumination self-calibration and sparse integration, characterized in that, Includes the following steps: A multi-source imaging system was constructed, and a standard spherical reflector was arranged in the imaging field of view for self-calibration of the illumination direction; Under calibrated lighting conditions, multi-angle illumination acquisition is performed on the finger surface to obtain a sequence of finger images under multiple lighting changes. The reconstruction region of the finger image sequence is defined using a mask image; The imaging model based on photometric stereo method is used to vectorize the pixel brightness changes of the reconstructed region under multiple illumination conditions, and a globally continuous normal gradient field is obtained based on the surface normal vector and reflectivity of each pixel. The normal gradient field is transformed into a depth constraint equation, which is then solved using sparse matrix optimization and least squares integration to achieve globally consistent reconstruction of the depth map and obtain the 3D reconstruction result.

2. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 1, characterized in that, Constructing a multi-source imaging system includes: Multiple sets of LED point light sources with constant brightness are selected and arranged around the finger sample to be tested at a fixed angle, ensuring that the camera optical axis is approximately perpendicular to the sample surface.

3. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 2, characterized in that, Let the reflectance at any point on the surface of the finger sample be . ,in Indicates the different light source numbers, These are pixel coordinates; According to the photometric stereo imaging model, the reflectance at any point on the surface of the finger sample to be tested is expressed by Lambert's law of reflection as follows: in, For pixel coordinates The reflectivity of the point, The surface unit normal vector, For the first i Each light source direction vector.

4. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 1, characterized in that, Arranging a standard spherical reflector in the imaging field of view for self-calibration of illumination direction includes: A standard spherical reflector is set up in the imaging field of view. By analyzing the positional distribution of the specular reflection points on the standard sphere, the spatial incident direction of each light source is calculated, and an illumination direction matrix is ​​established. Let the radius of the sphere be... r The center coordinates are The location of the highlight detected in the image is Then the surface normal vector corresponding to that point is: According to the law of reflection, the direction of light illumination... relative to the camera's incident direction The relationship is: By detecting the highlight region corresponding to each light source and calculating its average coordinates, the spatial direction vectors of all light sources are calculated, thus forming a complete illumination matrix: in, l nx , l ny , l nz These are the components of the light source direction vector along the x, y, and z axes, respectively.

5. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 1, characterized in that, The reconstructed region of the finger image sequence is defined using a mask image, including: The reconstructed region is filtered using threshold segmentation and binary masking operations; Let the input image be Its grayscale range is Define the maximum grayscale value as Then the mask function It can be represented as: Among them, the threshold coefficient Between 0.05 and 0.

1.

6. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 1, characterized in that, The surface normal vector and reflectivity of each pixel are expressed as follows: in, L is the illumination matrix, and I is the pixel intensity.

7. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 1, characterized in that, The normal gradient field is transformed into a depth-constrained equation, which is then solved using sparse matrix optimization and least squares integration, including: An integral optimization model based on sparse matrix construction is adopted; Assuming there are a total of m Given _{x} effective pixels, each forming two gradient constraints in the x and y directions, the overall linear equation is: in, for The sparse difference matrix, The gradient observation vector contains information; The depth vector is obtained by least squares optimization: 。 8. The method for reconstructing three-dimensional fingerprint microstructures based on illumination self-calibration and sparse integration according to claim 1, characterized in that, The three-dimensional reconstruction results include a normal map, a reflectance map, a depth map, and a height map.

9. A three-dimensional fingerprint microstructure reconstruction system based on illumination self-calibration and sparse integration, the system being used to implement the three-dimensional fingerprint microstructure reconstruction method based on illumination self-calibration and sparse integration as described in any one of claims 1 to 8, characterized in that, The system comprises: a self-calibration module for illumination, an acquisition module, an image preprocessing module, a normal vector calculation module, and a depth reconstruction module, connected in sequence; among them, The illumination self-calibration module is used to construct a multi-source imaging system and to arrange standard spherical reflectors in the imaging field of view for illumination direction self-calibration. The acquisition module is used to acquire multi-angle illumination data of the finger surface under calibrated lighting conditions, and obtain a sequence of finger images under multiple lighting changes. The image preprocessing module is used to define the reconstruction region of the finger image sequence using a mask image; The normal solution module is used to perform vectorized solution of the pixel brightness change of the reconstructed region under multiple illumination conditions according to the imaging model of photometric stereo method, and obtain a global continuous normal gradient field based on the surface normal vector and reflectivity of each pixel. The depth reconstruction module is used to transform the normal gradient field into a depth constraint equation, and solve it using sparse matrix optimization and least squares integration to achieve globally consistent reconstruction of the depth map and obtain the three-dimensional reconstruction result.

10. The three-dimensional fingerprint microstructure reconstruction system based on illumination self-calibration and sparse integration according to claim 9, characterized in that, The image preprocessing module includes a mask extraction unit, a threshold segmentation unit, and a noise removal unit; The mask extraction unit is used to extract the reconstructed region using a mask image; The threshold segmentation unit is used to segment the reconstructed region through threshold segmentation to reduce noise interference caused by uneven illumination. The noise removal unit is used to effectively remove low-brightness, shadow and high-reflection areas, reduce the interference of abnormal pixels on the solution results, and make the reconstruction process more robust.

Images