Polarization microscopic three-dimensional reconstruction algorithm fused with Fourier light field depth

Abstract

The invention discloses a Fourier light field depth-fused polarization microscopic three-dimensional reconstruction algorithm, which comprises the following steps of: acquiring an image sequence on a polarization imaging branch, and calculating a normal zenith angle and a polarization normal azimuth angle of each pixel point; fourier light field three-dimensional reconstruction is carried out based on an image collected by the Fourier light field imaging branch, and a reconstructed depth map is obtained; performing spatial registration on the visual field of the double branches, and judging and correcting a polarization normal azimuth angle; and constructing a convex optimization objective function to obtain an optimal solution target depth map Z. According to the method, absolute depth information provided by a Fourier light field is utilized to carry out auxiliary correction on an azimuth angle of a normal in a polarization three-dimensional reconstruction process, and the problem of reconstruction result local direction uncertainty caused by pi ambiguity is relieved. A joint model fusing the polarization normal and the light field depth is constructed and solved, and more accurate and continuous absolute depth reconstruction is realized by introducing global guidance and surface smoothness constraint of the light field depth.

Description

Technical Field

[0001] This invention relates to the fields of image processing technology and three-dimensional reconstruction technology, and more specifically, to a polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth. Background Technology

[0002] Polarization-based 3D reconstruction is a non-contact imaging method for estimating surface normals and reconstructing 3D topography based on the polarization characteristics of light waves. This method observes the polarization state of reflected light from an object's surface, extracts physical parameters such as the degree of polarization and polarization azimuth, and, combined with a Fresnel reflection model, establishes a correspondence between these parameters and the zenith and azimuth angles of the surface micro-element normals. Then, by integrating the normal field, 3D topography reconstruction is achieved. However, due to the ambiguity of the Fresnel model under different reflection types, the same polarization parameters may correspond to multiple normal directions, leading to misjudgments of the reconstructed topography's concavity and convexity. Furthermore, this method relies on the relative integration of normals and lacks the ability to perceive absolute depth information, limiting its applicability in high-precision reconstruction.

[0003] In 2023, Shi et al. combined light field microscopy with polarization measurement, proposing a method to incorporate Stokes vector measurement into a Fourier light field microscopy system. This effectively overcame the limitation of vertical resolution in traditional light field microscopy systems, enabling three-dimensional reconstruction of microstructured samples. Mao et al. proposed a three-dimensional reconstruction method for micro / nano-scale surfaces based on polarization modulation microscopy (PMM). By improving the polarization bidirectional reflection distribution function (PBRDF) model, correcting the ambiguity of the surface normal azimuth angle π, and using a coarse depth obtained through atomic force microscopy and downsampling with added noise as a priori, they constructed a variational optimization-based fusion framework, ultimately achieving higher-precision three-dimensional morphology restoration of microscale samples.

[0004] Light field microscopy (LFM) is an emerging microscopic 3D reconstruction technique, first proposed by Levoy et al. in 2006. By introducing a microlens array into the traditional microscope imaging path, LFM can simultaneously capture the spatial and angular information of incident light rays in a single exposure, thus recording a four-dimensional light field. Subsequently, 3D reconstruction can be achieved by combining deconvolution processing with the point spread function (PSF). This feature of not requiring scanning and possessing single-frame 3D reconstruction capability provides a new solution for the measurement of microstructures.

[0005] Subsequently, Broxton et al. introduced wave optics modeling into the LFM imaging model and used a three-dimensional deconvolution algorithm to significantly improve the signal-to-noise ratio and spatial detail recovery. Li et al. further improved the quality of the light field image through optical design optimization and system calibration, achieving three-dimensional reconstruction of micron-level samples. However, as research progressed, the inherent limitations of traditional LFM gradually became apparent: due to the trade-off between acquiring angle and spatial information in a single exposure, the lateral resolution of the reconstructed image was relatively low, and the resolution decreased unevenly along the axis with increasing depth of field, resulting in regular grid-like artifacts in the reconstruction results. In addition, the massive four-dimensional data acquired by LFM requires complex calculations to reconstruct three-dimensional volume data, resulting in low computational efficiency. To address these issues, Llavador et al. proposed a new technique, Fourier Light Field Microscopy (FLFM), in 2016. This technique moves the microlens array to the Fourier plane of the microscopy imaging system, records the Fourier domain light field information of the sample on the sensor, and allows the entire three-dimensional imaging process to be represented by a unified three-dimensional point spread function, significantly reducing the computational complexity of reconstruction. Meanwhile, because the light field recording method of the Fourier plane allocates spatial and angular information in a consistent aliasing manner, it avoids the artifact problem caused by redundant sampling in traditional LFM, making it a microscopic 3D reconstruction method with higher imaging quality and faster reconstruction speed. In 2018, Scrofani et al. systematically compared the imaging performance of LFM and FLFM, and found that the latter performed better in terms of resolution and depth-of-field consistency.

[0006] Subsequently, researchers continued to optimize FLFM's reconstruction strategies and light field manipulation mechanisms, constantly expanding its imaging performance and applicability. In 2018, Moreschini et al. designed a shear domain continuous refocusing method, improving the continuity of depth reconstruction and axial resolution. In 2019, Lu et al. introduced a phase space deconvolution algorithm to map the light field into a four-dimensional phase space for modeling, enhancing detail restoration capabilities. Palmieri et al. combined multi-view matching and focal plane stacking information to enhance the structural stability of weakly textured regions. In 2020, Liu et al. expanded the imaging scenarios of FLFM using speckle modulation coding and sparse reconstruction techniques. In 2021, Hua et al. constructed a high-resolution HR-FLFM system and effectively reduced the number of sub-aperture images acquired in the system design, achieving accurate reconstruction of micron-level three-dimensional structures using only three sub-viewpoints.

[0007] For polarization imaging, due to its inherent periodicity, the inversion process of the normal zenith angle and azimuth angle exhibits inherent ill-conditions, making accurate and stable 3D reconstruction difficult to achieve solely based on polarization information. Therefore, research in this field commonly employs a fusion approach of "polarization + auxiliary techniques," such as using depth cameras to acquire depth information, to compensate for the shortcomings of polarization imaging in depth calculation and ambiguity resolution. However, depth maps typically have low spatial resolution. Furthermore, for light field microscopic 3D imaging techniques, such as FLFM, the lateral resolution bottleneck caused by spatial-angular sampling conflicts remains a key challenge restricting the further development of light field 3D reconstruction technology.

[0008] Chinese patent document (application number: 202310816237.6, application date: 2023.07.04) discloses a light field three-dimensional imaging method and system based on polarization and data fusion. The method includes: Step 1, constructing a Fourier light field microscopic three-dimensional imaging system with polarization image acquisition function; Step 2, acquiring polarization light field images of the sample under test when the linear polarizer is at several different angles under the same conditions; Step 3, calculating the light field three-dimensional point cloud data of the sample and the polarization angle AoP and polarization degree DoP data of the sample based on the acquired image data; Step 4, obtaining the surface normal vector based on polarization three-dimensional imaging through the polarization data and three-dimensional point cloud data in Step 3, and fusing the object surface normal vector containing more detailed depth variation information with the light field three-dimensional point cloud data to further fit the three-dimensional point cloud data of the sample under test, and finally obtaining the light field microscopic three-dimensional imaging result with higher longitudinal resolution and reconstruction accuracy. This scheme uses the same optical path for both light field microscopy and polarization 3D imaging, but it does not solve the problem that light field microscopy and polarization 3D imaging each have their own separate optical paths.

[0009] The existing technology has the following problems: 1. The polarization normal has "π ambiguity" in the polarization imaging process, which causes misjudgment of the reconstructed morphology direction; 2. Single polarization imaging generally has the problem of low spatial resolution of depth map, while single light field microscopy has the problem of insufficient lateral resolution. Therefore, how to solve the above-mentioned problems has become an urgent problem to be solved in this field. Summary of the Invention

[0010] In view of this, the present invention provides a polarization microscopy 3D reconstruction algorithm that integrates Fourier light field depth to solve the above-mentioned technical problems: 1. The polarization normal has "π ambiguity" in the polarization imaging process, causing misjudgment of the reconstructed topography direction; 2. Single polarization imaging generally suffers from low spatial resolution of depth maps, while single light field microscopy suffers from insufficient lateral resolution. This method first utilizes the absolute depth information provided by the Fourier light field to assist in correcting the azimuth angle of the normal during polarization 3D reconstruction, thereby alleviating the local directional uncertainty of the reconstruction results caused by "π ambiguity". Furthermore, it constructs and solves a joint convex optimization model that integrates the polarization normal and the light field depth. By introducing global guidance of the light field depth and surface smoothness constraints, it achieves more accurate and continuous absolute depth reconstruction, compensating for the shortcomings of polarization imaging in depth measurement accuracy.

[0011] This application provides a polarization microscopy 3D reconstruction algorithm that integrates Fourier light field depth. The polarization microscopy 3D reconstruction algorithm is used in a microscopy imaging system containing independent polarization imaging branches and Fourier light field imaging branches.

[0012] The polarization microscopy three-dimensional reconstruction algorithm includes:

[0013] In the polarization imaging branch, by adjusting the rotation angle of the polarizer, image sequences at four polarization angles of 0°, 45°, 90°, and 135° are acquired respectively. Based on the polarization inversion model, the normal zenith angle θ and polarization normal azimuth angle of each pixel are calculated.

[0014] Based on the images acquired by the Fourier light field imaging branch, Fourier light field three-dimensional reconstruction is performed to obtain the depth conversion factor Δz of the microscopic imaging system. 物空间 Based on the depth conversion factor and the light intensity distribution of the 3D object to be reconstructed, a light field reconstruction depth map Z with absolute depth information is obtained. f ;

[0015] Spatially register the fields of view of the polarization imaging branch and the Fourier light field imaging branch, and reconstruct the depth map Z based on the light field. f Derive the corresponding azimuth angle of the light field normal. The polarization normal azimuth angle in the polarization imaging branch Discrimination and correction are performed; the Frankot–Chellappa algorithm is used to integrate the corrected polarization branch normal field to obtain a three-dimensional polarization reconstruction image with concavity / convexity correction, wherein:

[0016] Spatial registration of the field of view of the polarization imaging branch and the Fourier light field imaging branch includes: reconstructing the depth map Z using the light field. fAs a reference, the region with the largest inscribed square is selected as the registration target region. The XFeat algorithm is used to realize the mapping from the polarization image coordinate system to the light field reconstruction depth map Z. f The spatial mapping of the registration target region in the image yields a set of affine matrices T describing the spatial mapping relationship between the polarization image and the light field image. reg , wherein the affine matrix T reg In the diagram, the first row contains scaling, rotation, and translation coefficients along the x-axis; the second row contains the corresponding parameters along the y-axis; and the third row contains the normalization terms for the homogeneous coordinate affine transformation. A bicubic interpolation algorithm is used to reconstruct the depth map Z from the light field. f Sampling processing is performed to reconstruct the depth map Z of the light field. f It maintains the same resolution as the registered polarization image;

[0017] The depth map Z reconstructed based on the light field f Derive the corresponding azimuth angle of the light field normal. include:

[0018] Select the light field reconstruction depth map Z f For any pixel q = (x, y) on the surface, the neighborhood of the pixel is considered as a parametric surface, and the azimuth angle of the light field normal is... According to Equation 1:

[0019]

[0020] Repeat the above light field normal azimuth angle The calculation yields the light field reconstruction depth map Z. f The corresponding light field normal azimuth angle of each pixel The azimuth angle of the light field normal With respect to the polarization normal azimuth angle Perform pixel-by-pixel comparison, normal azimuth angle The correction should be made according to Equation 2:

[0021]

[0022] The corrected normal azimuth angle The surface normal is reconstructed by the zenith angle θ of the normal, and the normal integral is completed by the Frankot-Chellappa algorithm to obtain a polarization 3D reconstruction map with concavity and convexity correction.

[0023] A convex optimization objective function with multiple constraints is constructed, and the Split-Bregman multiplier method is used as the optimization algorithm to obtain the optimal solution, which is the target depth map Z; including:

[0024] The convex optimization objective function is:

[0025]

[0026] in, The depth gradient is calculated from the corrected polarization normal. λ1 represents the weight factor of the global depth consistency term; λ2 is the local gradient constraint term, and λ2 is the weighting factor of the local gradient constraint term; λ3 is the regularization term, and λ3 is the weight factor of the regularization term; Calculate according to equations 4-6:

[0027]

[0028] in, For surface continuity regularization, For TV regularization terms; α∈[0,1].

[0029] Optionally, the normal zenith angle θ and polarization normal azimuth angle of each pixel are calculated based on the polarization inversion model. include:

[0030] For each pixel, based on Malus's law, a polarization modulation curve is constructed by fitting the maximum and minimum light intensity values ​​received by the detector within one rotation cycle of the polarizer. Using this polarization modulation curve, the polarization phase angle φ and polarization degree ρ are extracted. Based on the mapping relationship between the polarization degree ρ and the normal zenith angle θ, the solution line zenith angle θ and the polarization normal azimuth angle are calculated. Among them, the polarization normal azimuth angle The polarization phase angle φ satisfies or

[0031] The mapping relationship between the degree of polarization ρ and the normal zenith angle θ satisfies Equation 7:

[0032]

[0033] In Equation 7, n is the refractive index of the three-dimensional object.

[0034] Optionally, the Fourier light field three-dimensional reconstruction based on the image acquired by the Fourier light field imaging branch includes:

[0035] The imaging response of a three-dimensional object at different depth positions is recorded. PSF slices are acquired at fixed intervals, and each slice is preprocessed sequentially, including background subtraction and binarization calibration, to generate the measured three-dimensional point spread function data h of the microscopic imaging system. 3D(x″,y″,z″), based on an ideal microscopic imaging model, the light intensity distribution g(x,y,z) of the three-dimensional object to be reconstructed is obtained by using the RL deconvolution algorithm.

[0036] Optionally, the depth conversion factor Δz of the microscopic imaging system is obtained. 物空间 ,include:

[0037] In the PSF slice, the layer with the highest brightness is selected as the image space location corresponding to the geometric center. Analysis is then performed layer by layer upwards from the layer with the highest brightness, until the m-th layer... i The top reflection signal was detected for the first time in the layer, determining the m-th layer. i Layers correspond to the positions of the highest points of a three-dimensional object in image space.

[0038]

[0039] Among them, R 样品 The radius of the three-dimensional object.

[0040] Optionally, the step of using the Split-Bregman multiplier method as the optimization algorithm to obtain the optimal solution includes:

[0041] Introducing auxiliary variable d x d y Replace the gradient terms of the depth map in the x and y directions, where:

[0042]

[0043] Introducing the Bregman multiplier b x b y The convex optimization objective function is transformed into the following Lagrangian form:

[0044]

[0045] The Split-Bregman method is used to solve the problem.

[0046] Compared with existing technologies, the polarization microscopy three-dimensional reconstruction algorithm that integrates Fourier light field depth provided by this invention achieves at least the following beneficial effects:

[0047] First, this invention derives the reference azimuth angle through the prior depth provided by the Fourier light field, and corrects the "π ambiguity" of the polarization normal pixel by pixel, effectively avoiding misjudgment of the direction of the reconstructed morphology, successfully breaking through the ill-conditioned limitations of traditional polarization imaging, and solving the core technical problem that has long existed in this field.

[0048] Second, by combining the high-frequency detail capture capability of polarization imaging with the absolute depth measurement advantage of Fourier light field, it not only makes up for the shortcoming of polarization imaging lacking absolute depth perception, but also alleviates the problem of insufficient lateral resolution of light field imaging, thus breaking through the performance bottlenecks of each single-mode technology.

[0049] Third, the hybrid regularization term introduced by the convex optimization model can effectively suppress noise and boundary deformation, while ensuring overall geometric consistency through global depth constraints, making the reconstruction results more accurate and continuous, and greatly improving the accuracy and stability of 3D reconstruction.

[0050] Fourth, the RL deconvolution algorithm for Fourier light field is based on the measured three-dimensional point spread function (3DPSF), which simplifies the computational complexity while improving the consistency of axial resolution. Compared with traditional light field reconstruction, it reduces artifacts and data redundancy, and balances reconstruction efficiency and imaging quality.

[0051] Of course, any product implementing this invention does not necessarily need to achieve all of the technical effects described above at the same time.

[0052] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0054] Figure 1 This is a schematic diagram of the steps of a polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth according to the present invention;

[0055] Figure 2 This is a flowchart of the overall algorithm for a polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth according to the present invention.

[0056] Figure 3 This is a diagram showing the process of fusing and reconstructing a 5μm densely packed polystyrene microsphere sample with continuous structure and fluctuating morphology using the polarization microscopic three-dimensional reconstruction algorithm of this invention.

[0057] Figure 4 This is a schematic diagram of dual-optical-path field-of-view registration in the polarization microscopy three-dimensional reconstruction algorithm of this invention;

[0058] Figure 5 These are comparison images of the registration results after dual-optical-path field-of-view registration in the polarization microscopy three-dimensional reconstruction algorithm of this invention. In the image, (a) is the Fourier light field depth image, and (b) is the polarization image after registration.

[0059] Figure 6This invention presents the azimuth correction process of the polarization microscopic three-dimensional reconstruction algorithm, wherein (a) is the polarization normal azimuth angle; (b) is the light field normal azimuth angle; (c) is the corrected polarization normal azimuth angle; (d) is the corrected normal; (e) is the original polarization three-dimensional reconstruction result; and (f) is the three-dimensional reconstruction result after polarization normal azimuth angle correction.

[0060] Figure 7 This is a schematic diagram of the polarization modulation curve of pixel B in the polarization microscopic three-dimensional reconstruction algorithm of the present invention. (a) is a schematic diagram of the position of pixel B; (b) is the fitted polarization modulation curve of pixel B; and (c) is an image of different polarization angles (0°, 30°, 60°, 90°, 150°). Detailed Implementation

[0061] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0062] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0063] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0064] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0065] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0066] In existing polarization-based 3D reconstruction techniques, the Fresnel model exhibits multiple solutions under different reflection types, meaning the same polarization parameter may correspond to multiple normal directions. This leads to misjudgments of the reconstructed topography's convexity and concavity. Relying solely on polarization information is insufficient for achieving accurate and stable 3D reconstruction. A fusion approach combining polarization and auxiliary techniques is employed to compensate for the shortcomings of polarization imaging in depth calculation and ambiguity resolution; however, the spatial resolution of depth maps is typically low. For light field microscopic 3D imaging techniques, such as FLFM, the lateral resolution bottleneck caused by spatial-angular sampling conflicts remains a key challenge hindering the further development of light field 3D reconstruction technology.

[0067] This invention proposes a polarization-based microscopic 3D reconstruction algorithm that integrates Fourier light field depth. This method is designed for imaging systems with both polarization and light field branches. The polarization branch acquires high-resolution polarization images of the sample surface to support subsequent fine-grained normal estimation and topography reconstruction. The light field branch, based on the principle of Fourier light field microscopy, acquires Fourier light field images with spatial-angular information and can estimate the absolute depth information of the sample surface using the algorithm. The two light paths are structurally independent but acquire data synchronously. System calibration and spatial registration unify the two modal data in both time and space dimensions. The method first utilizes the absolute depth information provided by the Fourier light field to assist in correcting the azimuth angle of the normal during polarization 3D reconstruction, thereby alleviating the local directional uncertainty in the reconstruction results caused by "π ambiguity." Furthermore, it constructs and solves a joint convex optimization model that integrates polarization normals and light field depth. By introducing global guidance from the light field depth and surface smoothness constraints, it achieves more accurate and continuous absolute depth reconstruction, compensating for the shortcomings of polarization imaging in depth measurement accuracy.

[0068] Reference Figures 1-7 As shown, Figure 1 This is a schematic diagram of the steps of a polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth according to the present invention; Figure 2 This is a flowchart of the overall algorithm for a polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth according to the present invention. Figure 3 This is a diagram showing the process of fusing and reconstructing a 5μm densely packed polystyrene microsphere sample with continuous structure and fluctuating morphology using the polarization microscopic three-dimensional reconstruction algorithm of this invention. Figure 4 This is a schematic diagram of dual-optical-path field-of-view registration in the polarization microscopy three-dimensional reconstruction algorithm of this invention; Figure 5 These are comparison images of the registration results after dual-optical-path field-of-view registration in the polarization microscopy three-dimensional reconstruction algorithm of this invention. In the image (a), Fourier light field depth image is shown, and in the image (b), the polarization image after registration is shown. Figure 6 This invention presents the azimuth correction process of the polarization microscopic three-dimensional reconstruction algorithm, wherein (a) is the polarization normal azimuth angle; (b) is the light field normal azimuth angle; (c) is the corrected polarization normal azimuth angle; (d) is the corrected normal; (e) is the original polarization three-dimensional reconstruction result; and (f) is the three-dimensional reconstruction result after polarization normal azimuth angle correction. Figure 7 This is a schematic diagram of the polarization modulation curve of pixel B in the polarization microscopic three-dimensional reconstruction algorithm of the present invention. (a) is a schematic diagram of the position of pixel B; (b) is the fitted polarization modulation curve of pixel B; and (c) is an image of different polarization angles (0°, 30°, 60°, 90°, 150°).

[0069] like Figure 1 , Figure 2As shown, this embodiment proposes a polarization microscopy 3D reconstruction algorithm that integrates Fourier light field depth. The polarization microscopy 3D reconstruction algorithm is used in microscopy imaging systems containing independent polarization imaging branches and Fourier light field imaging branches.

[0070] Polarization microscopy 3D reconstruction algorithms include:

[0071] S100. On the polarization imaging branch, by adjusting the rotation angle of the polarizer, image sequences at four polarization angles of 0°, 45°, 90°, and 135° are acquired respectively. Based on the polarization inversion model, the normal zenith angle θ and polarization normal azimuth angle of each pixel are calculated.

[0072] S200: Based on images acquired by the Fourier light field imaging branch, perform Fourier light field three-dimensional reconstruction to obtain the depth conversion factor Δz of the microscopic imaging system. 物空间 Based on the depth conversion factor and the light intensity distribution of the 3D object to be reconstructed, a light field reconstruction depth map Z with absolute depth information is obtained. f ;

[0073] S300. Spatial registration of the field of view of the polarization imaging branch and the Fourier light field imaging branch (S301), based on the light field reconstruction depth map Z. f Derive the corresponding azimuth angle of the light field normal. Azimuth angle of polarization normal in polarization imaging branch Discrimination and correction are performed (S302); the Frankot–Chellappa algorithm is used to integrate the corrected positive branch normal field to obtain the polarization 3D reconstruction map with concavity / convexity correction completed (S303), wherein:

[0074] Spatial registration of the field of view of the polarization imaging branch and the Fourier light field imaging branch includes: reconstructing the depth map Z using the light field. f As a reference, the region with the largest inscribed square is selected as the registration target region. The XFeat algorithm is used to realize the conversion from the polarization image coordinate system to the light field reconstruction depth map Z. f The spatial mapping of the registration target region in the image yields a set of affine matrices T describing the spatial mapping relationship between the polarization image and the light field image. reg , where the affine matrix T reg In the diagram, the first row contains scaling, rotation, and translation coefficients along the x-axis; the second row contains the corresponding parameters along the y-axis; and the third row contains the normalization terms for the homogeneous coordinate affine transformation. A bicubic interpolation algorithm is used to reconstruct the depth map Z from the light field. f Sampling processing is performed to reconstruct the depth map Z of the light field. f It maintains the same resolution as the registered polarization image;

[0075] Depth map Z reconstructed from light fieldf Derive the corresponding azimuth angle of the light field normal. include:

[0076] Select the light field reconstruction depth map Z f For any pixel q = (x, y) on the surface, the neighborhood of the pixel is considered as a parametric surface, and the azimuth angle of the light field normal is... According to Equation 1:

[0077]

[0078] Repeat the above calculations to obtain the light field reconstruction depth map Z. f The corresponding light field normal azimuth angle of each pixel The azimuth of the light field normal With polarization normal azimuth angle Perform pixel-by-pixel comparison, normal azimuth angle The correction should be made according to Equation 2:

[0079]

[0080] Corrected normal azimuth The surface normal is reconstructed by the normal zenith angle θ, and the normal integral is completed by the Frankot–Chellappa algorithm to obtain a polarization 3D reconstruction map with concavity and convexity correction.

[0081] S400. Construct a convex optimization objective function with multiple constraints, and use the Split-Bregman multiplier method as the optimization algorithm to obtain the optimal solution, which is the target depth map Z; including:

[0082] The objective function for convex optimization is:

[0083]

[0084] in, The depth gradient is calculated from the corrected polarization normal. λ1 represents the weight factor of the global depth consistency term; λ2 is the local gradient constraint term, and λ2 is the weighting factor of the local gradient constraint term; λ3 is the regularization term, and λ3 is the weight factor of the regularization term; Calculate according to equations 4-6:

[0085]

[0086] in, For surface continuity regularization, For TV regularization terms; α∈[0,1].

[0087] Specifically, the method of the present invention includes:

[0088] (1) In the polarization imaging branch, image sequences at four polarization angles of 0°, 45°, 90° and 135° were acquired respectively. The normal zenith angle and azimuth angle information of each pixel were calculated based on the polarization inversion model, and the surface depth was recovered by the normal integration method to obtain the preliminary polarization three-dimensional reconstruction results.

[0089] (2) Based on the images acquired by the Fourier light field imaging branch, perform Fourier light field three-dimensional reconstruction to obtain a reconstructed depth map containing absolute depth information, providing auxiliary priors for the correction of polarization three-dimensional reconstruction results.

[0090] (3) Spatial registration is performed between the field of view of the polarization imaging branch and the Fourier light field imaging branch. Based on the depth inversion of the light field reconstruction, the azimuth information of the normal is used to assist in the discrimination and correction of the ambiguity of the azimuth "π" in the polarization normal. Subsequently, the Frankot-Chellappa algorithm is used to integrate the corrected normal field to finally obtain the polarization three-dimensional reconstruction result with concavity and convexity correction.

[0091] (4) Since the 3D topography obtained by integrating the polarization normal is only a relative depth distribution, the absolute depth of the optical field is further introduced as prior information to construct a joint convex optimization model that integrates the polarization normal and the optical field depth. By introducing a global depth guiding term and surface continuity constraints, higher accuracy in 3D topography reconstruction is achieved.

[0092] It should be noted that in the microscopic imaging system targeted by this invention, polarization images and Fourier light field images are acquired by separate channels.

[0093] Specifically, step S100 mainly includes three steps: polarization image acquisition, S102 polarization parameter solution, and S103 normal vector calculation. In the S101 image acquisition stage, by adjusting the rotation angle of the linear polarizer, polarization image sequences of the sample in four directions (0°, 45°, 90°, and 135°) are acquired sequentially. The Olympus microscope's built-in depth-of-field extension mode is used in the experiment to obtain full depth-of-field images. Subsequently, in the S102 polarization parameter solution stage, for each pixel, a polarization modulation curve is constructed by fitting the light intensity of multiple frames, and two key parameters, polarization phase angle φ and polarization degree ρ, are extracted according to Malus's law. The polarization phase angle φ represents the projection direction of the reflected photoelectric vector direction onto the image plane and is used to calculate the surface normal azimuth angle. The degree of polarization ρ reflects the proportion of polarization components in the reflected light. It is affected by both surface roughness and reflection angle, and can be used to invert the zenith angle θ of the surface normal.

[0094] Specifically, in polarization 3D reconstruction, the solved polarization normal azimuth angle There is an inherent ambiguity surrounding π, namely or The normal directions represented by these vectors are the same on the projection perpendicular to the image plane, making it difficult to calculate a unique value from the polarization information itself. This leads to a "bump reversal" shape error in the reconstructed depth map. To solve this problem, this invention introduces the light field reconstructed depth map Z obtained from Fourier light field 3D reconstruction. f As geometric prior information, it is used to guide the correction of the polarization normal azimuth angle.

[0095] Step S300 mainly includes view matching and normal azimuth correction:

[0096] S301, Vision matching includes:

[0097] The largest inscribed square region was selected as the registration target region (ROI). The XFeat algorithm was used to assess feature matching performance. This algorithm automatically selects reference targets with structural consistency in the two images for corresponding point marking, and combines an affine transformation model to perform geometric alignment of the polarization images, including scale adjustment, position translation, and minor rotation correction operations, to achieve spatial mapping from the polarization image coordinate system to the ROI region of the light field image.

[0098] After completing the above registration operation, a set of affine matrices T describing the spatial mapping relationship between the polarization image and the light field image can be obtained. reg The first row contains scaling, rotation, and translation coefficients along the x-axis; the second row contains the corresponding parameters along the y-axis; and the third row contains the normalization term for the homogeneous coordinate affine transformation. To achieve scale uniformity, a bicubic interpolation algorithm is further used to upsample the light field depth image, ensuring that it maintains the same resolution as the registered polarization image.

[0099] For example, using the depth map reconstructed from the Fourier light field image as a reference, the largest inscribed square region is selected as the registration target region (ROI), with a size of 350×350 pixels, while the original polarization channel image resolution is 4104×2174. To achieve high-precision registration, this invention employs the XFeat algorithm, which boasts superior feature matching performance. This algorithm automatically selects reference targets with structural consistency in the two images for corresponding point marking, and combines an affine transformation model to perform geometric alignment of the polarization images, including scale adjustment, position translation, and minor rotation correction operations, achieving spatial mapping from the polarization image coordinate system to the ROI region of the light field image, such as... Figure 4 As shown.

[0100] After completing the above registration operation, a set of affine matrices T describing the spatial mapping relationship between the polarization image and the light field image can be obtained. reg Its expression is as follows:

[0101]

[0102] The first row contains scaling, rotation, and translation coefficients along the x-axis; the second row contains the corresponding parameters along the y-axis; and the third row contains the normalization terms for the homogeneous coordinate affine transformation. To achieve scale uniformity, a bicubic interpolation algorithm is further used to upsample the light field depth image, ensuring it maintains the same resolution as the registered polarization image. The final size is uniformly set to 1400×1400 pixels. The registration result is shown below. Figure 5 As shown.

[0103] S302, Normal azimuth correction includes:

[0104] After completing the polarization-Fourier light field image field registration, in order to assist in correcting the polarization normal azimuth angle... The π ambiguity problem requires the depth map Z reconstructed from the Fourier light field. f The azimuth angle of the corresponding light field normal is derived from (x,y). For the depth map Z of light field reconstruction f For any pixel q = (x, y) on the surface, and considering its neighborhood as a parametric surface, the corresponding normal vector can be calculated using the gradient of this surface, and its unnormalized surface normal vector n f (q) is usually written as:

[0105]

[0106] The first and second components of the above equation represent the first-order rate of change of depth in the x and y directions, while the third component includes not only the depth itself, Z. f (q) also includes additional terms introduced due to the changes in the horizontal coordinates (x-x0) and (y-y0). Furthermore, the normal vector can be converted into the azimuth angle of the light field normal using spherical coordinates. The equation for the zenith angle θ is:

[0107]

[0108] and θ f (q) represents the azimuth and zenith angles of the light field normal at the pixel location, respectively. Combining these two formulas, the azimuth angle can be directly calculated. With zenith angle θ f (q). Wherein, for the azimuth angle... The solution can be determined by the projection of the unit normal vector onto the x-y plane, and is commonly represented by the arctan 2 function:

[0109]

[0110] in and These represent the partial derivatives with respect to x and y, respectively. After the above calculations, the depth map Z... f The azimuth angle of the light field normal can be obtained for each pixel. Subsequently, the reference azimuth angle and the azimuth angle obtained from the polarization channel were used. Perform pixel-by-pixel comparison.

[0111] If the angle between the two exceeds This indicates that the azimuth direction obtained from polarization is inconsistent with the optical field verification, and the normal azimuth angle needs to be adjusted. Revised to:

[0112]

[0113] S303, Corrected normal azimuth angle The surface normal is reconstructed using the zenith angle θ obtained from the original polarization reconstruction, and then the Frankot–Chellappa algorithm is used to perform normal integration to recover the final 3D topography. The specific steps are as follows:

[0114] First, convert the zenith angle and azimuth angle into unit normal vectors. Right now:

[0115]

[0116] Let p and q be the gradient fields of the normal in the x and y directions, respectively, which can be expressed as:

[0117]

[0118] The surface height function z(x,y) can be viewed as a function expansion composed of a set of orthogonal Fourier functions:

[0119] z(x,y)=∑ ω c(ω x ,ω y )·Φ(x,y;ω x ,ω y );

[0120] Where Φ(x,y;ω) x ,ω y ) are two-dimensional Fourier orthogonal basis functions, in the form of:

[0121]

[0122] coefficient c(ω) x ,ωy ) represents the projection intensity of the basis function in z(x,y), and is the Fourier coefficient to be determined.

[0123] In Fourier space, the derivative energy of the basis functions with respect to x and y is defined as follows:

[0124]

[0125] By using frequency domain projection, let By performing a Fourier transform, we can obtain the expression for the Fourier coefficients:

[0126]

[0127] Substituting the projection results into the basis function expansion of z, and combining the expressions, we can obtain the final frequency domain integral form:

[0128]

[0129] Figure 6 This demonstrates a comparison of the 3D reconstruction results on the microsphere sample before and after correction. Figure 6 It is evident that the polarization reconstruction before correction exhibits a significant local "morphological inversion" phenomenon, where structures that should be convex are incorrectly reconstructed as concave. However, after azimuth correction, the morphology of this region is significantly improved, and the structural transition is more natural and coherent, verifying the effectiveness of this method in eliminating polarization ambiguity.

[0130] Specifically, as mentioned above, the depth information reconstructed using the Fourier light field effectively corrects the π ambiguity problem of the normal azimuth angle in polarization reconstruction, improving the topography reversal problem caused by incorrect normal direction in polarization 3D reconstruction. However, since polarization reconstruction essentially obtains the surface topography from the normal field integral, it cannot obtain the true depth that matches the object space. While the Fourier light field reconstruction results have limited spatial resolution, they can provide relatively accurate absolute depth information. Therefore, in step S400, the fusion optimization adopts a convex optimization depth fusion strategy. By constructing a convex optimization objective function containing multiple constraints, it seeks the optimal solution between the high-frequency details provided by polarization and the depth reference provided by the light field, and solves it using a variational model, specifically including:

[0131] Let Z be the target depth map obtained by the final optimization solution, and let Z be the depth map obtained by light field reconstruction. f The depth gradient calculated from the normal after polarization reconstruction correction is: Construct the following convex optimization objective function:

[0132]

[0133] Where λ1, λ2, and λ3 are the weighting factors among the three terms. The first term is the global depth consistency term, which controls the consistency between the overall geometric shape of the fusion result and the depth of the light field reconstruction; the second term is the local gradient constraint term, which preserves the matching between the fused morphology and the polarization estimation normal in the gradient direction; the third term is the regularization term. This method is used to suppress high-frequency noise and boundary deformation commonly found in integral reconstruction, thereby enhancing the smoothness and coherence of the surface morphology. Two regularization terms are employed to accommodate different morphological features, and a weighting factor α∈[0,1] is used to control the trade-off between smoothness and edge preservation.

[0134] (1) Surface continuity regularization term: This term is often used in the regularization constraints of surface reconstruction to adjust the smoothness of the surface.

[0135]

[0136] (2) TV regularization: This item is often used for edge extraction and is suitable for high-frequency areas that preserve details.

[0137]

[0138] The final regular expression is expressed as:

[0139]

[0140] The objective function described above is a constrained convex optimization problem, which can be solved iteratively. The Split-Bregman multiplier method is then used as the optimization algorithm.

[0141] In some optional embodiments provided by the present invention, the normal zenith angle θ and normal azimuth angle of each pixel are calculated based on the polarization inversion model. (S103) includes:

[0142] For each pixel, based on Malus's law, a polarization modulation curve is constructed by fitting the maximum and minimum light intensity values ​​received by the detector within one rotation cycle of the polarizer. From this curve, the polarization phase angle φ and polarization degree ρ are extracted. Based on the mapping relationship between polarization degree ρ and the normal zenith angle θ, step S103 is performed to solve for the normal zenith angle θ and the polarization normal azimuth angle. Azimuth of polarization normal The polarization phase angle φ satisfies or

[0143] The mapping relationship between the degree of polarization ρ and the normal zenith angle θ satisfies Equation 7:

[0144]

[0145] In Equation 7, n is the refractive index of the object.

[0146] Specifically, in S101, in the polarization imaging branch, the depth-of-field extension mode built into the Olympus microscope is used to acquire full depth-of-field image sequences at four polarization angles: 0°, 45°, 90°, and 135°. Subsequently, in the polarization parameter solving stage (S102), for each pixel, a polarization modulation curve is constructed using simplified linear fitting based on the light intensity of multiple frames.

[0147]

[0148] Where, θ pol I is the polarization angle, φ is the polarization phase angle, and I max and I min These represent the maximum and minimum light intensity values ​​received by the detector during one rotation cycle of the polarizer, respectively. The polarization modulation curve is shown below. Figure 7 As shown, for any pixel B on the image plane (e.g. Figure 7 As shown in Figure (a), multiple images with different polarization angles (such as...) can be obtained. Figure 7 The intensity values ​​shown in Figure (c) are used to fit a polarization modulation curve containing surface polarization information (as shown in Figure (c)). Figure 7 (as shown in Figure (b)).

[0149] Simultaneously, the degree of polarization ρ is obtained by the following formula:

[0150]

[0151] S103, Solve for the polarization normal azimuth angle The normal zenith angle θ describes the angle between the surface normal vector and the image plane normal (z-axis). Its solution is closely related to the polarization degree ρ of the outgoing light. The polarization degree ρ describes the proportion of polarized components in the outgoing light. A mapping relationship between the polarization degree ρ and the normal zenith angle θ can be established through a physical model, thus indirectly solving for the normal zenith angle θ. Its expression is:

[0152]

[0153] Where n is the refractive index of the three-dimensional object. Due to its monotonicity and continuity, the normal zenith angle θ can be solved by using the degree of polarization ρ.

[0154] For the polarization normal azimuth angle Its value is equal to φ, but due to the periodicity of trigonometric functions, the estimated polarization normal azimuth angle is... There is uncertainty regarding π, that is... and Both solutions satisfy the conditions. Further prior depth information needs to be introduced to further determine and correct the normal direction.

[0155] In some optional embodiments provided by the present invention, performing Fourier light field three-dimensional reconstruction based on images acquired by the Fourier light field imaging branch includes:

[0156] The imaging response of the sample object at different depth positions was recorded. PSF slices were acquired at fixed intervals, and each slice was preprocessed sequentially, including background subtraction and binarization calibration, to generate the measured three-dimensional point spread function data h of the microscopic imaging system. 3D (x″,y″,z″), based on an ideal microscopic imaging model, the light intensity distribution g(x,y,z) of the three-dimensional object to be reconstructed is obtained by using the RL deconvolution algorithm.

[0157] It should be noted that, regarding the 3D reconstruction of the Fourier light field and the recording of the imaging response of an object at different depths, this embodiment uses a 0.2 μm diameter polystyrene microsphere as the point spread function (PSF) measurement sample. Since the microsphere's size is much smaller than the system's spatial resolution, it can be approximated as an ideal point scattering source. An isolated microsphere is selected in the field of view, and Fourier light field images are acquired layer by layer along the axial direction from 4 μm below the focal plane to 4 μm above it, with a step size of 0.2 μm. The imaging response of the microsphere at different depths is recorded, and a total of 41 PSF slices are acquired at intervals. Subsequently, preprocessing operations such as background subtraction and binarization calibration are performed on each slice sequentially to finally generate the system's measured 3D point spread function (3DPSF) data h. 3D (x″, y″, z″). Since the ideal microscopic imaging model can be expressed as:

[0158] O(x″,y″)=∫h 3D (x″,y″,z″)g(x,y,z)dz;

[0159] Among them, h 3D (x″,y″,z″) represents the known 3D PSF kernel, and g(x,y,z) represents the light intensity distribution of the 3D object to be reconstructed. Therefore, the Richardson–Lucy (RL) deconvolution algorithm is used for calculation. The specific steps are as follows:

[0160] (i) Initialization: Let the initial 3D object estimate be g. (0) (x,y,z) uses a uniform small field to simply stretch the original light field image into a three-dimensional volume data as an initial guess.

[0161] (ii) Forward projection: Based on the object estimate g in the current k-th iteration (k) (x,y,z), the simulated imaging process yields the predicted camera planar image. Mathematically, this step is for g(k) Perform convolution projection, that is, for each axial slice z, g (k) (x,y,z) and the point spread function h at the corresponding depth 3D Convolve (x″, y″; z) and then sum over z. The formula is expressed as:

[0162]

[0163] in, This represents a two-dimensional convolution operation on the (x,y) plane. This refers to a simulated camera image generated based on the currently estimated object.

[0164] (iii) Calculate the residual ratio: Compare the actual obtained camera image O(x″,y″) with the simulated image Divide pixel by pixel to obtain the ratio image:

[0165]

[0166] This ratio reflects the under-probability of the current estimate at each pixel. When R > 1, it means that the estimated image is insufficiently bright at that pixel, indicating that the object estimation needs to enhance the intensity of the corresponding depth; when R < 1, it means that the estimated image is too bright.

[0167] (iv) Back projection correction: Adjust the residual ratio R (k) (x″,y″) is solved back to object space using PSF, and g (k) Corrections are made. Since the image is formed by the superposition of convolutions at various depths, during backprojection, the ratio image needs to be convolved with the conjugate of the PSF and then distributed to each depth. For each candidate object voxel (x, y, z), the correction factor Δg is calculated. (k) (x,y,z) is:

[0168]

[0169] in, This represents a 180° rotation of the point spread function in the (x″,y″) plane, i.e., the transpose of the imaging matrix H. Errors are distributed back to individual voxel locations by convolving the residual ratio image with the PSF, using the residual ratio as weights.

[0170] (v) Iterative update: Multiply the original object estimate by the above correction factor to obtain the object estimate for the next iteration:

[0171] g (k+1) (x,y,z)=g (k) (x,y,z)×Δg (k) (x,y,z);

[0172] This multiplicative update ensures non-negativity and, according to maximum likelihood theory, increases the likelihood of the result. After 200 iterations, g (k) It will gradually converge to the point where the simulated image The three-dimensional distribution that best matches the actual image O.

[0173] The above iterative process based on RL deconvolution can be simplified to matrix form:

[0174] g (k+1) =diag[diag(H T Hg (k) ) -1 (H T O)]g (k) ;

[0175] Here, the operator diag[·] indicates diagonalization of the matrix, H is the PSF matrix of the imaging system, H T It is its transpose. The RL algorithm guarantees the likelihood function in each iteration, that is, the likelihood of the observed image relative to the current object estimate continuously increases until convergence.

[0176] In some optional embodiments provided by the present invention, the depth conversion factor Δz of the microscopic imaging system is obtained. 物空间 ,include:

[0177] In the PSF slice, the layer with the highest brightness is selected as the image space location corresponding to the geometric center. Analysis is then performed layer by layer upwards from the layer with the highest brightness, until the m-th layer... i The top reflection signal was detected for the first time in layer m, determining the m-th layer. i Layers correspond to the positions of the highest points of a three-dimensional object in image space.

[0178]

[0179] Where R is the radius of the three-dimensional object.

[0180] Specifically, to achieve the ability to acquire absolute depth information of a sample using Fourier light field microscopy, it is necessary to establish a conversion relationship between the number of image space layers and the actual depth of the object space. Based on this, a depth conversion factor Δz for the system is established. 物空间 This refers to the actual object space depth corresponding to each deconvolutional image layer in the image space. The specific calibration method is as follows:

[0181] A calibration sphere is selected, and the true height D of the sample is obtained using an atomic force microscope, thereby determining the radius R of the sample sphere. 样品 =D 样品 / 2. In the obtained slices, the layer with the highest brightness is first selected as the image space location corresponding to the geometric center of the microsphere. Then, the analysis proceeds layer by layer upwards from this layer, until the m-th layer... iThe layer first detected a top reflection signal, determining its position in image space corresponding to the highest point of the sphere. Based on this, the system's depth conversion factor is established as follows:

[0182]

[0183] For the microsphere to be detected, the signal first appears above the core layer m through reflections from the tops of multiple microspheres. j The corresponding depth h after conversion should be:

[0184] h = m j ×Δz 物空间 ;

[0185] In this embodiment, 2 μm polystyrene microspheres were used as reference samples. Their positions and orientations were marked during the preparation process, and their true height relative to the mica substrate was measured to be 2.168 μm using atomic force microscopy (AFM). The vertical distance from the top of the microsphere to the largest cross-section is the radius, 1.084 μm. In the 41-layer slice obtained by deconvolution, the layer with the highest brightness was first selected as the image space position corresponding to the geometric center of the microsphere. Subsequently, layer-by-layer analysis was performed upwards from this layer. The top reflection signal was first detected in the 9th layer, determining its position in image space corresponding to the highest point of the microsphere. The depth conversion factor of the system was thus established as follows:

[0186]

[0187] That is, the actual object space depth corresponding to each layer of the deconvolution image is approximately 0.1204 μm. This conversion factor can be used as a benchmark for the reconstruction depth of the system, to map the layers of all stacked images to absolute depth values ​​with physical dimensions.

[0188] In some optional embodiments provided by the present invention, step S400, employing the Split-Bregman multiplier method as the optimization algorithm, includes:

[0189] Introducing auxiliary variable d x d y Replace the gradient terms of the depth map in the x and y directions, where:

[0190]

[0191] Introducing the Bregman multiplier b x b y The convex optimization objective function is transformed into the following Lagrangian form:

[0192]

[0193] The Split-Bregman algorithm is used to solve the problem.

[0194] Specifically, the Split-Bregman multiplier method is used as an optimization algorithm, including:

[0195] Introducing auxiliary variable d x d y The gradient terms of the depth map in the x and y directions are replaced, i.e.:

[0196]

[0197] Simultaneously, the Bregman multiplier b is introduced. x b y The original convex optimization objective function is transformed into the following Lagrangian form:

[0198]

[0199] Finally, the solution process of the Split-Bregman algorithm is as follows:

[0200] Input: Light field depth map Z f Polarization estimation gradient Hyperparameters λ1, λ2, λ3, α, convergence threshold ε;

[0201] Output: The fused and optimized 3D reconstructed depth map z;

[0202] (i) Initialize variables.

[0203] Let Z0 = Z f d x =d y =0, b x =b y =0, iteration step k=0.

[0204] (ii) Constructing the variational Lagrange form:

[0205]

[0206] (iii) Solve iteratively until ||Z|| is satisfied. k+1 -Z k ||<ε:

[0207] (1) Update d x ,d y :

[0208]

[0209] (2) Update the Bregman multiplier b x b y :

[0210]

[0211] (3) Update the main variable Z:

[0212]

[0213] (4) Output the final optimization result Z k+1 .

[0214] The reconstruction target was a 5μm densely packed polystyrene microsphere sample with continuous structure and varied morphology. The hyperparameters were configured as follows: λ1 = 0.02, λ2 = 0.5, λ3 = 0.1, and the regularization weighting factor α = 0.5. The fusion process is described in [link to fusion process]. Figure 3 The evaluation metrics after fusion are shown in Table 1. The reconstruction performance of different methods was compared and analyzed using four metrics: structural similarity (SSIM), mean square error (MSE), mean absolute error (MAE), and root mean square error (G-RMSE). The bolded parts in Table 1 represent the optimal results for each metric. As can be seen from Table 1, the fusion reconstruction achieved the best performance across all four evaluation metrics, and its overall reconstruction quality is significantly superior to other methods.

[0215] Table 1

[0216] method SSMI MSE MAE G-RMSE Polarization 3D Reconstruction 0.54 620.72 24.92 4.43 Fourier light field three-dimensional reconstruction 0.63 445.85 19.61 2.51 Integration and Reconstruction 0.75 257.40 9.71 1.98

[0217] The core of this invention focuses on technological innovation in dual-modal data fusion and accurate reconstruction. Its core lies in constructing a joint application framework for polarization microscopy and Fourier light field imaging. Through a specific process, it achieves complementary advantages between the two types of data, forming a unique 3D reconstruction solution. Key protection points include an innovative correction method for the polarization normal azimuth angle, which derives the reference azimuth angle from the depth map reconstructed from the Fourier light field, specifically addressing the inherent "π ambiguity" problem in polarization imaging. Simultaneously, it employs a multi-constraint convex optimization fusion model. This model integrates global depth consistency constraints, local gradient constraints, and a hybrid regularization term composed of surface continuity regularization and TV regularization, achieving a balance between high-frequency detail and smoothness.

[0218] As can be seen from the above embodiments, the polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth provided by the present invention achieves at least the following beneficial effects:

[0219] 1. This invention derives the reference azimuth angle from the prior depth provided by the Fourier light field and corrects the “π ambiguity” of the polarization normal pixel by pixel, effectively avoiding misjudgment of the direction of the reconstructed morphology. It successfully breaks through the ill-conditioned limitations of traditional polarization imaging and solves the core technical problem that has long existed in this field.

[0220] 2. By combining the high-frequency detail capture capability of polarization imaging with the absolute depth measurement advantage of Fourier light field, it not only makes up for the shortcoming of polarization imaging lacking absolute depth perception, but also alleviates the problem of insufficient lateral resolution of light field imaging, thus breaking through the performance bottlenecks of each single-mode technology.

[0221] 3. The hybrid regularization term introduced by the convex optimization model can effectively suppress noise and boundary deformation, while ensuring overall geometric consistency through global depth constraints, making the reconstruction results more accurate and continuous, and greatly improving the accuracy and stability of 3D reconstruction.

[0222] 4. The RL deconvolution algorithm for Fourier light field is based on the measured three-dimensional point spread function (3DPSF). It simplifies the computational complexity while improving the consistency of axial resolution. Compared with traditional light field reconstruction, it reduces artifacts and data redundancy, and balances reconstruction efficiency and imaging quality.

[0223] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A polarization microscopic three-dimensional reconstruction algorithm that integrates Fourier light field depth, characterized in that, The polarization microscopic three-dimensional reconstruction algorithm is used in a microscopic imaging system containing independent polarization imaging branches and Fourier light field imaging branches. The polarization microscopy three-dimensional reconstruction algorithm includes: In the polarization imaging branch, by adjusting the rotation angle of the polarizer, image sequences at four polarization angles of 0°, 45°, 90°, and 135° are acquired respectively. Based on the polarization inversion model, the normal zenith angle θ and polarization normal azimuth angle of each pixel are calculated. Based on the images acquired by the Fourier light field imaging branch, Fourier light field three-dimensional reconstruction is performed to obtain the depth conversion factor Δz of the microscopic imaging system. 物空间 Based on the depth conversion factor and the light intensity distribution of the 3D object to be reconstructed, a light field reconstruction depth map Z with absolute depth information is obtained. f ; Spatially register the fields of view of the polarization imaging branch and the Fourier light field imaging branch, and reconstruct the depth map Z based on the light field. f Derive the corresponding azimuth angle of the light field normal. The polarization normal azimuth angle in the polarization imaging branch Discrimination and correction are performed; the Frankot–Chellappa algorithm is used to integrate the corrected polarization branch normal field to obtain a three-dimensional polarization reconstruction image with concavity / convexity correction, wherein: Spatial registration of the field of view of the polarization imaging branch and the Fourier light field imaging branch includes: reconstructing the depth map Z using the light field. f As a reference, the region with the largest inscribed square is selected as the registration target region. The XFeat algorithm is used to realize the mapping from the polarization image coordinate system to the light field reconstruction depth map Z. f The spatial mapping of the registration target region in the image yields a set of affine matrices T describing the spatial mapping relationship between the polarization image and the light field image. reg , wherein the affine matrix T reg In the diagram, the first row contains scaling, rotation, and translation coefficients along the x-axis; the second row contains the corresponding parameters along the y-axis; and the third row contains the normalization terms for the homogeneous coordinate affine transformation. A bicubic interpolation algorithm is used to reconstruct the depth map Z from the light field. f Sampling processing is performed to reconstruct the depth map Z of the light field. f It maintains the same resolution as the registered polarization image; The depth map Z reconstructed based on the light field f Derive the corresponding azimuth angle of the light field normal. include: Select the light field reconstruction depth map Z f For any pixel q = (x, y) on the surface, the neighborhood of the pixel is considered as a parametric surface, and the azimuth angle of the light field normal is... According to Equation 1: Repeat the above light field normal azimuth angle The calculation yields the light field reconstruction depth map Z. f The corresponding light field normal azimuth angle of each pixel The azimuth angle of the light field normal With respect to the polarization normal azimuth angle Perform pixel-by-pixel comparison, normal azimuth angle The correction should be made according to Equation 2: The corrected normal azimuth angle The surface normal is reconstructed by the zenith angle θ of the normal, and the normal integral is completed by the Frankot-Chellappa algorithm to obtain a polarization 3D reconstruction map with concavity and convexity correction. A convex optimization objective function with multiple constraints is constructed, and the Split-Bregman multiplier method is used as the optimization algorithm to obtain the optimal solution, which is the target depth map Z; including: The convex optimization objective function is: in, The depth gradient is calculated from the corrected polarization normal. λ1 represents the weight factor of the global depth consistency term; λ2 is the local gradient constraint term, and λ2 is the weighting factor of the local gradient constraint term; λ3 is the regularization term, and λ3 is the weight factor of the regularization term; Calculate according to equations 4-6: in, For surface continuity regularization, For TV regularization terms; α∈[0,1].

2. The polarization microscopy three-dimensional reconstruction algorithm according to claim 1, characterized in that, The normal zenith angle θ and polarization normal azimuth angle of each pixel are calculated based on the polarization inversion model. include: For each pixel, based on Malus's law, a polarization modulation curve is constructed by fitting the maximum and minimum light intensity values ​​received by the detector within one rotation cycle of the polarizer. Using this polarization modulation curve, the polarization phase angle φ and polarization degree ρ are extracted. Based on the mapping relationship between the polarization degree ρ and the normal zenith angle θ, the normal zenith angle θ and the polarization normal azimuth angle are solved. Among them, the polarization normal azimuth angle The polarization phase angle φ satisfies or The mapping relationship between the degree of polarization ρ and the normal zenith angle θ satisfies Equation 7: In Equation 7, n is the refractive index of the three-dimensional object.

3. The polarization microscopy three-dimensional reconstruction algorithm according to claim 1, characterized in that, The Fourier light field three-dimensional reconstruction based on the images acquired by the Fourier light field imaging branch includes: The imaging response of a three-dimensional object at different depth positions is recorded. PSF slices are acquired at fixed intervals, and each slice is preprocessed sequentially, including background subtraction and binarization calibration, to generate the measured three-dimensional point spread function data h of the microscopic imaging system. 3D (x″,y″,z″), based on an ideal microscopic imaging model, the light intensity distribution g(x,y,z) of the three-dimensional object to be reconstructed is obtained by using the RL deconvolution algorithm.

4. The polarization microscopy three-dimensional reconstruction algorithm according to claim 3, characterized in that, The depth conversion factor Δz of the microscopic imaging system is obtained. 物空间 ,include: In the PSF slice, the layer with the highest brightness is selected as the image space location corresponding to the geometric center. Analysis is then performed layer by layer upwards from the layer with the highest brightness, until the m-th layer... i The top reflection signal was detected for the first time in the layer, determining the m-th layer. i Layers correspond to the positions of the highest points of a three-dimensional object in image space. Among them, R 样品 The radius of the three-dimensional object.

5. The polarization microscopy three-dimensional reconstruction algorithm according to claim 1, characterized in that, The method of using the Split-Bregman multiplier method as the optimization algorithm to obtain the optimal solution includes: Introducing auxiliary variable d x d y Replace the gradient terms of the depth map in the x and y directions, where: Introducing the Bregman multiplier b x b y The convex optimization objective function is transformed into the following Lagrangian form: The Split-Bregman method is used to solve the problem.

Images