Dark field aperture dislocation correction method for reflective Fourier lamination microscopy

Abstract

The invention discloses a dark field aperture dislocation correction method for reflective Fourier lamination microscopy, which comprises the following steps of: based on a mapping relation between an LED (light-emitting diode) and a frequency domain aperture, positioning a bright field aperture, correcting aberration, calculating and comparing structural similarity (SSIM) to obtain a first circle of dark field aperture center correction result, eliminating abnormal points in the first circle of dark field aperture center correction result by using an RANSAC circle fitting algorithm, and correcting the second circle of dark field aperture center correction result; and through linear extrapolation based on an LED arrangement rule, the position of a dark field aperture in a frequency domain is positioned. According to the method, the problem that an aperture correction algorithm based on a transmission-type FPM system cannot adapt to a reflection-type FPM system is solved, object dark field information does not need to be iteratively reconstructed in the correction process, the positioning process of each aperture is mutually independent and can be processed in parallel, rapid positioning of the dark field aperture of the reflection-type FPM system is achieved, and the method is suitable for large-scale popularization and application. And a foundation is laid for applying the reflective FPM technology to high-precision, high-throughput, low-cost and quantitative industrial detection.

Description

Technical Field

[0001] This invention belongs to the field of optical microscopy imaging technology, specifically a dark-field aperture misalignment correction method for reflective Fourier layered microscopy. Background Technology

[0002] In many key fields such as life science research, clinical case diagnosis, and industrial non-destructive testing, there is an urgent need for imaging methods that balance large field of view and high resolution to acquire more comprehensive structural and functional data without sacrificing detailed information. However, traditional microscopic imaging systems are constrained by Lagrange invariants, making it difficult to expand the spatial bandwidth product (SBP) of the system, resulting in limited information acquisition and severely restricting the efficiency and capability of high-throughput detection and accurate diagnosis. To overcome this physical bottleneck, methods such as spatial scanning stitching and interferometric synthetic aperture have been proposed. These methods can improve the system's SBP to some extent, but they usually rely on complex optical structures and high-precision mechanical control, leading to complex system design and significantly increased costs, making it difficult to meet the comprehensive requirements of stability and efficiency in practical applications. Fourier stacked microscopy (FPM), as an important breakthrough in the field of computational optics, provides a solution for achieving high SBP imaging without significantly increasing hardware complexity. By controlling multi-angle parallel light illumination, it enables objects to form different diffraction patterns on the back focal plane of the objective lens under different illuminations, and achieves the synthesis of multiple sub-apertures in the frequency domain, ultimately reconstructing a high-resolution image far exceeding the intrinsic numerical aperture of the objective lens.

[0003] Depending on the detection requirements of samples with different characteristics, FPM systems can be divided into transmissive and reflective types. The former is suitable for transparent or weakly absorbing samples, while the latter is for opaque or highly reflective materials such as metals, semiconductors, and ceramics, offering broader material compatibility. However, compared to the continuous LED light source arrangement and easily controllable illumination parameters in transmissive mode, reflective FPM faces more stringent calibration challenges. Its typical characteristic lies in the lack of spatial continuity and structural correlation between bright-field and dark-field light sources. Each LED illumination angle corresponds to a sub-aperture in the frequency domain; errors in light source position directly lead to spectral mismatch, which in turn introduces phase mismatch and information overlap during spectral stitching, significantly reducing reconstruction quality. This spectral misalignment problem is particularly prominent in reflective FPM, becoming a key bottleneck restricting its high-resolution imaging capabilities. Although aperture correction algorithms based on transmissive and reflective systems exist, the former heavily relies on the assumption of light source continuity, making it difficult to apply to reflective imaging modes with separate bright-field and dark-field light sources. The latter's correction strategy still requires synchronous iterative reconstruction of complete object information, without processing the correction process independently, increasing the algorithm's complexity. Summary of the Invention

[0004] The purpose of this invention is to propose a dark field aperture misalignment correction method for reflective Fourier stacked microscopy, which solves the problem of the incompatibility of traditional correction algorithms based on transmissive FPM systems in reflective FPM systems, and achieves stable and efficient correction of the dark field aperture position in the frequency domain.

[0005] The technical solution to achieve the purpose of this invention is: a dark-field aperture misalignment correction method for reflective Fourier layer microscopy, comprising:

[0006] Step 1: Acquire bright-field and dark-field light intensity maps using a reflective Fourier layered microscope system;

[0007] Step 2: Perform threshold denoising on the dark field intensity map;

[0008] Step 3: Use an edge detection algorithm to locate the bright-field aperture, and use an adaptive aberration compensation algorithm to correct aberrations and reconstruct bright-field object information simultaneously;

[0009] Step 4: Calculate and predict the dark field intensity map based on the corrected bright field spectrum;

[0010] Step 5: Calculate the SSIM value between each denoised dark field intensity map and the corresponding predicted dark field intensity map, compare and search for the global maximum SSIM value. The frequency domain coordinates corresponding to the maximum SSIM value are the true frequency domain aperture center coordinates. Construct a set of true frequency domain aperture center coordinates.

[0011] Step 6: Use the RANSAC circle fitting algorithm to remove outliers from the set of true frequency domain aperture center coordinates;

[0012] Step 7: Based on the arrangement of LEDs, obtain the center coordinates of the remaining frequency domain aperture by linear extrapolation.

[0013] Compared with the prior art, the present invention has the following significant advantages: (1) Compared with the traditional correction scheme, the present invention can correct the dark field aperture separately without relying on the continuity of the LED arrangement. (2) There is no need to iteratively reconstruct the dark field information of the object during the correction process, and the correction of each aperture is independent of each other and can be processed in parallel, with a faster correction speed.

[0014] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the working principle of a reflection Fourier layered microscopy system.

[0016] Figure 2 Flowchart of a dark-field aperture misalignment correction method for reflective Fourier layer microscopy

[0017] Figure 3Evaluation function residual distribution. (a) represents the 8 apertures being corrected. (b) represents the evaluation function residual distribution within the search range of each of the 8 apertures being corrected.

[0018] Figure 4 Imaging results of a standard USAF resolution plate. (a) Reconstructed intensity map of the standard USAF resolution plate after dark-field aperture correction using the present invention. (b) Reconstructed intensity map of the standard USAF resolution plate without dark-field aperture correction.

[0019] Figure 5 Imaging results of the chip. (a) Intensity map reconstructed by FPM using only bright-field information. (b) Intensity map reconstructed by FPM after adding dark-field information and performing aperture correction. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings.

[0021] This invention provides a method for correcting dark-field aperture misalignment in reflective Fourier layer microscopy, comprising the following steps:

[0022] Step 1: Acquire bright-field and dark-field light intensity maps using a reflective Fourier layered microscopy system.

[0023] In a further embodiment, the reflective Fourier multilayer microscopy system is as follows: Figure 1 As shown, it includes (1) a bright-field LED ring array; (2) a beam splitter; (3) a microscope objective; (4) a dark-field LED ring array; (5) the sample to be tested; (6) a tube mirror; and (7) a camera.

[0024] The specific implementation process is as follows: The bright-field LED ring array (1) is lit sequentially to provide a bright-field illumination beam. The beam is reflected by the beam splitter (2), illuminates the sample (5) through the microscope objective (3), and is then reflected. The reflected beam passes sequentially through the microscope objective (3), beam splitter (2), and tube mirror (6) and converges to the camera (7), acquiring N bright-field light intensity images. Subsequently, the dark-field LED ring array (4) is lit sequentially to provide a dark-field illumination beam, directly illuminating the sample (5). The reflected beam passes sequentially through the microscope objective (3), beam splitter (2), and tube mirror (6) and converges to the camera (7), acquiring M1+ M2+…+ M Q Zhang, the dark field LED ring array has Q rings, and the number of LEDs in each ring is M. q , q=1,2,...,Q.

[0025] Step 2: Threshold Denoising. Without turning on the illumination source, acquire a system noise map recorded by the camera. Take the average value of all pixel values ​​in the system noise map and use this value as a threshold to denoise the dark-field intensity map. Compare all pixel values ​​in the dark-field intensity map with the threshold; pixel values ​​greater than the threshold remain unchanged, while pixel values ​​less than the threshold are set to 0.

[0026] Step 3: Brightfield Aperture Localization and Aberration Correction. This process employs an adaptive aberration correction algorithm to iteratively reconstruct the brightfield spectrum and aperture function.

[0027] Step 3.1: Process the acquired brightfield intensity map Perform a Fourier transform to obtain the sub-spectrum corresponding to each intensity map. Define the initial aperture function. ,in The numerical aperture of the objective lens. This is the system's operating wavelength. Each LED generates a tilted plane wave at a different angle, resulting in different frequency domain shifts superimposed on the sub-spectrum corresponding to the intensity map. .

[0028] Step 3.2: Spectrum Initialization. Remove the frequency domain shift of all bright-field sub-spectrums, and then use the initial aperture function. Sampling of the sub-spectrum and superposition of all sampled values ​​yields the initial spectrum. The formula is:

[0029]

[0030] Step 3.3: Iterative reconstruction and updating of the bright-field spectrum. The acquired bright-field intensity maps are sequentially input into the corresponding frequency domain apertures. The neutron spectrum of the aperture before updating is... Updated aperture neutron spectrum The formula is:

[0031]

[0032] In the formula, Indicates Fourier transform, .

[0033] Substitute all updated sub-spectrums into the initial spectrum Each updated sub-spectrum is used as one iteration, and a total of iterations are performed. Round, the bright-field spectrum after each iteration is The aperture function is Finally, the reconstructed bright-field spectrum was obtained. and the corrected aperture function The iterative formula is:

[0034]

[0035] In the formula, Indicates conjugate.

[0036] The corrected aperture function includes a phase term obtained by weighted summation of Zernike polynomials. That is, the aberration of the aperture function, specifically expressed as:

[0037]

[0038] In the formula, The specific form is:

[0039]

[0040] In the formula, For specific terms of the Zernike polynomial, For each term, a weighted coefficient is used. Assuming the first K terms are used to represent aberrations, the K weighted coefficients are updated simultaneously. All weighted coefficients are updated synchronously with the bright-field spectrum and initialized to 0. The specific update formula is as follows:

[0041]

[0042] After each iteration, the cost function is calculated once as a convergence criterion. The cost function is:

[0043]

[0044] when When the iteration ends, Based on the actual system settings.

[0045] Step 4: Calculate and predict the dark field intensity map based on the corrected bright field spectrum. First, process the dark field intensity map collected from the first ring of dark field LEDs. Then, use the corrected bright field spectrum... Given the quantity, use the aperture function obtained in step 3. With step size Search for samples in the frequency domain, starting from the initial frequency domain position. Starting point ( (Based on actual system settings) Nine samples can be taken within the range. The sampled spectrum is subjected to inverse Fourier transform to obtain the predicted dark field complex amplitude. The square of the complex amplitude modulus is calculated to obtain the predicted dark field intensity map. The first ring of LEDs corresponds to M1 acquired dark field intensity maps. The above operation is performed on each acquired dark field intensity map. The formula for generating the predicted dark field intensity map is:

[0046]

[0047]

[0048] In the formula, For spatial coordinates, The corrected aperture function, The search step size in the frequency domain. This is used to collect the sequence number of the dark field light intensity map. Each time a search is completed... , .

[0049] Threshold denoising of the acquired dark field light intensity map yields , , ..., Each dark-field intensity map corresponds to a frequency-domain aperture center coordinate; the aperture radius is known. When searching for the coordinates of the aperture center in the frequency domain, the brightness of the predicted dark field intensity map is normalized based on the denoised dark field intensity map corresponding to that coordinate:

[0050]

[0051] In the formula, This is the predicted dark field intensity map after brightness normalization. and Equal brightness.

[0052] Step 5: Calculate and compare SSIM (Structural Similarity) values ​​and determine the actual aperture location. Calculate the intensity of each denoised dark-field image. The corresponding predicted dark field intensity map after brightness normalization The SSIM values ​​between the given values ​​are compared, and the global maximum SSIM value is searched. When this value is found, its corresponding frequency domain coordinates are the true frequency domain aperture center coordinates. The results are recorded as a set. .

[0053] Step 6: Remove outliers using the RANSAC circle fitting algorithm. The actual acquired dark-field light intensity map has a low signal-to-noise ratio and is easily affected by noise. Threshold denoising will filter out some effective information, causing the SSIM comparison method to malfunction. Therefore, based on the characteristic of the illumination source used in this invention being arranged in a circular pattern, the RANSAC circle fitting algorithm can remove possible outliers. The specific formula is as follows:

[0054] Randomly select a set From the three sets of coordinates in the equation, the equation of the circular locus passing through these three points can be calculated. The formula is:

[0055]

[0056] Let the coordinates be the center of the circle. Given the radius of the circle, calculate the Euclidean distance from the remaining coordinates to the center of the circle:

[0057]

[0058] When satisfied At this point, the point is considered an interior point, and the set of results with the most interior points is selected to fit the final circular trajectory equation. The corrected dark field aperture center falls on this circular trajectory equation. Thus, the first round of dark field frequency domain aperture center coordinate correction is completed.

[0059] Step 7: Based on the LED arrangement pattern, linearly extrapolate to obtain the remaining frequency domain aperture center coordinates. In a reflective Fourier layered microscope system, each dark-field LED corresponds to a dark-field intensity map, and each dark-field intensity map corresponds to a frequency domain aperture center coordinate. The arrangement trajectory of each ring of the dark-field LED ring array is a perfect circle, and the LEDs on each ring are distributed at equal angular intervals. Therefore, after determining the radius of each ring trajectory and the number of LEDs, the positions of the remaining LEDs can be obtained by linear extrapolation based on the corrected positions of the first ring of LEDs. The corresponding frequency domain aperture center coordinates can then be calculated based on the positions of the LEDs.

[0060] This invention is based on structural similarity comparison. It measures the structural similarity between the predicted and measured images using SSIM (Structural Similarity Model), and combines this with the RANSAC spatial fitting algorithm to robustly locate the frequency domain sub-aperture, thereby accurately deducing the position parameters of each dark-field LED. This method eliminates the need for iterative reconstruction, significantly improving the localization efficiency of dark-field sub-apertures while maintaining high accuracy. It provides crucial support for the rapid deployment and robust operation of reflective FPMs, expanding their application boundaries in complex material imaging and real-time detection.

[0061] Figure 1 This is a diagram illustrating the system's working principle.

[0062] Figure 2 This is a schematic diagram of the workflow of this method.

[0063] Figure 3 This is the residual distribution of the evaluation function. (a) represents the eight apertures being corrected. (b) represents the residual distribution of the evaluation function within the search range of each of the eight apertures being corrected. The residual distribution shows that it has a globally unique maximum value, and the frequency domain coordinates corresponding to this maximum value are the frequency domain center coordinates of the true aperture.

[0064] Figure 4 These are the imaging results of a standard USAF resolution plate. (a) is the reconstructed light intensity map of the standard USAF resolution plate after dark-field aperture correction using this invention. (b) is the reconstructed light intensity map of the standard USAF resolution plate without dark-field aperture correction. The box in the figure represents the ninth line pair in the standard resolution plate. A comparison of the two shows that this invention can effectively improve the reconstruction quality of FPM.

[0065] Figure 5These are the imaging results from the chip. (a) is the FPM-reconstructed intensity map using only bright-field information. (b) is the FPM-reconstructed intensity map after adding dark-field information and performing aperture correction. A comparison shows that adding dark-field information and performing iterative reconstruction at the correct aperture position results in clearer imaging details.

Claims

1. A method for correcting dark-field aperture misalignment in reflective Fourier layer microscopy, characterized in that, The application relates to a method for obtaining a real aperture center coordinate set of a dark field LED ring array of a reflection type Fourier ptychographic microscope system. Step 1: collecting bright field light intensity images and dark field light intensity images by using the reflection type Fourier ptychographic microscope system; Step 2: threshold denoising the dark field light intensity images; Step 3: positioning the bright field aperture by using an edge detection algorithm, correcting aberration by using an adaptive aberration compensation algorithm and synchronously reconstructing bright field object information; Step 4: calculating a predicted dark field light intensity image according to the corrected bright field spectrum; Step 5: calculating the SSIM value between each denoised dark field light intensity image and the corresponding predicted dark field light intensity image, comparing and searching for the global maximum SSIM value, and taking the frequency domain coordinate corresponding to the maximum SSIM value as the real frequency domain aperture center coordinate, and constructing a real frequency domain aperture center coordinate set; Step 6: removing abnormal values in the real frequency domain aperture center coordinate set by using a RANSAC circle fitting algorithm; Step 7: linearly extending the remaining frequency domain aperture center coordinates according to the arrangement rule of the LED.

2. The method for dark-field aperture misregistration correction for reflective Fourier ptychographic microscopy of claim 1, wherein, The reflection type Fourier ptychographic microscope system comprises a bright field LED ring array (1), a beam splitter (2), a microscope objective (3), a dark field LED ring array (4), a measured sample (5), a tube lens (6) and a camera (7), and the specific process of collecting the bright field light intensity images and the dark field light intensity images by using the reflection type Fourier ptychographic microscope system is as follows: the bright field LED ring array (1) is sequentially lighted to provide a bright field illumination light beam, the light beam is reflected by the beam splitter (2), passes through the microscope objective (3) to illuminate the measured sample (5) and is then reflected, the reflected light beam sequentially passes through the microscope objective (3), the beam splitter (2) and the tube lens (6) and is converged to the camera (7) to collect the bright field light intensity images; the dark field LED ring array (4) is sequentially lighted to provide a dark field illumination light beam, directly illuminates the measured sample (5), and the reflected light beam sequentially passes through the microscope objective (3), the beam splitter (2) and the tube lens (6) and is converged to the camera (7) to collect the dark field light intensity images.

3. The method for dark-field aperture misregistration correction for reflective Fourier ptychographic microscopy of claim 1, wherein, The specific method for threshold denoising the dark field light intensity images is as follows: in the case that the illumination light source is not turned on, an image is collected by using the camera as a system noise image; all pixel values in the system noise image are averaged to obtain a threshold value; all pixel values in the dark field light intensity image are compared with the threshold value, the pixel values greater than the threshold value are kept unchanged, and the pixel values less than the threshold value are taken as 0.

4. The method for dark-field aperture misregistration correction for reflective Fourier- laminated microscopy of claim 1, wherein, The specific method for positioning the bright field aperture by using the edge detection algorithm, correcting aberration by using the adaptive aberration compensation algorithm and synchronously reconstructing the bright field object information is as follows: Step 3.1: Fourier transform the collected bright-field light intensity maps to obtain the corresponding bright-field sub-spectra for each light intensity map; define an initial aperture function ; Step 3.2: Remove the frequency domain shift of all brightfield sub-spectra, use the initial aperture function Sample the sub-spectra, stack all the sample values to get the initial spectrum ; Step 3.3: The collected bright-field light intensity map is sequentially brought into the corresponding frequency domain aperture, and the sub-spectrum in the updated aperture is updated Specifically: ； wherein denotes the Fourier transform, , is the bright-field intensity map, is the neutron spectrum in the aperture before the update; all updated sub-spectrum bands into the initial spectrum , all updated sub-spectrum bands into as an iteration, the bright-field spectrum after each iteration is , the aperture function is , the final reconstructed bright-field spectrum is and the corrected aperture function is , and the iteration formula is: ； wherein represents conjugation; The corrected aperture function contains phase terms that are weighted sums of Zernike polynomials i.e. the aberrations of the aperture function, which are expressed as ； In the formulae, In a specific form, the compound of formula (I) is: ； wherein is a specific term of the Zernike polynomials, is a weighting coefficient for each term; assuming that the first K terms are used to represent the aberration, the K term weighting coefficients are simultaneously updated, all the weighting coefficients are synchronously updated with the bright field spectrum and are initialized as 0, and the specific updating formula is as follows: ； after each round of iteration, a cost function is calculated to serve as a convergence criterion, and the cost function is as follows: ； When the iteration is ended, according to the actual system settings.

5. The method for dark-field aperture misregistration correction for reflective Fourier- laminated microscopy of claim 1, wherein, The specific method for calculating the predicted dark field light intensity image according to the corrected bright field spectrum is as follows: first, the dark field light intensity image collected by the first circle of dark field LEDs is processed, including: with the corrected bright-field spectrum as known quantities, using the aperture function obtained in step 3 in steps of searching for samples in the frequency domain, with the initial frequency domain position as starting point according to the actual system setup, co-sampling nine times within the range the obtained spectrum is subjected to inverse Fourier transform to obtain a predicted dark field complex amplitude, and the square of the complex amplitude modulus is calculated to obtain a predicted dark field light intensity image; the first circle of LEDs corresponds to M1 collected dark field light intensity images, and the above operation is conducted on each collected dark field light intensity image to generate a predicted dark field light intensity image, and the specific operation is as follows: ； ； wherein is the spatial coordinate, is the corrected aperture function, is the search step in the frequency domain, is the acquisition dark-field intensity map number, each time a search is completed, , ; Each de-noised dark field light intensity map corresponds to a frequency domain aperture center coordinate, and when searching for a frequency domain aperture center coordinate, the de-noised dark field light intensity map corresponding to the frequency domain aperture center coordinate is taken as a reference to perform brightness normalization on the predicted dark field light intensity map. ； In the formula, is the normalized predicted dark-field light intensity map, and and the brightness is equal.

6. The method for dark-field aperture misregistration correction for reflective Fourier- ptychographic microscopy of claim 1, wherein, The specific method for removing outliers from the set of real frequency domain aperture center coordinates by using the RANSAC circle fitting algorithm is as follows: Three points are randomly selected from the set of real frequency domain aperture center coordinates, and a circular track equation passing through the three points is calculated, and the formula is as follows: ； is the center of the circle, is the radius of the circle, , ) are the coordinates of the three selected points; , ) are the coordinates of the three selected points; , ) are the coordinates of the three selected points; The Euclidean distance of the remaining points to the center of the circle is calculated. ； When the following conditions are met The point is considered as an inlier, the final circle trajectory equation is fitted by selecting the group of results with the most inliers, and the corrected dark-field aperture center falls on the final circle trajectory equation, thereby completing the first circle dark-field frequency-domain aperture center coordinate correction.

Images