Efficient annular LED misalignment correction method

Abstract

The invention discloses an efficient annular LED misalignment correction method which is used for high-temporal-spatial-resolution light intensity diffraction tomography. An annular LED is used as an illumination light source, the LED is sequentially lightened to irradiate a sample from different angles to obtain an intensity image, and an improved Fourier-Mellin algorithm is used for performing feature registration on a two-dimensional frequency spectrum of an actual shot image and an aperture of an ideal model in a reconstruction algorithm to correct the wavelength and the position of the light source. Meanwhile, in combination with a global position misalignment model of the annular LED, the accuracy and robustness of registration are further improved. In addition, the importance of physical position correction is further emphasized on the basis of algorithm calibration, so that the resolution and quantitation of three-dimensional refractive index reconstruction are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to three-dimensional refractive index imaging technology, and specifically provides a high-efficiency annular LED misalignment correction method. Background Art

[0002] Microscopy provides a unique and powerful method for observing cells and molecules across time and space. However, visualizing cellular structures is challenging because biological samples are mostly composed of water, which has poor refractive properties. Computational microscopy, an emerging microscopy technique, leverages the intrinsic refractive index of transparent biological samples as a natural contrast mechanism. By combining the power of optical hardware and computational algorithms, it can reconstruct images from indirect measurements without the need for labeling or staining. Recent studies and analysis of imaging system transfer functions have shown that annular matched illumination schemes offer unique advantages for high-temporal and spatial resolution imaging due to their optimized spectral response and data acquisition efficiency. This technique is relatively simple to implement and has been widely applied to quantitative phase imaging and diffraction tomography by adding a simple annular programmable LED illumination module as the light source to commercial microscopes. Notably, these high-resolution imaging techniques place extremely high demands on the precise matching between experimental parameters and algorithmic models. In particular, in three-dimensional IDT imaging, the two-dimensional spectrum of the measured image must be projected onto a three-dimensional Ewald shell. Even slight deviations from the model can lead to significant reconstruction artifacts and severely compromise reconstruction accuracy.

[0003] To ensure consistency between the inverse algorithm and the experimental setup and improve reconstruction quality, researchers have developed various LED self-calibration algorithms. Simulated annealing is a commonly used state-of-the-art self-calibration method. As a joint estimation solution, it can eliminate LED misalignment artifacts, which typically manifest as low-frequency noise. However, when the LED array is significantly misaligned, the simulated annealing algorithm may converge to a local optimum due to the limited number of random frequency shifts it can search for per LED element. Increasing the number of random frequency shifts per LED element during the simulated annealing process, or introducing more parameters to be corrected (e.g., wavelength correction), can slow the solver runtime by an order of magnitude or more, making the reconstruction computational cost prohibitive. To further improve the speed of the self-calibration algorithm, a method combining brightfield preprocessing and iterative spectral estimation has been proposed, achieving good calibration results across various scenarios. Unfortunately, these methods all rely on iterative steps in the reconstruction algorithm and often require significant user expertise, making the setup difficult for non-experts to replicate, thus undermining their universal applicability. Furthermore, existing LED calibration methods only "calibrate" position at the algorithmic level and do not involve "correcting" for physical displacements. Therefore, even if the position calibration is correct, the problem of reconstruction artifacts and inaccurate refractive index caused by the misalignment of the light source and the optical axis still exists. Summary of the Invention

[0004] The object of the present invention is to provide a high-efficiency annular LED misalignment correction method.

[0005] The technical solution to achieve this invention is: a highly efficient ring-shaped LED misalignment correction method, the steps are as follows:

[0006] Step 1: Illuminate the sample from different angles to obtain intensity images;

[0007] Step 2: Perform Fourier transform on the acquired intensity image to convert the image from the spatial domain to the frequency domain to obtain the actual pupil;

[0008] Step 3: Obtain an ideal spectrum based on the initial parameters of the microscopic imaging system, and crop the ideal spectrum to obtain an ideal pupil;

[0009] Step 4: Use the improved Fourier-Mellin algorithm to perform feature registration between the actual pupil and the ideal pupil to correct the wavelength and position of the light source and obtain the offset distance of each pupil;

[0010] Step 5: Fit all pupil offset distances to obtain the spectrum horizontal offset distance and spectrum vertical offset distance, and calibrate the LED coordinates to obtain the corrected actual pupil;

[0011] Step 6: Perform a second precise calibration of the wavelength based on the calibrated LED coordinates;

[0012] Step 7: Calculate the actual LED movement distance based on the spectrum shift and the corrected wavelength and perform mechanical compensation on the LED.

[0013] Preferably, a light intensity transmission diffraction tomography microscopy system is used to irradiate the sample to obtain an intensity image. The light intensity transmission diffraction tomography microscopy system includes a programmable LED, a microscope objective, a reflector, an imaging tube lens, and a camera. The center of the programmable LED is coaxial with the microscope objective lens and is placed at a predetermined height above the sample to be measured. The camera is placed on the imaging focal plane. The programmable LED emits an illumination light beam that passes through the sample to be measured from different directions, and converges to the camera plane after passing through the microscope objective lens, the reflector, and the imaging tube lens, thereby recording a series of original light intensity images on the camera plane.

[0014] Preferably, the initial parameters of the microscopic imaging system include the radius of the programmable LED, the wavelength of the LED, and the distance from the LED to the sample being measured.

[0015] Preferably, the improved Fourier-Mellin algorithm is used to perform feature registration between the actual pupil and the ideal pupil to correct the wavelength and position of the light source. The specific method for obtaining the offset distance of each pupil is:

[0016] Step 4.1: Perform Fourier transform on the actual pupil g(x) and the ideal pupil f(x) to obtain the actual frequency domain image. and ideal frequency domain image And take the center of the frequency domain image as the origin (x0, y0), transform the coordinates of each point (x, y) in the frequency domain into the polar coordinate system, and obtain the actual logarithmic polar coordinate image g′1(x) and the ideal logarithmic polar coordinate image f1′(x);

[0017] In step 4.2, a normalized cross-correlation operation is performed on the actual logarithmic polar coordinate image and the ideal logarithmic polar coordinate image to obtain a cross-correlation logarithmic polar coordinate graph. The specific formula for the normalized cross-correlation operation is:

[0018]

[0019] Where m = ln(r), n = θ, NCC(m,n) represents the normalized cross-correlation value at (m,n), f1′(m',n') represents the pixel value in the ideal log-polar coordinate image, and g′1(m'-m,n'-n) represents the pixel value in the actual log-polar coordinate image.

[0020] Step 4.3, according to the center of the cross-correlation logarithmic polar coordinate diagram (ln(r0),θ0) and the peak point coordinates (ln(r max ),θ max ) Calculate the scaling ratio between images:

[0021]

[0022] Where s represents the image scaling ratio;

[0023] The original ideal pupil is scaled according to the scaling ratio to achieve preliminary wavelength correction;

[0024] Step 4.4: Perform phase correlation calculation on the corrected ideal pupil and the actual pupil to obtain the phase correlation coefficient:

[0025]

[0026] in, is the Fourier transform of the corrected ideal pupil, is the Fourier transform of the actual pupil, for conjugation of;

[0027] Step 4.5, according to the peak coordinate (x max ,y max ) Calculate the spectrum offset:

[0028]

[0029] where x shift and y shift Represents the offset on the x-axis and y-axis respectively, M is the number of pixels on the x-axis, and N is the number of pixels on the y-axis. The spectrum corresponding to each LED is calculated in turn to obtain the offset distance of each pupil (x shift1 ,y shift1 ) to (x shiftn ,y shiftn ).

[0030] Preferably, the specific method for performing secondary precise correction of the wavelength according to the calibrated LED coordinates is:

[0031] Step 6.1, rotate the actual pupil G(x) to the position where the pupil center coordinate is X=0;

[0032] Step 6.2: Project the rotated actual pupil G(x) along the y-axis to obtain one-dimensional data, and perform a second derivative to obtain the cutoff frequency.

[0033] Step 6.3, calculate the wavelength value after accurate correction, the specific formula is:

[0034] f cut =NA obj / λ

[0035] where f cut is the cutoff frequency, NA obj is the numerical aperture of the objective lens, and λ is the wavelength value after precise correction.

[0036] Preferably, the distance the LED needs to move in step 7 is:

[0037]

[0038] Where (Δx, Δy) is the required movement distance of the LED, λ is the wavelength value after precise calibration, r is the radius of the ring LED, and h is the distance from the ring LED to the sample.

[0039] Compared with the prior art, the present invention has the following significant advantages:

[0040] (1) The present invention uses a normalized cross-correlation algorithm to replace the phase correlation algorithm in the Fourier-Mellin transform workflow, solving the problem that the traditional Fourier-Mellin transform algorithm requires a high overlap rate and a good signal-to-noise ratio between images, making it possible to correct the position and wavelength of the LED.

[0041] (2) The present invention uses the least squares method to optimize the global position misalignment model, which can achieve robust correction even in the presence of severe misalignment and obtain the LED position offset parameters.

[0042] (3) The present invention significantly improves the resolution and accuracy of three-dimensional refractive index reconstruction by combining algorithmic "calibration" with physical position "correction". BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of an efficient ring LED misalignment correction method.

[0044] Figure 2 This is a schematic diagram of the microscopic imaging system device based on the efficient annular LED misalignment correction method.

[0045] Figure 3 This is a schematic flow chart of the algorithm correction portion of the efficient annular LED misalignment correction method.

[0046] Figure 4 The three-dimensional refractive index distribution rendering comparison diagram of Hepg2 cells was reconstructed using the IDT algorithm based on the method of the present invention, the IDT algorithm with only LED correction by the algorithm, and the IDT algorithm without LED correction.

[0047] Figure 5 This is a rendering of the C166 cell healing process using high spatiotemporal resolution reconstructed using the method of the present invention. DETAILED DESCRIPTION

[0048] like Figure 1 As shown, an efficient ring LED misalignment correction method includes the following seven steps:

[0049] Step 1: Light up the LEDs in sequence to illuminate the sample from different angles to obtain intensity images;

[0050] like Figure 2 As shown, the present invention is based on a light intensity transmission diffraction tomography microscopic imaging system for acquiring an intensity image, wherein the microscopic imaging system comprises a programmable LED (1), a sample to be measured (2), a microscope objective lens (3), a reflector (4), an imaging tube lens (5), and a camera (6); the center of the programmable LED (1) is coaxial with the microscope objective lens (3) and is placed at a predetermined height above the sample to be measured; the camera is placed on the imaging focal plane;

[0051] The specific implementation process is as follows: a programmable LED (1) emits an illumination beam that passes through a sample to be measured (2) from different directions, passes through a microscope objective lens (3), a reflector (4) and an imaging tube lens (5), and converges to a camera (6) plane, thereby recording a series of original light intensity images on the camera (6) plane.

[0052] Step 2: Perform Fourier transform on the acquired intensity image to convert the image from the spatial domain to the frequency domain to obtain the actual pupil g(x);

[0053] Step 3: Based on the initial parameters of the microscopic imaging system, the ideal spectrum is obtained. The initial parameters of the microscopic imaging system include the radius of the programmable LED, the wavelength of the LED, and the distance from the LED to the sample under test. The ideal spectrum is then clipped to obtain the ideal pupil f(x), as shown in FIG. Figure 3 (a)

[0054] Step 4: Use the improved Fourier-Mellin algorithm to perform feature registration between the actual pupil and the ideal pupil to correct the wavelength and position of the light source, as shown in Figure 3 As shown in (b), the process is as follows:

[0055] Step 4.1: Perform Fourier transform on the actual pupil g(x) and the ideal pupil f(x) to obtain the frequency domain image. and And with the center of the frequency domain image as the origin (x0, y0), transform the coordinates of each point in the frequency domain (x, y) into the polar coordinate system (ln(r), θ):

[0056]

[0057] Obtain the actual logarithmic polar coordinate image g1′(x) and the ideal logarithmic polar coordinate image f1′(x);

[0058] Step 4.2, perform normalized cross-correlation operation on the log-polar coordinate image:

[0059]

[0060] Where m = ln(r), n = θ, NCC(m,n) represents the normalized cross-correlation value at (m,n), f1′(m',n') represents the pixel value in the ideal log-polar image, and g1′(m'-m,n'-n) represents the pixel value in the actual log-polar image. This formula quantifies the similarity between the template and the target image to determine the matching position of the template in the target image. This is calculated for each pixel value to obtain a cross-correlation log-polar coordinate plot.

[0061] Step 4.3, take the center of the cross-correlation logarithmic polar coordinate graph as the origin (ln(r0),θ0) and the peak point coordinates (ln(r max ),θ max ) Calculate the scaling ratio between images:

[0062]

[0063] Where s represents the image scaling ratio;

[0064] The original ideal pupil is scaled according to the scaling ratio to achieve preliminary wavelength correction;

[0065] Step 4.4: Perform phase correlation calculation on the corrected ideal pupil and the actual pupil to obtain the phase correlation coefficient:

[0066]

[0067] in is the Fourier transform of the corrected ideal pupil, is the Fourier transform of the actual pupil, for conjugation of;

[0068] Step 4.5, find the peak coordinate (x max ,y max ), the spectrum offset can be calculated:

[0069]

[0070] where x shift and y shift Represents the offset on the x-axis and y-axis respectively, M is the number of pixels on the x-axis, and N is the number of pixels on the y-axis. By calculating the spectrum corresponding to each LED in turn, the offset distance of each pupil (x shift1 ,y shift1 ) to (x shiftn ,y shiftn ).

[0071] Step 5: Fit all pupil offset distances obtained to obtain more accurate correction results; Figure 3 As shown in (c), the fitting needs to meet two physical prior constraints, namely: 1) the LED distribution follows a circular geometry and is arranged at equal intervals; 2) the LED is located on a horizontal plane perpendicular to the optical axis, that is, according to the formula Convert each LED coordinate from polar coordinates to Cartesian coordinates, where x' is the horizontal coordinate of the LED, y' is the vertical coordinate of the LED, r is the radius of the ring LED, and θ is the angle corresponding to each LED; then shift1 ,y shift1 ) to (x shiftn ,y shiftn ) is fitted by the least square method to obtain the spectrum lateral offset distance dx and the spectrum longitudinal offset distance dy, and the LED coordinates are calibrated to obtain the corrected actual pupil G(x).

[0072] Step 6: Perform a second precise calibration of the wavelength based on the calibrated LED coordinates, as shown in the following example: Figure 3 As shown in (d), the process is as follows:

[0073] Step 6.1, rotate the actual pupil G(x) to the position where the pupil center coordinate is X=0;

[0074] Step 6.2: Project the rotated actual pupil G(x) along the y-axis to obtain one-dimensional data, and perform a second derivative to obtain the cutoff frequency.

[0075] Step 6.3, according to formula f cut =NA obj / λ is calculated to obtain the wavelength value after accurate correction;

[0076] where f cut is the cutoff frequency, NA obj is the numerical aperture of the objective lens, and λ is the wavelength value after precise correction.

[0077] Step 7: Calculate the actual LED movement distance based on the programmable LED spectrum lateral offset distance dx and spectrum longitudinal offset distance dy obtained in step 5 and the corrected wavelength, and perform mechanical compensation on the LED:

[0078]

[0079] Where λ is the precisely calibrated wavelength, r is the radius of the ring LED, and h is the distance from the ring LED to the sample.

[0080] Figure 4 The three-dimensional refractive index distribution rendering comparison images of Hepg2 cells reconstructed using the IDT algorithm based on the method of the present invention, the IDT algorithm using only the algorithm-corrected LED, and the IDT algorithm without LED correction are respectively; the three-dimensional refractive index result reconstructed by the IDT algorithm without LED correction appears obviously blurred and the resolution is significantly reduced; although the reconstruction result of the IDT algorithm using only the algorithm-corrected LED can restore certain image details, the problem of misalignment between the annular LED light source and the optical axis still exists, and the problem of reconstructed refractive index distortion (as shown in the white dotted circle in the figure) is still unavoidable; the IDT algorithm based on the method of the present invention has a significantly better reconstruction effect and can obtain clear subcellular structures and cell contours.

[0081] Figure 5 The rendering of the apoptosis process of C166 cells reconstructed based on the method of the present invention has a maximum acquisition speed of 212 Hz (corresponding to a volume speed of 7.5 Hz), and reconstructs high-resolution results, which can obtain clear organelle movement and cell healing processes, verifying the accuracy and reliability of this method.

[0082] The present invention is a highly efficient annular LED misalignment correction method for use in high spatiotemporal resolution IDT imaging. Unlike the positioning correction method developed using traditional stacked iterative technology, the present invention first uses an improved Fourier-Mellin algorithm with stronger noise resistance to perform initial correction on the wavelength and spatial position of the LED, and then solves the optimization problem of the global position misalignment model using the least squares method to obtain a more accurate correction result. In addition, combined with the mechanical displacement compensation of the LED position and the annular matching illumination scheme, the present invention can significantly improve the imaging quality while maintaining a volume rate of 7.5 Hz and a lateral resolution of 350 nm. Experimental results show that the present invention is a universal and efficient self-calibration method, which is of great significance to the scientific research and industrial promotion of computational optical microscopy, and can provide strong technical support for biomedical and life science imaging.

Claims

1. An efficient ring-shaped LED misalignment correction method, characterized in that: Here are the steps: Step 1: Illuminate the sample from different angles to obtain intensity images; Step 2: Perform Fourier transform on the acquired intensity image to convert the image from the spatial domain to the frequency domain to obtain the actual pupil; Step 3: Obtain an ideal spectrum based on the initial parameters of the microscopic imaging system, and crop the ideal spectrum to obtain an ideal pupil; Step 4: Use the improved Fourier-Mellin algorithm to perform feature registration between the actual pupil and the ideal pupil to correct the wavelength and position of the light source and obtain the offset distance of each pupil; Step 5: Fit all pupil offset distances to obtain the spectrum horizontal offset distance and spectrum vertical offset distance, and calibrate the LED coordinates to obtain the corrected actual pupil; Step 6: Perform a second precise calibration of the wavelength based on the calibrated LED coordinates; Step 7: Calculate the actual LED movement distance based on the spectrum shift and the corrected wavelength and perform mechanical compensation on the LED.

2. The high-efficiency annular LED misalignment correction method according to claim 1, characterized in that: A light intensity transmission diffraction tomography microscopy imaging system is used to illuminate a sample to obtain an intensity image. The light intensity transmission diffraction tomography microscopy imaging system comprises a programmable LED (1), a microscope objective lens (3), a reflector (4), an imaging tube lens (5), and a camera (6). The center of the programmable LED (1) is coaxial with the microscope objective lens (3) and is placed at a predetermined height above the sample to be measured. The camera (6) is placed on the imaging focal plane. The programmable LED (1) emits an illumination light beam that passes through the sample to be measured (2) from different directions, passes through the microscope objective lens (3), the reflector (4), and the imaging tube lens (5), and then converges to the camera (6) plane, thereby recording a series of original light intensity images on the camera (6) plane.

3. The high-efficiency annular LED misalignment correction method according to claim 1, characterized in that: The initial parameters of the microscopic imaging system include the radius of the programmable LED, the wavelength of the LED, and the distance from the LED to the sample being measured.

4. The high-efficiency annular LED misalignment correction method according to claim 1, characterized in that: The improved Fourier-Mellin algorithm is used to align the actual pupil with the ideal pupil to correct the wavelength and position of the light source. The specific method for obtaining the offset distance of each pupil is as follows: Step 4.1: Perform Fourier transform on the actual pupil g(x) and the ideal pupil f(x) to obtain the actual frequency domain image. and ideal frequency domain image And take the center of the frequency domain image as the origin (x0, y0), transform the coordinates of each point (x, y) in the frequency domain into the polar coordinate system, and obtain the actual logarithmic polar coordinate image g1′(x) and the ideal logarithmic polar coordinate image f1′(x); In step 4.2, a normalized cross-correlation operation is performed on the actual logarithmic polar coordinate image and the ideal logarithmic polar coordinate image to obtain a cross-correlation logarithmic polar coordinate graph. The specific formula for the normalized cross-correlation operation is: Where m = ln(r), n = θ, NCC(m,n) represents the normalized cross-correlation value at (m,n), f1′(m',n') represents the pixel value in the ideal log-polar coordinate image, and g1′(m'-m,n'-n) represents the pixel value in the actual log-polar coordinate image. Step 4.3, according to the center of the cross-correlation logarithmic polar coordinate diagram (ln(r0),θ0) and the peak point coordinates (ln(r max ),θ max ) Calculate the scaling ratio between images: Where s represents the image scaling ratio; The original ideal pupil is scaled according to the scaling ratio to achieve preliminary wavelength correction; Step 4.4: Perform phase correlation calculation on the corrected ideal pupil and the actual pupil to obtain the phase correlation coefficient: in, is the Fourier transform of the corrected ideal pupil, is the Fourier transform of the actual pupil, for conjugation of; Step 4.5, according to the peak coordinate (x max ,y max ) Calculate the spectrum offset: where x shift and y shift Represents the offset on the x-axis and y-axis respectively, M is the number of pixels on the x-axis, and N is the number of pixels on the y-axis. The spectrum corresponding to each LED is calculated in turn to obtain the offset distance of each pupil (x shift1 ,y shift1 ) to (x shiftn ,y shiftn ).

5. The high-efficiency annular LED misalignment correction method according to claim 1, characterized in that: The specific method for performing secondary precise wavelength correction based on the calibrated LED coordinates is: Step 6.1, rotate the actual pupil G(x) to the position where the pupil center coordinate is X=0; Step 6.2: Project the rotated actual pupil G(x) along the y-axis to obtain one-dimensional data, and perform a second derivative to obtain the cutoff frequency. Step 6.3, calculate the wavelength value after accurate correction, the specific formula is: f cut =NA obj / λ where f cut is the cutoff frequency, NA obj is the numerical aperture of the objective lens, and λ is the wavelength value after precise correction.

6. The high-efficiency annular LED misalignment correction method according to claim 1, characterized in that: The specific distance the LED needs to move in step 7 is: Where (Δx, Δy) is the required movement distance of the LED, λ is the wavelength value after precise calibration, r is the radius of the ring LED, and h is the distance from the ring LED to the sample.

Images