Robust anti-motion structured light illumination super-resolution microscopy method based on principal component analysis
By compensating for pixel offset and phase error using a principal component analysis-based method, efficient and real-time super-resolution imaging of live cells in complex environments is achieved. This solves the imaging artifact problem caused by environmental disturbances and sample motion in traditional methods and is suitable for nanoscale observation of live cells.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2022-12-09
- Publication Date
- 2026-05-05
AI Technical Summary
Traditional structured illumination micro-imaging methods are easily affected by environmental disturbances and sample movement under complex and unstable conditions, leading to imaging artifacts and reconstruction errors, making it difficult to achieve high-quality super-resolution observation of live cells.
A principal component analysis-based approach is employed to acquire original structured light illumination images from three different directions, compensate for pixel shifts and phase errors caused by motion, perform precise spectral separation and merging, and reconstruct the super-resolution image using Wiener deconvolution.
It achieves more robust imaging quality under complex and unstable conditions, and provides a robust, efficient, real-time, flexible, and low-optical-damage super-resolution observation method, which is suitable for studying the nanoscale subcellular structural features, motion state, and protein function of living cells.
Smart Images

Figure CN116106274B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of super-resolution fluorescence microscopy technology, specifically a robust super-resolution microscopy method based on principal component analysis that resists moving structured light illumination. It is used to perform real-time super-resolution observation of living cells in a more compatible and flexible manner under complex and unstable conditions. Background Technology
[0002] Studying the dynamic changes of subcellular structures at the nanoscale is of great significance for exploring the nature of life and major diseases. In the past few decades, super-resolution techniques have bypassed the limitations of Abbe diffraction, enabling the visualization of biomolecules at the nanoscale to single-molecule level, and have become a powerful tool for life science research [PS Weiss, "Nobelprizes for super-resolution imaging," (2014).]. Thanks to its advantages such as fast wide-area imaging, low optical damage, and non-specific requirements for fluorescent molecules, structured illumination microscopy (SIM) has stood out among many super-resolution techniques, especially suitable for real-time long-term dynamic observation of living cells [J. Qian, Y. Cao, K. Xu, Y. Bi, W. Xia, Q. Chen, and C. Zuo, "Robust frame-reduced structured illumination microscopy with accelerated correlation-enabled parameter estimation," Appl. Phys. Lett. 121, 153701 (2022).].
[0003] SIM typically uses spatial structured light illumination to modulate high-frequency information that is originally above the system cutoff frequency, thereby expanding the lateral resolution [G.Wen,S.Li,L.Wang,X.Chen,Z.Sun,Y.Liang,X.Jin,Y.Xing,Y.Jiu,Y.Tang et al.,“High-fidelity structured illumination microscopy by point-spread-function engineering,”Light.Sci.&Appl.10,1–12(2021).]. However, in SIM image reconstruction, even slight estimation errors can lead to severe reconstruction artifacts. Therefore, to achieve high-quality super-resolution image reconstruction, accurate estimation of illumination parameters is necessary [X. Huang, J. Fan, L. Li, H. Liu, R. Wu, Y. Wu, L. Wei, H. Mao, A. Lal, P. Xi et al., “Fast, long-term, super-resolution imaging with hessian structured illumination microscopy,” Nat. biotechnology 36, 451–459 (2018).]. Since illumination parameters are highly susceptible to environmental disturbances, sample shifts, and other motion factors, if imaging conditions are not strictly kept stable, illumination parameters must be re-extracted for each reconstruction to enable dynamic observation of live cells [K. Wicker, “Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in fourier space,” Opt. express 21, 24692–24701 (2013).].Traditional parameter estimation methods, such as the most widely used iterative cross-correlation (COR), typically perform initial spectral separation with a known constant phase shift in each illumination direction, and then rely on overlapping sample signals of different spectral components to compensate for residual parameters [MG Gustafsson, L. Shao, PM Carlton, C.R. Wang, IGNolubovskaya, W.Z. Cande, D.A. Gard, and J.W.S. Detat, “Three dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. Journal 94, 4957–4970 (2008).]. However, in practical applications, environmental changes and human interference during experiments can cause motion of the sample and illumination, failing to meet the basic assumption of uniform phase shift in traditional algorithms, thus leading to severe imaging artifacts. Summary of the Invention
[0004] The purpose of this invention is to provide a robust, motion-resistant structured light illumination-based super-resolution microscopy imaging method based on principal component analysis. This method provides a robust, efficient, real-time, flexible, convenient, and low-light-damage super-resolution observation tool for studying the nanoscale subcellular structural features, motion states, interactions, and protein functions in living cells under complex and unstable conditions.
[0005] The technical solution to achieve the objective of this invention is: a robust super-resolution microscopy method based on principal component analysis and resistant to motion-induced structured light illumination, the specific process of which is as follows:
[0006] Step 1: Acquire raw structured light illumination images from three different directions;
[0007] Step 2: Compensate for pixel shifts caused by motion in the original structured light illumination image;
[0008] Step 3: Compensate for phase errors caused by motion in the original structured light illumination image;
[0009] Step 4: Perform precise spectral separation on the compensated structured light illumination image;
[0010] Step 5: Merge the separated spectra and reconstruct a real-time super-resolution image using Wiener deconvolution.
[0011] Preferably, the specific method for acquiring the original structured light illumination image in step 1 is as follows:
[0012] The sample was illuminated from three different directions using a structured lighting microsystem, and three-step phase-shift sinusoidal illumination images were acquired in each direction.
[0013] Preferably, the original structured light illumination image D in a certain direction n (n=1,2,3) represents the original three-step phase-shift sinusoidal illumination image of the structure in this direction, specifically:
[0014]
[0015] Where r is the image spatial coordinate, S is the sample information, and r n and This refers to pixel offset and phase error caused by motion. Given a constant phase shift, P is the point spread function. For convolution operations, k ex and m are the modulation frequency and modulation index of the structured light, respectively.
[0016] Preferably, the specific method for compensating for pixel shifts caused by motion in the original structured light illumination image in step 2 is as follows:
[0017] Step 2.1: Perform a Fourier transform on the original structured light illumination image acquired in Step 1 to obtain the following spectral image:
[0018]
[0019] Where k is the frequency coordinate, the superscript ~ represents the Fourier transform of the original object, the subscripts 0 and ±1 represent the spectral order of the sample, and O is the optical transfer function;
[0020] Step 2.2: Calculate the normalized cross-correlation power spectrum of the original structured light illumination image with the pixel offset image and the first illumination direction image to obtain an approximation of the pixel offset:
[0021]
[0022] In the formula, ψ is the obtained normalized cross-correlation power spectrum, the superscript * is the complex conjugate of the matrix, and Han is the Hanning filter;
[0023] Step 2.3: Extract the principal energy of the normalized cross-correlation power spectrum using a dual-window mask, specifically:
[0024] Perform an inverse Fourier transform on the normalized cross-correlation power spectrum ψ and locate the integer pixel peak of the inverse Fourier transform spectrum.
[0025] Centering on the normalized cross-correlation power spectrum, the principal energy of the spectrum is extracted using a mask operator and then subjected to a Fourier transform:
[0026]
[0027]
[0028]
[0029] In the formula, NaN represents invalid points, and r x,min r y,min These represent the left and lower boundaries of the signal window in the dual-window mask along the vertical axis (x) and vertical axis (y), respectively. x,max r y,max Let r be the right and top boundaries of the signal window in the dual-window mask along the x-axis and y-axis, respectively, and let R be the size of the blank window in the dual-window mask along the x-axis and y-axis. x r y Let r be the spatial coordinates along the vertical axis x and the vertical axis y, respectively, where the signal window size r is... x,min r y,min r x,max r y,max And the value of the blank window size R, in max{N 2 S / (r x(y),max -r x(y),min +2R+1) 2 The value is obtained when N is the size of the complete spectrum and S is the ratio between the signal window amplitude and the complete amplitude.
[0030] Step 2.4: Perform singular value decomposition on the principal energy of the normalized cross-correlation power spectrum obtained in Step 2.3 to extract the offset r of motion disturbance. n The normalized cross-correlation power spectrum after decomposition is as follows:
[0031] ψ(k)=U1Λ1V1 T
[0032] In the formula, U1 and V1 are the left and right singular matrices of the normalized cross-correlation power spectrum, respectively, Λ1 is its eigenvalue matrix, and the superscript T is the transpose of the matrix.
[0033] Step 2.5: Extract the principal components of the normalized cross-correlation power spectrum, specifically:
[0034]
[0035] In the formula, This represents a matrix where the first element in the first row and the first column is 1, and all other elements are 0.
[0036] Step 2.6: Perform phase expansion on the first row elements of the left singular matrix U1, and use the least squares method to linearly fit it to obtain the slope. This slope is the sub-pixel precision offset r caused by motion. nThe component in the horizontal x-direction; performing a phase expansion on the first row elements of the right singular matrix V1, and using the least squares method to linearly fit it to obtain the slope, the fitted slope being the sub-pixel precision offset r caused by motion. n The component in the vertical y-direction;
[0037] Step 2.7: Offset r caused by the extracted motion n Compensation is performed on the center spectrum of the +1 level spectrum of the sample:
[0038]
[0039] Preferably, the radius of the Hanning filter is The peak distance between level 0 and level 1 spectrum.
[0040] Preferably, the specific method for compensating for phase errors caused by motion in the original structured light illumination image in step 3 is as follows:
[0041] Step 3.1: Determine the first-order integer pixel peak of the spectrum of the original structured light illumination image after pixel compensation. Using the first-order integer pixel peak as the center, extract the principal energy of the spectrum using a dual-window mask, perform an inverse Fourier transform, and take the phase to obtain the e-exponent term of the phasor factor of the illumination image.
[0042]
[0043] In the formula, exp is an exponential function with the natural constant e as the base, and angle is the phase function with respect to the complex number. For inverse Fourier transform, C1 is the first-order spectral principal energy extracted using a double-window mask;
[0044] Step 3.2: Perform singular value decomposition on the e-exponential term of the phasor factor obtained in Step 3.1. The decomposition is as follows:
[0045]
[0046] In the formula, U2 and V2 are the left and right singular matrices of the phasor factor, respectively, Λ2 is its eigenvalue matrix, and the superscript T is the transpose of the matrix;
[0047] Step 3.3: Extract the principal components of the phasor factor, take the phase of the principal components of the phasor factor at r=0, and obtain the phase error caused by motion.
[0048] Preferably, the specific method for accurately separating the spectrum in step 4 is as follows:
[0049]
[0050] In the formula, k exand m represent the modulation frequency and modulation degree of the structured light, respectively; k is the frequency coordinate; O is the optical transfer function; S is the sample information; the superscript ~ represents the Fourier transform of the original object; and the subscripts 0 and ±1 represent the spectral order of the sample. The phase error is caused by motion. As a constant phase shift is set, D' is the center spectrum of the sample +1 level spectrum after compensating for pixel offset and phase error.
[0051] Preferably, the specific method for reconstructing the super-resolution image in step 5 is to merge the separated spectra and reconstruct the real-time super-resolution image using Wiener deconvolution:
[0052]
[0053] In the formula, the subscript i represents the spectral order, i = 0, +1, -1, the subscript d represents the d-th illumination component, and k d,ex Let d be the wave vector of the d-th illumination direction. Let w be the i-th order spectral component of the d-th illumination direction without optical transfer function and illumination parameter components, and w be the Wiener constant.
[0054] Compared with existing technologies, the significant advantages of this invention are: it can effectively compensate for non-uniform pixel movement and phase errors in each original illumination image; it achieves more robust imaging quality under complex and unstable conditions, and is expected to achieve more compatible and flexible live cell imaging; it provides a robust, efficient, real-time, flexible, convenient, and low-light-damage super-resolution observation method for studying the nanoscale subcellular structural features, motion state, interactions, and protein functions in living cells, which is of great significance to research in life science-related fields.
[0055] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0056] Figure 1 This is a flowchart of the present invention.
[0057] Figure 2 This paper presents the results of super-resolution reconstruction of cell images using the method described in this invention and conventional methods with artificially introduced motion errors. The reconstructions are then performed on cells at different signal-to-noise ratios using the method described in this invention. a) Wide-field image and super-resolution image obtained by this invention. b, c) Spectra separated using conventional methods and this invention. d, e) Spectra merged using conventional methods and this invention. fi) Wide-field image of the region within the box in a, obtained using conventional methods and this invention. j, k) Phase and offset estimation errors of this invention at different signal-to-noise ratios. Figure 3The images show the results of super-resolution imaging of immobilized COS-7 cells in a stable environment using the method described in this invention and conventional methods, respectively. a) Wide-field and super-resolution images obtained using this invention. b) and d) Enlarged wide-field and super-resolution images of actin within the box in a, obtained using different methods. e) Intensity curve along the blue line in b.
[0058] Figure 4 The images show the results of super-resolution imaging of immobilized COS-7 cells in a moving environment using the method described in this invention and conventional methods, respectively. a) Wide-field and super-resolution images obtained using this invention. b, c) Enlarged super-resolution images of the region within the white box in a, obtained using different methods. d-fa) Enlarged results of actin in the region within the yellow box. g) Intensity curve along the blue line in d.
[0059] Figure 5 This invention and conventional methods were used to perform real-time super-resolution observations of mitochondrial dynamic tubule events in live COS-7 cells under motion conditions. ac obtained wide-field and super-resolution images using this invention. df obtained magnified wide-field and super-resolution images of the region within the box in a using different methods. gk obtained a magnified super-resolution image of the region within the box in a using this invention. Detailed Implementation
[0060] A robust super-resolution microscopy method based on principal component analysis (PCA) for motion-resistant structured light illumination is presented. This method utilizes PCA-based dimensionality reduction tools to extract pixel shifts and phase errors caused by motion, as well as the "first principal component" dominated by illumination parameters, effectively compensating for pixel shifts and phase errors caused by motion. Experimental results demonstrate that, compared to traditional methods, this invention achieves more robust image quality under complex and unstable conditions, and holds promise for more compatible and flexible live-cell super-resolution imaging. The flowchart of this invention is shown below. Figure 1 As shown, the specific steps are as follows:
[0061] Step 1: Illuminate the sample from three different directions using a structured illumination microscopy (SIM) system, and acquire three-step phase-shift sinusoidal illumination images for each direction. The original structured illumination image D for one direction is shown below. n (n = 1, 2, 3) can be represented as:
[0062]
[0063] Where r represents the image spatial coordinates, and S represents the sample information. n and Let represent the pixel offset and phase error caused by motion (pixel offset is referenced to the first frame image, i.e., r1 = 0), φ be the set constant phase shift, and P be the point spread function. For convolution operations, k ex and m are the modulation frequency and modulation index of the structured light, respectively.
[0064] Step 2: Compensate for pixel shifts caused by motion in the three-step phase-shift sinusoidal illumination image, as follows:
[0065] Step 2.1: Perform a Fourier transform on the original structured light illumination image acquired in Step 1 to obtain the following spectral image:
[0066]
[0067] Where k is the frequency coordinate, the superscript ~ represents the Fourier transform of the original object, the subscripts 0 and ±1 represent the spectral order of the sample, and O is the optical transfer function;
[0068] Step 2.2: Calculate the normalized cross-correlation power spectrum between the original structured light illumination image with pixel offset and the first illumination direction image:
[0069]
[0070] In the formula, ψ is the obtained normalized cross-correlation power spectrum, i.e., the phase correlation matrix; the superscript * denotes the complex conjugate of the matrix; Han is the Hanning filter with radius . The peak distance between level 0 and level 1 spectra is used to attenuate non-level 0 spectra in the central region of the image;
[0071] Step 2.3: Extract the principal energy of the phase correlation matrix using a dual-window mask, specifically as follows:
[0072] Perform an inverse Fourier transform on the phase correlation matrix ψ and locate the integer-pixel peak of the inverse Fourier transform spectrum; then, using this peak as the center, extract the principal energy of the spectrum using a mask operator and perform a Fourier transform.
[0073]
[0074]
[0075]
[0076] In equation (5), NaN is an invalid point, r x,min r y,min These represent the left and lower boundaries of the signal window in the dual-window mask along the vertical axis (x) and vertical axis (y), respectively. x,max r y,maxLet r be the right and top boundaries of the signal window in the dual-window mask along the x-axis and y-axis, respectively, and let R be the size of the blank window in the dual-window mask along the x-axis and y-axis. x r y These are the spatial coordinates along the vertical axis (x) and the vertical axis (y), respectively. The signal window size r... x,min r y,min r x,max r y,max And the value of the blank window size R, in max{N 2 S / (r x(y),max -r x(y),min +2R+1) 2 The value is obtained when N is the size of the complete spectrum and S is the ratio between the signal window amplitude and the complete amplitude.
[0077] Step 2.4: Perform singular value decomposition on the principal energy of the phase correlation matrix obtained in Step 2.3 to extract the offset r of motion disturbance. n The phase correlation matrix after decomposition is as follows:
[0078] ψ(k)=U1Λ1V1 T (7)
[0079] In the formula, U1 and V1 are the left and right singular matrices of the phase correlation matrix, respectively, Λ1 is its eigenvalue matrix, and the superscript T is the transpose of the matrix;
[0080] Step 2.5: Extract the principal components of the phase correlation matrix. The extracted principal components are as follows:
[0081]
[0082] In the formula, This represents a matrix where the first element in the first row and the first column is 1, and all other elements are 0.
[0083] Step 2.6: Perform phase expansion on the first row elements of the left singular matrix U1, and then use the least squares method to linearly fit it to obtain the slope. This slope is the sub-pixel precision offset r caused by motion. n The component in the horizontal x-direction; the same operation is performed on the right singular matrix V1, and the final fitted slope is the sub-pixel precision offset r caused by motion. n The component in the vertical y-direction;
[0084] Step 2.7: Offset r caused by the extracted motion n The center spectrum of the +1 level spectrum of the sample is compensated.
[0085]
[0086] Offsets of non-integer pixels cause the spectral peaks of the inverse Fourier transform of ψ to spread into the neighborhood. Combined with other interfering factors such as noise and spectral residuals of other orders, this leads to offset estimation errors. Step 2 eliminates these errors to some extent, resulting in a more accurate estimate of the pixel offset r caused by motion. n .
[0087] Step 3: Compensate for phase errors caused by motion in the three-step phase-shifted sinusoidal illumination image, as follows:
[0088] Step 3.1: Take the first-order integer pixel peak of the spectrum of the original structured light illumination image after pixel compensation. Using this peak as the center, extract the principal energy of the spectrum using a dual-window mask, then perform an inverse Fourier transform and take its phase to obtain the e-exponent term of the phasor factor of the illumination image.
[0089]
[0090] In the formula, exp is an exponential function with the natural constant e as the base, and angle is the phase function with respect to the complex number. For inverse Fourier transform, C1 is the first-order spectral principal energy extracted using a double-window mask;
[0091] Step 3.2: Perform singular value decomposition on the e-exponential term of the phasor factor obtained in Step 5.1. The decomposition is as follows:
[0092]
[0093] In the formula, U2 and V2 are the left and right singular matrices of the phasor factor, respectively, Λ2 is its eigenvalue matrix, and the superscript T is the transpose of the matrix;
[0094] Step 3.3: Extract the principal components of the phasor factor. Take its phase at r=0 to obtain the phase error caused by motion.
[0095] Step 4: Precise Spectrum Separation. Given the pixel offset and phase error, precise spectrum separation is performed on the compensated structured light illumination image, specifically as follows:
[0096]
[0097] In the formula, k ex and m represent the modulation frequency and modulation degree of the structured light, respectively; k is the frequency coordinate; O is the optical transfer function; S is the sample information; the superscript ~ represents the Fourier transform of the original object; and the subscripts 0 and ±1 represent the spectral order of the sample. The phase error is caused by motion. As a constant phase shift is set, D' is the center spectrum of the sample +1 level spectrum after compensating for pixel offset and phase error.
[0098] Step 5: Reconstruct the super-resolution image. The separated spectra are merged, and steps 2-4 are performed on the other two illumination directions. Finally, Wiener deconvolution is used to reconstruct the real-time super-resolution image, specifically:
[0099]
[0100] In the formula, the subscript i (i = 0, +1, -1) represents the spectral order, the subscript d represents the d-th illumination component, and k d,ex Let d be the wave vector of the d-th illumination direction. Let w be the i-th order spectral component of the d-th illumination direction without optical transfer function and illumination parameter components, and w be the Wiener constant (usually determined empirically).
[0101] Example
[0102] To verify the effectiveness and noise resistance of this invention, super-resolution reconstruction of cell images was first performed using both the method described in this invention (Motion-resistant structured illumination microscopy based on principal component analysis, mrPCA-SIM) and the conventional method (structured illumination microscopy based on principal component analysis, PCA-SIM) with artificially introduced motion errors. Then, the method described in this invention was used to reconstruct cells at different signal-to-noise ratios. Figure 2 As can be seen in the white circled area in b, motion-induced errors result in significant crosstalk from other spectrum levels in the spectral components separated using PCA-SIM. In contrast, the method described in this invention avoids this crosstalk, achieving accurate spectrum separation and merging. Figure 2 c-2e), thus reconstructing a high-quality, artifact-free super-resolution image ( Figure 2 f-2i). Meanwhile, by Figure 2 Quantization data at different signal-to-noise ratios in j-2k show that the phase accuracy of this invention is less than 0.02 rads and the offset accuracy is less than 0.05 pixels at a normal signal-to-noise ratio (the relatively large phase errors in the second and third images are caused by offset errors). Furthermore, this invention exhibits excellent noise resistance at low signal-to-noise ratios. The experiment was independently repeated 10 times, with similar results.
[0103] To further verify the advancement of this invention, super-resolution reconstructions of DAPI-labeled nuclei, Alexa Fluor™ 568-labeled actin, and MitoTracker™ Green FM-labeled mitochondria in fixed COS-7 (CV-1in Origin Simian-7) cell samples were performed again in stable and dynamic environments using the method described in this invention and conventional methods. Figure 3 b-3d, due to accurate phase estimation, under stable imaging conditions, this invention can achieve the same resolution enhancement performance as conventional methods. Figure 4 It is evident that, even with the addition of non-uniform phase-shifted illumination and slight movement of the microscope mount, the present invention can still maintain high imaging quality and distinguish fine actin structures, while traditional methods exhibit significant image distortion and reconstruction artifacts.
[0104] One of the main applications of this invention is to address the environmental fluctuations caused by motion in real-time live-cell observation. To verify this, real-time super-resolution observations of mitochondrial dynamic tubulation (MDT) in live COS-7 cells were performed using both this invention and conventional methods under motion conditions. Figure 5 As can be seen from a-5c, traditional methods are highly susceptible to motion, resulting in severe reconstruction artifacts. However, due to the precise phase error and pixel offset of this invention, these artifacts can still be effectively suppressed even under low signal-to-noise ratio conditions caused by severe background defocus. Figure 5 d-5f). Meanwhile, such as Figure 5 As shown in f-5k, the mitochondria on the right extend and drag the elongated mitochondrial tubules towards the left, demonstrating the invention's ability to reconstruct dynamic mitochondrial tubule events with high quality under complex environments. This is valuable for studying mitochondrial network formation. Furthermore, in terms of speed, the C++-based software developed using this invention on a Dell XPS 8930 computer runs in approximately 40 milliseconds. This embodiment fully verifies the significant advantages of this invention in studying living cell structures and dynamic processes under complex conditions, and its significance for cell research.
Claims
1. A robust super-resolution microscopy method based on principal component analysis and resistant to motion-induced structured light illumination, characterized in that, The specific process is as follows: Step 1: Acquire raw structured light illumination images from three different directions; Step 2: Compensate for pixel shifts caused by motion in the original structured light illumination image; Step 3: Compensate for phase errors caused by motion in the original structured light illumination image. The specific method is as follows: Step 3.1: Determine the first-order integer pixel peak of the spectrum of the original structured light illumination image after pixel compensation. Using the first-order integer pixel peak as the center, extract the principal energy of the spectrum using a dual-window mask, perform an inverse Fourier transform, and take the phase to obtain the e-exponent term of the phasor factor of the illumination image. In the formula, exp is an exponential function with the natural constant e as the base, and angle is the phase function with respect to the complex number. For inverse Fourier transform, C1 is the first-order spectral principal energy extracted using a double-window mask; Step 3.2: Perform singular value decomposition on the e-exponential term of the phasor factor obtained in Step 3.
1. The decomposition is as follows: In the formula, U2 and V2 are the left and right singular matrices of the phasor factor, respectively. Let T be its eigenvalue matrix, and let T be the transpose of the matrix; Step 3.3: Extract the principal components of the phasor factor, take the phase of the principal components of the phasor factor at r=0, and obtain the phase error caused by motion. ; Step 4: Perform precise spectral separation on the compensated structured light illumination image. The specific method is as follows: In the formula, k ex and m represent the modulation frequency and modulation degree of the structured light, respectively; k is the frequency coordinate; O is the optical transfer function; S is the sample information; the superscript ∼ indicates the Fourier transform of the original object; and the subscripts 0 and ±1 indicate the spectral order of the sample. The phase error is caused by motion. For the set constant phase shift, The center spectrum of the +1 level spectrum of the sample after compensation for pixel offset and phase error; Step 5: Merge the separated spectra and reconstruct a real-time super-resolution image using Wiener deconvolution.
2. The robust, motion-resistant structured light illumination-resistant super-resolution microscopy method based on principal component analysis according to claim 1, characterized in that, The specific method for acquiring the original structured light illumination image in step 1 is as follows: The sample was illuminated from three different directions using a structured lighting microsystem, and three-step phase-shift sinusoidal illumination images were acquired in each direction.
3. The robust, motion-resistant structured light illumination-based super-resolution microscopy method based on principal component analysis according to claim 2, characterized in that, The original structured light illumination image D from a certain direction n n=1,2,3 represents the original three-step phase-shift sinusoidal illumination image of the structure in this direction, specifically: Where r is the image spatial coordinate, S is the sample information, and r n and This refers to pixel offset and phase error caused by motion. Given a constant phase shift, P is the point spread function. For convolution operations, k ex and m are the modulation frequency and modulation index of the structured light, respectively.
4. The robust, motion-resistant structured light illumination-based super-resolution microscopy method based on principal component analysis according to claim 3, characterized in that, The specific method for compensating for pixel shifts caused by motion in the original structured light illumination image in step 2 is as follows: Step 2.1: Perform a Fourier transform on the original structured light illumination image acquired in Step 1 to obtain the following spectral image: Where k is the frequency coordinate, the superscript ∼ represents the Fourier transform of the original object, the subscripts 0 and ±1 represent the spectral order of the sample, and O is the optical transfer function; Step 2.2: Calculate the normalized cross-correlation power spectrum of the original structured light illumination image with the pixel offset image and the first illumination direction image to obtain an approximation of the pixel offset: In the formula, To obtain the normalized cross-correlation power spectrum, the superscript * denotes the complex conjugate of the matrix, and Han represents the Hanning filter; Step 2.3: Extract the principal energy of the normalized cross-correlation power spectrum using a dual-window mask, specifically: Normalized cross-correlation power spectrum Perform inverse Fourier transform and locate the integer pixel peak of the inverse Fourier transform spectrum. Centered on the peak value of the normalized cross-correlation power spectrum, the principal energy of the spectrum is extracted using a mask operator and then subjected to a Fourier transform: In the formula, NaN represents invalid points. , These represent the left and lower boundaries of the signal window in the dual-window mask along the vertical axis (x) and the vertical axis (y), respectively. , Here, R represents the right and top boundaries of the signal window in the dual-window mask along the vertical axis (x) and vertical axis (y), respectively, while R represents the size of the blank window in the dual-window mask along the vertical axis (x) and vertical axis (y). , These represent the spatial coordinates along the vertical axis (x) and the vertical axis (y), respectively, where the signal window size is... , , , And the value of the blank window size R, in The value is obtained at that time, where N is the size of the complete spectrum and S is the ratio between the signal window amplitude and the complete amplitude; Step 2.4: Perform singular value decomposition on the principal energy of the normalized cross-correlation power spectrum obtained in Step 2.3 to extract the offset r of motion disturbance. n The normalized cross-correlation power spectrum after decomposition is as follows: In the formula, U1 and V1 are the left and right singular matrices of the normalized cross-correlation power spectrum, respectively. Let T be its eigenvalue matrix, and the superscript T is the transpose of the matrix; Step 2.5: Extract the principal components of the normalized cross-correlation power spectrum, specifically: In the formula, , represents a matrix where the element in the first column of the first row is 1, and all other elements are 0; Step 2.6: Perform phase expansion on the first row elements of the left singular matrix U1, and use the least squares method to linearly fit it to obtain the slope. This slope is the sub-pixel precision offset r caused by motion. n The component in the horizontal x-direction; performing a phase expansion on the first row elements of the right singular matrix V1, and using the least squares method to linearly fit it to obtain the slope, the fitted slope being the sub-pixel precision offset r caused by motion. n The component in the vertical y-direction; Step 2.7: Offset r caused by the extracted motion n Compensation is performed on the center spectrum of the +1 level spectrum of the sample: 。 5. The robust, motion-resistant structured light illumination-based super-resolution microscopy method based on principal component analysis according to claim 4, characterized in that, The radius of the Hanning filter is The peak distance between level 0 and level 1 spectrum.
6. The robust, motion-resistant structured light illumination-based super-resolution microscopy method based on principal component analysis according to claim 1, characterized in that, Step 5 involves reconstructing the super-resolution image by merging the separated spectra and then using Wiener deconvolution to reconstruct the real-time super-resolution image. In the formula, the subscript i represents the spectral order, i = 0, +1, -1, and the subscript d represents the d-th illumination component. Let d be the wave vector of the d-th illumination direction. Let w be the i-th order spectral component of the d-th illumination direction without optical transfer function and illumination parameter components, and w be the Wiener constant.
Citation Information
Patent Citations
Tensor hyperspectral image spectrum-space dimensionality reduction method based on deep convolutional neural network
CN106023065A
Super-resolution reconstruction method based on structured light illumination
CN111308682A