A method for extracting the initial phase of structured light in structured light illumination microscopy imaging

Abstract

The present invention proposes a method for extracting the initial phase of structured light in structured light illumination microscopic imaging, and belongs to the field related to optical imaging. Based on the image recombination transformation method, the present invention establishes a general form of image recombination transformation formula, which does not require equal step size and special phase shift value π / 2, and realizes the extraction of the initial phase of the cosine structured light field under any unknown phase shift value. It makes up for the shortcomings of the image recombination transformation method. The method Fourier transforms three recorded structured light illumination images with different initial phases into three spectrum images. Then, based on iterative optimization and estimation of phase shift values, two high-frequency spectrum components introduced into the spectrum image due to cosine structured light illumination are separated, and the maximum intensity difference of the two separated spectrum components at the structured light spatial frequency is calculated and found. At this time, the corresponding estimated phase shift value is the actual phase shift value between the structured light fields. Based on the actual phase shift value between the images and the three recorded images, the initial phase value is obtained by a general form of recombination transformation method.

Description

Technical Field

[0001] The invention belongs to the field of optical imaging, and relates to a method for extracting the initial phase of structured light in structured light illumination microscopic imaging. Background Art

[0002] Optical microscopy has been widely used in research in fields such as biology and life sciences, but the imaging resolution is limited by the optical diffraction limit and cannot meet the needs of observing fine microbial structures. In order to break the limitations of the diffraction limit, a variety of super-resolution optical microscopy techniques have been developed in recent years. Among them, the Structured Illumination Microscopy (SIM) proposed by Gustafsson in 2000 is based on the "Moiré fringe" principle. It uses cosine structured light fringe illumination to encode high-frequency information into the low-frequency area and be received by the detector, achieving a resolution twice that of ordinary wide-field imaging. SIM only changes the illumination part of the imaging system, and other parts do not need to be changed. It is compatible with a variety of imaging systems and has become the most widely used super-resolution imaging technology.

[0003] SIM is not a direct super-resolution imaging technology. It usually requires the use of a phase shift method to collect three structured light illumination images with different initial phases and jointly reconstruct a super-resolution image. How to accurately extract the initial phases of three different illumination structured light fields in SIM super-resolution reconstruction is the most critical issue in obtaining high-quality super-resolution reconstruction imaging. Currently, the commonly used methods for extracting the initial phase of the structured light field are the peak phase method, the autocorrelation method, and the image reconstruction transformation method, but they all have certain limitations. Among them, the peak phase method and the autocorrelation method both obtain approximate solutions to the initial phase of the structured light under the premise of satisfying the approximate conditions. However, since the approximation limits the conditions that the modulation degree of the structured light field is high enough and that the imaging does not contain defocused background information without structured light fringes, the peak phase method and the autocorrelation method are not suitable for imaging of low modulation degrees of the structured light field and thick samples. The image reconstruction transformation method is an accurate initial phase extraction method that is not limited by the premise of approximate conditions. Its basic extraction process is as follows:

[0004] When the phase shift step length between structured light fields is equal and is a special value of π / 2, three images of samples illuminated by structured light with different initial phases are collected. The mathematical expression can be written as:

[0005]

[0006] Where: r represents the two-dimensional coordinate in the spatial domain; m, p and I0 are the modulation, spatial frequency and average light intensity of the illumination cosine structured light respectively; is the initial phase of the structured light field and D1(r), D2(r), and D3(r) are the collected images; Sin (r) indicates the in-focus sample information; S out (r) represents the out-of-focus background information; H(r) represents the point spread function of the imaging system, Represents a convolution operation.

[0007] The above three images D1(r), D2(r), and D3(r) are recombined into a new image by the following formula:

[0008]

[0009] In the formula, j is an imaginary unit. The spectrum of the image obtained by Fourier transform can be expressed as:

[0010]

[0011] In the formula, k represents the two-dimensional coordinate in the frequency domain; ~ represents the Fourier transform.

[0012] At the spatial frequency p of the cosine structured light, there is only one complex exponential term The initial phase of the light field can be calculated for:

[0013]

[0014]

[0015] From the above process of extracting the initial phase, it can be found that the image reconstruction transformation method removes the influence of the defocused background by subtracting images, and the whole process does not make any approximation, is not affected by the structured light modulation, and can extract the precise value of the initial phase of the cosine structure light field. However, this method is only applicable to the case where the step size is equal and is a special value of π / 2. In the actual phase shift process, the phase shift value between the cosine structure fringes is often not the ideal phase shift value of π / 2 due to air disturbances, environmental vibrations, and inevitable errors of the phase shifter. The resulting phase shift error will seriously reduce the accuracy of the traditional image reconstruction transformation method in obtaining the initial phase. Summary of the invention

[0016] In order to solve the deficiencies in the prior art, the present invention proposes a method for extracting the initial phase of structured light in structured light illumination microscopy imaging. Based on the image reconstruction transformation method, the image reconstruction method is improved, and the image reconstruction transformation method is extended to a more general form, so as to realize the extraction of the initial phase of the structured light field under any unknown phase shift value.

[0017] The technical route of the present invention is: when imaging a sample with structured light illumination, two additional high-frequency spectrum components will be introduced and overlapped with the low-frequency spectrum components. Three images with structured light illumination of different initial phases are recorded, and the overlapped spectrum components are separated in combination with the phase shift value between the images. If the actual phase shift value between the images is accurately known, then the two high-frequency spectrum components can be accurately separated, and at this time, the intensity difference between the two separated high-frequency spectrum components at the structured light spatial frequency p is the largest, but if the actual phase shift value between the images is not accurately known, then the intensity difference at the spatial frequency p is small. Based on this characteristic, the actual phase shift value between images can be determined by iteratively optimizing the estimated phase shift value. The three recorded images are recombined using the actual phase shift value between the acquired images, and the initial phase of the illumination structured light field is extracted at the p position of the recombined image spectrum diagram.

[0018] The specific steps are as follows:

[0019] Step 1: Record three structured light illumination images with different initial phases:

[0020] Through laser illumination, the cosine structured light field generated on the sample surface by spatial light modulator can be expressed as:

[0021]

[0022] Where: I(r) is the cosine structured light field; r represents the two-dimensional coordinate in the spatial domain; m, p and I0 are the modulation, spatial frequency and average light intensity of the illumination cosine structured light respectively; is the initial phase of the cosine structure light field;

[0023] The spatial light modulator is controlled to change the phase shift value of the structured light field, and three structured light fields with different initial phases are generated in sequence. The structured light illuminates the sample and is imaged by the microscope system. Three structured light illumination images 1, 2, and 3 with different initial phases are recorded in sequence by the CCD on the image plane of the imaging system. The expression of the image is:

[0024]

[0025] Where: D1(r), D2(r), D3(r) are recorded images 1, 2, 3 respectively; is the initial phase of image 1, is the phase shift value; S in (r) indicates the in-focus sample information; S out (r) represents the out-of-focus background information; H(r) represents the point spread function of the imaging system; Represents a convolution operation.

[0026] Step 2: Get the actual phase shift value between images:

[0027] Perform Fourier transform on the three collected images to obtain three spectrum images 4, 5, and 6. The frequency domain image expression is:

[0028]

[0029] Where: k represents the two-dimensional coordinate in the frequency domain; ~ represents Fourier transform; j is the imaginary unit; Two additional high-frequency spectral components introduced for structured illumination imaging;

[0030] Set the initial estimated phase shift value The estimated phase shift value is used to construct the inverse matrix for the three mutually aliased spectrum components contained in spectrum images 4, 5, and 6. For separation, the specific formula is as follows:

[0031]

[0032] Construct the following formula to calculate the intensity difference SD of the separated spectral components ψ2(k) and ψ3(k) at the spatial frequency p:

[0033] SD=ψ2(p)-ψ3(p) (10)

[0034] Based on the characteristics of the intensity difference between spectral components, the estimated phase shift value is continuously optimized and iteratively And calculate the value of SD. When SD reaches the maximum value, and The corresponding value is the actual phase shift value and

[0035] Step 3: Extract the initial phase of the illumination structured light:

[0036] Using the actual phase shift value between images obtained in step 2 and The images 1, 2, and 3 collected in step 1 are recombined by the following formula to obtain a new image 7 with the out-of-focus background image removed:

[0037]

[0038] Perform Fourier transform (FFT) on the reconstructed image 7 to obtain its spectrum image 8, and calculate the value of the spectrum image 8 at the spatial frequency p, which can be expressed as:

[0039]

[0040] By taking the arg function arg in MATLAB to calculate the arg value of the above formula, the initial phase can be calculated. The mathematical formula for the value is:

[0041]

[0042] Using the actual phase shift value obtained by Calculate the other two initial phase values

[0043]

[0044] The beneficial effects of the present invention are:

[0045] (1) The present invention avoids the need to limit the modulation degree of the structured light field to be sufficiently high and ignore the approximate conditions such as the defocused background in the peak phase method and the autocorrelation method. The present invention does not perform approximate processing, is not limited by the modulation degree of the structured light field, and removes the influence of the defocused background by image reconstruction.

[0046] (2) The present invention gets rid of the requirement for a specific phase shift value in the image reconstruction transformation method. In the present invention, the actual phase shift value between images is first obtained by separating the characteristics between the spectral intensity differences, and then the images are reconstructed using the measured actual phase shift value, and then the initial phase is extracted, which broadens the scope of application of the image reconstruction transformation method and greatly reduces the requirements for the performance of the imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the method of the present invention;

[0048] Figure 2 It is a fluorescence microscopy imaging system based on laser interferometric structured light illumination;

[0049] Figure 3 To separate the intensity distribution of spectral components as the difference between the estimated phase shift value and the accurate phase shift value changes;

[0050] Figure 4(a)-Figure 4(d) The results of extracting the initial phase of structured light by the method of the present invention and the traditional method are compared under different structured light modulation degrees.

[0051] Attached Figure 2 Marked as follows: 1—laser light source, 2—first lens, 3—second lens, 8—third lens, 10—fourth lens, 4—first half-wave plate, 6—second half-wave plate, 5—reflective spatial light modulator SLM, 7—reflector, 9—spatial filter, 11—dichroic mirror, 12—microscope objective, 13—stage, 14—filter, 15—tube lens, 16—digital camera CCD. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0053] The initial phase extraction process of the present invention is as follows Figure 1As shown. The structured light illumination sample is imaged, and three structured light illumination images with different initial phases are recorded. The recorded images are Fourier transformed to obtain three spectrum images containing unknown phase shift values. In order to obtain the actual value of the unknown phase shift value, the two high-frequency spectrum components introduced into the spectrum image due to cosine structured light illumination are separated by iteratively setting the initial estimated phase shift value, and the intensity difference between the two separated high-frequency spectrum components at frequency p is calculated. When the negative value of the absolute value of the intensity difference is the smallest, the corresponding estimated phase shift value is output, which is the actual phase shift value between the structured light fields. The actual phase shift value obtained is then used to recombine the three recorded original images to obtain a new reconstructed image. The reconstructed image is Fourier transformed into the frequency domain, and the angle is taken at the frequency p in the spectrum diagram to calculate the value of the initial phase.

[0054] Fluorescence microscopy imaging system based on laser interferometric structured light illumination Figure 2 As shown: it includes a laser light source 1, a beam expansion and collimating lens group (lens 2 and lens 3) arranged behind the laser light source 1, a half-wave plate 4 arranged behind the beam expansion and collimating lens group, a reflective spatial light modulator SLM5 arranged behind the half-wave plate 4, a half-wave plate 6 arranged behind the reflective spatial light modulator SLM5, a reflector 7 arranged behind the half-wave plate 6, a lens 8 arranged behind the reflector 7, a spatial filter 9 arranged behind the lens 8, a lens 10 arranged behind the spatial filter 9, a dichroic mirror 11 arranged behind the lens 10, a microscope objective lens 12 and a stage 13 arranged behind the dichroic mirror 11, a filter 14 arranged in the fluorescent light path, a tube lens 15 and a digital camera CCD 16.

[0055] Based on the above process, the present invention is further explained in the following examples using fluorescent microspheres as samples, but the present invention is not limited to this example.

[0056] Step 1: Record three structured light illumination images with different initial phases:

[0057] like Figure 2 In the fluorescence microscopy imaging system based on laser interferometric structured light illumination, a 457nm laser is modulated in polarization state by a half-wave plate 4 and irradiated onto a reflective spatial light modulator SLM5 loaded with a cosine grating. The reflection generates multi-order diffraction beams, and the diffraction beams other than the ±1st order are filtered out by a spatial filter 9. The dichroic mirror reflects the beams into the microscope objective lens and interferes with each other to generate two-dimensional structured light to illuminate the sample.

[0058] The fluorescent microsphere sample is placed on the stage, and after structured light excitation illumination, the fluorescent microsphere emits fluorescent light and the image is recorded by the CCD of the digital camera as image 1.

[0059] The spatial light modulator is used to complete a phase shift of π / 2 steps on the structured light, and the image is recorded by CCD, which is recorded as image 2. The spatial light modulator is used again to complete a phase shift of π / 2 steps on the structured light, and the image is recorded by CCD, which is recorded as image 3. Images 1, 2, and 3 can be expressed by mathematical formulas:

[0060]

[0061] Where: r represents the two-dimensional coordinate in the spatial domain; m, p and I0 are the modulation, spatial frequency and average light intensity of the illumination cosine structured light respectively; is the initial phase of the cosine structured light field. D1(r), D2(r), and D3(r) are the collected images 1, 2, and 3 respectively; S in (r) indicates the in-focus sample information; S out (r) represents the out-of-focus background information; H(r) represents the point spread function of the imaging system; represents the convolution operation. Considering the influence of air disturbance, environmental vibration and other factors, the actual phase shift value is not the ideal π / 2, which is recorded as

[0062] Step 2: Get the actual phase shift value between images:

[0063] The subsequent process is completed by coding in the computer Matlab programming environment.

[0064] Perform Fourier transform (FFT) on the recorded images 1, 2, and 3 to obtain three spectrum images 4, 5, and 6:

[0065]

[0066] In the formula: k represents the two-dimensional coordinate in the frequency domain; ~ represents Fourier transform; j is the imaginary unit. In the example, the value of the spatial frequency p of the illumination structured light is determined by the peak finding method.

[0067] Set the initial estimated phase shift value Construct an inverse matrix and separate the mutually aliased spectral components in spectrum images 4, 5, and 6 using the following formula.

[0068]

[0069] Construct the following formula to calculate the intensity difference SD of the separated spectral components ψ2(k) and ψ3(k) at the spatial frequency p:

[0070] SD=ψ2(p)-ψ3(p) (18)

[0071] Figure 3The intensity distribution of the separated spectral components ψ2(k) and ψ3(k) varies with the difference between the estimated phase shift value and the actual phase shift value. When the estimated phase shift value is equal to the actual phase shift value, the intensity difference SD of the two spectral components at the spatial frequency p is the largest.

[0072] based on Figure 3 Characteristics of the mid-spectral components, iteratively find the actual phase shift value The ideal phase shift values ​​π / 2 and π are set as the iterative starting points of the estimated phase shift value, and the negative value of the absolute value of the intensity difference -SD is substituted into the MATLAB built-in optimization function fminsearch to estimate the phase shift value. Iterate and optimize to find the minimum value of -SD, and the corresponding estimated phase shift value and The actual phase shift value and

[0073] Step 3: Extract the initial phase of the cosine structured light field:

[0074] In step 2, the actual phase shift value between images is obtained and The size of the actual ground phase shift value obtained and The images 1, 2, and 3 collected in step 1 are recombined by the following formula to obtain a new image 7 with the out-of-focus background image removed:

[0075]

[0076] Perform Fourier transform (FFT) on the reconstructed image 7 to obtain its spectrum image 8, and calculate the value of the spectrum image 8 at the spatial frequency p, which can be expressed as:

[0077]

[0078] By taking the argument function arg in MATLAB and calculating the argument value of the above formula, the initial phase can be obtained. The mathematical formula for the value is:

[0079]

[0080] Using the actual phase shift value obtained in step 2 Calculate the other two initial phase values

[0081]

[0082] The following computer simulation is performed, assuming that the ideal phase shift values ​​are π / 2 and π, and the actual phase shift values ​​are 1.8 rad and 3.5 rad. Figures 4(a), 4(b), 4(c), and 4(d) respectively show the results of extracting the initial phase using the peak phase method, the autocorrelation method, the image reconstruction transformation method, and the method of the present invention when the structured light modulation degree m is 0.01, 0.05, 0.1, and 0.2. The comparison results show that the initial phase extraction result of the method of the present invention is closer to the actual value under the low-key system than the peak phase method and the autocorrelation method, and is not affected by the change of the structured light modulation degree, and the initial phase extraction result is more stable; compared with the image reconstruction transformation method, the method of the present invention can still accurately extract the initial phase of the structured light when there is a phase shift error between the structured lights.

[0083] The present invention proposes a method for extracting the initial phase of structured light in structured light illumination microscopy imaging. The method is not limited by any prerequisites and does not require equal and precise phase shifts, which greatly reduces the requirements for imaging system performance and provides a more universal and convenient initial phase extraction method for structured light illumination imaging.

Claims

1. A method for extracting the initial phase of structured light in structured light illumination microscopy imaging, characterized by: The following steps are involved: Step 1: Record three structured light illumination images with different initial phases: Through laser illumination, the cosine structured light field generated on the sample surface by spatial light modulator can be expressed as: Where: I(r) is the cosine structured light field; r represents the two-dimensional coordinate in the spatial domain; m, p and I0 are the modulation, spatial frequency and average light intensity of the illumination cosine structured light respectively; is the initial phase of the cosine structure light field; The spatial light modulator is controlled to change the phase shift value of the structured light field, and three structured light fields with different initial phases are generated in sequence. The structured light illuminates the sample and is imaged by the microscope system. Three structured light illumination images 1, 2, and 3 with different initial phases are recorded in sequence by the CCD on the image plane of the imaging system. The expression of the image is: Where: D1(r), D2(r), D3(r) are the acquired images 1, 2, 3 respectively; is the initial phase of image 1, is the phase shift value; S in (r) indicates the in-focus sample information; S out (r) represents the out-of-focus background information; H(r) represents the point spread function of the imaging system; Represents the convolution operation; Step 2: Get the actual phase shift value between images: Perform Fourier transform on the three collected images to obtain three spectrum images 4, 5, and 6. The frequency domain image expression is: Where: k represents the two-dimensional coordinate in the frequency domain; ~ represents Fourier transform; j is the imaginary unit; Two additional high-frequency spectral components introduced for structured illumination imaging; Set the initial estimated phase shift value The estimated phase shift value is used to construct an inverse matrix, and the three mutually aliased spectrum components contained in the spectrum images 4, 5, and 6 are analyzed by the constructed inverse matrix. For separation, the specific formula is as follows: Construct the following formula to calculate the intensity difference SD of the separated spectral components ψ2(k) and ψ3(k) at the spatial frequency p: SD=|ψ2(p)|-|ψ3(p)| (5) Based on the characteristics of the intensity difference between spectral components, the estimated phase shift value is continuously optimized and iteratively And calculate the value of SD. When SD reaches the maximum value, and The corresponding value is the actual phase shift value and Step 3: Extract the initial phase of the illumination structured light: Using the actual phase shift value between images obtained in step 2 and The images 1, 2, and 3 collected in step 1 are recombined by the following formula to obtain a new image 7 with the out-of-focus background image removed: Perform Fourier transform (FFT) on the reconstructed image 7 to obtain its spectrum image 8, and calculate the value of the spectrum image 8 at the spatial frequency p, which can be expressed as: By taking the arg function arg in MATLAB to calculate the arg value of the above formula, the initial phase can be calculated. The mathematical formula for the value is: Using the actual phase shift value obtained Calculate the other two initial phase values 2. A method for extracting the initial phase of structured light in structured light illumination microscopy according to claim 1, characterized in that: In step 1, the phase shift step lengths between the three structured light illumination images with different initial phases do not need to be equal and are a special value of π / 2.

Images