A spatial phase-shifted structured light super-resolution imaging system and method

Abstract

The present invention discloses a spatial phase-shifted structured light super-resolution imaging system and method. The system forms structured light to illuminate an object to be measured by coherently superimposing two linearly polarized light beams with mutually orthogonal polarization directions. After the structured light is split by a non-polarizing beam splitter prism, it is received by a corresponding first image sensor, a second image sensor, and a third image sensor respectively through a first polarizer, a second polarizer, and a third polarizer at different angles. The system generates a spatial phase shift by using the polarization characteristics of coherent light, and simultaneously collects three image information through three image sensors, thereby greatly compressing the system image collection time, and providing a new solution for improving the imaging efficiency and processing speed of the structured light super-resolution system.

Description

Technical Field

[0001] The present invention relates to the technical field of microscopic three-dimensional morphology measurement, and in particular to a spatial phase-shifted structured light super-resolution imaging system and method. Background Art

[0002] With the increase in application requirements, micro-nano devices are required to have a larger information capacity, which has prompted the development of micro-nano devices from two-dimensional and micron-level to three-dimensional and nano-level, and has put forward an urgent demand for nano-level three-dimensional structure detection technology. Among the many measurement methods, optical measurement is widely used due to its advantages such as non-contact, non-destructive, high measurement speed, simple system structure, and strong environmental adaptability. However, due to the existence of the optical diffraction limit, the lateral resolution of traditional optical microscopes is greatly limited. How to break through the optical diffraction limit and realize super-resolution optical measurement has always been one of the hot and difficult issues that the scientific community is concerned about.

[0003] To address this problem, researchers have developed a variety of optical super-resolution microscopy techniques, including stimulated emission depletion microscopy (STED) based on point spread function modification, photoactivated localization microscopy (PALM), stochastic optical reconstruction microscopy (STROM), and structured illumination microscopy (SIM). Among them, structured illumination microscopy is currently the most widely used technology in super-resolution optical microscopy of living cells because of its non-specific demand for fluorescent dyes and fast wide-field imaging advantages. However, the traditional structured light super-resolution demodulation algorithm requires at least three structured light images with phase shift differences at the same angle. At the same time, in order to achieve super-resolution effects in all directions of the image, at least three directions of structured light images need to be collected, that is, at least nine image acquisitions are required to achieve super-resolution effects, and the acquisition efficiency is low. Such acquisition efficiency greatly limits the real-time performance of the structured light super-resolution method. Therefore, how to improve the acquisition efficiency of structured light super-resolution measurement is an urgent problem to be solved. Summary of the invention

[0004] At least one of the purposes of the present invention is to provide a spatial phase-shifted structured light super-resolution imaging system and method to overcome the problems of low acquisition efficiency of structured light super-resolution measurement in the above-mentioned prior art, which can effectively improve the image acquisition efficiency and greatly shorten the time of super-resolution imaging.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention includes the following aspects.

[0006] A spatial phase-shifted structured light super-resolution imaging system, comprising a computer and a light source, a microscope objective lens, a quarter-wave plate, a third condenser lens, a second non-polarizing beam splitter prism, a first polarizer, a first image sensor, a third non-polarizing beam splitter prism, a second polarizer, a second image sensor, a third polarizer and a third image sensor, which are sequentially arranged along an optical path.

[0007] The light source is configured to generate coherent superposition of two linearly polarized light beams with mutually orthogonal polarization directions to form structured light to illuminate the object to be measured;

[0008] The center of each optical element in the optical path is kept on the optical axis, so that the light beam emitted by the light source passes through the object to be measured, and then passes through the microscope objective lens, the quarter wave plate, and the third condenser lens in sequence to be incident on the second non-polarized beam splitter prism, and is divided into a first light beam and a second light beam with the same energy; the first light beam passes through the second non-polarized beam splitter prism, is received by the first image sensor through the first polarizer, the second light beam is reflected by the second non-polarized beam splitter prism, and is divided into a third light beam and a fourth light beam through the third non-polarized beam splitter prism; the third light beam is received by the second image sensor through the second polarizer, and the fourth light beam is received by the third image sensor through the third polarizer;

[0009] The computer is connected to the first image sensor, the second image sensor, and the third image sensor in communication, and is configured to demodulate the original high-frequency information, reconstruct the object light wave, and obtain three structured light images with fixed phase differences.

[0010] Furthermore, it also includes a He-Ne laser, an attenuation plate, a beam expander, a first non-polarized beam splitter, a spatial light modulator, a first condensing lens, a filter, and a second condensing lens, which are sequentially arranged along the optical path direction;

[0011] The light source is generated by a He-Ne laser and is configured such that the generated light beam passes through an attenuation plate and a beam expander in sequence, passes through a first non-polarized beam splitter prism, enters a spatial light modulator, is modulated by the spatial light modulator, generates a light beam incident in multiple directions, the modulated light beam is reflected back to the first non-polarized beam splitter prism, is reflected by the first non-polarized beam splitter prism, enters a 4f system composed of a first condensing lens and a second condensing lens, a filter is placed at the Fourier plane of the 4f system, the filter is a special filter with two orthogonal polarization films, and after the light beam passes through the 4f system, two orthogonal polarized light beams are left to illuminate the object to be measured.

[0012] The first image sensor, the second image sensor and the third image sensor are all area array color cameras, linear array color cameras, area array black and white cameras or linear array black and white cameras, and their photosensitive devices are CMOS or CCD.

[0013] Furthermore, the multi-directional incident light beams are generated by a spatial light modulator, a physical grating or a double prism.

[0014] Furthermore, the spatial light modulator is loaded with a binary grating, the grating period T=81 μm, the grating is set at angles of 0°, 60° and 120° respectively, and the light beam passes through the spatial light modulator to generate light beams of three diffraction orders -1, 0 and +1.

[0015] Preferably, the angle between the quarter wave plate and the x-axis direction is 45°, and the deflection angles of the first polarizer, the second polarizer and the third polarizer relative to the horizontal direction are set to be different.

[0016] Furthermore, the structured light image of the object to be measured is formed by reflection or transmission.

[0017] Furthermore, the positions of the quarter wave plate and the third condensing lens can be interchanged.

[0018] Preferably, the wavelength of the light beam generated by the He-Ne laser is 632.8 nm, and the beam expander expands the diameter of the light spot by 2 to 20 times.

[0019] A spatial phase shift structured light super-resolution imaging method comprises the following steps:

[0020] Step 1: Obtain two beams of orthogonal polarized light carrying object light information O(x, y);

[0021] Step 2: Two beams of orthogonal polarized light pass through a quarter wave plate, and the linear polarized light becomes left-handed or right-handed circularly polarized light;

[0022] Step 3: The object light and the reference light pass through the first polarizer, the second polarizer and the third polarizer respectively, and the corresponding first image sensor, the second image sensor and the third image sensor collect three structured light illumination images;

[0023] Step 4: Perform structured light demodulation on the three structured light illumination images collected, and obtain the three separated spectrums by transforming the equation: IGL(f), IGN(f), and IGL(f). Move the misplaced spectrums IGL(f) and IGH(f) back to the correct position, and superimpose and fuse the three parts of information in the correct position to expand the frequency domain information.

[0024] Step 5: Load diffraction gratings in different directions on the spatial light modulator SLM (Spatial Light Modulator, SLM), repeat steps 1 to 4, and obtain super-resolution measurement results in all directions.

[0025] In summary, due to the adoption of the above technical solution, the present invention has at least the following beneficial effects:

[0026] By setting a first polarizer, a second polarizer and a third polarizer, and setting the first polarizer, the second polarizer and the third polarizer at different angles respectively, and modulating the structured light in a polarization manner, the phase of the structured light can be changed, and the time phase shift acquisition form is converted into a space phase shift acquisition form, so that the first image sensor, the second image sensor and the third image sensor receive different image information; by setting the first image sensor, the second image sensor and the third image sensor, three structured light images can be acquired at the same time, the system improves the acquisition efficiency by three times, and shortens the time of super-resolution imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the structure of a spatial phase-shifted structured light super-resolution imaging system according to an exemplary embodiment of the present invention.

[0028] Figure 2 FIG. 4 is a schematic diagram of the structure of a spatial phase-shifted structured light super-resolution imaging system according to another exemplary embodiment of the present invention.

[0029] Figure 3 Diffraction grating patterns in different directions loaded on the spatial light modulator of an exemplary embodiment of the present invention.

[0030] Figure 4 Figure 2 is a flow chart of the demodulation algorithm.

[0031] Symbols in the figure: 1-He-Ne laser, 2-attenuation plate, 3-beam expander, 4-first non-polarizing beam splitter prism, 5-spatial light modulator, 6-first focusing lens, 7-filter, 8-second focusing lens, 9-object to be measured, 10-microscope objective, 11-quarter wave plate, 12-third focusing lens, 13-second non-polarizing beam splitter prism, 14-first polarizer, 15-first image sensor, 16-third non-polarizing beam splitter prism, 17-second polarizer, 18-second image sensor, 19-third polarizer, 20-third image sensor. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments to make the purpose, technical solutions and advantages of the present invention more clearly understood. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] refer to Figure 1The spatial phase-shifted structured light super-resolution imaging system of the exemplary embodiment of the present invention mainly includes a light source, an object to be measured 9, a microscope objective 10, a quarter-wave plate 11, a third condensing lens 12, a second non-polarizing beam splitter prism 13, a first polarizer 14, a first image sensor 15, a third non-polarizing beam splitter prism 16, a second polarizer 17, a second image sensor 18, a third polarizer 19 and a third image sensor 20, which are arranged in sequence along the light path; the center of each optical element in the light path is kept on the optical axis.

[0034] The light source is configured to generate coherent superposition of two linearly polarized light beams with orthogonal polarization directions to form structured light illumination of the object to be measured 9. After the image generated by the illumination is magnified by the microscope objective 10, it is transmitted through the quarter-wave plate 11 and the third condensing lens 12 respectively, and finally enters the second non-polarized beam splitter prism 13 to form a first light beam and a second light beam with the same energy. The first light beam passes through a first polarizer 14 with a horizontal angle of 0° and enters a first image sensor 15; the second light beam enters a third non-polarized beam splitter prism 16 to form a third light beam and a fourth light beam, wherein the third light beam passes through a second polarizer 17 with a horizontal angle of 30° and enters a second image sensor 18, wherein the fourth light beam passes through a third polarizer 19 with a horizontal angle of 60° and enters a third image sensor 20; the computer is connected to the first image sensor, the second image sensor, and the third image sensor through communication, and the original high-frequency information is demodulated through a structured light demodulation algorithm, and the object light wave is reconstructed to obtain three structured light images with a fixed phase difference.

[0035] refer to Figure 2 The spatial phase shift structured light super-resolution imaging system of the exemplary embodiment of the present invention further includes: a He-Ne laser 1, an attenuation plate 2, a beam expander 3, a first non-polarization beam splitter prism 4, a spatial light modulator 5, a first condensing lens 6, a filter 7 and a second condensing lens 8 arranged in sequence along the optical path direction; the He-Ne laser 1 generates a light beam, the light beam passes through the attenuation plate 2 and the beam expander 3, and enters the spatial light modulator 5 through the first non-polarization beam splitter prism 4, and is modulated by the spatial light modulator 5. The modulated light beam is reflected back to the first non-polarization beam splitter prism 4, and is reflected by the first non-polarization beam splitter prism 4 into a 4f system composed of a first condensing lens 6 and a second condensing lens 8. A filter 7 is placed at the Fourier plane of the 4f system, and the filter 7 is a special filter with two orthogonal polarization films (refer to Figure 2 The filter 7 indicated by the middle arrow, after the light beam passes through the 4f system, only two orthogonal polarized light beams are left to form structured light illumination for the object to be measured 9.

[0036] The wavelength of the light beam generated by the He-Ne laser 1 is 632.8 nm. The attenuation plate 2 is used to attenuate the intensity of the light beam. The beam expander 3 is used to generate a light beam with a larger diameter, which can expand the diameter of the light beam by 2 to 20 times. The spatial light modulator 5 is used to generate an incident light beam in multiple directions (at least 3 directions). The spatial light modulator 5 is loaded with a binary grating with a grating period of T=81 μm. The light beam passes through the spatial light modulator 5 to generate three diffraction order light beams of -1, 0, and +1, such as Figure 3 As shown, the grating setting angles are 0°, 60° and 120° respectively. By setting different angles, structured light illumination in different directions can be generated, so that high-frequency information of the object in all directions can enter the image sensor.

[0037] The deflection angles of the first polarizer 14, the second polarizer 17 and the third polarizer 19 relative to the horizontal direction are not limited to 0°, 30° and 60°, and can also be other deflection angles. By setting different deflection angles, different phase shifts can be introduced into the structured light, so that the first image sensor 15, the second image sensor 18 and the third image sensor 20 receive different image information; the first image sensor 15, the second image sensor 18 and the third image sensor 20 can all be area array color cameras, linear array color cameras, area array black and white cameras or linear array black and white cameras, and their photosensitive devices use CMOS or CCD; by arranging the first image sensor 15, the second image sensor 18 and the third image sensor 20 in the direction of the optical path, three structured light images with fixed phase differences can be collected at the same time, so that structured light images in three directions can be collected by only performing three image collections, thereby achieving a super-resolution effect and improving the collection efficiency of structured light super-resolution measurement.

[0038] In the spatial phase-shifted structured light super-resolution imaging system of the present invention, the multi-directional incident light beam is a coherent linearly polarized light, and the multi-directional incident light beam can be generated by a spatial light modulator, or by a physical grating, a double prism, etc., and the direction of the incident light beam can be adjusted to achieve super-resolution measurement of the object to be measured; the angle between the quarter-wave plate 11 and the x-axis direction (horizontal direction) is 45°, and the positions of the quarter-wave plate 11 and the third focusing lens 12 can be interchanged; the structured light image of the object to be measured is formed by reflection or transmission, except for the use of Figure 2 In addition to being formed by the method of forming, it can also be formed by other methods.

[0039] The spatial phase-shifted structured light super-resolution imaging system of the present invention demodulates the original high-frequency information through a structured light demodulation algorithm, reconstructs the object light wave, and simultaneously obtains three structured light images with fixed phase differences. The method for obtaining the structured light images includes the following steps:

[0040] Step 1: Two orthogonal polarized light beams carrying object light information O(x, y) have Jones matrices that can be expressed as:

[0041]

[0042]

[0043] In the above formula, a represents the amplitude of polarized light, They represent the phase of polarized light respectively;

[0044] Step 2: The angle between the fast axis (or slow axis) of the quarter wave plate and the x-axis is 45°. The Jones matrix of the quarter wave plate is:

[0045]

[0046] When two orthogonal polarized lights pass through a quarter wave plate, the linearly polarized light becomes circularly polarized light (left-handed circularly polarized light or right-handed circularly polarized light), and its Jones matrix is:

[0047]

[0048]

[0049] Step 3: Make the object light and the reference light pass through a polarizer whose transmission direction and the x-axis direction have an angle α. Its Jones matrix can be expressed as:

[0050]

[0051] The Jones matrix of the final outgoing light is:

[0052]

[0053] The final interference light intensity of the fast axis and the slow axis in the transmission direction is:

[0054]

[0055] In the above formula, I is the light intensity, When the included angles of the first polarizer 14, the second polarizer 17 and the third polarizer 19 are 0°, 30° and 60° respectively, the interference light intensities of the structured light illumination images collected by the corresponding image sensor are:

[0056]

[0057]

[0058]

[0059] Step 4: Perform structured light demodulation on the three structured light illumination images obtained. The process is as follows: Figure 4 As shown: Since the two spectrum information overlap each other, to fully restore the spectrum information, the three separated spectrum parts can be obtained by transforming the equation:

[0060]

[0061] Then the displaced spectra IGL(f), IGH(f) (refer to Figure 4 The spectrum separation part in the graph, IGH(f) is located to the left of IGN(f) and IGL(f) is located to the right of IGL(f)) is moved back to the correct position (refer to Figure 4 In the spectrum shift and superposition part, IGL(f) is located on the left side of IGL(f), and IGN(f) is located on the right side of IGN(f)), and the three parts of information in the correct position are superimposed and fused to expand the frequency domain information;

[0062] Step 5: Load diffraction gratings in different directions on the spatial light modulator SLM, repeat steps 1 to 4, and obtain super-resolution measurement results in all directions.

[0063] The above is only a detailed description of the specific implementation of the present invention, rather than a limitation of the present invention. Various substitutions, modifications and improvements made by those skilled in the relevant art without departing from the principle and scope of the present invention should be included in the protection scope of the present invention.

Claims

1. A spatial phase shift structured light super-resolution imaging system, characterized in that: The invention comprises a computer and a light source, a microscope objective lens (10), a quarter wave plate (11), a third condenser lens (12), a second non-polarizing beam splitter prism (13), a first polarizing plate (14), a first image sensor (15), a third non-polarizing beam splitter prism (16), a second polarizing plate (17), a second image sensor (18), a third polarizing plate (19) and a third image sensor (20) which are sequentially arranged along the optical path direction; The light source is configured to generate coherent superposition of two linearly polarized light beams with mutually orthogonal polarization directions to form structured light to illuminate the object to be measured (9); The center of each optical element in the optical path is maintained on the optical axis, so that the light beam emitted by the light source passes through the object to be measured (9), and then passes through the microscope objective lens (10), the quarter wave plate (11), and the third condenser lens (12) in sequence to be incident on the second non-polarization beam splitter prism (13), and is divided into a first light beam and a second light beam with the same energy; the first light beam passes through the second non-polarization beam splitter prism (13), is received by the first image sensor (15) via the first polarizer (14), the second light beam is reflected by the second non-polarization beam splitter prism (13), and is divided into a third light beam and a fourth light beam via the third non-polarization beam splitter prism (16); the third light beam is received by the second image sensor (18) via the second polarizer (17), and the fourth light beam is received by the third image sensor (20) via the third polarizer (19); The computer is in communication connection with the first image sensor (15), the second image sensor (18), and the third image sensor (20), and is configured to demodulate the original high-frequency information, reconstruct the object light wave, and obtain three structured light images with fixed phase differences.

2. The spatial phase shift structured light super-resolution imaging system according to claim 1, characterized in that: It also includes a He-Ne laser (1), an attenuation plate (2), a beam expander (3), a first non-polarized beam splitter (4), a spatial light modulator (5), a first condensing lens (6), a filter (7) and a second condensing lens (8) which are sequentially arranged along the optical path. The light source is generated by a He-Ne laser (1), and is configured such that the generated light beam passes through an attenuation plate (2), a beam expander (3), a first non-polarized beam splitter prism (4), and enters a spatial light modulator (5). The light beam is modulated by the spatial light modulator (5) to generate a light beam incident in multiple directions. The modulated light beam is reflected back to the first non-polarized beam splitter prism (4), and is reflected by the first non-polarized beam splitter prism (4) to enter a 4f system composed of a first condensing lens (6) and a second condensing lens (8). A filter (7) is placed at the Fourier plane of the 4f system. The filter (7) is a special filter with two orthogonal polarization films. After the light beam passes through the 4f system, two orthogonal polarized light beams are left to illuminate the object to be measured (9).

3. The spatial phase shift structured light super-resolution imaging system according to claim 1, characterized in that: The first image sensor (15), the second image sensor (18) and the third image sensor (20) are all area array color cameras, linear array color cameras, area array black-and-white cameras or linear array black-and-white cameras, and their photosensitive devices are CMOS or CCD.

4. The spatial phase shift structured light super-resolution imaging system according to claim 2, characterized in that: The multi-directional incident light beams are generated by a spatial light modulator (5), a physical grating or a double prism.

5. The spatial phase shift structured light super-resolution imaging system according to claim 2, characterized in that: The spatial light modulator (5) is loaded with a binary grating, the grating period T=81 μm, the grating is set at angles of 0°, 60° and 120° respectively, and the light beam passes through the spatial light modulator (5) to generate light beams of three diffraction orders -1, 0 and +1.

6. The spatial phase shift structured light super-resolution imaging system according to claim 1, characterized in that: The angle between the quarter wave plate (11) and the x-axis direction is 45°, and the deflection angles of the first polarizer (14), the second polarizer (17) and the third polarizer (19) relative to the horizontal direction are set to be different.

7. The spatial phase shift structured light super-resolution imaging system according to claim 1, characterized in that: The structured light image of the object to be measured (9) is formed by reflection or transmission.

8. The spatial phase shift structured light super-resolution imaging system according to claim 1, characterized in that: The positions of the quarter wave plate (11) and the third condensing lens (12) can be interchanged.

9. The spatial phase shift structured light super-resolution imaging system according to claim 2, characterized in that: The wavelength of the light beam generated by the He-Ne laser (1) is 632.8 nm, and the beam expander (3) expands the diameter of the light spot by 2 to 20 times.

10. A spatial phase shift structured light super-resolution imaging method, characterized in that: A spatial phase shift structured light super-resolution imaging system according to any one of claims 1 to 9 is used, and comprises the following steps: Step 1: Obtain two beams of orthogonal polarized light carrying object light information O(x, y); Step 2: Two beams of orthogonal polarized light pass through a quarter wave plate, and the linear polarized light becomes left-handed or right-handed circularly polarized light; Step 3: The object light and the reference light pass through the first polarizer, the second polarizer and the third polarizer respectively, and the corresponding first image sensor, the second image sensor and the third image sensor collect three structured light illumination images; Step 4: Perform structured light demodulation on the three structured light illumination images collected, and obtain the three separated spectrum parts by transforming the equation: IGL(f), IGN(f), IGL(f), move the misplaced spectrum IGL(f) and IGH(f) back to the correct position, and superimpose and fuse the three parts of information in the correct position to expand the frequency domain information; Step 5: Load diffraction gratings in different directions on the spatial light modulator SLM, repeat steps 1 to 4, and obtain super-resolution measurement results in all directions.

Images