Dual-camera alternating acquisition based structured light illumination three-dimensional microscopic method and system

Abstract

The application discloses a kind of based on the structure light illumination three-dimensional microscopic system and method of double camera alternate acquisition, system includes illumination projection module, microscopic imaging module, acquisition module, image reconstruction module and control module, wherein, illumination projection module generates the stripe structure light of different phase shift amount;Microscopic imaging module is imaged to the different axial slice position of sample surface;Acquisition module acquires the two-dimensional intensity image generated at different axial slice position of sample under the illumination of stripe structure light;Control module is according to the preset control timing to illumination projection module, microscopic imaging module and acquisition module are synchronously controlled;Image reconstruction module reconstructs and three-dimensional stack to the two-dimensional intensity image at different axial slice position acquired, obtains the three-dimensional image of sample.The application is by layer by layer regulation and control sample axial position and repeat imaging procedure, ultimately realize full three-dimensional tomography reconstruction, can realize the acquisition of multiple phase shift data in high-speed axial scanning.

Description

Technical Field

[0001] This invention belongs to the field of microscopic imaging technology, specifically relating to a three-dimensional microscopic method and system based on structured light illumination with alternating acquisition by dual cameras. Background Technology

[0002] Optical three-dimensional microscopy not only presents observation results intuitively and three-dimensionally, allowing for a more comprehensive and realistic view of targets, but also provides accurate three-dimensional models and data for the establishment of digital sample databases, possessing significant scientific and application value. In the industrial field, non-transparent reflective industrial three-dimensional microscopy is of great importance for evaluating processing quality and improving processing techniques.

[0003] Various techniques have been proposed for the three-dimensional topography inspection of industrial components. Currently, mainstream techniques include laser confocal microscopy (LSCM), digital holographic microscopy (DHM), white light interferometry (WLI), microscopic fringe projection measurement (MFPP), and structured light microscopy (SIM). LSCM uses a focused spot to perform point-by-point three-dimensional scanning of the sample, while simultaneously using a pinhole to filter the reflected / scattered light from the focused spot on the sample, ultimately achieving three-dimensional tomographic imaging. However, this technique requires point-by-point scanning of the sample, resulting in a relatively slow imaging speed. DHM reconstructs the three-dimensional topography information of the measured object's surface by recording a hologram formed by the reflected light from the sample surface under coherent illumination and a reference light. This method has the advantage of high imaging speed, but suffers from low image signal-to-noise ratio and a small axial measurement range. WLI exhibits excellent sub-nanometer axial resolution in topography measurement; however, its extreme sensitivity to environmental vibrations and stringent experimental requirements severely limit its application in industrial environments. MFPP (Multi-Fold Pattern Propagation) analyzes the three-dimensional morphological distribution of a sample by sequentially projecting sinusoidal fringes of different frequencies and phase values ​​onto the sample and recording the distorted fringes on the surface of the object using a camera. This technique relies on a complex phase unwrapping algorithm, which is highly susceptible to significant reconstruction errors when dealing with surfaces exhibiting abrupt changes or geometric discontinuities.

[0004] Optical section structured light illumination microscopy (OS-SIM) utilizes fringe structured light to illuminate samples and records intensity images of the samples under structured light illumination at different directions and phase shifts, enabling optical section microscopic imaging. Compared to the techniques mentioned above, OS-SIM stands out due to its advantages such as fast imaging speed and minimal sample damage. To improve the speed of 3D imaging, the Hilbert Transform reconstruction algorithm has been introduced. This method reduces the number of original data formats required to reconstruct a single optical section image from three to two, significantly shortening the data acquisition time. However, traditional OS-SIM requires acquiring multiple phase-shifted images when the sample is stationary, which conflicts with high-speed axial scanning of the sample and limits its high-speed 3D imaging capabilities. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a method and system for three-dimensional microscopy using structured light illumination based on alternating dual-camera acquisition. The technical problem to be solved by this invention is achieved through the following technical solution: One aspect of the present invention provides a structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition, comprising an illumination projection module, a microscopic imaging module, an acquisition module, an image reconstruction module, and a control module, wherein... The illumination projection module is used to generate striped structured light with different phase shifts; The microscopic imaging module is used to image different axial slice positions on the surface of the sample under test using the striped structured light with different phase shifts; The acquisition module is used to acquire two-dimensional intensity images generated at different axial slice positions on the sample surface based on striped structured light with different phase shifts. The control module is used to synchronously control the illumination projection module, the microscopic imaging module and the acquisition module according to a preset control sequence. The image reconstruction module is used to reconstruct and stack two-dimensional intensity images at different axial slice positions acquired by the acquisition module to obtain a three-dimensional image of the sample.

[0006] Another aspect of the present invention provides a structured light illumination three-dimensional microscopy method based on alternating acquisition by dual cameras, comprising: S1: Using the structured light illumination three-dimensional microscopy system based on alternating acquisition by dual cameras, two two-dimensional intensity images are obtained at each axial slice position of the sample to be tested; S2: The two-dimensional intensity images at different axial slice locations are reconstructed and stacked in three dimensions using a light slice image reconstruction algorithm based on Hilbert transform to obtain a three-dimensional image of the sample.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The structured light illumination 3D microscopy system based on alternating dual-camera acquisition proposed in this invention dynamically generates and rapidly switches between two sets of structured light images with phase shifts of 0 and π during high-speed axial scanning of the sample. Simultaneously, two high-frame-rate cameras sequentially acquire the original 2D intensity images of the sample under corresponding structured light illumination, reconstructing the light slice image at the current axial position. By adjusting the axial position of the sample layer by layer and repeating the above imaging process, rapid 3D tomographic imaging is finally achieved. This method can realize multi-frame phase shift data acquisition during high-speed axial scanning and is expected to be widely used in high-speed, 3D morphology imaging of industrial components. Its application scenarios can cover precision manufacturing, microstructure device inspection, functional material surface characterization, and microelectronic device quality assessment.

[0008] 2. This invention integrates the extended light source and DMD into a modular illumination and projection unit, which not only reduces the optical path volume but also improves light energy utilization and system robustness. Furthermore, by employing extended light source illumination and DMD light intensity modulation, this invention can generate axially extremely thin striped structured light, ultimately achieving optimal light slicing accuracy.

[0009] 3. This invention utilizes dual-camera staggered imaging, significantly improving imaging speed. Traditional OS-SIM typically uses a single camera to sequentially acquire images with different phase shift intensities, and the image readout time of the camera itself greatly limits its imaging speed. This invention proposes using dual-camera staggered imaging; while the first CMOS camera reads out the image, the second CMOS camera begins exposure, eliminating image readout time. The switching delay between the two structured light images is only the camera exposure time, eliminating image transfer time and achieving structured light switching without axial displacement in high-speed axial sample scanning, thus increasing the 3D imaging speed of large-size samples by an order of magnitude.

[0010] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0011] Figure 1 This is a block diagram of a structured light illumination three-dimensional microscopy system based on alternating acquisition by dual cameras, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the optical path of a structured light illumination three-dimensional microscopy system based on alternating acquisition by dual cameras, provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of structured light microscopy illumination based on an extended light source provided in an embodiment of the present invention; Figure 4 This is a system control timing diagram provided in an embodiment of the present invention; Figure 5 This is a structured light-illuminating micro-slice image provided in an embodiment of the present invention; Figure 6 This is a three-dimensional reconstruction image of a micro-slice with obvious structured illumination provided in an embodiment of the present invention.

[0012] Explanation of reference numerals in the attached figures: 101-Illumination and projection module; 102-Microscopic imaging module; 103-Acquisition module; 104-Image reconstruction module; 105-Control module; 1-Illumination and projection unit; 2-First lens; 3-Polarizer; 4-First mirror; 5-Second lens; 6-Attenuator; 7-Second mirror; 8-Third lens; 9-Unpolarized beam splitter cube; 10-Microscopic objective lens; 11-Sample; 12-Transfer stage; 13-Third mirror; 14-Fourth lens; 15-Unpolarized flat beam splitter; 16-Fourth mirror; 17-First CMOS camera; 18-Second CMOS camera; 19-Computer; 20-Transfer stage controller. Detailed Implementation

[0013] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a method and system for three-dimensional microscopy based on alternating acquisition by dual cameras using structured light illumination.

[0014] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0015] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0016] Example 1 Please see Figure 1 , Figure 1This is a block diagram of a structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition, provided by an embodiment of the present invention. The structured light illumination three-dimensional microscopy system includes an illumination projection module 101, a microscopic imaging module 102, an acquisition module 103, an image reconstruction module 104, and a control module 105. The illumination projection module 101 generates striped structured light with different phase shifts; the microscopic imaging module 102 images different axial (i.e., the thickness direction of the sample) slices on the sample surface using striped structured light with different phase shifts; the acquisition module 103 acquires two-dimensional intensity images generated at different axial slice positions on the sample surface based on striped structured light with different phase shifts; the control module 105 synchronously controls the illumination projection module 101, the microscopic imaging module 102, and the acquisition module 103 according to a preset control sequence; and the image reconstruction module 104 reconstructs and stacks the two-dimensional intensity images acquired by the acquisition module 103 at different axial slice positions to obtain a three-dimensional image of the sample.

[0017] Please see Figure 2 , Figure 2 This is a schematic diagram of the optical path of a structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition, provided by an embodiment of the present invention. The illumination projection module 101 of this embodiment includes an illumination projection unit 1, a first lens 2, a polarizer 3, a second lens 5, an attenuator 6, a third lens 8, a non-polarized beam splitter cube 9, and a microscope objective 10, arranged sequentially along the optical path. The illumination projection unit 1 includes an extended light source and a digital micromirror device (DMD) disposed on the optical axis of the extended light source. The extended light source is used to generate narrowband, incoherent extended illumination light to improve the axial resolution of three-dimensional imaging. The digital micromirror device is used to quickly load and switch structured light images with different phase shifts, and to modulate the extended illumination light into striped structured light with different phase shifts. The polarizer... Lens 3 is used to modulate the polarization direction of the current fringe structured light to be parallel to the fringe direction to optimize the contrast of the fringe structured light; the first lens 2 and the second lens 5 form the first 4f relay system, which images the fringe structured light generated by the DMD onto the attenuator 6; the attenuator 6 is used to reduce the light intensity of the fringe structured light to within the linear dynamic range of the camera in the acquisition module 103; the third lens 8 and the microscope objective 10 form the second 4f relay system, and the non-polarized beam splitter cube 9 is set in the optical path between the third lens 8 and the microscope objective 10, which is used to reflect the fringe structured light to the microscope objective 10, and project it onto the sample surface through the microscope objective 10. The DMD is used to load and switch the first structured light image and the second structured light image under the control of the control module 105, and the phase shifts of the first structured light image and the second structured light image are 0 and 0, respectively. .

[0018] Furthermore, the illumination projection module 101 also includes a first reflector 4 and a second reflector 7. The first reflector 4 is obliquely disposed in the optical path between the polarizer 3 and the second lens 5, and is used to reflect the fringe structured light from the polarizer 3 to the second lens 5. The second reflector 7 is obliquely disposed in the optical path between the attenuator 6 and the third lens 8, and is used to reflect the fringe structured light from the attenuator 6 to the third lens 8. The function of the first reflector 4 and the second reflector 7 is to change the direction of the optical path, making the structure of the entire microscopy system more compact.

[0019] Please see Figure 3 , Figure 3 This is a schematic diagram of structured light microscopy illumination based on an extended light source provided in an embodiment of the present invention, wherein, Figure 3 Figure (a) in the figure is a schematic diagram of axially localized structured light generation under extended illumination; Figure 3 Figure (b) shows the intensity distribution of the striped light as the divergence angle of the light source increases sequentially. As can be seen from the figure, when using extended light source illumination, extremely thin axially thin striped structured light can be generated through DMD intensity modulation. The larger the divergence angle of the light source illumination, the shorter the axially distributed structured light stripes, and the better the light slicing effect. Therefore, the extended light source in this embodiment uses a maximum divergence angle of 20°. o An LED light source is used to illuminate the DMD at a 24° incident angle. The DMD dynamically loads structured light images with different phase shifts, generating striped structured light (diffracted beams) on the DMD. The LED light source has stable output power and produces a flat and uniform illumination background. The DMD is used to generate structured light illumination and phase shift operations, and has the advantages of high precision, fast refresh rate, and no mechanical vibration error.

[0020] Accordingly, the fringe structured light is emitted in a direction perpendicular to the DMD surface, then propagates along the principal optical axis, and passes through polarizer 3 to optimize the fringe contrast of the fringe structured light. In this embodiment, polarizer 3 is a thin-film linear polarizer to ensure the generation of structured light illumination fringes with optimal contrast.

[0021] To accurately project the fringe structured light onto the sample, the beam passes through a cascaded optical relay structure. Specifically, the first lens 2 and the second lens 5 form the first 4f relay system, imaging the fringe structured light generated by the DMD onto the central conjugate surface of the DMD, i.e., the back focal plane of the second lens 5. An attenuator 6 is positioned between the second lens 5 and the second reflector 7 to reduce the light intensity within the camera's linear dynamic range, ensuring that the tonal gradations of the image can be captured. The non-polarized beam splitter 9 splits the incident beam from the third lens 8 into two beams in a 50:50 ratio, while maintaining the original polarization state of the incident beam unchanged. That is, 50% of the fringe structured light from the third lens 8 is reflected by the non-polarized beam splitter 9 to the microscope objective 10. The microscope objective 10 and the third lens 8 form the second 4f relay system, accurately projecting the fringe structured light onto the current axial slice position of the sample.

[0022] In this embodiment, the first lens 2, the second lens 5, and the third lens 8 are all achromatic cemented lenses, which can greatly correct chromatic aberration and suppress spherical aberration.

[0023] See also Figure 2 The microscopic imaging module 102 of this embodiment includes a displacement stage 12, a fourth lens 14, and a non-polarizing flat beam splitter 15. The displacement stage 12 is disposed at the back focal plane of the microscope objective 10, and the sample 11 is disposed on the displacement stage 12. Under the control of the control module 105, the displacement stage 12 can move the sample 11 along the axial direction (i.e., Figure 2 The Z-direction of the light source is moved so that the striped structured light from the illumination projection module 101 can illuminate different axial slice positions of the sample 11; the non-polarized beam splitter cube 9 is also used to transmit the light signal carrying the sample surface information from the sample 11 to the fourth lens 14; the non-polarized flat beam splitter 15 is used to split the light signal from the fourth lens 14 and transmit it to different cameras of the acquisition module 103 for image acquisition.

[0024] Furthermore, the microscopic imaging module 102 also includes a third reflecting mirror 13, which is obliquely disposed in the optical path between the transmission direction of the non-polarized beam splitter cube 9 and the fourth lens 14, for reflecting the light signal transmitted by the non-polarized beam splitter cube 9 to the fourth lens 14.

[0025] In this embodiment, sample 11 can be an industrial centimeter-sized sample. The displacement stage 12 carries sample 11 and moves it sequentially along a pre-set XYZ direction to acquire different positional information. Here, the Z direction is the optical axis, and the X and Y directions are perpendicular to each other and both perpendicular to the Z direction. The unpolarized flat beam splitter 15 in this embodiment splits the incident beam from the fourth lens 14 into two beams (transmission and reflection) in a 50:50 ratio while maintaining the original polarization state. The third reflecting mirror 13 changes the optical path direction, making the entire microscopic system more compact. The fourth lens 14 is an achromatic cemented lens, which greatly corrects chromatic aberration and suppresses spherical aberration.

[0026] See also Figure 2 The acquisition module 103 in this embodiment includes a first CMOS camera 17 and a second CMOS camera 18. The first CMOS camera 17 is used to receive the optical signal of the sample 11 under striped structured light illumination with a phase shift of 0, under the control of the control module 105. The second CMOS camera 18 is used to receive the optical signal of the sample 11 under striped structured light illumination with a phase shift of 0, under the control of the control module 105. The light signal of sample 11 under striped structured light illumination. The microscopic imaging module 102 of this embodiment also includes a fourth reflecting mirror 16, which is obliquely disposed in the optical path between the reflection direction of the unpolarized flat beam splitter 15 and the first CMOS camera 17, for reflecting the light beam from the unpolarized flat beam splitter 15 to the first CMOS camera 17, so as to improve the compactness of the entire microscopic system structure.

[0027] Specifically, the light signal carrying sample surface information reflected at the current axial slice of sample 11 is collected by the microscope objective 10, passes through the non-polarized beam splitter cube 9 and is deflected by the third reflecting mirror 13, then is split into two equal paths by the fourth lens 14 and the non-polarized flat beam splitter 15, and finally converges onto the sensors of the first CMOS camera 17 and the second CMOS camera 18, respectively, thereby realizing image acquisition. In this embodiment, the first CMOS camera 17 and the second CMOS camera 18 are exactly the same model, with trigger modes and appropriate grayscale levels, pixel sizes and pixel counts.

[0028] Furthermore, the control module 105 of this embodiment includes a computer 19 and a displacement stage controller 20. The computer 19 is used to sequentially switch the first structured light image and the second structured light image loaded on the DMD according to a preset control timing sequence, and to control the exposure time of the first CMOS camera 17 and the second CMOS camera 18 to be synchronized with the loading time of the first structured light image and the second structured light image, respectively, so that the first CMOS camera 17 acquires a two-dimensional intensity image of the sample under striped structured light conditions based on the first structured light image, and the second CMOS camera 18 acquires a two-dimensional intensity image of the sample 11 under striped structured light conditions based on the second structured light image. After the first CMOS camera 17 and the second CMOS camera 18 sequentially complete the acquisition of the corresponding two-dimensional intensity images at the current axial slice position, the displacement stage controller 20 is used to control the displacement stage 12 to move along the optical axis to the next axial slice position for the next acquisition.

[0029] In this embodiment, computer 19 controls DMD to sequentially switch between loading two binary gratings with a period of 4 pixels and a phase shift interval of π to generate sinusoidal illumination fringes.

[0030] In one specific embodiment, the system uses a data acquisition card (DAQ) to synchronously control the operation of the DMD, the first CMOS camera 17, the second CMOS camera 18, and the displacement stage controller 20 through a Matlab program in computer 19, ensuring good timing synchronization of the system. This synchronization is crucial for acquiring the three-dimensional dataset of sample 11.

[0031] Please see Figure 4 , Figure 4 This is a system control timing diagram provided in an embodiment of the present invention. Taking the current axial slice position as an example, in this timing diagram, starting from t0, the DMD is triggered to project two structured light images sequentially, which are synchronized with the exposure times of the two cameras CMOS1 (i.e., the first CMOS camera) and CMOS2 (i.e., the second CMOS camera). Wherein, T1 is the projection duration of a single frame structured light pattern (i.e., the first or second structured light image), and the solid and dashed lines represent the time for continuous projection of two structured light patterns onto the DMD, respectively. T2 is the camera readout time. At time t1, the acquisition of the original two-dimensional intensity images under the two striped structured light illuminations is completed. Subsequently, the displacement stage 12 moves the sample 11 to the next adjacent axial slice position. The system enters the next DMD-camera cycle modulation-shooting cycle (corresponding to...). Figure 4(Control timing from t2 to t3). This system adopts the alternating trigger synchronization principle of high-speed pulse drive of the data acquisition card, aiming to reduce image storage time and the time difference between two structured light patterns through dual-channel parallel acquisition mode, and ultimately realize rapid structured light switching and original image acquisition of sample 11 in high-speed axial scanning.

[0032] In terms of timing control, a National Instruments (NI) card is used as the core clock reference. When the system issues a command at time t0, it synchronously triggers the loading of the first structured light pattern on the DMD for a duration of T1, synchronized with the shutter opening (exposure) of the CMOS1 camera. After the exposure time T1 (T1 = 0.40 ms), the CMOS1 camera requires a relatively long internal image transfer time (T2 = 5.8 ms). However, the system of this invention triggers the loading of the second structured light image on the DMD immediately after T1 and begins the exposure and shooting of the CMOS2 camera. Since the two-dimensional intensity images generated by the two striped structured lights are captured "simultaneously" by the two cameras, the time point at which the CMOS2 camera captures the optical signal falls within the internal image transfer time period of the CMOS1 camera, thereby effectively shortening the shooting time difference between the two images. This allows the OS-SIM system of this invention to instantly acquire two intensity images during high-speed axial scanning of the sample, realizing high-speed three-dimensional light slicing imaging.

[0033] The image reconstruction module 104 in this embodiment uses a light slicing image reconstruction method based on Hilbert Transform (HT) to reconstruct the image. The specific reconstruction principle will be described in detail in the following embodiment 2.

[0034] This invention proposes a structured light illumination 3D microscopy system based on alternating dual-camera acquisition. During high-speed axial scanning of the sample, a digital micromirror device (DMD) dynamically generates and rapidly switches between two sets of structured light images with phase shifts of 0 and π. Simultaneously, two high-frame-rate cameras sequentially acquire the original 2D intensity images of the sample under corresponding structured light illumination, reconstructing the light slice image at the current axial position. By adjusting the axial position of the sample layer by layer and repeating the above imaging process, rapid 3D tomographic imaging is finally achieved. This method enables multi-frame phase shift data acquisition during high-speed axial scanning and is expected to be widely used in high-speed, 3D morphology imaging of industrial components. Its application scenarios can cover precision manufacturing, microstructure device inspection, functional material surface characterization, and microelectronic device quality assessment.

[0035] This invention integrates an extended light source and a DMD into a modular illumination projection unit, which not only reduces the size of the optical path but also improves light energy utilization and system robustness. Furthermore, by employing extended light source illumination and DMD light intensity modulation, this invention can generate axially extremely thin striped structured light, ultimately achieving optimal light slicing accuracy. This invention utilizes dual-camera staggered imaging, significantly improving imaging speed. Traditional OS-SIM typically requires acquiring multiple frames (e.g., three-step phase shift, five-step phase shift), and a single camera must sequentially acquire images of different phase shift intensities. The camera's image transfer time severely limits its imaging speed. This invention, however, proposes using dual-camera staggered imaging, making the switching delay between the two structured light images only the camera exposure time, eliminating image transfer time, and achieving axial displacement-free structured light switching in high-speed axial sample scanning, thus increasing the 3D imaging speed of large-size samples by an order of magnitude.

[0036] Example 2 Based on Example 1, this example provides a structured light illumination three-dimensional microscopy method based on alternating acquisition by dual cameras, including: S1: Using the structured light illumination three-dimensional microscopy system based on alternating acquisition by dual cameras as described in Example 1, two two-dimensional intensity images are obtained at each axial slice position of the sample; S2: Using the light slice image based on Hilbert transform, the two-dimensional intensity images at different axial slice positions are reconstructed and stacked in three dimensions to obtain a three-dimensional image of the sample.

[0037] Specifically, for real-valued functions in two-dimensional space Its one-dimensional Hilbert transform is defined as a real-valued function in two-dimensional space. and Convolution operation: (1) In structured illumination micro-imaging, the structural information of the sample (i.e., the distribution of absorption / reflectance of the sample surface to the illumination light) can be obtained using... This indicates that the structural information of the sample will be displayed. With one-dimensional cosine modulation term Multiplication operations are performed to obtain the modulation structure, thereby encoding the image information. Represents the coordinates of a two-dimensional image. For spatial frequency, Given the initial phase, the Hilbert transform of this modulation structure can be expressed as: (2) In optical microscopy systems, a single frame of structured light illumination image (i.e., a two-dimensional intensity image) is acquired. It can be broken down into in-focus components and out-of-focus components: (3) in, =1 or 2, For modulation depth, Indicates the composition of the coke. Indicates the defocused component. .

[0038] Meanwhile, a wide-field image can be represented as a simple linear superposition of in-focus and out-of-focus components: (4) Expanding equation (3) and substituting it into equation (4), the structured light illumination image can be rewritten in a form based on a wide-field image: (5) Subtract both sides of equation (5) It is possible to obtain modulation terms that contain only focus information: (6) Using the properties described in equation (2), for Perform a one-dimensional Hilbert transform to obtain the phase shift. Orthogonal signals: (7) Then, using the sum of squares of trigonometric identities, modulation fringes are eliminated through square root operations, and the focal component is extracted. : (8) Substituting the original structured light illumination image, the final formula for reconstructing the light slice image based on the Hilbert transform is: (9) In this embodiment, the numerical aperture of the microscope objective 10 is NA=0.45, and the imaging wavelength is... =488 nm, at which point the axial resolution of the system is measured to be σ. Z =4.71 m.

[0039] In a traditional OS-SIM system, two CMOS cameras (1920×1200 pixels, pixel size 5.86) m) After an exposure time T1 = 0.40 ms, a relatively long internal image transfer time T2 = 5.8 ms is required.

[0040] In the OS-SIM system proposed in this invention, the alternating shooting by the dual cameras results in a switching time of the two structured light images (and intensity map shooting) of [time value missing]. T= 0.40 ms. In contrast, the traditional method uses a single camera to sequentially capture two intensity images under structured light illumination. Each frame must undergo exposure-readout, and the switching time between the two structured light images (and the intensity image capture) is T1+T2=6.2 ms. Therefore, compared to the traditional single-camera mode, the system proposed in this invention improves the acquisition efficiency by (T1+T2) / T1=15.4 times. Furthermore, when using a DMD with a higher refresh rate (e.g., 9523Hz), the acquisition efficiency per frame will be further improved.

[0041] The following discussion compares the acquisition speeds of the traditional OS-SIM and the OS-SIM method proposed in this invention under two acquisition modes: (1) Axial "Move-Stop" Step-by-Step Shooting Mode: The maximum axial acceleration of the displacement stage 12 is 5 mm / s². 2 The maximum speed is 5 mm / s. Assume the axial step size of the displacement stage 12 is... m, the acceleration time of the displacement stage With deceleration time Same, that is ms, where, Indicating acceleration, the time taken to capture two frames is: ms. Based on the dual-camera OS-SIM mode proposed in this invention, using continuous axial motion, the total time required to record two phase-shift intensity images is: Therefore, compared to traditional single-camera acquisition, the acquisition speed of the dual-camera acquisition mode of this invention is increased by 11.9 times (73.8 / 6.2). When the step size is larger, the stage time will increase further, and the acquisition efficiency of the dual-camera acquisition mode will be further improved compared to the traditional single-camera mode.

[0042] (2) Continuous shooting mode during axial movement: When the axial displacement of the displacement stage 12... When m, the effect of axial displacement can be ignored. To ensure that the effect of axial displacement during continuous axial movement of the sample is negligible, the maximum movement speed of the displacement stage 12 in single-camera mode is... Maximum motion speed of the displacement stage in dual-camera mode At this point, compared to single-camera mode, the method of the present invention is 15.8 times faster.

[0043] In actual operation, the incident light emitted by the 456 nm wavelength LED light source in the illumination projection unit 1 is modulated by the DMD to generate a striped structured light field. After passing through two 4f relay systems (telescope systems) consisting of the first lens 2, the second lens 5, the third lens 8, and the microscope objective 10, the striped structured light is accurately projected onto the sample 11. In this embodiment, the focal length of the first lens 2 is 60 mm, the focal length of the second lens 5 is 75 mm, the focal length of the third lens 8 is 125 mm, the magnification of the microscope objective 10 is 20x, and the numerical aperture NA is 0.45.

[0044] Subsequently, the imaging signal emitted by sample 11 (i.e., the light signal carrying information about the sample surface) first passes through the microscope objective 10, then is separated from the illumination light path by the non-polarized beam splitter cube 9, and is then imaged by the microscope objective 10 and the fourth lens 14 into two identical first CMOS cameras 17 and second CMOS cameras 18 respectively.

[0045] In actual operation, the phase shift is projected sequentially as 0 and 0 by the DMD. Two structured light images are captured, and the duration of each image is synchronized with the exposure of the first CMOS camera 17 and the second CMOS camera 18, respectively. In other words, the two cameras sequentially record two original two-dimensional intensity images under structured light illumination. After acquiring the two original two-dimensional intensity images under structured light illumination, the displacement stage is moved axially so that the striped structured light illuminates the second axial slice position of the sample. Then, the system enters the next DMD-camera cycle modulation-shooting cycle until the system completes the imaging of the entire axial scanning range of the sample. Finally, using formula (9), the light slice image of the sample can be reconstructed layer by layer to complete the three-dimensional image reconstruction.

[0046] The following experiments further illustrate the effectiveness of the structured light illumination three-dimensional microscopy system and method based on alternating dual-camera acquisition of the present invention.

[0047] In the first experiment, a three-dimensional image of a metallic pentagram was created using the structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition, as described in this invention. When sequentially loaded onto the DMD... Figure 5 Figure a in the middle and Figure 5 When capturing the two structured light images shown in Figure b, the original intensity images acquired in parallel by camera CMOS1 and camera CMOS2 are as follows: Figure 5 Figure c in the middle and Figure 5 As shown in Figure d, high-frequency modulation fringes can be observed by magnifying the focused area. These dense fringes are absent in the defocused area. Figure 5The e-image in the image is a conventional wide-field (WF) image obtained by averaging the original image. Because wide-field images are filled with a large amount of out-of-focus background scattered light, the overall contrast is low, and details are obscured. In contrast, Figure 5 The image f in the figure is the image (OS) after being processed by the optical slicing image reconstruction algorithm based on Hilbert transform. It can be seen that the image processed by the optical slicing image reconstruction algorithm shows a significant performance improvement. The optical slicing technology can accurately extract useful signals in the focal plane and completely eliminate out-of-focus blur. In the OS image, the microstructural features of the sample (such as fiber texture and edge details) are clearly presented, and the background light is suppressed to almost black, which significantly improves the signal-to-noise ratio of the image.

[0048] based on Figure 5 The optical section image reconstruction algorithm shown in Figure f is used in this experiment to further acquire a series of optical section images of the sample at different depths (axial sections). Subsequently, professional 3D imaging software was used to perform spatial 3D reconstruction on these high-contrast 2D data, which had been optimized by the algorithm and had out-of-focus blur eliminated. The final result is a 3D topographic image of the sample and a schematic diagram of its orthogonal cross-section analysis, as shown in Figure f. Figure 6 As shown. In Figure 6 In Figure a, the overall morphology of the pentagram sample was reproduced with high fidelity using 3D imaging software. Figure 6 Figure b in the figure is the xz plane cross-section of the sample, which clearly shows the cross-sectional profile of the sample along the horizontal center line. The measurement results show that the star-shaped structure is not planar, but has a distinct upward arched arc shape, and its maximum vertical displacement is precisely quantified as 160 μm. Figure 6 Figure c in the diagram is a longitudinal cross-section of the corresponding sample, which further supplements the information of the side view and provides key data support for three-dimensional spatial geometric analysis.

[0049] This invention integrates an LED light source with a DMD to generate incoherent, high-contrast structured light stripes, and is equipped with a shielding shell to improve light energy utilization, mechanical stability, and signal-to-noise ratio. At the detection end, a dual-camera alternating trigger acquisition mode is employed, using the DAQ data acquisition card as the clock reference. CMOS1 and CMOS2 cameras are exposed alternately under the control of the DMD exposure pulses, ensuring that the exposure of one camera falls within the data buffer cycle of the other camera. This eliminates the frame rate bottleneck and dead time of single-camera hardware. In this acquisition mode, the displacement stage can perform "flying" imaging in continuous uniform motion, eliminating the need for intermittent step-stop-exposure movements, thereby eliminating mechanical vibration and significantly improving 3D imaging efficiency.

[0050] Furthermore, the image reconstruction module employs a light slicing image reconstruction algorithm based on Hilbert transform: by performing Hilbert transform on two images to construct orthogonal signals, and then directly extracting the current focal plane information through square root operations, without requiring multiple phase shifts. In summary, the structured light illumination three-dimensional microscopy method and system based on alternating dual-camera acquisition proposed in this invention not only possess advantages such as fast acquisition speed, large imaging depth, and strong background suppression capability, but also have a relatively simple system structure, enabling high signal-to-background ratio rapid three-dimensional imaging of thick samples. Therefore, it has promising application prospects in industrial fields such as precision manufacturing, microelectronic device quality assessment, and material surface characterization.

[0051] In the several embodiments provided by this invention, it should be understood that the apparatus and methods disclosed in this invention can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0052] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0053] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition, characterized in that, It includes an illumination projection module (101), a microscopic imaging module (102), an acquisition module (103), an image reconstruction module (104), and a control module (105), wherein, The illumination projection module (101) is used to generate striped structured light with different phase shifts; The microscopic imaging module (102) is used to image different axial slice positions on the sample surface using the striped structured light with different phase shifts; The acquisition module (103) is used to acquire two-dimensional intensity images generated at different axial slice positions on the sample surface based on striped structured light with different phase shifts. The control module (105) is used to synchronously control the illumination projection module (101), the microscopic imaging module (102) and the acquisition module (103) according to a preset control timing sequence; The image reconstruction module (104) is used to reconstruct and stack two-dimensional intensity images at different axial slice positions acquired by the acquisition module (103) to obtain a three-dimensional image of the sample. The illumination projection module (101) includes an illumination projection unit (1), a first lens (2), a polarizer (3), a second lens (5), an attenuator (6), a third lens (8), a non-polarizing beam splitter cube (9), and a microscope objective (10) arranged sequentially along the optical path. The illumination projection unit (1) includes an extended light source and a digital micromirror device disposed on the optical axis of the extended light source. The extended light source is used to generate narrowband, incoherent extended illumination light to improve the axial resolution of three-dimensional imaging. The digital micromirror device is used to load and switch structured light images with different phase shifts and to modulate the extended illumination light into striped structured light with different phase shifts. The digital micromirror device is used to load and switch between a first structured light image and a second structured light image under the control of the control module (105), wherein the phase shifts of the first structured light image and the second structured light image are 0 and 0, respectively. ; The microscopic imaging module (102) includes a displacement stage (12), a fourth lens (14), and a non-polarized flat beam splitter (15). The displacement stage (12) is located at the back focal plane of the microscope objective (10), and the sample is placed on the displacement stage (12). The displacement stage (12) can move the sample along the axial direction under the control of the control module (105), so that the striped structured light from the illumination projection module (101) can illuminate different axial slice positions of the sample. The non-polarized beam splitter cube (9) is also used to transmit the light signal carrying the sample surface information from the sample to the fourth lens (14). The non-polarized flat beam splitter (15) is used to split the light signal carrying the sample surface information from the fourth lens (14) and transmit it to different cameras of the acquisition module (103) for image acquisition. The acquisition module (103) includes a first CMOS camera (17) and a second CMOS camera (18), wherein the first CMOS camera (17) is used to receive the light signal of the sample under striped structured light illumination with a phase shift of 0 under the control of the control module (105); the second CMOS camera (18) is used to receive the light signal of the sample under striped structured light illumination with a phase shift of 0 under the control of the control module (105). The optical signal of the sample under striped structured light illumination.

2. The structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition according to claim 1, characterized in that, The polarizer (3) modulates the polarization direction of the current stripe structure light to be parallel to the stripe direction in order to optimize the contrast of the stripe structure light; The first lens (2) and the second lens (5) constitute the first 4f relay system, which images the striped structured light generated by the digital micromirror device onto the attenuator (6); The attenuator (6) is used to reduce the light intensity of the striped structured light to within the linear dynamic range of the camera in the acquisition module (103); The third lens (8) and the microscope objective (10) constitute a second 4f relay system. The non-polarized beam splitter cube (9) is disposed in the optical path between the third lens (8) and the microscope objective (10) to reflect the striped structure light to the microscope objective (10) and project it onto the sample surface through the microscope objective (10).

3. The structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition according to claim 2, characterized in that, The lighting projection module (101) further includes a first reflector (4) and a second reflector (7), wherein, The first reflector (4) is tilted and disposed in the optical path between the polarizer (3) and the second lens (5) to reflect the striped structure light from the polarizer (3) to the second lens (5); The second reflector (7) is tilted and disposed in the optical path between the attenuator (6) and the third lens (8) to reflect the striped structured light from the attenuator (6) to the third lens (8).

4. The structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition according to claim 1, characterized in that, The microscopic imaging module (102) also includes a third reflecting mirror (13), wherein, The third mirror (13) is tilted and disposed in the optical path between the transmission direction of the non-polarized beam splitter cube (9) and the fourth lens (14) to reflect the light signal carrying sample surface information transmitted by the non-polarized beam splitter cube (9) to the fourth lens (14).

5. The structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition according to claim 1, characterized in that, The microscopic imaging module (102) further includes a fourth reflecting mirror (16), which is tilted and disposed in the optical path between the reflection direction of the unpolarized flat beam splitter (15) and the first CMOS camera (17), for reflecting the light beam from the unpolarized flat beam splitter (15) to the first CMOS camera (17).

6. The structured light illumination three-dimensional microscopy system based on alternating dual-camera acquisition according to claim 1, characterized in that, The control module (105) includes a computer (19) and a displacement stage controller (20), wherein, The computer (19) is used to sequentially switch the first structured light image and the second structured light image loaded on the digital micromirror device according to a preset control timing sequence, and to control the exposure time of the first CMOS camera (17) and the second CMOS camera (18) to be synchronized with the loading time of the first structured light image and the second structured light image, respectively, so that the first CMOS camera (17) acquires the two-dimensional intensity image of the sample under the striped structured light condition of the first structured light image, and the second CMOS camera (18) acquires the two-dimensional intensity image of the sample under the striped structured light condition of the second structured light image; At the current axial slice position, after the first CMOS camera (17) and the second CMOS camera (18) have completed the corresponding two-dimensional intensity image acquisition in sequence, the displacement stage controller (20) is used to control the displacement stage (12) to move along the optical axis to the next axial slice position for the next acquisition.

7. A three-dimensional microscopy method based on structured light illumination using alternating dual-camera acquisition, characterized in that, include: S1: Using any one of the structured light illumination three-dimensional microscopy systems based on alternating dual-camera acquisition as described in claims 1 to 6, two two-dimensional intensity images are obtained at each axial slice position of the sample to be tested; S2: The two-dimensional intensity images at different axial slice locations are reconstructed and stacked in three dimensions using a light slice image reconstruction algorithm based on Hilbert transform to obtain a three-dimensional image of the sample.

Images