Rapid light intensity diffraction tomography device and method

Abstract

The invention provides a rapid light intensity diffraction tomography device and method, which are applied to the technical field of light intensity diffraction tomography, and the device comprises a multi-angle lighting device which comprises a low-coherence laser used for emitting low-coherence laser; the parallel collimation optical system is arranged on an emergent light path of the low-coherence laser and is used for generating parallel light beams with uniform space; the two-dimensional galvanometer assembly is arranged on an emergent light path of the parallel collimation optical system, and light beam angle deflection is achieved through voltage adjustment; the 4f relay optical system is arranged on the reflection light path of the two-dimensional galvanometer assembly, so that the galvanometer reflection surface of the two-dimensional galvanometer assembly and the plane where the sample is located form optical conjugation; the sample objective table is used for bearing a sample; the detection objective lens is used for collecting transmission light penetrating through the sample; the imaging tube lens is used for focusing and imaging the transmission light; and the camera is used for acquiring a transmitted light intensity image. According to the invention, the imaging speed and the spatial resolution capability of the IDT system can be comprehensively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of light intensity diffraction tomography, and in particular to a rapid light intensity diffraction tomography device and method. BACKGROUND

[0002] The existing intensity diffraction tomography (IDT) system generally adopts a light-emitting diode (LED) ring array structure as a multi-angle illumination light source in the design of the illumination module to realize sample transmission image acquisition under different incident directions.

[0003] Such an LED array is usually fixed at a certain distance above the sample to meet the plane wave approximation condition, so that the incident light in each direction maintains good collimation during propagation. However, since the LED is a non-coherent divergent light source, its light intensity rapidly decreases with the propagation distance during propagation. In order to ensure that the sample can receive sufficient illumination to make the image have a proper signal-to-noise ratio, the system often needs to prolong the exposure time of each image, and the exposure time of a single light intensity image is usually between 0.1 seconds and 1 second. This not only greatly increases the time cost of single three-dimensional data acquisition, but also seriously limits the overall imaging speed of the system, making it difficult to meet the higher requirements of imaging time in application scenarios such as high-speed imaging, dynamic process monitoring or real-time feedback control. SUMMARY

[0004] The present application provides a rapid light intensity diffraction tomography device and method to solve the defects of the existing IDT system in the illumination module structure and light source arrangement and the limitation of the solving algorithm, and to comprehensively improve the imaging speed and spatial resolution of the IDT system.

[0005] The present application provides a rapid light intensity diffraction tomography device, comprising a multi-angle illumination device, wherein the multi-angle illumination device comprises: a low-coherence laser for emitting low-coherence laser; a parallel collimation optical system arranged on the exit light path of the low-coherence laser for generating a spatially uniform parallel light beam; a two-dimensional galvanometer assembly arranged on the exit light path of the parallel collimation optical system for realizing light beam angle deflection through voltage adjustment; a 4f relay optical system arranged on the reflection light path of the two-dimensional galvanometer assembly, so that the mirror reflection surface of the two-dimensional galvanometer assembly and the plane where the sample is located form optical conjugation; a sample stage arranged on the exit light path of the multi-angle illumination device for carrying the sample; a detection objective arranged behind the sample stage for collecting transmission light through the sample; an imaging tube lens coaxial with the detection objective for focusing and imaging the transmission light; and a camera arranged on the imaging plane of the imaging tube lens for collecting transmission light intensity images.

[0006] According to a fast light intensity diffraction tomography imaging device provided by the present invention, the device also includes: a synchronization control device electrically connected to the multi-angle lighting device and the camera, for coordinating the timing synchronization of lighting angle switching and camera exposure.

[0007] The present invention also provides a rapid light intensity diffraction tomography method, comprising the following steps: Based on the galvanometer scanning voltage value, a light source distribution corresponding to each scanning angle is constructed; based on the light source distribution and preset optical device parameters, a three-dimensional transfer function for each scanning angle is determined; a sample transmitted light intensity map corresponding to each scanning angle is obtained; the sample transmitted light intensity map is preprocessed and two-dimensionally Fourier transformed to obtain a light intensity spectrum map for each scanning angle; the three-dimensional transfer function for each scanning angle and the light intensity spectrum map for each scanning angle are deconvolved and mapped through Tikhonov regularization to obtain a three-dimensional imaging result of the sample refractive index data.

[0008] According to a fast light intensity diffraction tomography method provided by the present invention, the optical device parameters include: low coherence laser center wavelength, overall system magnification, medium refractive index, detection objective numerical aperture, and camera pixel size; determining the three-dimensional transfer function for each scanning angle based on the light source distribution and preset optical device parameters includes: constructing a pupil function based on the light source distribution, the low coherence laser center wavelength, the overall system magnification, medium refractive index, the detection objective numerical aperture, and the camera pixel size; determining the phase propagation kernel of each layer according to a preset layer thickness; and determining the three-dimensional transfer function for each scanning angle based on the light source distribution, the pupil function, and the phase propagation kernel.

[0009] According to a fast light intensity diffraction tomography method provided by the present invention, determining the phase propagation kernel of each layer according to a preset layer thickness includes: determining the phase propagation kernel of each layer using a phase propagation kernel formula: in, represents the phase propagation kernel, represents the refractive index of the medium surrounding the sample, Indicates the Axial distance of the slice position, represents the central wavelength of the low-coherence laser, Represents the horizontal spatial frequency coordinates in the frequency domain, corresponding to the frequency components in the x-direction and y-direction in the image plane, Represents an imaginary unit.

[0010] According to the application, a fast light intensity diffraction tomography method is provided, and the sample transmission light intensity map corresponding to each scanning angle is obtained, comprising: repeatedly performing the following steps until the transmission light intensity map acquisition of all preset angles is completed, wherein the single-angle exposure time is less than a preset value: driving the two-dimensional galvanometer assembly to scan the sample at a preset angle by a synchronous control device; when the two-dimensional galvanometer assembly scans to a target angle, triggering the camera exposure by the synchronous control device to obtain the transmission light intensity map of the target angle; after triggering the camera exposure, switching the two-dimensional galvanometer assembly to the next scanning angle by the synchronous control device.

[0011] According to the application, a fast light intensity diffraction tomography method is provided, and the sample transmission light intensity map is preprocessed and two-dimensional Fourier transform, the construction of the three-dimensional transfer function and the deconvolution operation of the Tikhonov regularization are executed in parallel on a graphics processor.

[0012] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the fast light intensity diffraction tomography method according to any one of the above when executing the program.

[0013] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the fast light intensity diffraction tomography method according to any one of the above.

[0014] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the fast light intensity diffraction tomography method according to any one of the above.

[0015] The fast light intensity diffraction tomography device and method provided by the application realize millisecond-level high-speed multi-angle illumination switching under the premise of maintaining the fixed position of the sample surface illumination spot through the synergistic effect of a low coherence laser, a two-dimensional galvanometer and a 4f relay optical system. Meanwhile, the axial spectral blind area of the traditional ring light source is filled by the flexible scanning of the galvanometer, the axial resolution and reconstruction accuracy of three-dimensional imaging are significantly improved, and high signal-to-noise ratio and motion artifact-free original intensity image data are provided for subsequent non-iterative fast reconstruction algorithms. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the application or prior art, the drawings needed in the embodiments or prior art description will be briefly introduced one by one below. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is a structural schematic diagram of a rapid light intensity diffraction tomography device provided by the present application.

[0018] Figure 2 is a rapid light intensity diffraction tomography method provided by the present application.

[0019] Figure 3 is a whole flowchart of the rapid light intensity diffraction tomography method provided by the present application.

[0020] Figure 4 is a different scheme solving time comparison diagram provided by the present application.

[0021] Figure 5 is a different scheme solving result comparison diagram provided by the present application.

[0022] Figure 6 is a physical structure schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] Intensity Diffraction Tomography (IDT) technology is a new three-dimensional non-labeled refractive index imaging method, which has attracted widespread attention in the field of optical microscopy in recent years. As a kind of computational optical imaging method, IDT combines multi-angle illumination and inversion algorithm, which can reconstruct the three-dimensional refractive index distribution inside the sample without introducing additional staining agent, so as to realize the in-depth analysis of the structure and composition of transparent sample.

[0025] Optical microscopic imaging technology plays an important role in many fields such as life science, material science and medical diagnosis due to its non-invasiveness, real-time and high resolution. However, in the traditional bright field microscope, due to the fact that many biological samples (such as living cells) hardly absorb light intensity, the imaging contrast is extremely low, and the formed image usually presents a "transparent" state, which makes it difficult to distinguish the cell morphology and internal organelle substructure information.

[0026] To improve the imaging contrast, the prior art generally uses dyeing or fluorescent labeling to enhance the image signal. However, these labeling methods usually involve chemical dyes or fluorescent probes, and the attachment process may change the native state of the cells, and there is a certain risk of phototoxicity and chemical toxicity, which limits its application in long-term, dynamic or in vivo imaging. In addition, fluorescent labeling has problems such as photobleaching, which affects the repeatability and consistency of the experiment.

[0027] In view of the above problems, label-free imaging has become a hot direction in recent years. Compared with traditional labeling imaging technology, IDT uses multi-angle illumination and computational inversion strategy, and only by recording two-dimensional intensity images at different angles, three-dimensional refractive index information can be reconstructed, which effectively overcomes the problem of dyeing requirement and imaging damage. This technology can not only be used for non-destructive imaging of transparent cell structures, but also be suitable for quantitative analysis of complex samples.

[0028] The current mainstream IDT imaging reconstruction method usually relies on multiple iterative optimization processes to gradually approach the three-dimensional refractive index distribution of the sample. Such algorithms are generally implemented and calculated on a Central Processing Unit (CPU) platform using matlab or python language. In practical applications, IDT needs to process a large amount of image data obtained by multi-angle illumination. These images not only have a large amount of data, but also involve multiple physical models and numerical calculation steps in the reconstruction process, but the calculation complexity of a single pixel is not high. Due to the limited parallel computing capability of traditional CPU, the number of cores is relatively small, and it is difficult to efficiently cope with such intensive operation requirements, resulting in limited overall calculation speed. According to the resolution of the image, the complexity of the target structure and the reconstruction algorithm used, the entire solving process may take from several minutes to several hours, which seriously restricts the application potential of IDT technology in high-throughput or real-time imaging scenarios. The deployment and control ability of matlab or python script language on system resources (such as memory) is insufficient, which limits the data size and speed that can be processed.

[0029] The existing intensity diffraction tomography (IDT) system generally adopts a light-emitting diode (LED) ring array structure as a multi-angle illumination light source in the design of an illumination module to realize sample transmission image acquisition under different incident directions. The LED array is usually fixed at a certain distance above the sample to meet the plane wave approximation condition, so that the incident light in each direction maintains good collimation during propagation. However, since the LED is a non-coherent divergent light source, the light intensity rapidly decreases with the propagation distance during propagation. In order to ensure that the sample can receive sufficient illumination to make the image have a proper signal-to-noise ratio, the system often needs to prolong the exposure time of each image, and the exposure time of a single light intensity image is usually between 0.1 seconds and 1 second. This not only greatly increases the time cost of single three-dimensional data acquisition, but also seriously limits the overall imaging speed of the system, making it difficult to meet the higher requirements of imaging time in application scenarios such as high-speed imaging, dynamic process monitoring or real-time feedback control.

[0030] In addition, the existing LED ring has a limited range of incident angle regulation due to its fixed physical structure, and can only complete a circle of preset angle scanning. This scanning method has a natural sampling blind area in the three-dimensional Fourier frequency spectrum space, especially in the high-frequency region and the Z-axis direction spectrum information acquisition. The incompleteness of this frequency domain sampling will directly lead to stretching distortion in the Z-axis direction during image reconstruction, thereby reducing the axial resolution of the final image and affecting the quantitative accuracy and structure restoration of the imaging. Therefore, the existing IDT system still has obvious limitations in the structure of the illumination module and the arrangement of the light source, and an improved illumination scheme that can improve the illumination efficiency, reduce the exposure time, and realize a larger range of angle scanning is needed to comprehensively improve the imaging speed and spatial resolution of the IDT system.

[0031] The present application provides a fast intensity diffraction tomography (IDT) method based on graphics processing unit (GPU) acceleration without iteration process and a matching multi-angle free illumination device. The present application comprehensively considers the multiple limitations of the existing IDT system in imaging efficiency and reconstruction speed, aiming to solve the key technical bottlenecks of the traditional IDT imaging technology, such as long image acquisition time, multiple iteration times of reconstruction algorithm, large calculation resource consumption, and poor system real-time performance, so as to realize efficient, fast and low resource consumption three-dimensional quantitative imaging.

[0032] Reference Figure 1 , Figure 1It is the structural schematic view of the quick light intensity diffraction tomography device provided by the application, wherein, it comprises: a multi-angle free lighting device 1, a sample stage 2, a detection objective lens 3, an imaging tube lens 4 and a camera 5.

[0033] The multi-angle lighting device 1 comprises: A low coherence laser 1-1 is used for emitting low coherence laser, a parallel collimating optical system 1-2 is arranged on the light path of the low coherence laser and is used for generating a spatially uniform parallel light beam, a two-dimensional galvanometer assembly 1-3 is arranged on the light path of the parallel collimating optical system 1-2 and is used for realizing light beam angle deflection through voltage adjustment, and a 4f relay optical system 1-4 is arranged on the reflection light path of the two-dimensional galvanometer assembly 1-3, so that the mirror reflection surface of the two-dimensional galvanometer assembly 1-3 and the plane where the sample is located are optically conjugated. The sample stage 2 is arranged on the light path of the multi-angle lighting device 1 and is used for carrying the sample. The detection objective lens 3 is arranged behind the sample stage 2 and is used for collecting transmitted light through the sample. The imaging tube lens 4 is coaxial with the detection objective lens 3 and is used for focusing and imaging the transmitted light. The camera 5 is arranged on the imaging plane of the imaging tube lens 4 and is used for collecting the transmitted light intensity image.

[0034] In the embodiment of the application, the multi-angle lighting device 1 comprises a low coherence laser 1-1, the light emitted by the laser is adjusted to be parallel light after passing through the parallel collimating optical system (expanding and collimating device) 1-2, and the light beam aperture size is matched with the size of the galvanometer lens. Then the light is transmitted to the plane where the sample is located through the two-dimensional galvanometer assembly (two-axis galvanometer) 1-3 and the 4f relay optical system 1-4 by respectively adjusting the voltage of the two-axis galvanometer to control the lighting angle.

[0035] The sample stage 2 is used for fixing and clamping the sample and can move in the XY plane. The illumination light is received by the detection objective lens 3 after passing through the sample, and then is focused on the target surface of the camera 5 after passing through the imaging tube lens 4.

[0036] Specifically, the application adopts a low-coherence laser as a light source, and combines a high-speed galvo scanner to construct a controllable multi-angle illumination system. By driving the galvo scanner, the incident angle of the light beam is accurately and quickly adjusted. Compared with the traditional fixed LED ring illumination method, this scheme not only greatly improves the flexibility and spatial coverage of angle scanning, but also significantly reduces the time required for single three-dimensional image acquisition, and can meet the needs of high-speed imaging and dynamic process capture. The galvo scanner system can set the angle scanning path according to the actual needs, so as to flexibly fill in the areas in the three-dimensional spectral space that cannot be covered by the conventional system, especially obtaining more complete frequency domain information in the Z-axis direction. Therefore, under this system, there is no need to collect sample transmission images at different Z-axis positions, which greatly reduces the time required for image acquisition while ensuring the spatial resolution and refractive index restoration capability of the reconstructed image in the axial direction, and avoids the artifacts caused by the movement of the displacement table and the sample.

[0037] The multi-angle free-illumination device is composed of a low-coherence laser, a parallel collimation optical system, a two-dimensional galvo scanner assembly, and a 4f relay optical system. The low-coherence laser serves as a light source, and the output laser beam is first processed by the parallel collimation system to form a parallel light beam with uniform spatial distribution. The parallel light beam is then irradiated to the two-dimensional galvo scanner reflection surface, and by adjusting the scanning angle of the galvo scanner, the angle of the illumination light beam in space can be controlled and deflected. Then, through the 4f relay optical system, the scanning angle of the galvo scanner is effectively transmitted to the sample plane, realizing the conjugate imaging relationship between the galvo surface and the sample surface. This optical configuration ensures that the sample surface illumination spot position is fixed while dynamically adjusting the illumination angle of the sample, thereby achieving rapid switching of multi-angle illumination.

[0038] After the illumination light passes through the sample to be measured, the transmission information of the sample is collected by a high-numerical-aperture detection objective lens, and then the intensity image is collected by imaging on the imaging target surface of the camera through the tube lens.

[0039] Through the embodiment of the application, by the synergistic effect of the low-coherence laser, the two-dimensional galvo scanner, and the 4f relay optical system, the millisecond-level high-speed multi-angle illumination switching is realized while maintaining the fixed position of the sample surface illumination spot. At the same time, the axial spectral blind area of the traditional ring light source is filled by the flexible scanning of the galvo scanner, which significantly improves the axial resolution and reconstruction accuracy of three-dimensional imaging, and provides high signal-to-noise ratio and motion artifact-free original intensity image data for subsequent non-iterative fast reconstruction algorithms.

[0040] According to the fast light intensity diffraction tomography device provided by the application, the device further comprises: A synchronous control device is electrically connected with the multi-angle illumination device and the camera, and is used for coordinating the timing synchronization of the illumination angle switching and the camera exposure.

[0041] In the embodiment of the present application, the configured synchronization control device is used to accurately coordinate the scanning position of the galvanometer and the exposure state of the camera, ensure that the illumination scanning and imaging process at each angle are highly synchronized, and thus improve the accuracy and efficiency of data acquisition.

[0042] Through the above structural design, the device of the present application can significantly improve the image acquisition speed of the IDT system and the control accuracy and integrity of the illumination angle, and provide stable and efficient front-end data support for subsequent fast non-iterative image reconstruction algorithms.

[0043] In terms of image reconstruction, the present application designs an efficient non-iterative reconstruction process to solve the problem of complex calculation and slow speed in the iterative solving mode of the IDT algorithm, and implements it through the parallel processing mechanism under the GPU architecture. The non-iterative algorithm significantly reduces the dependence on CPU computing resources while maintaining imaging accuracy, and can realize millisecond-level three-dimensional reconstruction, greatly shortening the overall processing flow from data acquisition to image output, and having good real-time processing capability and system scalability.

[0044] Reference Figure 2 , Figure 2 The present application provides a fast light intensity diffraction tomography imaging method, which specifically comprises the following steps.

[0045] Step 201, constructing the light source distribution corresponding to each scanning angle based on the galvanometer scanning voltage value.

[0046] In the embodiment of the present application, the galvanometer voltage value during scanning is read and converted into a scanning angle to construct the light source distribution corresponding to each angle.

[0047] By presetting the galvanometer scanning angle sequence and the corresponding voltage value through software, the voltage signal is converted into the transverse spatial frequency coordinates (the X-axis transverse frequency and the Y-axis transverse frequency are respectively denoted as and ).

[0048] Step 202, determining the three-dimensional transfer function of each scanning angle based on the light source distribution and the preset optical device parameters.

[0049] In the embodiment of the present application, the three-dimensional transfer function is generated based on the light source distribution and the optical system parameters according to the following process: According to the numerical aperture (NA) of the detection objective lens, the system magnification, and the camera pixel size, the maximum frequency domain range (para_kBoundPixel) that can be received by the objective lens is calculated; in the frequency domain grid, the region with a transverse frequency less than the NA cutoff frequency is assigned a value of 1, and the region exceeding the value is assigned a value of 0, forming a binary pupil function (pupil).

[0050] The phase propagation kernel formula is used to calculate the phase delay of each layer.

[0051] Multiply the source distribution, pupil function, and phase propagation kernel pixel by pixel; perform a 2D Fourier transform on the result, outputting a complex 3D transfer function.

[0052] According to a fast light intensity diffraction tomography method provided by the present invention, the optical device parameters include: low coherence laser center wavelength, system overall magnification, medium refractive index, detection objective lens numerical aperture and camera pixel size; Based on the light source distribution and the preset optical device parameters, the three-dimensional transfer function of each scanning angle is determined, including: Construct the pupil function based on the light source distribution, low-coherence laser center wavelength, system overall magnification, medium refractive index, detection objective lens numerical aperture, and camera pixel size; Determine the phase propagation kernel of each layer according to the preset slice thickness; Based on the light source distribution, pupil function, and phase propagation kernel, a three-dimensional transfer function is determined for each scanning angle.

[0053] In this embodiment of the present invention, a pupil function is constructed based on the constructed light source distribution and the basic optical system parameters, including the low-coherence laser center wavelength, the overall system magnification, the medium refractive index, the detection lens numerical aperture (NA), and the camera pixel size. The phase propagation kernel for each layer is then calculated based on the set slice thickness.

[0054] The three-dimensional propagation function of the layer under different illumination angles is obtained by convolution based on the obtained light source distribution, pupil function and phase propagation kernel.

[0055] Through the embodiments of the present invention, by fusing the light source distribution, pupil function and phase propagation kernel, a physically accurate three-dimensional transfer function is constructed, and the light wave diffraction and propagation process under multi-angle illumination is completely modeled in the frequency domain, thereby achieving high-resolution three-dimensional refractive index reconstruction simultaneously in a single focal plane acquisition, avoiding the axial mechanical scanning and iterative optimization required by traditional methods, and significantly improving imaging speed and axial accuracy.

[0056] According to a fast light intensity diffraction tomography method provided by the present invention, a phase propagation kernel of each layer is determined according to a preset layer thickness, comprising: The phase propagation kernel formula is used to determine the phase propagation kernel of each slice: in, represents the phase propagation kernel, represents the refractive index of the medium surrounding the sample, Indicates the axial distance of layer-cut position, denotes the central wavelength of low-coherence laser, denotes the transverse spatial frequency coordinate in the frequency domain, corresponding to the frequency components in the x direction and y direction in the image plane respectively, denotes the imaginary unit.

[0057] In the embodiments of the present application, denotes the refractive index of the medium around the sample, which is directly input by experimental configuration parameters according to the environment in which the sample is located. denotes the central wavelength of low-coherence laser, which is obtained by reading the nominal central wavelength of the low-coherence laser (for example, = 532 nm). denotes the first The axial distance of the layer-cut position is calculated according to the preset layer-cut thickness. denotes the transverse spatial frequency coordinate in the frequency domain, which is calculated by the camera pixel size and the magnification.

[0058] Through the closed-form analytical expression of the phase propagation kernel, the axial diffraction propagation process of light waves from the focal plane to each layer-cut surface in the frequency domain is accurately quantified, and a light field phase delay model in three-dimensional space is directly constructed.

[0059] Step 203, obtaining a sample transmission light intensity map corresponding to each scanning angle.

[0060] In the embodiments of the present application, a multi-angle free illumination device is used to scan the sample at different angles, and a camera is used to capture a sample transmission light intensity map of the clearest layer corresponding to each scanning angle.

[0061] According to the fast light intensity diffraction tomography method provided by the present application, a sample transmission light intensity map corresponding to each scanning angle is obtained, comprising: Repeat the following steps until the transmission light intensity map acquisition of all preset angles is completed, wherein the single-angle exposure time is less than a preset value: Drive the two-dimensional galvanometer assembly to scan the sample at the preset angle through the synchronous control device; When the two-dimensional galvanometer assembly scans to the target angle, trigger the camera exposure through the synchronous control device to obtain the transmission light intensity map of the target angle; After triggering the camera exposure, switch the two-dimensional galvanometer assembly to the next scanning angle through the synchronous control device.

[0062] In the embodiment of the present application, the hardware is coordinated by the synchronous control device to send an exposure trigger signal to the camera after the galvanometer scans to the target angle; the camera collects the transmission light intensity map of the clearest layer (focal plane) of the sample with an exposure time less than or equal to 1 millisecond; after the exposure is completed, the galvanometer immediately switches to the next angle, and the cycle is repeated until all the preset angles are collected. Finally, a set of 8-bit grayscale transmission light intensity images corresponding to the scanning angles are output.

[0063] Through the embodiment of the present application, the synchronous control device precisely coordinates the timing of galvanometer scanning and camera exposure, realizes millisecond-level multi-angle illumination switching and image acquisition under the condition of maintaining the absolute stillness of the sample position, eliminates the motion artifacts of traditional displacement scanning, and compresses the single-angle exposure time by using the high light intensity characteristics of low coherence laser.

[0064] Step 204, pre-processing and two-dimensional Fourier transform of the sample transmission light intensity map to obtain the light intensity spectrum map of each scanning angle.

[0065] In the embodiment of the present application, the collected sample transmission light intensity map is pre-processed and two-dimensional Fourier transformed into the frequency domain.

[0066] For example, the 8-bit grayscale image is converted into a floating-point complex number format (the real part is the normalized light intensity value, and the imaginary part is 0); the background noise is eliminated and the non-uniform illumination is corrected. The two-dimensional fast Fourier transform is performed on each pre-processed transmission light intensity map to output the frequency domain complex number data (i.e. the light intensity spectrum map).

[0067] Step 205, deconvolution and mapping of the three-dimensional transfer function of each scanning angle and the light intensity spectrum map of each scanning angle by Tikhonov regularization to obtain the three-dimensional imaging result of the sample refractive index data.

[0068] In the embodiment of the present application, the regularization coefficient is set, and the light intensity spectrum map and the three-dimensional transfer function of different angles are deconvoluted by Tikhonov (Tikhonov) regularization.

[0069] According to the mapping of the deconvoluted data according to the theoretical derivation, the three-dimensional imaging result of the sample refractive index data is finally obtained.

[0070] According to the fast light intensity diffraction tomography method provided by the present application, the pre-processing and two-dimensional Fourier transform of the sample transmission light intensity map, the construction of the three-dimensional transfer function and the deconvolution operation of the Tikhonov regularization are performed in parallel on the graphics processing unit.

[0071] The application uses low coherence laser combined with a galvanometer as a ring-shaped illumination system, significantly improves the sample position illumination intensity, reduces the camera acquisition time, can provide more complete Z direction spectral information, reduces the acquisition time and motion artifacts under the condition of ensuring the Z direction resolution. The CUDA algorithm written under the GPU framework significantly reduces the dependence on CPU computing resources, can realize millisecond-level three-dimensional reconstruction, greatly shortens the overall processing flow from data acquisition to image output, and provides the possibility for IDT real-time three-dimensional imaging.

[0072] The following illustrates an example of a fast light intensity diffraction tomography method provided by the application in practical application.

[0073] Reference Figure 3 , Figure 3 is the overall flowchart of the fast light intensity diffraction tomography method provided by the application, which comprises: reading the current scanning voltage and system information to construct a three-dimensional transfer function; using a multi-angle illumination device to scan the sample to obtain a transmission light intensity map; and performing three-dimensional refractive index reconstruction through an algorithm. Specifically, the following steps are included.

[0074] Step 1, read the current scanning voltage and system information to construct a three-dimensional transfer function, all data mentioned below, if not specially stated, are stored on the GPU video memory, and are calculated on the GPU using the CUDA language.

[0075] Step 1.1, set the number of scanning angles and the corresponding two-axis galvanometer voltages in the software, and then correspond the voltages to the scanning X-axis and Y-axis transverse frequencies, respectively denoted as and .

[0076] Step 1.2, calculate the pupil function pupil. According to the numerical aperture, magnification of the system used probe objective and the size of the camera pixel size, the maximum frequency range that the camera can accept under this system is calculated , the frequency domain interval corresponding to each pixel and the number of pixels corresponding to the maximum frequency range that the objective can collect .

[0077] When the transverse frequency is within the range that the objective can collect, the value of pupil is 1; when the transverse frequency exceeds the range that the objective can collect, the value of pupil is 0; read the i-th illumination angle and , since it is parallel light illumination, the illumination angle is set to 1 at the pixel position corresponding to the frequency spectrum and 0 at other positions, to obtain the light source function of the i-th illumination angle .

[0078] Step 1.3. According to the phase propagation theory Write CUDA kernel to generate phase propagation kernel .

[0079] where is the refractive index of the medium surrounding the sample, represents the Z-axis direction of the layer cutting distance of the h-th layer cutting position relative to the clearest layer of the shot, is the central wavelength of the low coherence laser, represents the distribution of frequency transverse and longitudinal, is a complex unit.

[0080] The kernel function is assigned a block size of (32, 32), which is a basic execution unit containing 32x32=1024 threads that can share memory and synchronize between threads. The grid is composed of multiple Blocks: X / Y dimensions: size (image width / height), Z dimension: size_Joint (total number of layer cutting), where the default size of a single image collected is a square of 1024x1024 pixels, is the total number of layer cutting set. The index corresponding to each pixel in the GPU can be calculated as follows: pixel column thread number can be calculated by (pixel row thread number, pixel layer number) and pixel layer number Similarly, the corresponding memory index position is calculated as (i.e. column index, row index, layer index) Other kernel functions mentioned later also use this method to establish the index of pixel and memory location.

[0081] Step 1.4. Write kernel and use cufftExecC2C function to perform forward two-dimensional Fourier transform on GPU to calculate three-dimensional transfer function i represents the i-th illumination angle, represents the light source distribution of the i-th angle, represents the pupil function, represents the phase propagation kernel, which includes basic four arithmetic operations (addition, subtraction, multiplication, and division) between matrix and matrix corresponding pixels (elements) in the kernel function, the matrix data type must be a type unique to CUDA, and there is no four arithmetic operations about this type itself or itself and float type in the CUFFT library, so a related structure is needed to perform operator overloading.

[0082] Step 1.5, (i represents the i-th lighting angle, h represents the h-th layer of the slice) The real and imaginary parts are respectively subjected to inverse 2D Fourier transform on the GPU using the cufftExecC2C function. In order to ensure the reasonable numerical value after the inverse Fourier transform, it is necessary to write a kernel function to normalize the inverse transformed matrix. At this point, the three-dimensional transfer function matrix for different angles is obtained. (stores the real part matrix of all angles and all slice transfer functions) and (Store the imaginary matrix of all angles and all slice transfer functions). If the real matrix and the imaginary part matrix The required video memory size is too large, causing the system's overall video memory to exceed the GPU's own video memory size, which will cause a sharp drop in computing speed. Therefore, the real matrix needs to be and the imaginary part matrix Part of the data is copied to the system memory, and then copied back to the GPU memory when needed to prevent the memory overflow and cause the calculation speed to decrease.

[0083] Step 2: Use a multi-angle illumination device to scan the sample to obtain a transmitted light intensity map. Turn on the low-coherence laser and use a galvanometer to scan at different angles in sequence. When the galvanometer scans an angle, it transmits a signal to the system synchronization control device, which sends a signal to trigger the camera to expose and capture the image. After the acquisition is completed, the camera sends a signal to the system synchronization control device, which then sends a signal to the galvanometer to move to another angle. Repeat the above steps to capture the transmitted light intensity map of the sample from multiple angles.

[0084] Step 3: Reconstruct the three-dimensional refractive index through an algorithm.

[0085] Step 3.1: First, the camera stores the captured 8-bit grayscale image on the hard disk. The program reads the multi-angle transmitted light intensity map of the sample into the system memory, and then copies it to the GPU memory and writes a kernel function to convert the light intensity map data type from 8-bit to type, and other preprocessing such as image normalization.

[0086] Step 3.2, use the cufftExecC2C function to perform a two-dimensional Fourier transform on the intensity map to obtain , Indicates the lighting angles.

[0087] Step 3.3, write a kernel function to calculate the distribution of the scattering potential in the frequency domain.

[0088] wherein denotes the illumination angle, denotes the layer of the layer cut, are the Tikhonov regularization coefficients, respectively, which can take the value 0, and need to be fine-tuned for a good image reconstruction.

[0089] wherein denotes the real part of the Scattering Potential in the frequency domain, denotes the imaginary part of the Scattering Potential in the frequency domain, denotes the total number of illumination angles, denotes the angle, denotes the two-dimensional Fourier transform (complex form) of the angle-dependent transmission intensity map, denotes the Tikhonov regularization parameter, which suppresses the noise amplification (typical values 0.001-0.1). denotes the angle, layer cut, denotes the complex conjugate of denotes the angle, layer cut, denotes the imaginary part of the three-dimensional transfer function, denotes the complex conjugate of .

[0090] Step 3.4 uses the cufftExecC2C function to perform a two-dimensional inverse Fourier transform on and writes a kernel function to normalize and map the values to the sample refractive index to obtain the three-dimensional distribution of the sample refractive index .

[0091] After that, without changing the galvanometer scanning mode, steps 2 and 3 can be repeated in a loop to collect and solve the information of different positions of the sample for reconstruction.

[0092] It should be noted that if only one collection and solution is performed, in order to reduce the memory occupation and will be released after participating in a summation operation.

[0093] Reference Figure 4 , Figure 4is a time comparison chart of different schemes provided by the present application, including: 3-1 and 3-2; 3-1 shows the exposure time required for a camera to take a picture using an LED light ring and the exposure time required for a camera using a low coherence laser. 3-2 shows the time comparison of using a C++ and CUDA language program and a matlab program to calculate once with a 1024*1024 pixel size, taking 24 angle transmission intensity maps. It can be seen that the fast light intensity diffraction tomography device and the calculation method provided by the present application can realize real-time imaging of IDT.

[0094] Reference Figure 5 , Figure 5 is a calculation result comparison chart of different schemes provided by the present application. Figure 5 are the final reconstruction results of the calculation program written by C++ and CUDA language and the calculation program written by matlab, respectively.

[0095] Figure 6 is a schematic diagram of the physical structure of an electronic device provided by the present application, as shown in Figure 6 The electronic device can include a processor 610, a communications interface 620, a memory 630 and a communications bus 640, wherein the processor 610, the communications interface 620 and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call the logical instructions in the memory 630 to execute the fast light intensity diffraction tomography method, which includes: constructing the light source distribution corresponding to each scanning angle based on the galvanometer scanning voltage value; determining the three-dimensional transfer function of each scanning angle based on the light source distribution and the preset optical device parameters; obtaining the sample transmission light intensity map corresponding to each scanning angle; pre-processing and two-dimensional Fourier transform of the sample transmission light intensity map to obtain the light intensity spectrum map of each scanning angle; and through Tikhonov regularization, deconvolution and mapping of the three-dimensional transfer function of each scanning angle and the light intensity spectrum map of each scanning angle to obtain the three-dimensional imaging result of the sample refractive index data.

[0096] In addition, the logic instructions in the memory 630 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0097] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being executable by a processor to cause a computer to perform the fast light intensity diffraction tomography method provided by the above-mentioned methods, the method comprising: constructing a light source distribution corresponding to each scanning angle based on a galvanometer scanning voltage value; determining a three-dimensional transfer function of each scanning angle based on the light source distribution and a preset optical device parameter; obtaining a sample transmitted light intensity map corresponding to each scanning angle; pre-processing and two-dimensional Fourier transforming the sample transmitted light intensity map to obtain a light intensity spectrum map of each scanning angle; and performing deconvolution and mapping on the three-dimensional transfer function of each scanning angle and the light intensity spectrum map of each scanning angle by Tikhonov regularization to obtain a three-dimensional imaging result of sample refractive index data.

[0098] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the fast light intensity diffraction tomography method provided by the above-mentioned methods, the method comprising: constructing a light source distribution corresponding to each scanning angle based on a galvanometer scanning voltage value; determining a three-dimensional transfer function of each scanning angle based on the light source distribution and a preset optical device parameter; obtaining a sample transmitted light intensity map corresponding to each scanning angle; pre-processing and two-dimensional Fourier transforming the sample transmitted light intensity map to obtain a light intensity spectrum map of each scanning angle; and performing deconvolution and mapping on the three-dimensional transfer function of each scanning angle and the light intensity spectrum map of each scanning angle by Tikhonov regularization to obtain a three-dimensional imaging result of sample refractive index data.

[0099] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of the embodiments or some parts of the embodiments.

[0101] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A fast light intensity diffraction tomography device, characterized in that: include: A multi-angle lighting device, comprising: A low-coherence laser is used to emit low-coherence laser light; a parallel collimating optical system is arranged on the output light path of the low-coherence laser and is used to generate a spatially uniform parallel light beam; a two-dimensional galvanometer mirror assembly is arranged on the output light path of the parallel collimating optical system and realizes the angle deflection of the light beam by voltage adjustment; a 4f relay optical system is arranged on the reflection light path of the two-dimensional galvanometer mirror assembly so that the galvanometer mirror reflection surface of the two-dimensional galvanometer mirror assembly forms an optical conjugate with the plane where the sample is located; A sample stage, arranged on the outgoing light path of the multi-angle illumination device, for carrying the sample; a detection objective lens, disposed behind the sample stage and configured to collect light transmitted through the sample; An imaging tube lens, coaxial with the detection objective lens, for focusing and imaging the transmitted light; The camera is arranged on the imaging plane of the imaging tube lens and is used to collect the transmitted light intensity image.

2. The fast light intensity diffraction tomography device according to claim 1, characterized in that: The device further comprises: A synchronization control device is electrically connected to the multi-angle lighting device and the camera, and is used to coordinate the timing synchronization of lighting angle switching and camera exposure.

3. A rapid light intensity diffraction tomography method, characterized in that: The fast light intensity diffraction tomography device according to any one of claims 1 to 2 comprises: Based on the scanning voltage value of the galvanometer, the light source distribution corresponding to each scanning angle is constructed; Determining a three-dimensional transfer function for each scanning angle based on the light source distribution and preset optical device parameters; Obtain the sample transmitted light intensity graph corresponding to each scanning angle; The sample transmitted light intensity map is pre-processed and subjected to two-dimensional Fourier transformation to obtain a light intensity spectrum map at each scanning angle; The three-dimensional transfer function of each scanning angle and the light intensity spectrum of each scanning angle are deconvolved and mapped through Tychonoff regularization to obtain a three-dimensional imaging result of the sample refractive index data.

4. The rapid light intensity diffraction tomography method according to claim 3, characterized in that: The optical device parameters include: low coherence laser center wavelength, overall system magnification, medium refractive index, detection objective lens numerical aperture and camera pixel size; Determining the three-dimensional transfer function of each scanning angle based on the light source distribution and preset optical device parameters includes: Constructing a pupil function based on the light source distribution, the central wavelength of the low-coherence laser, the overall magnification of the system, the refractive index of the medium, the numerical aperture of the detection objective lens, and the size of the camera pixel; Determine the phase propagation kernel of each layer according to the preset slice thickness; A three-dimensional transfer function for each scanning angle is determined based on the light source distribution, the pupil function, and the phase propagation kernel.

5. The rapid light intensity diffraction tomography method according to claim 4, characterized in that: The step of determining the phase propagation kernel of each layer according to the preset layer thickness includes: The phase propagation kernel formula is used to determine the phase propagation kernel of each slice: in, represents the phase propagation kernel, represents the refractive index of the medium surrounding the sample, Indicates the Axial distance of the slice position, represents the central wavelength of the low-coherence laser, Represents the horizontal spatial frequency coordinates in the frequency domain, corresponding to the frequency components in the x and y directions in the image plane, respectively. Represents an imaginary unit.

6. The rapid light intensity diffraction tomography method according to claim 3, characterized in that: The obtaining of a sample transmitted light intensity graph corresponding to each scanning angle includes: Repeat the following steps until all transmitted light intensity images at the preset angles are collected, and the exposure time for each angle is less than the preset value: The synchronous control device drives the two-dimensional galvanometer assembly to scan the sample at a preset angle; When the two-dimensional galvanometer assembly scans to a target angle, the synchronous control device triggers the camera exposure to obtain a transmitted light intensity map at the target angle; After triggering the camera exposure, the synchronization control device controls the two-dimensional galvanometer assembly to switch to the next scanning angle.

7. The rapid light intensity diffraction tomography method according to claim 3, characterized in that: The preprocessing of the sample transmitted light intensity map and the two-dimensional Fourier transform, the construction of the three-dimensional transfer function and the deconvolution operation of the Tikhonov regularization are performed in parallel on a graphics processor.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the fast light intensity diffraction tomography method according to any one of claims 3 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fast light intensity diffraction tomography method according to any one of claims 3 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the fast light intensity diffraction tomography method according to any one of claims 3 to 7 is implemented.

Images