A fast imaging method for wide-field optical sectioning based on virtual HiLo algorithm

Abstract

The present invention belongs to the field of optical imaging, and more specifically, to the field of optical sectioning imaging, and provides a wide-field optical sectioning fast imaging method based on a virtual HiLo algorithm, the process of which includes edge detection of the original image, and then combining the widely used HiLo algorithm to realize the virtual HiLo algorithm process. The present invention has an optical sectioning performance comparable to conventional optical sectioning imaging technology. The method of the present invention is characterized in that only one wide-field illuminated image is required to reconstruct an optical sectioning image. Compared with conventional optical sectioning methods, the method of the present invention can effectively improve the imaging speed, suppress motion artifacts, simplify the complexity of imaging and reduce the cost of imaging devices. In addition, the method of the present invention can be easily applied to imaging fields with requirements related to defocus blur noise suppression.

Description

Technical Field

[0001] The present invention belongs to the field of optical imaging, more specifically, the field of optical sectioning imaging. It involves edge detection, and in combination with the HiLo algorithm, a low-cost, wide-field optical sectioning fast imaging method based on a virtual HiLo algorithm is proposed. Technical Background

[0002] Wide-field imaging is a basic imaging method in optical imaging methods, which is widely popular because of its fast speed and low cost. But some common disadvantages are also obvious. When observing objects with depth, the out-of-focus signal is blurred into the background or noise, and the additional illumination light excitation may reduce the signal-to-noise ratio of the acquired image. This hinders the acquisition of high-contrast images. In order to suppress this out-of-focus noise, a variety of optical sectioning imaging techniques have been developed, such as confocal fluorescence microscopy, two-photon fluorescence microscopy, light sheet fluorescence microscopy, OS-SIM microscopy, HiLo microscopy, etc. But due to their special optical design, each technology has its advantages and disadvantages. Among them, confocal or two-photon fluorescence microscopy based on laser scanning optical design achieves optical sectioning capabilities at the expense of low imaging speed. Light sheet fluorescence microscopy requires expensive and inconvenient complex optical components. Conventional OS-SIM microscopy can simplify the optical system, but requires three images of sinusoidal illumination patterns with specific phase shifts to reconstruct optical sectioning images, which limits the imaging speed and the application of capturing fast dynamic events.

[0003] In recent years, HiLo technology with excellent optical sectioning capability based on two images (uniform illumination image and structured illumination image) has been widely used in traditional wide-field microscopes, wide-field endoscopes, wide-field multiphoton microscopes, light sheet microscopes, light-field microscopes, multifocal microscopes, incoherent holography, etc.

[0004] Although the wide-field imaging mode with only two captured images has greatly improved the imaging speed, the two captured images required, uniform illumination image and structured illumination image, still affect the improvement of imaging speed and imaging quality. First, it cannot fully utilize the imaging speed potential of scientific cameras because there is a switch between structured illumination and uniform illumination in HiLo. Therefore, its imaging speed is always less than half of the camera imaging speed. Second, the requirement of acquiring images twice means a compromise between imaging speed and imaging stability, otherwise artifacts will be introduced. For fast-moving objects, motion-induced artifacts between the two captured images are inevitable, which will affect the final imaging quality. Finally, HiLo is rarely used in label-free imaging, despite its excellent sectioning ability, inconvenient steps and redundant equipment are part of the reasons that limit its application scope. Therefore, there is currently no method that can simultaneously meet the conditions of fast, low-cost, artifact-free, convenient and easy to expand.

[0005] In order to overcome the above limitations of existing optical sectioning imaging technology, the present invention proposes a wide-field optical sectioning fast imaging method based on a virtual HiLo algorithm, which can simultaneously simplify the optical system and increase the imaging speed. This new method only requires one wide-field uniformly illuminated image, breaking through the imaging speed limitation of the traditional HiLo or OS-SIM method. For a single image, no artifacts caused by motion are introduced. In addition, the optical sectioning performance is comparable to that of current optical sectioning methods. Summary of the invention

[0006] The purpose of the present invention is to combine edge detection, HiLo algorithm and other technologies to provide a wide-field optical sectioning fast imaging method based on a virtual HiLo algorithm, namely V-HiLo-ED (Virtual HiLo based on edge detection), which overcomes the problems of existing optical sectioning methods such as slow imaging speed, high cost, complex experimental setup, and the presence of artifacts.

[0007] The present invention can reconstruct optical sectioning image results based on a wide-field uniformly illuminated image without any prior training or other conditions, significantly improving imaging speed, reducing experimental costs, simplifying experimental complexity, and avoiding motion artifacts.

[0008] In order to achieve the above object, the present invention adopts the following technologies. It should be understood that the Laplacian operator (LOG) in edge detection described herein is only used to explain the present invention and is not used to limit the present invention:

[0009] A wide-field optical sectioning rapid imaging method based on a virtual HiLo algorithm comprises the following steps:

[0010] Step 1: Apply the Laplacian operator (LOG) based edge detection method to the acquired uniformly illuminated wide-field image I u (x,y) to extract the edge detection image I L (x, y), in actual operation, the edge detection method based on the Laplace operator (LOG) can be directly used through the MATLAB image processing toolbox. 2 The formula for G(x,y) is as follows:

[0011]

[0012] Where (x, y) is the spatial coordinate, is the second-order partial derivative operation, σ is the standard deviation. The following is the edge detection method of the Laplace operator ▽ 2 G(x,y) acts on the acquired uniformly illuminated wide-field image I u(x,y) to obtain the edge detection image I L (x,y), the formula is as follows:

[0013]

[0014] In the formula, is the convolution operation.

[0015] Step 2: Reconstruct an optical section image I using the HiLo algorithm HiLo (x,y) requires a uniformly illuminated image I u (x,y) and a structured illumination image I S (x,y), where the edge detection image I u (x,y) can be replaced by the structured illumination image I S (x,y), the formula is as follows:

[0016]

[0017] The uniform illumination images I obtained in the previous two steps are u (x,y) and edge detection image I S (x, y) are respectively used as inputs of the HiLo algorithm for calculation, where the specific steps of the HiLo algorithm are as follows:

[0018] Step 3: In the HiLo algorithm, the high-frequency component H is calculated first. i (x,y),H i (x, y) can be directly obtained by performing a Gaussian high-pass filter operation on the uniformly illuminated image. The formula is as follows:

[0019] H i (x,y)=F -1 {HP fc {F[I u (x,y)]}} (4)

[0020] Among them, F and F -1 are Fourier and inverse Fourier transform operations, HP fc is a two-dimensional Gaussian high-pass filter operation, where f c Indicates the cutoff frequency of the high-pass filter.

[0021] Step 4: Calculate the low-frequency component L o (x,y), calculate L o Before (x,y), the difference image I needs to be calculated first dbp (x, y), the image I that will be uniformly illuminated u (x,y) and edge detection image I L (x, y) is subtracted and then applied to a bandpass filter to obtain a difference image Idbp (x,y), the formula is as follows:

[0022] I dbp (x,y)=F -1 {{F[I u (x,y)-I s (x,y)]}×BFP(f x ,f y )} (5)

[0023] In the formula, × is the multiplication operation, BFP(f x ,f y ) is a two-dimensional bandpass filter, (f x ,f y ) are the frequency domain coordinates.

[0024] Step 5: Based on the calculated difference image I dbp (x, y), we can then calculate the local contrast image W(x, y), which represents the rough intensity distribution of the in-focus part, expressed as:

[0025]

[0026] where Λ is the length of the square sampling window, which depends on the filter cutoff frequency k c Here, k c It can be defined as k c =1 / (2Λ). Λ (x,y) and μ Λ (x,y) are the difference images I dbp The standard deviation and mean of (x,y). The formula is as follows:

[0027]

[0028]

[0029] In the formula, * represents the relevant operation. Λ (x,y) is the kernel used to calculate W(x,y), N k is the window matrix N Λ The sum of (x,y).

[0030] Step 6: Based on the local contrast image W(x,y), the uniform illumination image I u (x, y) is multiplied with the local contrast image, and the result of the multiplication is subjected to Gaussian low-pass filtering to obtain the low-frequency component L o (x,y), the formula is as follows:

[0031] L o (x,y)=F-1 {LP fc {F[W(x,y)×I u (x,y)]}} (9)

[0032] Where LP fc It is a low pass filtering operation.

[0033] Step 7: The high frequency component H obtained by the above calculation i (x,y) image and low frequency component L o The (x, y) images are added with a certain weight η to obtain the final slice image I HiLo (x,y), in order to make the transition from low frequency to high frequency smoother, the weight value varies according to different samples. The adjustment range is between 0 and 3, and the formula is as follows:

[0034] I HiLo (x,y)=H i (x,y)+ηL o (x,y) (10)

[0035] The beneficial effects of the present invention are:

[0036] 1. The present invention combines edge detection technology, HiLo optical sectioning imaging technology and wide-field imaging, overcoming the problems of complex defocus information and low signal-to-noise ratio in traditional wide-field imaging, and achieving optical sectioning imaging results with high signal-to-noise ratio and no background.

[0037] 2. Compared with the existing conventional optical sectioning imaging technology, the present invention can increase the imaging speed by more than 2 times because it only requires one image.

[0038] 3. Compared with the existing conventional optical sectioning imaging technology, the present invention can effectively suppress artifacts introduced by moving objects due to taking multiple images, thanks to the fact that it only requires one image.

[0039] 4. Compared with existing confocal, two-photon and other optical sectioning imaging technologies, the present invention can effectively reduce experimental costs and simplify experimental complexity due to its wide-field imaging mode and simple experimental device requirements.

[0040] 5. Thanks to its wide-field imaging mode and simple experimental device requirements, the present invention can be easily expanded to various fields that require suppression of out-of-focus background. Its great scalability is conducive to promoting the development of various fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram showing a wide-field uniform imaging device provided by an embodiment of the present invention;

[0042] In the figure, 1-incoherent green LED light source, 2-lens 1, 3-lens 2, 4-object, 5-lens 3, 6-camera sCMOS;

[0043] Figure 2 Flow chart showing the optical sectioning method based on a single wide-field uniformly illuminated image provided by an embodiment of the present invention

[0044] Figure 3 Comparison diagram showing the improvement of the effect brought by the optical sectioning method of the present invention; (a) antibody-labeled mitochondrial image with wide-field uniform illumination; (b) edge detection image result of the wide-field uniform illumination image; (c) extracted low-frequency component Lo; (d) extracted high-frequency component Hi; (e) optical sectioning image result I reconstructed based on Hi and Lo HiLo ; (f) Comparison of light intensity distribution between wide-field uniform illumination image and optical sectioning image at the same position.

[0045] Figure 4 A comparison diagram showing the artifact suppression effect of the optical sectioning method of the present invention and the traditional HiLo method; (a) wide-field uniformly illuminated image with dense defocus and in-focus information; (b) wide-field structured light illumination image result of an object with motion displacement; (c) optical sectioning image with artifacts reconstructed based on (a), (b) and the conventional HiLo algorithm; (d) edge detection image result of the wide-field uniformly illuminated image (a); (e) optical sectioning image without artifacts reconstructed based on (a), (d) and the method of the present invention; DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the Laplacian operator (LOG) in edge detection described herein is only used to explain the present invention and is not used to limit the present invention.

[0047] Reference Figure 1 As shown, the present invention illustrates a wide-field imaging system for V-HiLo-ED, which uses a green incoherent light source LED with an optical fiber as an illumination light source to perform original wide-field imaging and the proposed V-HiLo-ED method imaging. The LED is converged by lens 1 and then diverged by lens 2 as non-collimated light. It is only necessary to ensure that the light beam can simultaneously illuminate object 1 and object 2 and is not blocked by the aperture of lens 3. Then, it is converged by lens 3 to the imaging camera sCMOS, and a single wide-field imaging can be completed to obtain a uniformly illuminated wide-field image I. u (x,y).

[0048] After obtaining the above-mentioned uniformly illuminated wide-field image I by wide-field imaging u(x,y) After that, please refer to the image processing flow Figure 2 As shown in the program flow chart, the main process steps are as follows:

[0049] Step 1: Apply the Laplacian operator (LOG) based edge detection method to the acquired uniformly illuminated wide-field image I u (x,y) to extract the edge detection image I L (x, y), in actual operation, the edge detection method based on the Laplace operator (LOG) can be directly used through the MATLAB image processing toolbox. 2 The formula for G(x,y) is as follows:

[0050]

[0051] Where (x, y) is the spatial coordinate, is the second-order partial derivative operation, σ is the standard deviation. The following is the edge detection method of the Laplace operator ▽ 2 G(x,y) acts on the acquired uniformly illuminated wide-field image I u (x,y) to obtain the edge detection image I L (x,y), the formula is as follows:

[0052]

[0053] In the formula, is the convolution operation.

[0054] Step 2: Reconstruct an optical section image I using the HiLo algorithm HiLo (x,y) requires a uniformly illuminated image I u (x,y) and a structured illumination image I S (x,y), where the edge detection image I u (x,y) can be replaced by the structured illumination image I S (x,y), the formula is as follows:

[0055]

[0056] The uniform illumination images I obtained in the previous two steps are u (x,y) and edge detection image I S (x, y) are respectively used as inputs of the HiLo algorithm for calculation, where the specific steps of the HiLo algorithm are as follows:

[0057] Step 3: In the HiLo algorithm, the high-frequency component H is calculated first. i (x,y),H i(x, y) can be directly obtained by performing a Gaussian high-pass filter operation on the uniformly illuminated image. The formula is as follows:

[0058] H i (x,y)=F -1 {HP fc {F[I u (x,y)]}} (14)

[0059] Among them, F and F -1 are Fourier and inverse Fourier transform operations, HP fc is a two-dimensional Gaussian high-pass filter operation, where f c Indicates the cutoff frequency of the high-pass filter.

[0060] Step 4: Calculate the low-frequency component L o (x,y), calculate L o Before (x,y), the difference image I needs to be calculated first dbp (x, y), the image I that will be uniformly illuminated u (x,y) and edge detection image I L (x, y) is subtracted and then applied to a bandpass filter to obtain a difference image I dbp (x,y), the formula is as follows:

[0061] I dbp (x,y)=F -1 {{F[I u (x,y)-I s (x,y)]}×BFP(f x ,f y )} (15)

[0062] In the formula, × is the multiplication operation, BFP(f x ,f y ) is a two-dimensional bandpass filter, (f x ,f y ) are the frequency domain coordinates.

[0063] Step 5: Based on the calculated difference image I dbp (x, y), we can then calculate the local contrast image W(x, y), which represents the rough intensity distribution of the in-focus part, expressed as:

[0064]

[0065] where Λ is the length of the square sampling window, which depends on the filter cutoff frequency k c Here, k c It can be defined as k c =1 / (2Λ). Λ(x,y) and μ Λ (x,y) are the difference images I dbp The standard deviation and mean of (x,y). The formula is as follows:

[0066]

[0067]

[0068] In the formula, * represents the relevant operation. Λ (x,y) is the kernel used to calculate W(x,y), N k is the window matrix N Λ The sum of (x,y).

[0069] Step 6: Based on the local contrast image W(x,y), the uniform illumination image I u (x, y) is multiplied with the local contrast image, and the result of the multiplication is subjected to Gaussian low-pass filtering to obtain the low-frequency component L o (x,y), the formula is as follows:

[0070] L o (x,y)=F -1 {LP fc {F[W(x,y)×I u (x,y)]}} (19)

[0071] Where LP fc It is a low pass filtering operation.

[0072] Step 7: The high frequency component H obtained by the above calculation i (x,y) image and low frequency component L o The (x, y) images are added with a certain weight η to obtain the final slice image I HiLo (x,y), in order to make the transition from low frequency to high frequency smoother, the weight value varies according to different samples. The adjustment range is between 0 and 3, and the formula is as follows:

[0073] I HiLo (x,y)=H i (x,y)+ηL o (x,y) (20)

[0074] Please refer to Figure 3 The figure shows an example of wide-field uniform illumination imaging of mitochondria labeled with antibodies taken by LED, and the optical sectioning image results after being processed by the V-HiLo-ED algorithm we proposed.

[0075] Please refer to Figure 4The figure shows an example of a fast-moving sample in a wide-field uniform illumination image taken by an LED, and an optical sectioning image after being processed by the V-HiLo-ED algorithm we proposed. Figure 4 (c) is an optical section image with artifacts, and Figure 4 (e) is an optical section image without artifacts reconstructed by the present invention

[0076] Therefore, based on the HiLo algorithm and combined with the edge detection method, it is possible to reconstruct an optical sectioning image using only one wide-field uniformly illuminated image. Compared with confocal, two-photon and other optical sectioning microscopes, this not only reduces the cost, but also utilizes its specific wide-field imaging mode to make up for the defect of slow imaging speed. Compared with the traditional HiLo algorithm and OS-SIM method that require at least two images to achieve optical sectioning imaging, the method of the present invention increases the imaging speed by at least two times, and for fast-moving objects, based on the advantage of single-shot imaging, it can effectively avoid the artifacts that are difficult to avoid in traditional methods due to the need for multiple images. In addition, the method of the present invention can effectively simplify the experimental steps for reconstructing optical sectioning images and reduce experimental costs. Thanks to its extremely low imaging conditions, the method of the present invention can be easily expanded to many fields that need to suppress out-of-focus blurred backgrounds.

[0077] The technical contents not elaborated in detail in the present invention belong to the common knowledge of those skilled in the art. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A fast imaging method for wide-field optical sectioning based on virtual HiLo algorithm, It is characterized in that The steps include: Step 1: Get a wide-field uniformly illuminated image I u (x, y), and perform edge detection to obtain edge detection image I L (x,y); Step 2: Edge detection image I L (x, y) and the structured light illumination image I required by the HiLo algorithm s (x, y) are replaced with each other, and the formula is as follows: Yo s (x,y)≈I L (x,y)(1) Step 3: Find the high-frequency component H required to reconstruct the optical section image i (x,y), the formula is as follows: H i (x,y)=F -1 {HP fc {F[I u (x,y)]}} (2) In the formula, F and F -1 are Fourier transform and inverse Fourier transform operations, HP fc It is a Gaussian high-pass filter operation; Step 4: Calculate the low-frequency component L required to reconstruct the optical section image o (x,y), the formula is as follows: L o (x,y)=F -1 {LP fc {F[W(x,y)×I u (x,y)]}} (3) Where W(x,y) is the local contrast image; LP fc is a Gaussian high-pass filter operation, where f c represents the cutoff frequency of the high-pass filter, and × is the multiplication operation; Step 5: Reconstruct the optical section image I HiLo , the formula is as follows: Yo Hilo (x,y)=H i (x,y)+ηL o (x,y) (4) Where η is the high frequency component H i (x,y) and low frequency component L o The weight ratio of (x,y) ranges from 0 to 1; In the step 4, the local contrast image W(x,y) is calculated, and the specific steps are as follows: Step 4.1, calculate the wide field uniform illumination image I u (x,y) and edge detection image I L The difference image I of (x,y) dbp (x,y), the formula is as follows: I dbp (x,y)=F -1 {{F[I u (x,y)-I S (x,y)]}×BFP(f x ,f y )} (5) In the formula, (f x ,f y ) is the frequency domain coordinate, BFP(f x ,f y ) is a Gaussian high-pass filter, and its formula is as follows: Where σ is the standard deviation of the low-pass filter; Step 4.2, calculate the local contrast image W(x,y), the formula is as follows: Where Λ is the length of the square sampling window, which depends on the cutoff frequency k of the filter. c , k c =1 / (2Λ);σ Λ (x,y) and μ Λ (x,y) are the difference images I dbp The standard deviation and mean of (x,y) are as follows: In the formula, * represents the relevant operation, N Λ (x,y) is the kernel used to calculate W(x,y), N k is the window matrix N Λ The sum of (x,y).

Images