A method for correcting image intensity in intensity diffraction tomography

Abstract

The present invention relates to the technical field of computational optical imaging, and specifically discloses a method for correcting the image plane intensity of intensity diffraction tomography. Under the condition of LED array illumination, the edge variance value is calculated using the light and dark transition characteristics of the image edge, and the image affected by the vignetting effect is screened and removed in combination with the threshold condition. The lamp bead position of the recorded image as a bright field image is subjected to an "AND" operation with the lamp bead position of the image affected by the vignetting effect to obtain the lamp bead position that needs to be corrected for the exposure time. According to the linear relationship between the image plane intensity and the camera exposure time, by recording two images with different exposure times under the illumination of each lamp bead, the intensity linear coefficient term and constant term at different lamp bead positions are obtained, and then according to the image plane intensity and exposure time when the center lamp bead is illuminated, the exposure time of the lamp beads at other positions is calculated, and finally the image plane intensity correction under different lamp bead illumination is achieved, effectively improving the reconstruction quality of intensity diffraction tomography.

Description

Technical Field

[0001] The present invention relates to the field of computational optical imaging technology, in particular to the field of non-interference intensity diffraction tomography three-dimensional imaging based on LED array illumination, and specifically to a method for correcting image plane intensity of intensity diffraction tomography. Background Art

[0002] Refractive index is a core indicator for studying the microscopic properties of biological samples. Slight changes in its value are closely related to the dry mass, morphological structure, and concentration of intracellular biomolecules of the cells. Traditional quantitative phase imaging (QPI) technology can only obtain two-dimensional phase distribution and cannot obtain true three-dimensional information of the sample. Therefore, based on tomography technology, researchers have proposed a variety of non-interference-based intensity diffraction tomography (IDT) technologies in recent years. This technology uses LED arrays as light sources to measure the intensity image of the sample under different illumination angles, and combines reconstruction algorithms to restore the three-dimensional refractive index distribution of the sample.

[0003] However, in the actual imaging process, using LED arrays as light sources will result in uneven distribution of recorded image intensity, which will lead to a decrease in the quality of the reconstructed image. The specific reasons include (θ represents the angle between the direction of each LED bead in the LED array to the sample and the normal direction of the LED bead surface): (1) When the illumination angle is large, the light intensity received by the sample increases with the increase of cos 2 (θ) decays; (2) The luminous characteristics of the lamp beads satisfy the Lambertian radiation model. Since there is an angle between the illumination direction and the normal direction of the lamp bead surface, the image intensity decays at a rate of cos(θ); (3) When the illumination is tilted, the illumination area of ​​the sample is affected by the illumination angle, causing the image intensity to decay at a rate of cos(θ); (4) The brightness of the lamp beads of the same batch of LED light sources fluctuates; (5) Problems with the light source control program or circuit cause the brightness of the lamp beads to decay.

[0004] Among them, reasons (1)-(3) are the uneven image intensity caused by the physical process, and reasons (4)-(5) are the uneven image intensity caused by the quality of the light source hardware. The image plane image intensity correction is essentially to increase the exposure time of some LED lamp beads or multiply by the correction factor so that the image intensity at each illumination angle remains consistent during the final imaging. There are two common correction strategies: the first is to make corrections by analyzing the relationship between the illumination angle of the LED lamp beads and the brightness of the light received by the sample, and the two satisfy I = I 0cos m(θ)(J. Yang, et al. “Intensity correction research for Fourierptychographic microscopy,” Laser Optoelectron. Prog. 54(3), 31101-31108(2017))。 If the exposure time of the central lamp bead of the LED light source is T , then the exposure time of the lamp bead that deviates from the optical axis is T / cos m (θ), where I 0 is the image intensity when the central lamp is lighting. θ is the angle between the lighting direction of each lamp bead and the normal direction of the lamp bead surface, m is a real number greater than 1. This solution takes into account the brightness attenuation caused by physical reasons, but cannot correct the brightness attenuation caused by the quality of the light source hardware. The second solution uses a high numerical aperture (NA) microscope objective to collect all bright field images in the absence of a sample. Since the objective NA is greater than the illumination NA, the image plane under all illumination angles is a bright field image. By multiplying each bright field image by the inverse of the normalized light intensity as a compensation factor, the light brightness under the low-magnification objective is compensated (A.Pan, et al. “Subwavelength resolution Fourier ptychography with hemisphericaldigital condensers,” Opt. Express 26(18), 23119-23131 (2018)). However, this solution can only ensure basic intensity consistency and cannot accurately correct the image plane intensity.

[0005] Furthermore, during IDT numerical reconstruction, halo blurring and reduced reconstruction quality can occur. This is primarily due to the inclusion of images with varying intensities that are neither bright nor dark field. These images are caused by the vignetting effect of the optical system. This is due to the gradual decrease in light entering the camera due to off-axis illumination, resulting in a nonlinear decay of the image plane illumination from the center to the edges. This results in uneven reception of light from different angles. Therefore, these alternating bright and dark images must be removed before numerical reconstruction. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention proposes a method for correcting image intensity in intensity diffraction tomography, which includes edge variance threshold calculation and exposure time correction to accurately correct image intensity based on LED array illumination.

[0007] Technical solution:

[0008] The idea of ​​the present invention is that when an image is affected by the vignetting effect, a half-bright and half-dark intensity change appears at the edge of the recorded image. By calculating the edge variance value of each image, the images affected by the vignetting effect are screened out. The lamp bead position where the image is recorded as a bright field image is "ANDed" with the lamp bead position where the image is affected by the vignetting effect to obtain the lamp bead position that needs to be corrected for exposure time. According to the linear relationship between the image plane intensity and the camera exposure time, by recording two images with different exposure times under the illumination of each lamp bead, the intensity linear coefficient term and constant term at different lamp bead positions are obtained, and then according to the image plane intensity I0 and exposure time T when the center lamp bead is illuminated, the exposure time of the lamp beads at other positions is calculated, and finally the image plane intensity correction under the illumination of different lamp beads is achieved.

[0009] In a first aspect, the present application provides a method for calibrating intensity of an intensity diffraction tomography image plane, characterized by comprising the following steps:

[0010] Step 1: Under clear imaging conditions, set the exposure time of the LED array to T1, set the preset conditions, and obtain the intensity images of the lamp beads at different positions through the intensity diffraction tomography system;

[0011] Step 2: Select a square mask, extract the edge value of each image, expand it into a one-dimensional vector, and calculate the edge strength variance value s 2 , combined with the threshold condition, the position of the lamp beads affected by the vignetting effect is obtained;

[0012] Step 3: Set the exposure time to T2, satisfying T2 ≠ T1, and repeat step 1 to obtain intensity images under illumination by the lamp beads at different positions. Combined with the images recorded at exposure time T1, calculate the linear correction coefficient and constant term of the image intensity at different lamp bead positions (m, n), where m and n represent the horizontal and vertical coordinates of the lamp bead position on the LED array, respectively.

[0013] Step 4: Calculate the position of the lamp bead in the bright field image and the position of the lamp bead in the image affected by the vignetting effect to obtain the position of the lamp bead that needs to be corrected for exposure time;

[0014] The bright field image refers to the incident light at different lighting angles emitted when the lamp beads at different positions are lit. illum Less than or equal to the numerical aperture NA of the microscope objective obj When the microscope is in the dark, the objective lens can collect all the incident light and form a bright field image;

[0015] Step 5: Light up the LED at the center and select exposure time T3, record the image intensity I0, and use the exposure time T of the LED at the position (m, n) that needs exposure time correction. m,nThe linear relationship between the image intensity I0 and the exposure time of the lamp beads in other positions is calculated to achieve image intensity correction under different lamp bead illumination.

[0016] In one embodiment, the preset conditions in step 1 include: adjusting the distance between the LED array and the stage to h, lighting up the lamp beads on the LED array in sequence, and when there is no sample on the stage, the light emitted by each lamp bead is incident on the microscope objective lens and reaches the target surface of the photoelectric imaging device after passing through the imaging lens.

[0017] In one embodiment, the size of the mask in step 2 is consistent with the image size, and the center of the mask is blank and the surrounding areas are filled with values.

[0018] In one embodiment, the formula for calculating the edge strength variance in step 2 is:

[0019] ;

[0020] in, Represents the edge value of the image taken out using the mask, a is the side length of the square mask, It represents the average value calculated after expanding the edge value of the image into a one-dimensional vector, and n1 represents the number of pixels of the edge value of the image.

[0021] In one embodiment, the threshold condition in step 2 is to set a threshold v for distinguishing images affected by the vignetting effect from images not affected by the vignetting effect; when the image is affected by the vignetting effect, the calculated edge variance value s 2 Large, when the marginal variance value s 2 When the value exceeds a preset threshold value v, it is determined that the image is affected by the vignetting effect.

[0022] In one embodiment, the step 3 calculates the linear correction coefficient k of the image intensity at different lamp bead positions (m, n) m,n and the constant term b m,n The specific process is:

[0023] ;

[0024] Among them, the LED array has x×y lamp beads, satisfying 1≤m≤x and 1≤n≤y, I1 m,n and I2 m,n They are the average image intensities at exposure times of T1 and T2 and at the lamp position (m, n).

[0025] In one embodiment, an “AND” operation is performed between the positions of the lamp beads in the bright field image recorded in step 4 and the positions of the lamp beads in the image affected by the vignetting effect.

[0026] In one embodiment, the exposure time of the lamp beads that do not need to be corrected in step 5 is set to T 3， The exposure time T of the position lamp bead (m, n) that needs exposure time correction m,n The linear relationship with the image plane intensity I0 is determined by the exposure time formula, which is: .

[0027] In a second aspect, the present application provides a system for implementing the above-mentioned method for correcting the intensity of an intensity diffraction tomography image plane.

[0028] Beneficial effects:

[0029] The proposed method for correcting image intensity in intensity diffraction tomography utilizes the edge transition characteristics of the image to calculate edge variance, thereby filtering and removing images affected by vignetting. Exposure time correction is achieved by exploiting the linear relationship between exposure time and image intensity. This approach is simple to operate, requires no objective lens replacement, and accurately corrects image intensity, effectively improving the reconstruction quality of intensity diffraction tomography. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is an optical path diagram of a non-interference intensity diffraction tomography system according to the present invention;

[0031] Figure 2 is a flow chart of image plane intensity correction of the present invention;

[0032] Figure 3 This is the correction process of the present invention. Figure 3 (a) is the distribution of the image intensity mean before correction along with the position of the lamp beads. Figure 3 (b) is the position distribution of the bright field image, Figure 3 (c) is the distribution of image edge variance values ​​along with the position of the lamp beads. Figure 3 (d) is the distribution of the image affected by the vignetting effect along with the position of the lamp beads. Figure 3 (e) is the position of the lamp bead that needs to be corrected. Figure 3 (f) shows the distribution of the mean image intensity after correction along with the position of the lamp beads.

[0033] Reference numerals:

[0034] 1-LED array, 2-stage, 3-microscope objective lens, 4-convex lens, 5-photoelectric imaging device. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0036] Example 1:

[0037] A method for correcting intensity of an intensity diffraction tomography image plane, characterized by comprising the following steps:

[0038] Step 1: Under clear imaging conditions, set the exposure time of the LED array to T1, adjust the distance between the LED array and the stage to h, and light up the LED beads on the LED array in sequence. When there is no sample on the stage, the light emitted by each lamp bead enters the microscope objective lens at a certain angle, passes through the imaging lens, and reaches the target surface of the photoelectric imaging device, obtaining the intensity image of the illumination of the lamp beads at different positions;

[0039] The certain angle is from 0° to θ max °, , where d represents the distance between adjacent lamp beads, and N represents the number of intervals between the center lamp bead and the farthest lamp bead.

[0040] Step 2: Select a square mask with a side length of a, extract the edge value of each image, expand it into a one-dimensional vector, and calculate the edge strength variance value s 2 , combined with the threshold condition, the position of the lamp beads affected by the vignetting effect is obtained; the mask is a matrix with a specific structure, the center of the structure is blank, the surrounding values ​​are fixed, and the size is consistent with the image size;

[0041] The formula for calculating the edge strength variance is:

[0042] ;

[0043] in, Represents the edge value of the image taken using the mask, It represents the average value calculated after expanding the edge value of the image into a one-dimensional vector, and n1 represents the number of pixels of the edge value of the image;

[0044] The threshold condition refers to setting a threshold v to distinguish images affected by the vignetting effect from those not affected. When the image is affected by the vignetting effect, the calculated edge variance value s 2 Especially large, therefore, when the marginal variance value s 2 When the value exceeds a preset threshold value v, it can be determined that the image is affected by the vignetting effect;

[0045] Step 3: Change the exposure time to T2 (T2≠T1), repeat step 1, and obtain the intensity images under the illumination of the lamp beads at different positions. Combined with the images recorded at exposure time T1, calculate the linear correction coefficient k of the image intensity at different lamp bead positions (m, n) m,n and the constant term b m,n , specifically:

[0046] ;

[0047] Among them, m and n represent the horizontal and vertical coordinates of the position of the lamp beads on the LED array respectively. The LED array has x×y lamp beads, satisfying 1≤m≤x and 1≤n≤y, I1 m,n and I2 m,n are the average image intensity at the lamp position (m, n) under exposure time T1 and T2 respectively;

[0048] Step 4: Perform an AND operation on the lamp bead position of the bright field image and the lamp bead position affected by the vignetting effect to obtain the lamp bead position that requires exposure time correction;

[0049] The position of the lamp beads in the bright field image refers to the incident light with different illumination angles when the lamp beads at different positions are lit. illum Less than or equal to the numerical aperture NA of the microscope objective obj When the microscope is in the dark, the objective lens can collect all the incident light and form a bright field image;

[0050] Step 5: Light up the LED at the center and select the appropriate exposure time T3. Record the image intensity I0 and use the exposure time formula to calculate the exposure time T for the LED at the position (m, n) that requires exposure time correction. m,n , and the exposure time of the lamp beads that do not need exposure time correction is set to T3;

[0051] The exposure time formula is: .

[0052] When an image is affected by the vignetting effect, the edges of the recorded image exhibit half-bright, half-dark intensity variations. By calculating the edge variance value for each image, images affected by the vignetting effect are screened out. The positions of the lamp beads where the image is recorded as a brightfield image are ANDed with the positions of the lamp beads where the image is affected by the vignetting effect to determine the lamp bead positions requiring exposure time correction. Based on the linear relationship between image plane intensity and camera exposure time, by recording two images with different exposure times for each lamp bead, the intensity linear coefficient term and constant term at different lamp bead positions are determined. Furthermore, based on the image plane intensity I0 and exposure time T when the center lamp is illuminated, the exposure time for lamp beads at other positions is calculated, ultimately achieving image plane intensity correction under different lamp bead illumination conditions.

[0053] Example 2:

[0054] This embodiment provides a system using the method of embodiment 1, involving a system for implementing a method for correcting the intensity of an intensity diffraction tomography image plane. The experimental optical path is shown in the attached figure. Figure 1 As shown, it includes: an LED array 1, a stage 2, a microscope objective lens 3, a convex lens 4, and a photoelectric imaging device 5.

[0055] Image plane intensity correction scheme, including edge variance threshold calculation and exposure time correction, the flow chart is as attached Figure 2 The specific workflow is as follows:

[0056] Adjust the distance between the LED array 1 containing 15×15 lamp beads and the stage 2 to 70 mm. Set the exposure time to 80 ms. No sample is placed on the stage 2. Starting from the center position (8, 8) of the LED array 1, record the first image; then record (8, 7) as the second image, (9, 7) as the third image, and then record the images in a counterclockwise order. Each lamp bead emits light with a central wavelength of 468 nm, which is incident on the 10× / 0.28NA microscope objective 3 at different angles. After passing through the convex lens 4, the incident light wave reaches the target surface 5 of the photoelectric imaging device, and the intensity images under the illumination of the lamp beads at different positions are obtained (as shown in the attached figure). Figure 3 (as shown in (a)).

[0057] In order to filter out the positions of the LEDs affected by the vignetting effect, a square mask with a side length of a (blank in the center and values ​​around it) is selected. The size of the mask is consistent with the image size. The edge value of each image is taken through the mask and calculated using the edge variance formula. Figure 3 (c) shows the edge variance of the image at different lamp bead positions. When the image is affected by the vignetting effect, the edge variance is particularly large, while the edge variance of the image not affected by the vignetting effect is very small. Set the threshold v = 100, when the edge variance value s 2 When it is greater than the threshold value v, the image is determined to be affected by the vignetting effect and the corresponding lamp position is marked as "0" (see the attached Figure 3 (d)), the images at these locations will be discarded and will not participate in the numerical reconstruction.

[0058] When the lamp beads at different positions are lit in sequence, the numerical aperture NA of the incident light at different angles is calculated using the spacing d=4 mm between adjacent lamp beads. illum , when NA illum Less than or equal to the numerical aperture NA of the microscope objective 3 obj When the microscope objective lens 3 can collect all the incident light and form a bright field image, the position of the lamp bead recording the bright field image (as shown in the attached figure) Figure 3 (b)) and the position of the lamp beads affected by the vignetting effect (as shown in the attached Figure 3 (d) to perform an “AND” operation to obtain the position of the lamp beads that need to be corrected for exposure time (as shown in the attached Figure 3 (e)).

[0059] Keeping other conditions unchanged, set the exposure time to 50 ms to obtain intensity images under the illumination of lamp beads at different positions. Combined with the images recorded when the exposure time is 80 ms, the linear correction coefficient k of the image intensity at different lamp bead positions (m, n) is calculated. m,n and the constant term b m,n Finally, light up the lamp at the center position (8, 8) and set the exposure time to 180 ms, and record the image intensity I0. Use the exposure time formula to calculate the exposure time T of the lamp at the position (m, n) that needs exposure time correction. m,n , and the exposure time of the lamp beads that do not need exposure time correction is set to 180 ms. Light up each LED lamp bead in turn, record the intensity image under the illumination of the lamp beads at different positions after the exposure time correction, and finally achieve the image plane intensity correction at different lamp bead positions (as shown in the attached figure). Figure 3 (f)).

[0060] The proposed method for correcting image intensity in intensity diffraction tomography utilizes the edge transition characteristics of the image to calculate edge variance, thereby filtering and removing images affected by vignetting. Exposure time correction is achieved by exploiting the linear relationship between exposure time and image intensity. This approach is simple to operate, requires no objective lens replacement, and accurately corrects image intensity, effectively improving the reconstruction quality of intensity diffraction tomography.

[0061] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. The replacement may be a replacement of a portion of a structure, device, or method step, or it may be a complete technical solution. Any equivalent replacement or modification based on the technical solution and inventive concept of the present invention shall be covered by the scope of protection of the present invention.

Claims

1. A method for correcting the intensity of an intensity diffraction tomography image plane, characterized in that The steps include: Step 1: Under clear imaging conditions, set the exposure time of the LED array to T1, set the preset conditions, and obtain the intensity images of the lamp beads at different positions through the intensity diffraction tomography system; Step 2: Select a square mask, extract the edge value of each image, expand it into a one-dimensional vector, and calculate the edge strength variance value s 2 , combined with the threshold condition, the position of the lamp beads affected by the vignetting effect is obtained; Step 3: Set the exposure time to T2, satisfying T2≠T1, and repeat step 1 to obtain intensity images under the illumination of lamp beads at different positions. Combined with the images recorded at exposure time T1, calculate the linear correction coefficient k of the image intensity at different lamp bead positions (m, n) m ,n and the constant term b m,n The specific process is: ； Among them, the LED array has x×y lamp beads, satisfying 1≤m≤x and 1≤n≤y, I1 m,n and I2 m,n are the average image intensities at the lamp position (m, n) under exposure times of T1 and T2, respectively. m and n represent the horizontal and vertical coordinates of the lamp position on the LED array, respectively. Step 4: Calculate the position of the lamp beads in the bright field image and the position of the lamp beads in the image affected by the vignetting effect to obtain the position of the lamp beads that need to be corrected for exposure time; the bright field image refers to the incident light at different illumination angles when the lamp beads at different positions are lit. When the numerical aperture NA of the incident light is illum Less than or equal to the numerical aperture NA of the microscope objective obj When the microscope is in the dark, the objective lens can collect all the incident light and form a bright field image; Step 5: Light up the LED at the center and select exposure time T3, record the image intensity I0, and use the exposure time T of the LED at the position (m, n) that needs exposure time correction. m,n The linear relationship with the image plane intensity I0 is determined by the exposure time formula. The exposure time of the lamp beads that do not need exposure time correction is set to T3, and the exposure time of the lamp beads in other positions is calculated to achieve image plane intensity correction under different lamp bead illumination; the exposure time formula is: .

2. The method for correcting intensity of an intensity diffraction tomography image according to claim 1, wherein: The preset conditions in step 1 include: adjusting the distance between the LED array and the stage to h, lighting up the lamp beads on the LED array in sequence, and when there is no sample on the stage, the light emitted by each lamp bead is incident on the microscope objective lens and reaches the target surface of the photoelectric imaging device after passing through the imaging lens.

3. The method for correcting image intensity of intensity diffraction tomography according to claim 1, characterized in that: The size of the mask in step 2 is consistent with the image size, and the center of the structure is blank and the surrounding areas are filled with values.

4. The method for correcting intensity of an intensity diffraction tomography image according to claim 1, wherein: The formula for calculating the edge strength variance in step 2 is: ； in, Represents the edge value of the image taken out using the mask, a is the side length of the square mask, It represents the average value calculated after expanding the edge value of the image into a one-dimensional vector, and n1 represents the number of pixels of the edge value of the image.

5. The method for correcting intensity of an intensity diffraction tomography image according to claim 1, wherein: The threshold condition in step 2 is to set a threshold v to distinguish images affected by the vignetting effect from those not affected; when the image is affected by the vignetting effect, the calculated edge variance value s 2 Large, when the marginal variance value s 2 When the value exceeds a preset threshold value v, it is determined that the image is affected by the vignetting effect.

6. The method for correcting intensity of an intensity diffraction tomography image according to claim 1, wherein: In step 4, an AND operation is performed between the lamp bead positions of the bright field image recorded and the lamp bead positions of the image affected by the vignetting effect.

7. A system for implementing the method for correcting intensity of an intensity diffraction tomography image plane according to any one of claims 1 to 6.

Images