A line illumination modulation dichromatic tomography imaging system

Abstract

The application discloses a line illumination modulation dichromatic tomography imaging system, belonging to the technical field of multicolor tomography imaging, comprising: a line illumination modulation module, comprising two monochromatic light sources and a modulation light path, each light beam is modulated to form a line light spot focused on the focal plane of an objective lens, the position parameter of the line light spot is located on the center of a pixel row or the intersection line of adjacent pixel rows; the distance between the position parameters of the two line light spots is an integer multiple of 0.5 pixels; an imaging module, a multi-element detector with n rows of pixels is used to continuously scan imaging along a first direction, n mixed images are obtained, each mixed image corresponds to a row of pixels, each mixed image contains the signals of 2 channels, n is a positive integer; an image demodulation module is used to subtract the mixed images of two symmetrical pixel rows, the signals of one channel are eliminated from the mixed images, monochromatic images are demodulated, and the monochromatic images contain the signals of another channel. The application realizes synchronous dichromatic tomography imaging on a single detector.

Description

Technical Field

[0001] This invention relates to the field of multicolor imaging technology, and in particular to a line illumination modulation dual-color tomography imaging system. Background Technology

[0002] Selectively labeling different structures in biological tissues with fluorophores of different colors and simultaneously acquiring multicolor fluorescence signals using multicolor microscopy can better resolve the spatial relationships and interactions between cells, organelles, and molecules in biological tissues. Current multicolor imaging methods typically use dichroic mirrors to spatially separate signals from each channel and transmit them to different monochrome cameras for detection. However, this method has several drawbacks: images acquired from commonly used wide-field imaging microscopes lack tomographic capabilities and are susceptible to strong defocus background interference; for fluorophores with overlapping emission spectra, dichroic mirrors cannot completely separate fluorescence signals from different channels, resulting in crosstalk between channels; using multiple monochrome cameras for detection leads to lateral image shifts and axial differences in the detection focal plane, necessitating complex registration processing of the original images; each channel requires a separate monochrome camera, limiting the number of channels for multicolor imaging due to system size and complexity; and the system's manufacturing cost increases with the number of monochrome cameras. Summary of the Invention

[0003] To overcome the problems of existing multicolor imaging methods, this invention provides a line illumination modulation dual-color tomography system. The technical solution is as follows:

[0004] Line illumination modulation dual-color tomography system includes:

[0005] The line illumination modulation module includes two monochromatic light sources and a modulation optical path. The two monochromatic light sources emit light beams with different wavelengths. Each light beam passes through the modulation optical path to form a line spot focused on the focal plane of the objective lens. The illumination intensity of the line spot on the focal plane of the objective lens is Gaussian distributed in a first direction, which is perpendicular to the extension direction of the line spot. The position parameter of the line spot is located at the center of a pixel row or on the intersection line of adjacent pixel rows. The distance between the position parameters of two line spots is an integer multiple of 0.5 pixels in the corresponding width in the object space. The line spots of the two channels are superimposed to form a dual-color line illumination light.

[0006] An imaging module is used to continuously scan and image along the first direction using a multi-element detector with n rows of pixels to obtain n mixed images under the illumination of the dual-color line illumination light. Each mixed image corresponds to one row of pixels, and each mixed image contains signals from two channels. The n ≥ 3 and is a positive integer.

[0007] The image demodulation module is used to perform subtraction processing on a mixed image of two symmetrical pixel rows, and demodulate a monochrome image from the mixed image. The position parameters of the two symmetrical pixel rows are symmetrical about the line spot of one channel, and the monochrome image corresponds to the signal of the other channel.

[0008] Furthermore, the image demodulation module includes:

[0009] The channel selection unit is used to select one channel as the target channel and another channel as the auxiliary channel;

[0010] A pixel row determination unit is used to determine a first pixel row and a second pixel row that are symmetrical about the position parameters of the auxiliary channel;

[0011] An image correction unit is used to perform de-biasing correction on the mixed image of the second pixel row;

[0012] The image demodulation unit is used to obtain a monochrome image of the target channel based on the mixed image of the first pixel row and the mixed image of the second pixel row after polarization correction.

[0013] Furthermore, the image correction unit includes:

[0014] A kernel acquisition subunit is used to acquire the debiasing correction convolution kernel between the first pixel row and the second pixel row;

[0015] A convolution kernel correction subunit is used to perform debiasing correction on the hybrid image of the second pixel row using the debiasing correction convolution kernel.

[0016] Furthermore, the kernel acquisition subunit is used for:

[0017] Acquire monochrome images of the first pixel row and the second pixel row with only the auxiliary channel illumination beam turned on;

[0018] Perform Fourier transform on the monochrome images of the first pixel row and the second pixel row to obtain the frequency domain images of the first pixel row and the second pixel row.

[0019] After dividing the frequency domain image of the first pixel row and the frequency domain image of the second pixel row, an inverse Fourier transform is performed to obtain the debiasing correction convolution kernel.

[0020] Furthermore, the kernel acquisition subunit is used for:

[0021] Calculate the effective point spread function of the first pixel row and the effective point spread function of the second pixel row;

[0022] The effective point spread function of the first pixel row and the effective point spread function of the second pixel row are subjected to Fourier transform to obtain the optical transfer function of the first pixel row and the optical transfer function of the second pixel row.

[0023] The optical transfer function of the first pixel row is divided by the optical transfer function of the second pixel row, and then an inverse Fourier transform is performed to obtain the debiasing correction convolution kernel.

[0024] Furthermore, the image correction unit includes:

[0025] The translation parameter acquisition subunit is used to acquire the translation parameters between the first pixel row and the second pixel row;

[0026] A translation correction subunit is used to perform debiasing correction on the blended image of the second pixel row using the translation parameters.

[0027] Furthermore, the modulation optical path includes a shaping optical path for shaping the beam into a line beam, a position adjustment optical path for adjusting the position parameters of each line spot, and a projection optical path for superimposing the line spots to form a two-color line illumination light.

[0028] Furthermore, the position adjustment optical path includes a second dichroic mirror, a third dichroic mirror, a first reflecting mirror, a second reflecting mirror, a third reflecting mirror, and a fourth reflecting mirror. The second dichroic mirror splits the incident light beam into two optical paths. One optical path passes through the second dichroic mirror and the third dichroic mirror in sequence, and the other optical path passes through the second dichroic mirror, the first reflecting mirror, the second reflecting mirror, and the third dichroic mirror in sequence. The light emitted from the third dichroic mirror passes through the third reflecting mirror and the fourth reflecting mirror in sequence.

[0029] Furthermore, the imaging module includes:

[0030] A scanning unit is used to continuously scan and image along a first direction using a multi-element detector with n rows of pixels, where n ≥ 3;

[0031] The image patch acquisition unit acquires the strip image patch of the i-th pixel row in each frame of a sample obtained in chronological order;

[0032] The stitching unit stitches together the strip image blocks of the i-th pixel row in each frame of a sample to obtain the mixed image of the i-th pixel row, where i∈n.

[0033] Furthermore, the line illumination modulation dual-color tomography system also includes a driving module for driving the line illumination modulation module and the sample to move relative to each other in three mutually perpendicular directions.

[0034] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0035] This invention provides a line-illumination modulation dual-color tomography system that uses a single multi-element detector for scanning imaging, reducing system cost and complexity compared to imaging methods requiring multiple cameras. It employs multiple pixel rows for detection imaging, with images between channels having a natural registration relationship, eliminating the need for additional registration processing. The demodulated image eliminates defocus background signals and naturally possesses tomographic capabilities. In fluorescence imaging, this line-illumination modulation dual-color tomography system utilizes the differences in illumination light wavelengths to excite different colors of fluorescence signals, avoiding emission spectrum crosstalk. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a functional block diagram of the line illumination modulation dual-color tomography imaging system of the present invention;

[0038] Figure 2 This is a schematic diagram of the optical path structure for line illumination modulation dual-color tomography of the present invention;

[0039] Figure 3 This is a schematic diagram of the linear illumination light distribution of the present invention;

[0040] Figure 4 This is a functional block diagram of the imaging module of the present invention;

[0041] Figure 5 This is a schematic diagram of the sample imaging acquisition process of the present invention;

[0042] Figure 6 This is a schematic diagram of the three-dimensional imaging of the present invention;

[0043] Figure 7 This is a functional block diagram of an image correction unit according to the present invention;

[0044] Figure 8 This is a functional block diagram of another image correction unit of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0046] like Figure 1 As shown, this embodiment of the invention provides a line illumination modulation dual-color tomography system, including a line illumination modulation module 10, an imaging module 20, and an image demodulation module 30.

[0047] The line illumination modulation module 10 includes two monochromatic light sources and a modulation optical path. The light beams emitted by the two monochromatic light sources have different wavelengths. Each light beam is shaped by the modulation optical path into a line spot focused on the focal plane of the objective lens. The illumination intensity of the line spot on the focal plane of the objective lens is Gaussian distributed in a first direction, which is perpendicular to the extension direction of the line spot. The position parameters of the line spots formed after different light beams are modulated are different. The position parameters of the line spots are located at the center of the pixel row or on the intersection of adjacent pixel rows. The distance between the position parameters of two line spots is an integer multiple of the width of 0.5 pixels in the object space.

[0048] Furthermore, the imaging module 20 includes a multi-element detector with n rows of pixels for multi-line detection imaging of the sample signal excited by the dual-color line illumination light. Since each channel line spot only excites the signal of one color on the sample, by continuously scanning and imaging along the first direction with the multi-element detector with n rows of pixels, n mixed images under dual-color line illumination light can be obtained. Each mixed image corresponds to one row of pixels, and each mixed image contains signals from 2 channels, where n≥3 and is a positive integer.

[0049] Furthermore, the image demodulation module 30 performs subtraction processing on the mixed image of two pixel rows that are symmetrical about the line spot position parameters of one channel, and demodulates a monochrome image from the mixed image, which corresponds to the signal of the other channel.

[0050] The line illumination modulation dual-color tomography system provided in this invention uses a single multi-element detector for scanning imaging. Compared to imaging methods that require multiple cameras, this reduces system cost and complexity. Furthermore, by employing multiple pixel rows for detection and imaging, the images between each channel have a natural registration relationship, eliminating the need for additional registration processing and achieving simultaneous dual-color imaging. Image demodulation is achieved by subtracting the mixed image of two pixel rows whose line spot position parameters are symmetrical about one channel. This eliminates defocus background signals in the demodulated image, which naturally possesses tomographic capabilities. In fluorescence imaging, this line illumination modulation dual-color tomography system utilizes the difference in illumination light wavelengths to excite different colors of fluorescence signals, avoiding emission spectrum crosstalk.

[0051] It should be noted that the monochromatic light source in this line-illumination modulated dual-color tomography system refers to a light source with a specific wavelength. This can be a monochromatic laser source, a monochromatic LED (Light-emitting diode) source, or a broadband light source combined with a narrow-band filter to obtain a monochromatic light source. Furthermore, this line-illumination modulated dual-color tomography system is applicable to various scenarios, including fluorescence microscopy and non-fluorescence microscopy (i.e., the illumination wavelength is the same as the detection wavelength), and this application does not impose any limitations on these applications.

[0052] Furthermore, the modulation optical path includes a shaping optical path for shaping the beam into a line beam, a position adjustment optical path for adjusting the position parameters of each line spot, and a projection optical path for superimposing the line spots to form a two-color line illumination light.

[0053] The following example, using two laser light sources, illustrates the optical path structure of a line-illumination modulation dual-color tomography system in fluorescence microscopy, as described in this application. Figure 2 As shown, the shaping optical path includes a first dichroic mirror 3 for combining two laser light sources, a first beam expander 4, a second beam expander 5, and a cylindrical lens 12 arranged sequentially along the beam propagation direction after beam combining. The position adjustment optical path includes a second dichroic mirror 6, a third dichroic mirror 7, a first reflecting mirror 8, a second reflecting mirror 9, a third reflecting mirror 22, and a fourth reflecting mirror 11. The projection optical path includes an illumination tube lens 13, an objective lens 14, and a fourth dichroic mirror 17 (in non-fluorescent imaging, this needs to be replaced with a semi-reflective beam splitter, or a combination of a polarizing beam splitter and a quarter-glass slide, or other devices that can both reflect illumination light and transmit probe light with the same wavelength as the illumination light), used to superimpose line spots to form a two-color line illumination light. The line illumination modulation two-color tomography imaging system in this application also includes an imaging optical path, specifically including an objective lens 14, a fourth dichroic mirror 17, an emission filter 18 (not needed in non-fluorescent imaging), a probe tube lens 19, and a multi-element detector 21.

[0054] Specifically, the laser beams emitted from the laser sources of the two channels are first combined by the first dichroic mirror 3, and then expanded by the first beam expander 4 and the second beam expander 5. Next, the illumination beams from the different channels are separated by the second dichroic mirror 6 and enter two optical paths respectively. One optical path passes through the second dichroic mirror 6 and the third dichroic mirror 7, while the other optical path passes through the second dichroic mirror 6, the first reflecting mirror 8, the second reflecting mirror 9, and the third dichroic mirror 7 in sequence. Finally, the illumination beams from the two optical paths are combined again by the third dichroic mirror 7. The light emitted from the third dichroic mirror 7 passes sequentially through the third reflecting mirror 22 and the fourth reflecting mirror 11. Finally, the combined illumination beam is shaped into a line beam by the cylindrical lens 12 and then projected onto the focal plane of the objective lens via the illumination tube lens 13, the fourth dichroic mirror 17, and the objective lens 14. This line beam is used to excite the fluorescence signal of the sample 15 on the three-dimensional motorized translation stage 16. The fluorescence signal passes sequentially through the objective lens 14, the fourth dichroic mirror 17, the emission filter 18, and the detector tube lens 19, and is finally received by the multi-element detector 21 for imaging.

[0055] Specifically, adjusting the angle of any one or both of the third reflector 22 or the fourth reflector 11 can simultaneously adjust the position parameters of the line spot in two channels (denoted as channel a and channel b). Adjusting the angle of any one or more of the second dichroic mirror 6, the first reflector 8, the second reflector 9, and the third dichroic mirror 7 can adjust the position parameters of the line spot in the corresponding channel (denoted as channel b) along the optical path. Therefore, during the experiment, the angle of any one or both of the third reflector 22 or the fourth reflector 11 can be adjusted first to shift the line spot of channel a along the x-direction until its position parameter is located at the center of a pixel row or on the intersection of adjacent pixel rows. Then, by adjusting the angle of any one or more of the second dichroic mirror 6, the third dichroic mirror 7, the first reflector 8, and the second reflector 9, the line spot of channel b can be shifted along the x-direction until its position parameter is also located at the center of a pixel row or on the intersection of adjacent pixel rows. Furthermore, the distance between the position parameters of the line spots in the two channels is an integer multiple of the width of 0.5 pixels in the object space. By adjusting the position of the optical path, the line spots on the two channels can be misaligned in the x-direction, thus achieving the purpose of separating the line spots of different channels in the x-direction.

[0056] In other embodiments, the second dichroic mirror 6 and the third dichroic mirror 7 can be replaced with a PBS (Polarization Beamsplitter) to achieve the function of beam splitting. Specifically, a half-wavelength glass slide is inserted before combining lasers of different wavelengths to adjust the polarization state of the lasers, so that the polarization states of the different wavelength lasers after beam combining are orthogonal. Then, the PBS can be used to achieve beam splitting of different wavelength lasers and subsequent beam combining.

[0057] In other embodiments, the modulation optical path can be achieved by directly generating line beams of different wavelengths from linear LEDs with different emission wavelengths, which are then projected onto the focal plane of the objective lens via a projection optical path. The positional parameters of the line spot on the focal plane of the objective lens can be changed by adjusting the positions of the different linear LEDs.

[0058] like Figure 3 As shown, this is a schematic diagram of the line illumination light distribution of the present invention. The illumination intensity of the line illumination beams in channels 1 and 2 follows a Gaussian distribution in the x-direction. The imaging area of ​​the multi-element detector is 8 pixel rows, and the position parameters of the line spot focused on the objective lens focal plane by each beam after passing through the modulation optical path are different. Figure 3 As shown in (A), the brightest point of the line spot in the x-direction of channel 1 is located on the intersection of the 4th and 5th pixel rows, meaning the position parameter of the line spot in channel 1 is located on the intersection of the 4th and 5th pixel rows. Similarly, the position parameter of the line spot in channel 2 is located on the intersection of the 5th and 6th pixel rows. Figure 3 As shown in (B), the line spot position parameter of channel 1 is located in the center of the 4th pixel row, and the line spot position parameter of channel 2 is located in the center of the 5th pixel row; as Figure 3 As shown in (C), the line spot position parameter of channel 1 is located on the intersection of the 4th and 5th pixel rows, and the line spot position parameter of channel 2 is located in the center of the 5th pixel row.

[0059] In this embodiment, the multi-element detector can be an area-array CCD (charge-coupled device) or an area-array CMOS (Complementary Metal Oxide Semiconductor) camera with sub-array or ROI (Region of Interest) functionality, or it can be a linear CCD or linear CMOS camera with area array mode functionality.

[0060] Furthermore, referring to Figure 4 The imaging module 20 includes:

[0061] The scanning unit 201 is used to continuously scan and image along a first direction using a multi-element detector with n rows of pixels, where n ≥ 3;

[0062] Image patch acquisition unit 202 acquires the strip image patch of the i-th pixel row in each frame image of a sample obtained in chronological order;

[0063] The stitching unit 203 stitches together the strip image blocks of the i-th pixel row in each frame of a sample to obtain the mixed image of the i-th pixel row, i∈n.

[0064] Figure 5This is a schematic diagram of the sample imaging acquisition process of the present invention, which is described below in conjunction with... Figure 5 The functions of the imaging module 20 are described below. To facilitate subsequent image demodulation, in this embodiment, the imaging area of ​​the multi-element detector is an n-pixel row arranged along the x-direction, and the movement direction of the sample is also along the x-direction, thereby facilitating the scanning and imaging of the sample under different pixels to achieve synchronous dual-color imaging.

[0065] During the imaging acquisition process, the three-dimensional electric translation stage 16 drives the sample 15 to move at a constant speed along the x-direction. The single-frame exposure time of the multi-element detector is equal to the time it takes for the sample 15 to move one pixel across the corresponding width in the object space. If we define any pixel row in a frame as a strip image block, then the multiple strip image blocks corresponding to that pixel row in multiple frames represent the sequential imaging of various parts of the sample. By stitching together these sequentially imaged strip image blocks obtained in time order, we can obtain a mixed image corresponding to n pixel rows. Each mixed image corresponds to one pixel row, and each mixed image contains signals from two channels, where n ≥ 3 and is a positive integer. Specifically, the i-th pixel row is selected from the n pixel rows. The image block acquisition unit 202 acquires the strip image block of the i-th pixel row in each frame image of a sample obtained in time order. The stitching unit 203 stitches together the strip image blocks of the i-th pixel row in each frame image of a sample to obtain the mixed image of the i-th pixel row.

[0066] Furthermore, the line illumination modulation dual-color tomography system of the present invention also includes a driving module 40 for driving the line illumination modulation module and the sample to move relative to each other in three mutually perpendicular directions.

[0067] Specifically, refer to Figure 6 The diagram illustrates the three-dimensional imaging of the present invention. The sample is defined as having four surfaces, each divided into four sample strips. A three-dimensional motorized translation stage 16 moves the sample 15 along the x-direction to image the first sample strip. Then, it moves along the y-direction by the width of one sample strip, followed by movement along the -x-direction to image the second sample strip. This process is repeated continuously to image the third and fourth sample strips. After scanning and imaging the first surface, the three-dimensional motorized translation stage 16 moves the sample 15 a certain distance along the z-direction, changing the focal depth of the sample from the first surface to the second surface. Similarly, the three-dimensional motorized translation stage 16 moves the sample 15 again along the x-direction to image the second surface. This cycle is repeated to achieve three-dimensional imaging of the sample.

[0068] The following mainly explains the demodulation principle and process in fluorescence imaging. The demodulation principle and process for non-fluorescence imaging are basically the same as those for fluorescence imaging, with the main difference being the different detection wavelengths. This results in a slight difference in the effective PSF of the system, which can be ignored, and the method is still applicable.

[0069] Similar to the imaging principle of line scanning confocal microscopy, in this method, the mixed image corresponding to each pixel row is the convolution of the system's effective PSF (point spread function) and the fluorescence signal distribution on the sample. The effective PSF of the system is the product of the illumination system PSF and the detection system PSF.

[0070] Under dual-color illumination, the mixed image contains fluorescence signals from two channels. Taking a single-channel fluorescence signal as an example, the effective PSF of the j-th channel in the i-th pixel row is:

[0071]

[0072] in, Indicates the lighting PSF, This indicates the detection PSF, where i represents the row number of each pixel in the imaging region, j represents the type number of the fluorescent protein, which is also the channel number, x, y, z represent the coordinates of each direction in the three-dimensional object space, and b... i,j This represents the distance between the i-th pixel row and the position parameter of the illumination line spot in the j-th channel.

[0073] Therefore, the signal of the j-th channel contained in the blended image obtained by the camera in the i-th pixel row can be represented as:

[0074]

[0075] Among them, f j (x,y,z) represents the relative concentration distribution of the j-th fluorescent protein on the sample, and "*" indicates convolution operation. As can be seen from formula (2), the fluorescence signal of the sample is modulated by both the illumination system PSF and the detection system PSF.

[0076] Furthermore, referring to Figure 1 The image demodulation module 30 includes:

[0077] The channel selection unit 301 is used to select one channel as the target channel and another channel as the auxiliary channel;

[0078] The pixel row determination unit 302 is used to determine a first pixel row and a second pixel row that are symmetrical about the position parameters of the auxiliary channel.

[0079] Image correction unit 303 is used to perform de-biasing correction on the mixed image of the second pixel row;

[0080] The image demodulation unit 304 is used to obtain a monochrome image of the target channel based on the mixed image of the first pixel row and the mixed image of the second pixel row after polarization correction.

[0081] When dual-channel simultaneous illumination occurs, the mixed image contains fluorescence signals from both channels. First, one channel is selected as the target channel and the other as the auxiliary channel using the channel selection unit 301. In formula (1), the intensity distribution of both the illumination PSF and the detection PSF in each z-plane is rotationally symmetric. Therefore, for any channel, the effective PSF shapes of two pixel rows that are symmetrical about the line spot position parameters are approximately the same (the imaging focal depth and modulation intensity of these two symmetrical pixel rows are the same). Thus, the images of this channel acquired by these two symmetrical pixel rows are also approximately the same.

[0082] Furthermore, due to the misalignment of the line spots in the two channels, two pixel rows symmetrical about the position parameters of the line spots in the auxiliary channel will exhibit modulation of different intensities under the influence of the line spots in the target channel. Therefore, by using the pixel row determination unit 302, two pixel rows symmetrical about the position parameters of the line spots in the auxiliary channel are selected as the first pixel row and the second pixel row. Then, the mixed image of the first pixel row and the second pixel row is subtracted, and the monochromatic image of the fluorescence signal corresponding to the target channel can be demodulated from the mixed image.

[0083] Specifically, the reduction process includes: performing debiasing correction on the mixed image of the second pixel row by the image correction unit 303, and then obtaining the monochrome image of the target channel by the image demodulation unit 304 based on the mixed image corresponding to the first pixel row and the mixed image of the second pixel row after debiasing correction.

[0084] by Figure 3 Taking the lighting condition shown in (A) as an example, the line spot position parameter of channel 1 is located on the intersection of the 4th and 5th pixel rows, and the line spot position parameter of channel 2 is located on the intersection of the 5th and 6th pixel rows. Since the 5th and 6th pixel rows of channel 2 are symmetrical about the center of the line spot of channel 2, the signal of channel 2 can be eliminated by processing the mixed image corresponding to the 5th and 6th pixel rows of channel 2, thereby obtaining the monochrome image of channel 1.

[0085] In some embodiments, the 4th and 7th pixel rows, and the 3rd and 8th pixel rows in channel 2 can also be selected for mixed image demodulation to obtain a monochrome image of channel 1.

[0086] In some embodiments, the monochrome images of channel 1 demodulated from multiple pairs of pixel rows can be added together to improve the signal-to-noise ratio of the demodulated image.

[0087] Similarly, Figure 3 (B) The 4th and 6th pixel rows of channel 2 are symmetrical about the line spot position parameters. Therefore, the signal of channel 2 can be eliminated by processing the mixed image corresponding to the 4th and 6th pixel rows of channel 2, and the monochrome image of channel 1 can be obtained. Figure 3 In (C), the 4th and 6th pixel rows of channel 2 are symmetrical about the position parameters of the line spot. Therefore, the signal of channel 2 can be eliminated by processing the mixed image corresponding to the 4th and 6th pixel rows of channel 2, and the monochrome image of channel 1 can be obtained.

[0088] Furthermore, according to formula (2), the blended image corresponding to the 5th pixel row and the 6th pixel row can be represented as:

[0089] I5(x,y,z)=h 5,1 (x,y,z)*f1(x,y,z)+h 5,2 (x,y,z)*f2(x,y,z) (3)

[0090] I6(x,y,z)=h 6,1 (x,y,z)*f1(x,y,z)+h 6,2 (x,y,z)*f2(x,y,z) (4)

[0091] In formulas (3) and (4), the effective PSFh of channel 2 on the 5th and 6th pixel rows is... 5,2 (x,y,z) and h 6,2 The shape of (x, y, z) in 3D space is symmetrical about the center of the line spot in channel 2. Figure 3 (As shown in illumination (A)), the images of channel 2 in the blended images corresponding to the 5th and 6th pixel rows are approximately the same, with only a slight offset. To eliminate channel 2 signals in the blended image as much as possible, the blended image needs to be debiased first.

[0092] Furthermore, such as Figure 7 As shown, the image correction unit 303 includes:

[0093] Convolution kernel acquisition subunit 3031 is used to acquire the debiasing correction convolution kernel between the first pixel row and the second pixel row;

[0094] The convolution kernel correction subunit 3032 is used to perform debias correction on the mixed image of the second pixel row using the debias correction convolution kernel.

[0095] In some embodiments, the debiasing correction convolution kernel between the first pixel row and the second pixel row in the auxiliary channel can be obtained through theoretical calculation.

[0096] Specifically, this can be achieved in the following ways:

[0097] S101, Calculate the effective point spread function of the first pixel row and the effective point spread function of the second pixel row;

[0098] S102, perform Fourier transform on the effective point spread function of the first pixel row and the effective point spread function of the second pixel row to obtain the optical transfer function of the first pixel row and the optical transfer function of the second pixel row.

[0099] S103, after dividing the optical transfer function of the first pixel row by the optical transfer function of the second pixel row, perform an inverse Fourier transform to obtain the debiased correction convolution kernel.

[0100] In some embodiments, the debiased correction convolution kernel can be obtained experimentally.

[0101] Specifically, this can be achieved in the following ways:

[0102] S201, with only the auxiliary channel illumination beam turned on, acquire the monochrome image of the first pixel row and the monochrome image of the second pixel row;

[0103] S202, Perform Fourier transform on the monochrome image of the first pixel row and the monochrome image of the second pixel row to obtain the frequency domain image of the first pixel row and the frequency domain image of the second pixel row.

[0104] S203, after dividing the frequency domain image of the first pixel row by the frequency domain image of the second pixel row, perform an inverse Fourier transform to obtain the debiased correction convolution kernel.

[0105] The following is based on Figure 3 Taking the lighting condition shown in (A) as an example, the demodulation process is explained in detail. Channel 1 is used as the target channel and Channel 2 is used as the auxiliary channel. The mixed image corresponding to the 5th and 6th pixel rows in Channel 2 is selected for demodulation.

[0106] To obtain the debiased convolution kernel for channel 2, the sample is first illuminated only by the laser of channel 2. At this time, the camera acquires a monochrome image containing only the fluorescence signal of channel 2. The monochrome images acquired in the 5th and 6th pixel rows can be represented as follows:

[0107] I 5,2 (x,y,z)=h 5,2 (x,y,z)*f2(x,y,z) (5)

[0108] I 6,2 (x,y,z)=h 6,2 (x,y,z)*f2(x,y,z) (6)

[0109] Performing a Fourier transform on the original spatial domain monochrome image above yields its frequency domain image, which is represented as follows:

[0110] O 5,2 (u,v,w)=H 5,2 (u,v,w)×F2(u,v,w) (7)

[0111] O 6,2 (u,v,w)=H 6,2 (u,v,w)×F2(u,v,w) (8)

[0112] Among them, H 5,2 (u,v,w),H 6,2 (u,v,w) and F2(u,v,w) represent h respectively. 5,2 (x,y,z), h 6,2 Fourier transforms of f(x,y,z) and f2(x,y,z). Dividing the frequency domain images of the 5th and 6th pixel rows and then performing an inverse Fourier transform yields the debiased convolution kernel.

[0113]

[0114] When this debiasing correction convolution kernel is applied to an image, the main difference between the images before and after convolution is the displacement in the x-direction, while the signal intensity at various locations within the image remains largely unchanged. As long as the position of the illumination beam relative to the camera remains constant, this debiasing correction convolution kernel does not change with the sample location.

[0115] After obtaining the bias correction convolution kernel, the bias correction kernel is used to correct the bias in the mixed image of the second pixel row. Specifically, the original spatial domain image of the 6th pixel row is convolved with the bias correction convolution kernel to obtain the corrected spatial domain image of the 6th pixel row.

[0116] I6 * (x,y,z)=I6(x,y,z)*K 56 (x,y,z) (10)

[0117] Finally, by subtracting the debiased image of the 6th row from the original spatial domain image acquired in the 5th row, we obtain an image containing only the fluorescence signal of channel 1, i.e.:

[0118]

[0119] Where g1(x,y,z) represents the demodulated image of channel 1.

[0120] In formula (11), the demodulated monochrome image contains the modulation term h. 5,1(x,y,z) and h 6,1 (x, y, z) can be written as:

[0121]

[0122] From these two formulas, we can conclude that near the focal plane, and There is a misalignment; the values ​​differ significantly at different x-coordinates. At the out-of-focus position, the illumination PSF rapidly decays to uniform illumination, i.e.:

[0123]

[0124] In addition, convolution kernel K 56 (x,y,z) does not change the intensity value of the convolution object. Therefore, at the out-of-focus position, the modulation term in formula (11) is further expressed as:

[0125] h 5,1 (x,y,z)-h 6,1 (x,y,z)*K 56 (x,y,z)≈0, when z is in the out-of-focus position (15)

[0126] This formula shows that in the monochrome image demodulated by formula (11), the defocused background is suppressed, and the image has tomographic capability. The line illumination modulation dual-color tomographic imaging system of this invention improves the axial resolution and tomographic capability of the imaging system while achieving dual-color imaging, making it suitable for dual-color three-dimensional imaging of thick samples.

[0127] The above embodiments provide a method for obtaining a three-dimensional debiasing correction convolution kernel and using the convolution kernel to perform two-color signal demodulation in three-dimensional space. In some embodiments, in order to improve the calculation speed of demodulation, two-color demodulation can also be performed on a two-dimensional plane.

[0128] For a single channel, the illumination PSF corresponding to each pixel row of the camera only shifts in the x-direction. Therefore, the effective PSF mainly changes in the x-direction. This allows for further approximation and simplification of the debiased correction convolution kernel in equation (9), namely:

[0129]

[0130] Among them O 5,2 (u,v) and O 6,2 (u,v) represent I 5,2 (x,y) and I 6,2 The Fourier transform of (x,y), I 5,2 (x,y) and I 6,2(x,y) represents the two-dimensional image of channel 2 captured by the camera at the 5th and 6th pixel rows when the sample is illuminated by a laser with only channel 2 enabled. 5,2 (u,v) and H 6,2 (u,v) represent h respectively. 5,2 (x,y) and h 6,2 The Fourier transform of (x,y), h 5,2 (x,y) and h 6,2 (x, y) represent the two-dimensional effective PSF (focal plane PSF) of channel 2 in the 5th and 6th pixel rows, respectively. Therefore, in single-channel illumination, only a single-layer two-dimensional image needs to be acquired, and then the two-dimensional debiasing correction convolution kernel can be calculated using formula (16). The two-dimensional debiasing correction convolution kernel can be used to demodulate the two-dimensional image of channel 1 in dual-channel illumination.

[0131]

[0132] in I5(x,y) and I6(x,y) represent the two-dimensional blended images acquired by the camera in the 5th and 6th pixel rows, respectively, under dual-channel illumination; h 5,1 (x,y) and h 6,1 (x,y) represent the two-dimensional effective PSF (focal plane PSF) of channel 1 in the 5th and 6th pixel rows, respectively; f1(x,y) represents the relative concentration distribution of the first fluorescent protein in the two-dimensional plane.

[0133] In some embodiments, the process of de-biasing correction of the blended image corresponding to the second pixel row can also be achieved through translation processing, as described above. Figure 8 The image correction unit 303 further includes:

[0134] Translation parameter acquisition subunit 3033 is used to acquire the translation parameters between the first pixel row and the second pixel row;

[0135] Translation correction subunit 3034 is used to perform debiasing correction on the mixed image of the second pixel row using the translation parameters.

[0136] Specifically, the mixed image of the 6th pixel row can be directly shifted, and then the monochrome image of channel 1 can be demodulated by subtracting the 5th pixel row from the 6th pixel row.

[0137] g1(x,y)=I5(x,y)-I6(x+d1,y) (18)

[0138] Where d1 represents the offset distance between the fluorescence signals corresponding to channel 2 in the mixed image corresponding to the 5th and 6th pixel rows.

[0139] The above embodiments describe the monochrome image demodulation process for channel 1. The image demodulation process for channel 2 is similar to the steps described above. (Refer to...) Figure 3 (A) shows the illumination condition. First, a debiasing convolution kernel is obtained between the mixed images of the 4th and 5th pixel rows under channel 1 monochromatic illumination. Then, this debiasing convolution kernel is used to eliminate the offset between the fluorescence signals corresponding to channel 1 in the mixed images of the 4th and 5th pixel rows under dual-channel illumination. Since the line illumination intensity of channel 2 is greater in the 5th pixel row than in the 4th pixel row, the mixed image of the 4th pixel row is finally subtracted from the debiased mixed image of the 5th pixel row to demodulate the monochromatic image of channel 2.

[0140] By achieving synchronous dual-color tomography on a single multi-element detector, the images between each channel have a natural registration relationship, eliminating the need for additional registration processing. The system can obtain a monochrome image containing the fluorescence signal of a single channel simply by processing the mixed image through a demodulation algorithm, greatly reducing the complexity of the system.

[0141] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A line illumination modulation dichromatic tomography imaging system, characterized by, The line illumination modulation dual-color tomography system comprises: A line illumination modulation module, comprising two monochromatic light sources and a modulation light path, the light beams emitted by the two monochromatic light sources are different in wavelength; each of the light beams passes through the modulation light path to form a line spot focused on the focal plane of an objective lens, the illumination intensity of the line spot on the focal plane of the objective lens is in Gaussian distribution in a first direction, the first direction is perpendicular to the extension direction of the line spot, and the position parameter of the line spot is located on the intersection of central pixel rows or adjacent pixel rows; the distance between the position parameters of the two line spots is an integer multiple of 0.5 pixel width in the object space, and the line spots of the two channels are superimposed to form dual-color line illumination light; An imaging module, configured to continuously scan imaging along the first direction by using a multi-element detector with n rows of pixels, to obtain n mixed images under the illumination of the dual-color line illumination light, each of the mixed images corresponds to a row of pixels, and each of the mixed images contains signals of two channels, wherein n is greater than or equal to 3 and is a positive integer; An image demodulation module, configured to perform subtraction processing on the mixed images of two symmetric pixel rows to demodulate monochromatic images from the mixed images, the two symmetric pixel rows are symmetric about the position parameter of the line spot of one channel, and the monochromatic images correspond to the signals of the other channel; The image demodulation module comprises: A channel selection unit, configured to select one channel as a target channel and the other channel as an auxiliary channel; A pixel row determination unit, configured to determine a first pixel row and a second pixel row that are symmetric about the position parameter of the auxiliary channel; An image correction unit, configured to perform de-skew correction on the mixed image of the second pixel row; An image demodulation unit, configured to obtain the monochromatic image of the target channel according to the mixed image of the first pixel row and the de-skew corrected mixed image of the second pixel row.

2. The line illumination modulation contrast imaging system of claim 1, wherein, The image correction unit comprises: A convolution kernel acquisition subunit, configured to acquire a de-skew convolution kernel between the first pixel row and the second pixel row; A convolution kernel correction subunit, configured to perform de-skew correction on the mixed image of the second pixel row by using the de-skew convolution kernel.

3. The line illumination modulation contrast imaging system of claim 2, wherein, The convolution kernel acquisition subunit is configured to: Acquire the monochromatic image of the first pixel row and the monochromatic image of the second pixel row in the case that only the illumination light beam of the auxiliary channel is turned on; Perform Fourier transform on the monochromatic image of the first pixel row and the monochromatic image of the second pixel row to obtain a frequency domain image of the first pixel row and a frequency domain image of the second pixel row; Perform inverse Fourier transform on the frequency domain image of the first pixel row and the frequency domain image of the second pixel row after division to obtain the de-skew convolution kernel.

4. The line illumination modulation contrast imaging system of claim 2, wherein, The convolution kernel acquisition subunit is configured to: Calculate an effective point spread function of the first pixel row and an effective point spread function of the second pixel row; Perform Fourier transform on the effective point spread function of the first pixel row and the effective point spread function of the second pixel row to obtain an optical transfer function of the first pixel row and an optical transfer function of the second pixel row; The optical transfer function of the first pixel row is divided by the optical transfer function of the second pixel row, and inverse Fourier transform is performed to obtain the de-skew correction kernel.

5. The line illumination modulation contrast imaging system of claim 1, wherein, The image correction unit comprises: A translation parameter acquisition subunit is configured to acquire a translation parameter between the first pixel row and the second pixel row. A translation correction subunit is configured to perform de-skew correction on the mixed image of the second pixel row by using the translation parameter.

6. The line illumination modulation contrast imaging system according to any one of claims 1 to 5, wherein, The modulation light path comprises a shaping light path for shaping a light beam into a line light beam, a position adjustment light path for adjusting a position parameter of each line light spot, and a projection light path for superimposing the line light spots to form a dual-color line illumination light.

7. The line illumination modulation contrast imaging system of claim 6, wherein, The position adjustment light path comprises a second dichroic mirror, a third dichroic mirror, a first reflecting mirror, a second reflecting mirror, a third reflecting mirror and a fourth reflecting mirror.

8. The line illumination modulation contrast imaging system according to any one of claims 1 to 5, wherein, The imaging module comprises: A scanning unit is configured to continuously scan imaging along a first direction by using a multi-element detector having n rows of pixels, where n≥3; An image block acquisition unit is configured to acquire a strip image block of an i-th pixel row in each frame of image of one sample obtained in time sequence; A splicing unit is configured to sequentially splice the strip image blocks of the i-th pixel row in each frame of image of one sample to obtain a mixed image of the i-th pixel row, where i∈n.

9. The line illumination modulation contrast imaging system according to any one of claims 1 to 5, wherein, The line illumination modulation dual-color tomography imaging system further comprises a driving module configured to drive the line illumination modulation module and the sample to move relative to each other in three directions perpendicular to each other.

Images