A Linear Illumination Modulation Multicolor Tomography System

Abstract

The present invention discloses a line illumination modulation multi-color tomography system, belonging to the technical field of multi-color tomography, comprising: a line illumination modulation module, comprising m monochromatic light sources and a modulation optical path, wherein each light beam is modulated to form an illumination line spot, and the illumination line spots of the m channels are superimposed to form an m-color line illumination light; a detection module, which divides the signals of the q channels excited by the m-color line illumination light into p+1 groups, wherein when q is an even number, the p+1 groups are all dual-channel signal groups, and when q is an odd number, the p groups are dual-channel signal groups, and the remaining group is a single-channel signal group, wherein the distance between the position parameters of the illumination line spots of the two channels corresponding to each dual-channel signal group is an integer multiple of 0.5 pixels; a scanning imaging module, which uses a multi-element detector for continuous scanning imaging to obtain n images, each image corresponding to a row of pixels; and an image demodulation module, which performs subtraction processing on a mixed image of two symmetrical pixel rows of the same dual-channel signal group and demodulates a monochromatic image from the mixed image. The present invention realizes synchronous multi-color tomography.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-color tomography, and in particular to a linear illumination modulation multi-color tomography system. Background Art

[0002] Using fluorophores of different colors to selectively label different structures in biological tissues and using multi-color microscopy technology to simultaneously acquire multi-color signals can better analyze the spatial relationships and interactions between cells, organelles, and molecules in biological tissues.

[0003] Current multicolor tomography methods typically use dichroic mirrors to spatially separate the signals of each channel and transmit them to different black-and-white cameras for detection. However, this method has some problems: images obtained from commonly used wide-field imaging microscopes have strong out-of-focus background interference signals. For fluorophores with overlapping emission spectra, the dichroic mirror cannot completely separate the signals of different channels, so there will be crosstalk between channels; when multiple black-and-white cameras are used for detection, the images of each channel are offset in the lateral direction and there are differences in the detection focal plane in the axial direction, so complex alignment processing of the original images is required; each channel requires a separate black-and-white camera, which is limited by the size and complexity of the system, and the number of channels for multicolor tomography is limited; the production cost of the system increases with the number of black-and-white cameras. Summary of the Invention

[0004] To address technical issues such as the limited number of channels, complex system structure, high system cost, and strong out-of-focus background signals in the acquired multi-color images of existing multi-color imaging methods, an embodiment of the present invention provides a linear illumination modulation multi-color tomography system. The technical solution is as follows:

[0005] The linear illumination modulation multi-color tomography system comprises:

[0006] A line illumination modulation module includes m monochromatic light sources and a modulation optical path, wherein the wavelength of the light beam emitted by each monochromatic light source is different; each light beam passes through the modulation optical path to form an illumination line spot focused on the focal plane of the objective lens, wherein the illumination intensity of the illumination line spot on the focal plane of the objective lens is Gaussian distributed in a first direction, wherein the first direction is perpendicular to the extension direction of the illumination line spot; the illumination line spots of the m light beams are superimposed to form m-color line illumination light, wherein m is ≥ 3 and is a positive integer; the m-color line illumination light excites q channels of signals, each channel signal has a different wavelength, and wherein q is ≥ m;

[0007] a detection module, configured to divide the signals of the q channels into p+1 groups, where p=floor(q / 2)-1, and the signals of the p+1 groups are respectively received by p+1 multi-element detectors, wherein the position parameters of the illumination line spot are located at the center of a pixel row of the multi-element detector that detects the excited signal or on the intersection of adjacent pixel rows; when q is an odd number, the p groups are dual-channel signal groups, and the remaining group is a single-channel signal group; when q is an even number, the p+1 groups are all dual-channel signal groups, wherein the single-channel signal group contains only a signal of one channel, and the dual-channel signal group contains signals of two channels, wherein the signals of the two channels are excited by illumination line spots of different wavelengths, and the distance between the position parameters of the illumination line spots is a positive integer multiple of 0.5 pixels;

[0008] a scanning imaging module, configured to continuously scan and image along the first direction using a multi-element detector having n rows of pixels, wherein the multi-element detector obtains n images, each image corresponding to a row of pixels, where n is ≥ 3 and is a positive integer, wherein the multi-element detector corresponding to the dual-channel signal group obtains a mixed image, and the multi-element detector corresponding to the single-channel signal group obtains a monochrome image;

[0009] An image demodulation module is used to perform subtraction processing on a mixed image of two symmetrical pixel rows of the same dual-channel signal group, and demodulate a monochrome image from the mixed image, wherein the two symmetrical pixel rows are symmetrical with respect to position parameters of the illumination line spot of one channel of the dual-channel signal group.

[0010] Furthermore, the detection module includes a detection imaging optical path, which includes p dichroic mirrors arranged in sequence along the signal transmission direction. The signals of q channels are separated into signals of two channels each time they pass through a dichroic mirror, and the signals separated into two channels are received by corresponding multi-channel detectors.

[0011] Furthermore, the modulation optical path includes a shaping optical path for shaping the light beam into a linear beam, a position adjustment optical path for adjusting the position parameters of each illumination line spot, and a projection optical path for superimposing the illumination line spots to form an m-color line illumination light.

[0012] Furthermore, the shaping light path includes a first dichroic mirror group for combining m light sources, and a first beam expander, a second beam expander and a cylindrical lens arranged in sequence along the transmission direction of the combined light beam. The first dichroic mirror group includes m-1 dichroic mirrors, and the m-1 dichroic mirrors are arranged along the transmission direction of the first light source. Each of the remaining light sources is merged into the first light source through one dichroic mirror to form a combined light beam.

[0013] Furthermore, the image demodulation module includes:

[0014] A channel selection unit is used to select one channel as the target channel and the other channel as the auxiliary channel;

[0015] A pixel row determining unit, configured to determine a first pixel row and a second pixel row that are symmetrical about a position parameter of the auxiliary channel;

[0016] an image correction unit, configured to perform de-skew correction on the mixed image of the second pixel row;

[0017] An image demodulation unit is used to obtain a monochrome image of the target channel based on the mixed image of the first pixel row and the mixed image of the second pixel row after de-skew correction.

[0018] Furthermore, the image correction unit includes:

[0019] a convolution kernel acquisition subunit, configured to acquire a de-skew correction convolution kernel between the first pixel row and the second pixel row;

[0020] A convolution kernel correction subunit is used to perform deflection correction on the mixed image of the second pixel row using the deflection correction convolution kernel.

[0021] Furthermore, the convolution kernel acquisition subunit is used to:

[0022] Acquire a monochrome image of the first pixel row and a monochrome image of the second pixel row when only the auxiliary channel illumination beam is turned on;

[0023] Performing Fourier transform on the monochrome image of the first pixel row and the monochrome image of the second pixel row to obtain a frequency domain image of the first pixel row and a frequency domain image of the second pixel row;

[0024] The frequency domain image of the first pixel row and the frequency domain image of the second pixel row are divided and then subjected to inverse Fourier transform to obtain the debiasing correction convolution kernel.

[0025] Furthermore, the convolution kernel acquisition subunit is used to:

[0026] calculating an effective point spread function of the first pixel row and an effective point spread function of the second pixel row;

[0027] Performing Fourier transformation on the effective point spread function of the first pixel row and the effective point spread function of the second pixel row to obtain an optical transfer function of the first pixel row and an optical transfer function of the second pixel row;

[0028] The optical transfer function of the first pixel row is divided by the optical transfer function of the second pixel row, and then an inverse Fourier transform is performed to obtain the deflection correction convolution kernel.

[0029] Furthermore, the image correction unit includes:

[0030] a translation parameter acquisition subunit, configured to acquire a translation parameter between the first pixel row and the second pixel row;

[0031] The translation correction subunit is used to perform deflection correction on the mixed image of the second pixel row using the translation parameter.

[0032] Furthermore, the linear illumination modulation multi-color tomography system further includes a driving module for driving the sample to move relative to each other in three directions perpendicular to each other.

[0033] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0034] The present invention provides a line illumination modulated multi-color tomography imaging system, which separates the detection light into multiple groups of signals at the detection end of the imaging system, each group of signals contains signals of two channels, and each group of signals is detected by a multi-element detector. For the two-color mixed image detected by each multi-element detector, demodulation is performed using subtraction processing to obtain a monochrome image corresponding to each channel. Compared with the traditional multi-color imaging method that only uses a dichroic mirror to separate the detection light and each monochrome channel corresponds to a multi-element detector, the present invention realizes multi-channel multi-color imaging, reduces the number of multi-element detectors, and reduces the cost and complexity of the system. At the same time, for the two-channel signals in each group of signals, multiple pixel rows are used for detection and imaging, and the images between each channel are naturally aligned, without the need for additional alignment processing. In addition, the unique demodulation algorithm enhances the multi-color tomography imaging capability of the system. In fluorescence imaging, the present line illumination modulated multi-color tomography imaging system uses illumination light of different wavelengths to differentially excite signals of different colors, avoiding the problem of emission spectrum crosstalk. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a functional module diagram of the linear illumination modulation multi-color tomography system of the present invention;

[0037] Figure 2 This is a schematic diagram of the optical path structure of the linear illumination modulation multi-color tomography of the present invention;

[0038] Figure 3 This is a schematic diagram of the online illumination light distribution of each multi-element detector (when detecting dual-channel signals) in the present invention;

[0039] Figure 4 is a functional module diagram of the scanning imaging module of the present invention;

[0040] Figure 5 is a schematic diagram of the sample imaging acquisition process of the present invention;

[0041] Figure 6 is a schematic diagram of three-dimensional imaging of the present invention;

[0042] Figure 7 It is a functional module diagram of an image correction unit of the present invention;

[0043] Figure 8 It is a functional module diagram of another image correction unit of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a line illumination modulation multi-color tomography system, which includes a line illumination modulation module 10 , a detection module 20 , a scanning imaging module 30 and an image demodulation module 40 .

[0046] The line illumination modulation module 10 includes m (m ≥ 3, where m is a positive integer) monochromatic light sources and a modulation optical path. Each monochromatic light source emits a light beam of different wavelength. Each light beam passes through the modulation optical path to form an illumination line spot focused on the focal plane of the objective lens. The illumination intensity of the illumination line spot on the focal plane of the objective lens has a Gaussian distribution in a first direction perpendicular to the extension direction of the illumination line spot. The illumination line spots of the m light beams are superimposed to form an m-color line illumination light. The m-color line illumination light excites q channels of signals, each with a different wavelength, where q ≥ m.

[0047] The detection module 20 is used to divide the signals of q channels into p+1 groups, where p = floor(q / 2)-1 (floor(·) indicates rounding down). The p+1 groups of signals are respectively received by p+1 multi-element detectors, and the position parameters of the illumination line spot are located at the center of the pixel row of the multi-element detector detecting the signal it excites or at the intersection of adjacent pixel rows. When q is an odd number, the p groups are dual-channel signal groups, and the remaining group is a single-channel signal group, which only contains the signal of one channel. When q is an even number, the p+1 groups are all dual-channel signal groups, which contain signals of two channels. The signals of the two channels are excited by illumination line spots of different wavelengths, and the distance between the position parameters of the illumination line spots is a positive integer multiple of 0.5 pixels.

[0048] The scanning imaging module 30 is configured to continuously scan and image along a first direction using a multi-element detector having n (n ≥ 3) rows of pixels. Each multi-element detector can obtain n images, each corresponding to a row of pixels. The multi-element detector corresponding to the dual-channel signal group obtains a mixed image, while the multi-element detector corresponding to the single-channel signal group obtains a monochrome image.

[0049] The image demodulation module 40 is used to perform subtraction processing on a mixed image of two symmetrical pixel rows of the same dual-channel signal group, and demodulate a monochrome image from the mixed image, wherein the two symmetrical pixel rows are symmetrical about the position parameters of the illumination line spot of one channel of the dual-channel signal group.

[0050] The linear illumination modulated multi-color tomography system provided by an embodiment of the present invention separates the detection light into multiple groups of signals at the detection end of the imaging system, each group of signals contains signals of two channels, and each group of signals is detected by a multi-element detector. For the two-color mixed image detected by each multi-element detector, demodulation is performed using subtraction processing to obtain a monochrome image corresponding to each channel. Compared with the traditional multi-color imaging method that only uses a dichroic mirror to separate the detection light and each monochrome channel corresponds to a multi-element detector, the present invention realizes multi-channel multi-color imaging, reduces the number of multi-element detectors, and reduces the cost and complexity of the system. At the same time, for the two-channel signals in each group, multiple pixel rows are used for detection and imaging, and the images between each channel are naturally aligned, without the need for additional alignment processing. In addition, the unique demodulation algorithm enhances the multi-color tomography capability of the system. In fluorescence imaging, this linear illumination modulated multi-color tomography system uses illumination light of different wavelengths to differentially excite signals of different colors, avoiding the problem of emission spectrum crosstalk.

[0051] It should be noted that the monochromatic light source in this linear illumination modulation multicolor tomography system refers to a light source with a fixed wavelength. This can be a monochromatic laser light source, a monochromatic LED (light-emitting diode) light source, or a monochromatic light source obtained by combining a broadband light source with a narrowband filter. Furthermore, this linear illumination modulation multicolor tomography system can be applied to a variety of scenarios, including fluorescence microscopy and non-fluorescence microscopy (i.e., where the illumination wavelength is the same as the detection wavelength), without limitation in this application.

[0052] Furthermore, since m-color line illumination can excite signals of q different colors (wavelengths), where q ≥ m, this system is applicable to situations where each wavelength of the illumination line spot can excite signals of one or more colors (wavelengths). For example, in fluorescence imaging, when each of the m wavelengths of the illumination line spot excites only one color of fluorescence signal on the sample, or in non-fluorescence imaging (where the illumination wavelength and the detection wavelength are the same), when q = m, this system can be used. For another example, in fluorescence imaging, when one or more of the m wavelengths of the illumination line spot can simultaneously excite multiple colors of fluorescence signals, and each fluorescence signal is excited by only one wavelength of the illumination line spot (where q > m), this method is also applicable. In the latter case, the multiple fluorescence signals excited by the same wavelength illumination line spot in the system are detected by different multi-element detectors, i.e., the two fluorescence signals detected on each multi-element detector are excited by illumination line spots of different wavelengths.

[0053] Furthermore, the modulation optical path includes a shaping optical path for shaping the light beam into a linear beam, a position adjustment optical path for adjusting the position parameters of each illumination line spot, and a projection optical path for superimposing the illumination line spots to form an m-color line illumination light.

[0054] Reference Figure 2, which is the optical path structure of the line illumination modulated multi-color tomography in fluorescence microscopy imaging of the present application. The shaping optical path includes a first dichroic mirror group 3.1, 3.2, ..., 3.(m-1) for combining m laser light sources, and a first beam expander 4, a second beam expander 5 and a cylindrical lens 12 arranged in sequence along the direction of light beam transmission after combining. The position adjustment optical path is located between the second beam expander 5 and the cylindrical lens 12, and includes a second dichroic mirror group 6.1, 6.2, ..., 6.(m-1), a third dichroic mirror group 7.1, 7.2, ..., 7.(m-1), a first reflector 8, a second reflector 9, a third reflector 10 and a fourth reflector 11. Each time a light beam passes through a dichroic mirror in the second dichroic mirror group, a light path is separated. The separated light paths are combined by the corresponding dichroic mirror in the third dichroic mirror group. The remaining light beam after passing through the second dichroic mirror group enters the third dichroic mirror group via the first reflector 8 and the second reflector 9 for combination. The combined light can adjust the position parameters of the light spots of each laser beam line in the combined light by passing through the third reflector 10 and the fourth reflector 11. The projection light path includes an illumination tube lens 13, an objective lens 14 and a fourth dichroic mirror 17 (in non-fluorescence imaging, it needs to be replaced with a semi-reflective and semi-transmissive beam splitter or other device that can both reflect and transmit the light irradiated thereon), which is used to superimpose the illumination line spots to form multi-color line illumination light.

[0055] Specifically, the first dichroic mirror group 3.1, 3.2, ..., 3.(m-1) is arranged along the direction of laser beam transmission. The laser beams emitted by the second laser light source 2.1, the third laser light source 2.2, ..., and the m-th laser light source 2.(m-1) are merged into the laser beam emitted by the first laser light source 1 through a dichroic mirror in the first dichroic mirror group to form a combined beam. The combined beams are then expanded by the first beam expander 4 and the second beam expander 5. The second dichroic mirror group 6.1, 6.2, ..., 6.(m-1) is then used to separate the illumination beams of different channels from each other and respectively enter different optical paths. The optical paths separated by the second dichroic mirror 6.1 are then combined by the third dichroic mirror 7.1. The remaining light beam after passing through the second dichroic mirror 6.1 is separated again by the second dichroic mirror 6.2, and the separated light paths are combined after passing through the third dichroic mirrors 7.2 and 7.1. The remaining light beam after passing through the second dichroic mirror 6.2 is separated again by the second dichroic mirror 6.3, and the separated light paths are combined after passing through the third dichroic mirrors 7.3, 7.2, and 7.1. Similarly, the light path separated by the second dichroic mirror 6.(m-1) passes through the third dichroic mirrors 7.(m-1), ..., 7.2, and 7.1, and then combined. The remaining light beam enters the third dichroic mirrors 7.(m-1), ..., 7.2, and 7.1 through the first reflector 8 and the second reflector 9 to be combined. The combined light beam is shaped into a line beam by the cylindrical lens 12. The third reflector 10 and the fourth reflector 11 can adjust the position parameters of the position of the line spots of each laser beam in the combined light beam. The combined light beam is projected onto the focal plane of the objective lens through the illumination tube lens 13, the fourth dichroic mirror 17 and the objective lens 14 to excite q kinds of signals of the sample 15 on the electric translation stage 16.

[0056] The detection module 20 includes a detection imaging optical path, which includes p dichroic mirrors arranged sequentially along the signal transmission direction. Each q-channel signal is separated into two channels by each dichroic mirror. The separated two-channel signals are then received by the corresponding multi-element detector. When q is an odd number, p groups of signals contain signals from two channels, and the remaining group of signals contains signals from one channel. When q is an even number, each of the p+1 groups of signals contains signals from two channels.

[0057] The multi-element detector in this embodiment can be a surface array CCD (Charge-coupled device) or surface array CMOS (Complementary Metal Oxide Semiconductor) camera with sub-array or ROI (Region of Interest) functions, or a linear array CCD or linear array CMOS camera with surface array mode functions.

[0058] Further, refer to Figure 2 The detection and imaging optical path includes an objective lens 14, a fourth dichroic mirror 17, a fifth dichroic mirror group 18.1, 18.2, ..., 18.p, arranged sequentially along the signal transmission direction, a detection tube lens group 19.1, 19.2, ..., 19.p, 19.(p+1), an emission filter group 20.1, 20.2, ..., 20.p, 20.(p+1) (in non-fluorescence imaging, an emission filter group is not required), and a multi-element detector group 21.1, 21.2, ..., 21.p, 21.(p+1). Each signal from each of the m channels is separated into two channels by a fifth dichroic mirror. These signals are then received by the multi-element detector through the corresponding detection tube lens and emission filter for imaging.

[0059] Specifically, after passing through objective lens 14 and fourth dichroic mirror 17, the signals are separated into p+1 groups of signals by fifth dichroic mirrors 18.1, 18.2, ..., and 18.p. Specifically, q signals pass through fifth dichroic mirror 18.1, where they are separated into two channels. These signals then pass through detection tube lens 19.1 and emission filter 20.1, where they are received by multi-element detector 21.1 for imaging. The remaining q-2 signals after passing through fifth dichroic mirror 18.1 pass through fifth dichroic mirror 18.2, where they are further separated into two channels. These signals then pass through detection tube lens 19.2 and emission filter 20.2, where they are received by multi-element detector 21.2 for imaging. Similarly, the q-4 signals after passing through fifth dichroic mirror 18.2 are sequentially separated into two channels by fifth dichroic mirrors 18.3, 18.4, ..., and 18.p, where they then pass through corresponding detection tube lenses and emission filters, where they are received by multi-element detector 21.2 for imaging. The remaining one or two signals after passing through the fifth dichroic mirror 18.p then pass through the detection tube lens 19.(p+1) and the emission filter 20.(p+1) and are received by the multi-element detector 21.(p+1) for imaging.

[0060] To facilitate subsequent demodulation, for a multi-element detector containing two channel signals, the position parameters of each channel illumination line spot must be located at the center of the pixel row of the multi-element detector detecting the channel signal or at the intersection of adjacent pixel rows, and the distance between the position parameters of the corresponding two channel illumination line spots in each multi-element detector must be a positive integer multiple of the corresponding width of 0.5 pixels in the object space.

[0061] Specifically, the angle of either the third reflector 10 or the fourth reflector 11 can be adjusted first, so that the illumination line spot of the wavelength transmitted from the second dichroic mirror 6.1 or the third dichroic mirror 7.1 is moved to a specified position. Then, the angle of either the second dichroic mirror 6.2 or the third dichroic mirror 7.2 can be adjusted to adjust the position parameters of the illumination line spot of the wavelength reflected from the second dichroic mirror 6.2 or the third dichroic mirror 7.2. Then, the angle of either the second dichroic mirror 6.3 or the third dichroic mirror 7.3 can be adjusted to adjust the position parameters of the illumination line spot of the wavelength reflected from the second dichroic mirror 6.3 or the third dichroic mirror 7.3. Similarly, the angles of the remaining dichroic mirrors in the second and third dichroic mirror groups are adjusted in sequence to adjust the position parameters of the illumination line spots of the wavelength reflected from them. Finally, the angle of either the first reflector 8 or the second reflector 9 can be adjusted to adjust the position parameters of the illumination line spots of the wavelength reflected from them.

[0062] In other embodiments, the modulated light path can be configured such that linear LEDs with different emission wavelengths directly generate linear beams of different wavelengths, which are then projected onto the focal plane of the objective lens via the projection light path. By adjusting the positions of the different linear LEDs, the position parameters of the linear spot illuminated on the focal plane of the objective lens can be changed.

[0063] like Figure 3 The figure shows the distribution of line illumination light when detecting dual-channel signals on each multi-element detector. The two channel signals detected on each multi-element detector are named channel 1 and channel 2. The illumination intensity of the line illumination beams of channel 1 and channel 2 is Gaussian distributed in the x-direction. The imaging area of ​​the multi-element detector is 8 pixel rows. The position parameters of the illumination line spot focused on the focal plane of the objective lens after each light beam passes through the modulated optical path are different. Figure 3 As shown in A, the brightest point of the illumination line spot of channel 1 in the x direction is located at the intersection of the 4th pixel row and the 5th pixel row, that is, the illumination line spot position parameter of channel 1 is located at the intersection of the 4th pixel row and the 5th pixel row. Similarly, the illumination line spot position parameter of channel 2 is located at the intersection of the 5th pixel row and the 6th pixel row. Figure 3 As shown in B, the illumination line spot position parameter of channel 1 is located in the center of the 4th pixel row, and the illumination line spot position parameter of channel 2 is located in the center of the 5th pixel row; Figure 3 As shown in C, the illumination line spot position parameter of channel 1 is located at the intersection of the 4th pixel row and the 5th pixel row, and the illumination line spot position parameter of channel 2 is located at the center of the 5th pixel row.

[0064] In addition, when q is an odd number, only one of the p+1 multi-element detectors acquires a single-channel signal, and there are no above requirements for the position parameters of the illumination line spot of this channel, that is, it does not need to be located in the center of the pixel row or on the intersection of adjacent pixel rows.

[0065] Further, refer to Figure 4 , the scanning imaging module 30 includes:

[0066] A scanning unit 301 is configured to continuously scan and image along a first direction using a multi-element detector having n rows of pixels, where n is a positive integer and is greater than or equal to 3;

[0067] An image block acquisition unit 302 acquires a strip image block of the i-th pixel row in each frame of an image of a sample obtained by each of the multi-element detectors in time sequence;

[0068] The splicing unit 303 sequentially splices the strip image blocks of the i-th pixel row in each frame image of a sample acquired by each of the multi-element detectors to obtain an image of the i-th pixel row, i∈n.

[0069] Specifically, scanning unit 301 uses p+1 multi-element detectors to continuously scan and image along a first direction. Image block acquisition unit 302 and stitching unit 303 process the images detected by each multi-element detector separately. The multi-element detectors corresponding to the dual-channel signal group obtain a two-color mixed image, while the multi-element detectors corresponding to the single-channel signal group obtain a monochrome image.

[0070] Figure 5 This is a schematic diagram of the sample imaging acquisition process of the present invention. Figure 5 The functions of the scanning imaging module 30 are described below. To facilitate subsequent image demodulation, the imaging area of ​​each multi-element detector in this embodiment is n pixel rows arranged along the x-direction. The sample's motion direction is also along the x-direction, facilitating separate scanning and imaging of the sample at different pixels, achieving synchronous dual-color imaging on each multi-element detector.

[0071] During the imaging acquisition process, the three-dimensional motorized translation stage 16 moves the sample 15 at a constant speed along the x-direction, and the single-frame exposure time of each multi-element detector is equal to the time it takes for the sample 15 to move one pixel in object space. For any multi-element detector, if the image corresponding to any pixel row in a frame is defined as a strip image block, then the multiple strip image blocks corresponding to that pixel row in multiple frames represent sequential, continuous imaging of various parts of the sample. These strip image blocks, obtained in chronological order, are spliced ​​together to produce a mixed image corresponding to each of the n pixel rows. Each mixed image corresponds to a pixel row, and each mixed image contains signals from two channels, where n ≥ 3 and is a positive integer. Specifically, the i-th pixel row is selected from the n pixel rows, and the strip image blocks for the i-th pixel row in each chronologically obtained frame of the sample are obtained. The strip image blocks for the i-th pixel row in each frame of the sample are then spliced ​​together to produce a mixed image for the i-th pixel row.

[0072] Furthermore, the linear illumination modulation multi-color tomography system of the present invention further includes a driving module 50 for driving the sample to move relative to each other in three directions perpendicular to each other.

[0073] Specifically, refer to Figure 6 , is a schematic diagram of three-dimensional imaging according to the present invention. The sample is configured as four surface layers, each of which is divided into four sample strips. The three-dimensional motorized translation stage 16 drives the sample 15 along the x-direction to image the first sample strip. It then moves along the y-direction by the width of one sample strip, and then along the -x-direction to image the second sample strip. This process is repeated continuously to complete imaging of the third and fourth sample strips. After completing the scanning and imaging of the first surface layer, the three-dimensional motorized translation stage 16 drives the sample 15 along the z-direction for a certain distance, changing the focal depth of the sample from the first surface layer to the second surface layer. Similarly, the three-dimensional motorized translation stage 16 drives the sample 15 along the x-direction again to image the second surface layer. This repetitive cycle completes three-dimensional imaging of the sample.

[0074] As mentioned above, the signals of q channels have been divided into p+1 groups according to different detection bands by the p dichroic mirrors in the detection module 20. When q is an odd number, the p group is a dual-channel signal group in which each signal group contains signals of two channels, and the remaining group is a single-channel signal group in which only one channel is contained. When q is an even number, the p+1 group is a dual-channel signal group in which each signal group contains signals of two channels. Among them, the multi-element detector corresponding to the dual-channel signal group obtains a two-color mixed image, and the multi-element detector corresponding to the single-channel signal group detects a monochrome image. The image detected by the single-channel signal group itself is a monochrome image, which will not be explained in detail here. It is only necessary to demodulate the two-color mixed image detected by the dual-channel signal group and separate it into a monochrome image to obtain m-channel monochrome images. Next, a method for demodulating the two-color mixed image detected by the detector of the dual-channel signal group is introduced.

[0075] The following mainly describes the demodulation principle and process for fluorescence imaging. The demodulation principle and process for non-fluorescence imaging are essentially the same as those for fluorescence imaging. The main difference is the difference in the detection wavelength, which results in a slight difference in the effective PSF of the system. This difference can be ignored and the same method still applies.

[0076] Similar to the imaging principle of line scanning confocal microscopy, the mixed image corresponding to each pixel row in this method is the convolution of the effective point spread function (PSF) of the system and the signal distribution on the sample, where the effective PSF of the system is the product of the PSF of the illumination system and the PSF of the detection system.

[0077] Taking the fluorescence signal of a single channel detected by a single multi-element detector as an example, the effective PSF of the j-th channel in the i-th pixel row is:

[0078]

[0079] in, represents the illumination PSF, represents the detection PSF, i represents the serial number of each pixel row on the imaging area, j represents the type serial number of the fluorescent protein, and also represents the channel serial number, x, y, z represent the coordinates of each direction in the three-dimensional object space, b i,j Represents the distance between the i-th pixel row and the j-th channel illumination line spot position parameter.

[0080] Therefore, the signal of the jth channel contained in the mixed image obtained by the i-th pixel row in the multi-element detector can be expressed as:

[0081]

[0082] Among them, f j (x, y, z) represents the relative concentration distribution of the jth fluorescent protein on the sample, and “*” represents the convolution operation. As can be seen from formula (2), the fluorescence signal of the sample is modulated by both the PSF of the illumination system and the PSF of the detection system.

[0083] Further, refer to Figure 1 , the image demodulation module 40 includes:

[0084] The channel selection unit 401 is used to select one channel as the target channel and the other channel as the auxiliary channel;

[0085] A pixel row determining unit 402 is configured to determine a first pixel row and a second pixel row that are symmetrical about the position parameters of the auxiliary channel;

[0086] An image correction unit 403, configured to perform de-skew correction on the mixed image of the second pixel row;

[0087] The image demodulation unit 404 is configured to obtain a monochrome image of the target channel according to the mixed image of the first pixel row and the mixed image of the second pixel row after de-skew correction.

[0088] For a two-color mixed image detected by a single multi-element detector, first, one channel is selected as the target channel and the other channel is selected as the auxiliary channel through the channel selection unit 401. In formula (1), whether it is the illumination PSF or the detection PSF, its intensity distribution in each z plane is rotationally symmetric, so for any channel, the effective PSF shapes of the two pixel rows that are symmetric about the line spot position parameters are approximately the same (the imaging focus depth and modulation intensity of these two symmetric pixel rows are the same), so the channel images obtained by these two symmetric pixel rows are also approximately the same

[0089] Furthermore, due to the misalignment of the line spots of the two channels, two pixel rows that are symmetrical about the auxiliary channel's line spot position parameters will be modulated to different intensities by the target channel's line spot. Therefore, pixel row determination unit 402 selects two pixel rows in the auxiliary channel that are symmetrical about the line spot position parameters as the first and second pixel rows. Subtraction processing is then performed on the mixed image of the first and second pixel rows to demodulate a monochrome image of the fluorescence signal corresponding to the target channel from the mixed image.

[0090] Specifically, the subtraction processing process includes: performing depolarization correction on the mixed image of the second pixel row through the image correction unit 403, and then obtaining a monochrome image of the target channel based on the mixed image corresponding to the first pixel row and the mixed image of the second pixel row after depolarization correction through the image demodulation unit 404.

[0091] by Figure 3 Taking the illumination condition shown in (A) as an example, the line spot position parameters for channel 1 are located at the intersection of pixel rows 4 and 5, and those for channel 2 are located at the intersection of pixel rows 5 and 6. Because pixel rows 5 and 6 of channel 2 are symmetrical about the line spot center of channel 2, the monochrome image of channel 1 can be obtained by processing the mixed image corresponding to pixel rows 5 and 6 of channel 2 and eliminating the signal from channel 2.

[0092] In some embodiments, the 4th pixel row and the 7th pixel row, and the 3rd pixel row and the 8th pixel row in channel 2 may be selected for mixed image demodulation to obtain a monochrome image of channel 1.

[0093] In some embodiments, the monochrome images of channel 1 demodulated from multiple pairs of pixel rows may be added together to improve the signal-to-noise ratio of the demodulated image.

[0094] Likewise, Figure 3 In (B), the 4th and 6th pixel rows of channel 2 are symmetrical about the line spot position parameters. Therefore, the mixed image corresponding to the 4th and 6th pixel rows of channel 2 can be processed to eliminate the signal of channel 2 and obtain the monochrome image of channel 1. Figure 3 In (C), the 4th and 6th pixel rows of channel 2 are symmetrical about the line spot position parameters. Therefore, the mixed image corresponding to the 4th and 6th pixel rows of channel 2 can be processed to eliminate the signal of channel 2 and obtain the monochrome image of channel 1.

[0095] Furthermore, according to formula (2), the mixed image corresponding to the 5th pixel row and the 6th pixel row can be expressed as:

[0096] I5(x,y,z)=h 5,1 (x,y,z)*f1(x,y,z)+h 5,2 (x,y,z)*f2(x,y,z) (3)

[0097] I6(x,y,z)=h 6,1 (x,y,z)*f1(x,y,z)+h 6,2 (x,y,z)*f2(x,y,z) (4)

[0098] In formula (3) and formula (4), the effective PSFh of channel 2 on the 5th and 6th pixel rows is 5,2 (x,y,z) and h 6,2 The shape of (x, y, z) in 3D space is symmetrical about the center of the line spot of channel 2 ( Figure 3 (A) Therefore, the images of channel 2 in the mixed image corresponding to pixel rows 5 and 6 are approximately identical, with only a slight offset. To minimize the channel 2 signal in the mixed image, debiasing correction is required.

[0099] Furthermore, if Figure 7 As shown, the image correction unit 403 includes:

[0100] A convolution kernel acquisition subunit 4031 is configured to acquire a de-skew correction convolution kernel between the first pixel row and the second pixel row;

[0101] The convolution kernel correction subunit 4032 is used to use the depolarization correction convolution kernel to perform depolarization correction on the mixed image of the second pixel row.

[0102] In some embodiments, the de-skew correction convolution kernel between the first pixel row and the second pixel row in the auxiliary channel can be obtained by theoretical calculation.

[0103] Specifically, this can be achieved by:

[0104] S101, calculating an effective point spread function of a first pixel row and an effective point spread function of a second pixel row;

[0105] S102, performing Fourier transform on the effective point spread function of the first pixel row and the effective point spread function of the second pixel row to obtain the optical transfer function of the first pixel row and the optical transfer function of the second pixel row;

[0106] S103 , dividing the optical transfer function of the first pixel row by the optical transfer function of the second pixel row and performing inverse Fourier transform to obtain a de-skew correction convolution kernel.

[0107] In some embodiments, the de-skew correction convolution kernel can be obtained experimentally.

[0108] Specifically, this can be achieved by:

[0109] S201, acquiring a monochrome image of a first pixel row and a monochrome image of a second pixel row with only the auxiliary channel illumination beam turned on;

[0110] S202, performing Fourier transform on the monochrome image of the first pixel row and the monochrome image of the second pixel row to obtain a frequency domain image of the first pixel row and a frequency domain image of the second pixel row;

[0111] S203 , dividing the frequency domain image of the first pixel row and the frequency domain image of the second pixel row, and then performing inverse Fourier transform to obtain a de-skew correction convolution kernel.

[0112] Below Figure 3 Taking the lighting condition shown in (A) as an example, the demodulation process is described in detail. Channel 1 is used as the target channel and channel 2 as the auxiliary channel. The mixed image corresponding to the 5th pixel row and the 6th pixel row in channel 2 is selected for demodulation.

[0113] In order to obtain the de-skew correction convolution kernel for channel 2, we first turn on the laser of channel 2 to illuminate the sample. At this time, the camera captures a monochrome image containing only the fluorescence signal of channel 2. The monochrome images obtained in the 5th and 6th pixel rows can be expressed as follows:

[0114] I 5,2 (x,y,z)=h 5,2 (x,y,z)*f2(x,y,z) (5)

[0115] I 6,2 (x,y,z)=h 6,2 (x,y,z)*f2(x,y,z) (6)

[0116] Perform Fourier transform on the above original spatial domain monochrome image to obtain its frequency domain image, which can be expressed as:

[0117] O 5,2 (u,v,w)=H 5,2(u,v,w)×F2(u,v,w) (7)

[0118] O 6,2 (u,v,w)=H 6,2 (u,v,w)×F2(u,v,w) (8)

[0119] Among them, H 5,2 (u,v,w),H 6,2 (u,v,w) and F2(u,v,w) represent h 5,2 (x,y,z),h 6,2 The Fourier transform of (x,y,z) and f2(x,y,z). Divide the frequency domain images of the 5th pixel row and the 6th pixel row and perform the inverse Fourier transform to obtain the de-biasing convolution kernel:

[0120]

[0121] When this de-skew correction kernel is applied to an image, the primary shift between the pre- and post-convolution images is in the x-axis, while the signal intensity at each position in the image remains unchanged. This de-skew correction kernel remains constant with sample position, as long as the illumination beam remains in the same position relative to the camera.

[0122] After obtaining the de-skew correction convolution kernel, the de-skew correction convolution kernel is used to de-skew the mixed image of the second pixel row. Specifically, the original spatial domain image collected by the 6th pixel row is convolved with the de-skew correction convolution kernel to obtain the offset-corrected spatial domain image of the 6th pixel row:

[0123]

[0124] Finally, the image containing only the fluorescence signal of channel 1 is obtained by subtracting the de-skewed image of the 6th pixel row from the original spatial domain image collected in the 5th pixel row, that is:

[0125]

[0126] Among them, g1(x,y,z) represents the demodulated image of channel 1.

[0127] In formula (11), the demodulated monochrome image contains the modulation term h 5,1 (x,y,z) and h 6,1 (x,y,z), can be written as:

[0128]

[0129]

[0130] From these two formulas, we can conclude that near the focal plane, and There is a misalignment, and at different x-coordinates, there are obvious differences between their values. At the out-of-focus position, the illumination PSF quickly decays to uniform illumination, that is:

[0131] When z is in the defocus position (14)

[0132] In addition, the convolution kernel K 56 (x, y, z) does not change the intensity value of the convolved object. So at the out-of-focus position, the modulation term in formula (11) is further expressed as:

[0133] h 5,1 (x,y,z)-h 6,1 (x,y,z)*K 56 (x,y,z)≈0, when z is out of focus (15)

[0134] This formula shows that in the monochromatic image demodulated by formula (11), the out-of-focus background is suppressed, and the image has tomographic capabilities. The linear illumination modulation two-color tomographic imaging system of the present invention not only achieves two-color imaging, but also improves the axial resolution and tomographic capabilities of the imaging system, making it suitable for two-color three-dimensional imaging of thick samples.

[0135] The above embodiment provides a method for obtaining a three-dimensional de-skew correction convolution kernel and using the convolution kernel to perform two-color signal demodulation in three-dimensional space. In some embodiments, in order to improve the calculation speed of demodulation, two-color demodulation can also be performed on a two-dimensional plane.

[0136] For a single channel, the illumination PSF corresponding to each pixel row of the camera only shifts in the x direction, so the effective PSF changes mainly in the x direction. The debiasing correction convolution kernel in formula (9) can be further approximated and simplified, namely:

[0137]

[0138] Among them O 5,2 (u,v) and O 6,2 (u,v) are I 5,2 (x,y) and I 6,2 The Fourier transform of (x,y), I 5,2 (x,y) and I 6,2 (x, y) is the two-dimensional image of channel 2 obtained by the 5th and 6th pixel rows of the camera when only the laser of channel 2 is turned on to illuminate the sample. 5,2 (u,v) and H 6,2 (u,v) are h 5,2 (x,y) and h6,2 The Fourier transform of (x,y), h 5,2 (x,y) and h 6,2 (x, y) represents the two-dimensional effective PSF (PSF of the focal plane) of channel 2 at the 5th and 6th pixel rows, respectively. Therefore, when single-channel illumination is used, it is only necessary to obtain a single-layer two-dimensional image, and then use formula (16) to calculate the two-dimensional depolarization correction convolution kernel. The two-dimensional depolarization correction convolution kernel can be used to demodulate the two-dimensional image of channel 1 in the two-color mixed image.

[0139]

[0140] in I5(x,y) and I6(x,y) represent the two-dimensional two-color mixed images obtained by the 5th and 6th pixel rows of the camera respectively; h 5,1 (x,y) and h 6,1 (x, y) represents the two-dimensional effective PSF (PSF of the focal plane) of channel 1 in the 5th and 6th pixel rows, respectively; f1(x, y) represents the relative concentration distribution of the first fluorescent protein on the two-dimensional plane.

[0141] In some embodiments, the process of performing de-skew correction on the mixed image corresponding to the second pixel row can also be achieved by translation processing, referring to Figure 8 , the image correction unit 403 further includes:

[0142] A translation parameter acquisition subunit 4033, configured to acquire a translation parameter between the first pixel row and the second pixel row;

[0143] The translation correction subunit 4034 is configured to perform de-skew correction on the mixed image of the second pixel row using the translation parameter.

[0144] Specifically, the mixed image of the 6th pixel row can be directly translated, and then the monochrome image of channel 1 can be demodulated by subtracting the 5th pixel row from the 6th pixel row, that is:

[0145] g1(x,y)=I5(x,y)-I6(x+d1,y) (18)

[0146] Wherein, d1 represents the offset distance between the fluorescence signals corresponding to channel 2 in the mixed image corresponding to the 5th pixel row and the 6th pixel row.

[0147] The above embodiment describes the monochrome image demodulation process of channel 1, and the image demodulation process of channel 2 is similar to the above steps. Figure 3For the illumination conditions shown in (A), a depolarization correction convolution kernel is first obtained for the mixed image of pixel rows 4 and 5 under monochromatic illumination of channel 1. This depolarization correction convolution kernel is then used to eliminate the offset between the fluorescence signals corresponding to channel 1 in the mixed image of pixel rows 4 and 5 under dual-channel illumination. Because the intensity of the line illumination light of channel 2 is greater in pixel row 5 than in pixel row 4, the mixed image of pixel row 4 is subtracted from the depolarization-corrected mixed image of pixel row 5 to demodulate the monochrome image of channel 2.

[0148] This method realizes synchronous two-color tomography on a single multi-element detector and, by combining a dichroic mirror and multiple multi-element detectors, further enhances the multi-color imaging capability of the system and reduces system complexity and cost.

[0149] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A linear illumination modulation multi-color tomography system, characterized in that: The linear illumination modulation multi-color tomography system comprises: A line illumination modulation module includes m monochromatic light sources and a modulation optical path, wherein the wavelength of the light beam emitted by each monochromatic light source is different; each light beam passes through the modulation optical path to form an illumination line spot focused on the focal plane of the objective lens; the illumination intensity of the illumination line spot on the focal plane of the objective lens is Gaussian distributed in a first direction, wherein the first direction is perpendicular to the extension direction of the illumination line spot; the illumination line spots of the m light beams are superimposed to form m-color line illumination light, where m is ≥ 3 and is a positive integer; the m-color line illumination light excites q channels of signals, each channel signal has a different wavelength, and q is ≥ m; A detection module is used to divide the signals of the q channels into p+1 groups, where p= floor (q / 2)-1, the p+1 groups of signals are received by p+1 multi-element detectors respectively, and the position parameters of the illumination line spot are located at the center of the pixel row of the multi-element detector detecting the excited signal or on the intersection of adjacent pixel rows; when q is an odd number, the p group is a dual-channel signal group, and the remaining group is a single-channel signal group; when q is an even number, the p+1 groups are all dual-channel signal groups, the single-channel signal group contains only one-channel signal, and the dual-channel signal group contains two-channel signals, the two-channel signals are excited by illumination line spots of different wavelengths, and the distance between the position parameters of the illumination line spots is a positive integer multiple of 0.5 pixels; a scanning imaging module, configured to continuously scan and image along the first direction using a multi-element detector having n rows of pixels, wherein the multi-element detector obtains n images, each image corresponding to a row of pixels, where n is ≥ 3 and is a positive integer, wherein the multi-element detector corresponding to the dual-channel signal group obtains a mixed image, and the multi-element detector corresponding to the single-channel signal group obtains a monochrome image; an image demodulation module, configured to perform subtraction processing on a mixed image of two symmetrical pixel rows of the same dual-channel signal group, and demodulate a monochrome image from the mixed image, wherein the two symmetrical pixel rows are symmetrical with respect to position parameters of the illumination line spot of one channel of the dual-channel signal group; The image demodulation module includes: A channel selection unit is used to select one channel as the target channel and the other channel as the auxiliary channel; A pixel row determining unit, configured to determine a first pixel row and a second pixel row that are symmetrical about a position parameter of the auxiliary channel; an image correction unit, configured to perform de-skew correction on the mixed image of the second pixel row; An image demodulation unit is used to obtain a monochrome image of the target channel based on the mixed image of the first pixel row and the mixed image of the second pixel row after de-skew correction.

2. The linear illumination modulation multi-color tomography system according to claim 1, characterized in that: The detection module includes a detection imaging optical path, which includes p dichroic mirrors arranged in sequence along the signal transmission direction. The signals of q channels are separated into two channels each time they pass through a dichroic mirror. The signals separated into two channels are received by corresponding multi-element detectors.

3. The linear illumination modulation multi-color tomography system according to claim 1, characterized in that: The modulation optical path includes a shaping optical path for shaping a light beam into a linear beam, a position adjustment optical path for adjusting the position parameters of each illumination line spot, and a projection optical path for superimposing the illumination line spots to form m-color line illumination light.

4. The linear illumination modulation multi-color tomography system according to claim 3, characterized in that: The shaping light path includes a first dichroic mirror group for combining m light sources, and a first beam expander, a second beam expander, and a cylindrical lens arranged in sequence along the transmission direction of the combined light beam. The first dichroic mirror group includes m-1 dichroic mirrors, and the m-1 dichroic mirrors are arranged along the transmission direction of the first light source. Each of the remaining light sources is merged into the first light source through one of the dichroic mirrors to form a combined light beam.

5. The linear illumination modulation multi-color tomography system according to claim 1, characterized in that: The image correction unit includes: a convolution kernel acquisition subunit, configured to acquire a de-skew correction convolution kernel between the first pixel row and the second pixel row; A convolution kernel correction subunit is used to perform deflection correction on the mixed image of the second pixel row using the deflection correction convolution kernel.

6. The linear illumination modulation multi-color tomography system according to claim 5, characterized in that: The convolution kernel acquisition subunit is used to: Acquire a monochrome image of the first pixel row and a monochrome image of the second pixel row when only the auxiliary channel illumination beam is turned on; Performing Fourier transform on the monochrome image of the first pixel row and the monochrome image of the second pixel row to obtain a frequency domain image of the first pixel row and a frequency domain image of the second pixel row; The frequency domain image of the first pixel row and the frequency domain image of the second pixel row are divided and then subjected to inverse Fourier transform to obtain the debiasing correction convolution kernel.

7. The linear illumination modulation multi-color tomography system according to claim 5, characterized in that: The convolution kernel acquisition subunit is used to: calculating an effective point spread function of the first pixel row and an effective point spread function of the second pixel row; Performing Fourier transformation on the effective point spread function of the first pixel row and the effective point spread function of the second pixel row to obtain an optical transfer function of the first pixel row and an optical transfer function of the second pixel row; The optical transfer function of the first pixel row is divided by the optical transfer function of the second pixel row, and then an inverse Fourier transform is performed to obtain the deflection correction convolution kernel.

8. The linear illumination modulation multi-color tomography system according to claim 1, characterized in that: The image correction unit includes: a translation parameter acquisition subunit, configured to acquire a translation parameter between the first pixel row and the second pixel row; The translation correction subunit is used to perform deflection correction on the mixed image of the second pixel row using the translation parameter.

9. The linear illumination modulation multi-color tomography system according to any one of claims 1 to 4, characterized in that: The linear illumination modulation multi-color tomography system further includes a driving module for driving the sample to move relative to each other in three directions perpendicular to each other.

Images