A Dual-channel Joint Reconstruction Method Based on Structured Illumination Microscopy

Abstract

The present invention belongs to the technical field of image processing, and specifically relates to a dual-channel joint reconstruction method based on a structured illumination microscope, including: obtaining dual-channel fluorescence images of organelles according to the structured illumination microscope; constructing a structure-function synchronous imaging model based on the dual-channel fluorescence images, where is the organelle structure information and is the organelle function information; adding a regularization term constraint of the organelle function information to the structure-function synchronous imaging model to obtain an objective function; iteratively optimizing the objective function to obtain the organelle structure information and function information; the present invention combines structure imaging and function imaging into the same objective function for modeling, realizes the synchronous reconstruction of structure information and function information, can better suppress reconstruction artifacts, and improve the fidelity; the present invention introduces continuous prior information into the reconstruction process, can better suppress the artifacts in the reconstructed super-resolution function information, and solve the reconstruction problem of low signal-to-noise ratio images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a dual-channel joint reconstruction method based on structured illumination microscopy. Background Art

[0002] In the field of fluorescence image super-resolution imaging, structured illumination microscopy is a powerful tool for observing the structural functions of organelles. However, the microscope cannot synchronously detect the functional information during live cell imaging. Therefore, obtaining functional information currently requires two steps: The first step is to perform structural imaging to acquire the dual-channel fluorescence images of the organelles, that is, the structural information. The second step is to perform functional imaging, and by means of numerical calculation, calculate the pixel-level intensity ratio of the dual-channel images, and finally map the intensity ratio to functional information such as pixel-level PH distribution and FRET efficiency values.

[0003] In terms of PH functional imaging, the genetically encoded single-protein red fluorescence sensor pHRed designed by Tantama et al. can map the intensity ratio of two structural images in different wavelength ranges to the pH distribution, and can be used to monitor the changes in intracellular acidity. In terms of FRET functional imaging, Huang Jin et al. provided a ratio-type fluorescent encoded microsphere based on FRET distance regulation and its preparation method. These technologies or inventions all belong to the framework of structural-functional step-by-step imaging. However, directly applying the above framework during SIM super-resolution imaging has two obvious disadvantages: One is that the calculated functional information is severely distorted. The functional information is calculated from the ratio of the structural information in the second step, so it will be interfered by the artifacts in the structural information reconstructed in the first step. Especially when SIM microscopy reconstructs the super-resolution structural information in the first step, various reconstruction artifacts will be generated, and these artifacts will cause serious distortion of the functional information calculated in the second step. The second disadvantage is that the universality of the structural-functional step-by-step reconstruction method is poor. The functional information of organelles includes various categories. When calculating different functional information, different numerical calculation algorithms need to be designed according to their respective physical and chemical characteristics, lacking a unified reconstruction framework and having poor universality.

[0004] Therefore, the functional information calculated by the above prior art is severely distorted and has poor universality. Summary of the Invention

[0005] To solve the above problems of the prior art, the present invention adopts a dual-channel joint reconstruction method based on structured illumination microscopy, including:

[0006] S1. Obtain the dual-channel fluorescence images D 1 and D 2 of the organelles according to the structured illumination microscopy; wherein, D 1 is the fluorescence image of channel 1, and D 2It is the fluorescence image of Channel 2;

[0007] S2. According to the fluorescence images D 1 and D 2 Construct a structure-function synchronous imaging model wherein, is the organelle structure information of Channel 1, is the ratio of the organelle structure information of Channel 2 and Channel 1, that is, the organelle function information;

[0008] S3. Add the regularization term constraint of the organelle function information to the structure-function synchronous imaging model to obtain the objective function;

[0009] S4. Iteratively optimize the objective function to obtain the organelle structure information and organelle function information of Channel 1.

[0010] Obtain the dual-channel fluorescence images D 1 and D 2 including:

[0011] Obtain the organelles labeled with two special dyes, illuminate the organelles labeled with two special dyes using structured light, and use two cameras to respectively receive the fluorescence signals of two wavelength ranges excited by the structured light in the organelles to obtain the dual-channel fluorescence images D 1 and D 2 .

[0012] Construct a structure-function synchronous imaging model including:

[0013]

[0014] wherein, I is the illumination light, H is the point spread function, is the fluorescence image of Channel 1, ζ is the likelihood term parameter, is the fluorescence image of Channel 2, d is the direction of the illumination light, is the phase parameter of the illumination light, and r is the three-dimensional space coordinate system.

[0015] Adding the regularization term constraint of the organelle function information to the structure-function synchronous imaging model includes:

[0016]

[0017] wherein, λ is the regularization term coefficient, and R() is the regularization term constraint.

[0018] The regularization term constraint is spatio-temporal continuity regularization.

[0019] Iterative optimization of the objective function includes: using the split Bregman algorithm to split the objective function and performing alternating iterative optimization on the split objective function.

[0020] Using the split Bregman algorithm to split the objective function includes:

[0021]

[0022] where v is a new variable introduced by the split Bregman algorithm, λ and β are regularization term coefficients, and b v is an intermediate variable.

[0023] Performing alternating iterative optimization on the split objective function includes:

[0024] S41. Optimize :

[0025]

[0026] S42. Optimize :

[0027]

[0028] S43. Optimize v:

[0029]

[0030] S44. When the preset maximum number of iterations is reached, obtain the final organelle structure information and organelle function information of channel 1; otherwise, return to step S41; where k is the number of iterations.

[0031] Intermediate variable The iterative update formula for is:

[0032]

[0033] where v (k) is the new variable introduced by the split Bregman algorithm in the k-th iteration, is the function information in the k-th iteration, is the intermediate variable in the (k - 1)-th iteration.

[0034] Beneficial effects:

[0035] 1. Compared with the method of step-by-step structure-function imaging, the present invention constructs a general framework applicable to various structure-function imaging based on the physical principles of super-resolution structure imaging and function imaging. Under this framework, only minor modifications need to be made according to the physical models of different function imaging, and then the same method can be used for reconstruction. 2. The present invention models by incorporating structure imaging and function imaging into the same objective function, realizing the synchronous reconstruction of structure information and function information, which can better suppress reconstruction artifacts and improve fidelity. 3. The present invention introduces the prior information of the continuity of organelle function information into the reconstruction process, establishes an iterative optimization reconstruction algorithm, which can better suppress the artifacts in the reconstructed super-resolution function information and solve the reconstruction problem of low signal-to-noise ratio images. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a structural diagram of a dual-channel joint reconstruction method based on structured illumination microscopy provided by an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the reconstruction results of structure information and function information based on simulation data provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] As Figure 1 shown, this embodiment adopts a dual-channel joint reconstruction method based on structured illumination microscopy, including the following contents:

[0040] S1. Obtain the dual-channel fluorescence images D 1 and D 2 of the organelles according to the structured illumination microscopy;

[0041] Build a hardware platform for structured illumination microscopy with dual-channel or multi-channel detection:

[0042] Currently, the "ratiometric method" is usually used in pH value functional super-resolution imaging, that is, the organelles labeled with two special dyes are illuminated with structured light, and then two cameras are used to receive the fluorescence signals of two wavelength ranges excited in the organelles respectively, so as to collect the dual-channel original images. Therefore, first, according to the imaging principle, build a hardware platform for structured illumination microscopy to detect the pH value inside the mitochondria with dual channels. The built hardware platform will use two cameras to receive the fluorescence images of the dual channels and obtain the fluorescence image D1 (r) and D 2 (r).

[0043] S2. According to the fluorescence image D 1 and D 2 Construct a structure-function synchronous imaging model wherein, is the organelle structure information of channel 1, is the ratio of the organelle structure information of channel 1 and channel 2, that is, the organelle function information;

[0044] When reconstructing the fluorescence images of the two channels collected in the present invention, instead of reconstructing them separately, a unified imaging model is established to jointly model the structure information and the function information as shown in Figure 1 . Further, the spatio-temporal continuity prior information is added to the model for joint iterative reconstruction, and finally the structure image and the function image of the dual-channel ratio can be synchronously reconstructed, avoiding the operation of first reconstructing the fluorescence images of the two channels and then dividing the pixel values.

[0045] First, the original dual-channel fluorescence images collected by the SIM microscopy system are and There are 9 original images for each channel, which are obtained by cosine structured light illumination in 3 directions d, with 3 phases in each direction . Although the 9 pieces of data of and in the two channels both contain super-resolution spectral information, they cannot be resolved by the human eye and need to be reconstructed by an algorithm to truly become a super-resolution image observable by the human eye. And the reconstruction algorithm is based on an accurate physical imaging model. Therefore, first, the physical model of the actual imaging process needs to be extracted from the structure and function synchronous imaging system in order to accurately reconstruct the original images with super resolution.

[0046] When imaging with a structured light illumination fluorescence microscope, the fluorescence data D(r) collected by the camera can be represented by the convolution of the excited fluorescence photon distribution E(r) and the point spread function H(r):

[0047]

[0048] wherein, r represents a three-dimensional spatial coordinate system, and the excited fluorescence photon distribution E(r) is generated after the fluorophore S(r) is excited by the illumination light I(r):

[0049] E(r) =S(r)I(r) (2)

[0050] The illumination light I(r) presents a cosine function pattern and is formed by double-beam interference:

[0051]

[0052] where p d represents the illumination light vector, d and represent the direction and phase parameter of the illumination light fringe respectively. Therefore, the imaging model of the classical structured illumination fluorescence microscope is:

[0053]

[0054] For the dual-channel imaging model of the present invention, the physical models of the fluorescence imaging of the two channels are both consistent with the imaging model of the above-mentioned structured illumination fluorescence microscope, that is, the original data acquired by channel 1

[0055]

[0056] where S 1 is the structural information of channel 1. For channel 2, the original data acquired by it

[0057]

[0058] where S 2 is the structural information of channel 2, and K represents the ratio of the structural information intensity S 2 of channel 2 to the structural information intensity S 1 of channel 1:

[0059] K = S 2 / S 1 (7)

[0060] That is, the functional information reflected by the ratio distribution.

[0061] If only the structural information is reconstructed, the structural information S 1 and S 2 of the two channels can be reconstructed by traditional SIM reconstruction algorithms such as Wiener inverse filtering respectively. However, in order to avoid the disadvantages of step-by-step structure-function reconstruction, the present invention does not adopt traditional inverse reconstruction methods such as Wiener inverse filtering, but designs a structure-function synchronous imaging model based on the actual imaging model, starting from formulas (5) and (6), to combine the structural information and functional information into one model:

[0062]

[0063] Both terms in the formula are likelihood terms, where Represents the structural information collected by channel 1 reconstructed iteratively. Represents the ratio distribution (i.e., functional information) of channel 2 and channel 1 reconstructed iteratively. ζ represents the likelihood term parameter.

[0064] S3. Add the functional information of organelles to the structural-functional synchronous imaging model with a regularization term constraint to obtain the objective function.

[0065] Since the functional information such as pH distribution is continuously changing under actual conditions, while the artifacts in the reconstructed images are random and discontinuous in space and time, based on this prior information, the present invention proposes a regularization term based on spatio-temporal continuity and combines it with the imaging physical model represented by formula (8) to establish a structural and functional synchronous reconstruction objective function applicable to low signal-to-noise ratio dual-channel images:

[0066]

[0067] Among them, the first and second terms in the objective function (9) are the likelihood terms in the reconstruction objective function, and the third term in the objective function is the regularization term, representing the prior constraint on the functional information By adding the regularization term constraint to the objective function (9), the artifacts in the reconstructed super-resolution functional information can be better weakened. The regularization term can be constrained using the spatio-temporal continuity prior or a self-supervised denoising network. Among them, the spatio-temporal continuity prior is:

[0068]

[0069] Among them, x, y, and z are the different directions of the three-dimensional space coordinate system, t is the time direction, r is the three-dimensional space coordinate system, σ 1 and σ 2 are respectively the parameters representing the continuity of the signal in the z direction and the continuity in the t direction. is the second-order partial derivative in the x direction. Similarly, etc. represent the second-order partial derivatives in other directions.

[0070] The iterative reconstruction process of the objective function (9) is as follows:

[0071] First, by using the split Bregman algorithm, the objective function (9) can be re-expressed as:

[0072]

[0073] Among them, v is the new variable introduced by the split Bregman algorithm, λ and β represent the regularization term coefficients, bv It is an intermediate variable for the optimization process, and its iterative update formula is shown in Formula (12):

[0074]

[0075] Secondly, according to the objective function expressed by Formula (11), the global iterative optimization processes shown in Formulas (13)-(15) can be obtained respectively:

[0076]

[0077]

[0078] By optimizing Formula (13), we get According to By optimizing Formula (14), we get According to By optimizing Formula (15), we get v. Alternately and iteratively optimize the above Formulas (13)-(15). When the preset maximum number of iterations is reached, the super-resolution structural information and super-resolution functional information collected by Channel 1 in the finally reconstructed image are obtained.

[0079] Figure 2 Figure is the reconstruction result diagram of the structural information and functional information based on simulation data using the method of the present invention, which preliminarily verifies the effectiveness of the super-resolution structure and function synchronous joint reconstruction. The original data used in the experiment are two dual-channel original images with a 1-fold difference in intensity generated by a forward simulation program, and then the super-resolution structural image of Channel 1 is reconstructed using this scheme ( Figure 2 a) and the ratio image between the two channels, that is, the functional information ( Figure 2 b). Histogram statistics are performed on the reconstructed functional information, and the intensities of the reconstructed functional information are all around 2 ( Figure 2 c), which fits the true value situation where the intensities of the two channels differ by 1-fold, indicating the effectiveness of the reconstruction algorithm. Compared with wide-field functional imaging, the functional information obtained by the SIM super-resolution imaging method has higher resolution and clarity than wide-field imaging.

[0080] The above scheme takes the pH distribution in functional imaging as an example to introduce the modeling process of structural and functional imaging. However, this scheme is actually a general reconstruction scheme and can be applied to the requirements of various functional imaging. Although different functional imaging has its own physical principles, when using the modeling and reconstruction framework proposed by the present invention, only according to the physical principles of different functional imaging, establish the corresponding likelihood term and substitute it into the likelihood term in Formula (9), then the requirements of various functional imaging and reconstruction can be met.

[0081] Taking pH functional imaging as an example, this embodiment jointly models and reconstructs structural and functional super-resolution imaging, which can simultaneously reconstruct super-resolution structural information and functional information, and has the advantages of high fidelity and generality. Compared with the previous method of first reconstructing structural information and then obtaining functional information through numerical operations, the solution proposed in this patent can reduce the influence of artifacts in the structural image on the functional information and improve the reconstruction fidelity. Moreover, this method introduces continuous prior information into the reconstruction process and establishes an iterative optimization reconstruction algorithm, which can further suppress the artifacts in the reconstructed super-resolution functional information, solve the reconstruction problem of low signal-to-noise ratio images, and improve the reconstruction fidelity. In addition, there is an additional benefit of improving the reconstruction fidelity in this invention, that is, when applied to actual biological experiments, it can further reduce the exposure duration and excitation light intensity required for actual imaging, thereby reducing the phototoxicity and photobleaching during live cell imaging, improving the time resolution of structural and functional imaging, and extending the total duration of continuous imaging.

[0082] The above embodiments further elaborate on the purpose, technical solutions, and advantages of the present invention. It should be understood that the above embodiments are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made to the present invention within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A dual-channel joint reconstruction method based on structured light illumination microscopy, characterized in that: include: S1. Obtain dual-channel fluorescence images of organelles using structured light illumination microscopy D 1 and D 2 Among them, D 1 is the fluorescence image of channel 1, D 2 is the fluorescence image of channel 2; S2. According to the fluorescence image D 1 and D 2 Constructing a structure-function simultaneous imaging model in, is the organelle structure information of channel 1, is the ratio of the organelle structure information of channel 2 to that of channel 1, i.e., the organelle function information; S3. In the structure-function simultaneous imaging model Adding organelle function information The regular term constraint of is used to obtain the objective function; S4, iteratively optimizing the objective function to obtain the organelle structure information and organelle function information of channel 1; Constructing a structure-function simultaneous imaging model include: Where I is the illumination light, H is the point spread function, is the fluorescence image of channel 1, ζ is the likelihood parameter, is the fluorescence image of channel 2, d is the direction of the illumination light, is the phase parameter of the illumination light, r is the three-dimensional space coordinate system; In the structure-function simultaneous imaging model Adding organelle function information The regularization constraints include: Among them, λ is the regularization term coefficient, R() is the regularization term constraint; The regularization term is constrained to be space-time continuity regularization.

2. The dual-channel joint reconstruction method based on structured light illumination microscopy according to claim 1, characterized in that: Acquire dual-channel fluorescence images of organelles 1 and D 2 include: Obtain the cell organelles marked by two special dyes, illuminate the cell organelles marked by two special dyes with structured light, use two cameras to receive the fluorescence signals of two wavelength ranges in the cell organelles excited by the structured light, and obtain a dual-channel fluorescence image D 1 and D 2 .

3. The dual-channel joint reconstruction method based on structured light illumination microscopy according to claim 1, characterized in that: Iterative optimization of the objective function includes: using the segmentation Bergman algorithm to segment the objective function, and performing alternating iterative optimization on the segmented objective function.

4. The dual-channel joint reconstruction method based on structured light illumination microscopy according to claim 3, characterized in that: Using the segmentation Bergmann algorithm, segmenting the target function includes: Among them, v is the new variable introduced by the segmentation Bergmann algorithm, λ and β are the regularization coefficients, and b v is an intermediate variable.

5. The dual-channel joint reconstruction method based on structured light illumination microscopy according to claim 4, characterized in that: The alternating iterative optimization of the segmented objective function includes: S41, yes To optimize: S42, yes To optimize: S43. Optimize v: S44. When the preset maximum number of iterations is reached, the final organelle structure information and organelle function information of channel 1 are obtained; otherwise, the process returns to step S41; wherein k is the number of iterations.

6. The dual-channel joint reconstruction method based on structured light illumination microscopy according to claim 5, characterized in that: Intermediate variables The iterative update formula is: Among them, v (k) The new variable introduced by the segmentation Bergmann algorithm for the kth iteration, is the functional information of the kth iteration, is the intermediate variable of the k-1th iteration.

Images