A method for reconstructing the spatial and angular distribution of fluorescent molecules based on the generalized Richardson-Lucy algorithm

Abstract

The present invention discloses a method for reconstructing the spatial and angular distribution of fluorescent molecules based on a generalized Richardson-Lucy algorithm, which introduces two dimensions, namely, excitation mode and angular distribution, into the traditional Richardson-Lucy algorithm, so that the spatial and angular distribution of fluorescent molecules can be better reconstructed through polarized fluorescence microscopy data, without the need to introduce a regularization term that will bring estimation bias in order to suppress noise. In addition, the present invention designs an operation process based on spherical harmonic domains, which greatly reduces the amount of calculation of front and back projections and spherical domain multiplication and division in the algorithm. Therefore, the present invention designs a generalized Richardson-Lucy algorithm for reconstructing the spatial angular distribution of molecules from polarized fluorescence microscopy data, which solves the problems of poor noise suppression ability, cumbersome manual parameter adjustment, low spatial resolution of reconstruction results, and inaccurate angular distribution in existing methods at a reasonable computational cost.

Description

Technical Field

[0001] The invention belongs to the technical field of polarized fluorescence microscopy imaging, and in particular relates to a method for reconstructing the spatial and angular distribution of fluorescent molecules based on a generalized Richardson-Lucy algorithm. Background Art

[0002] Most fluorescent molecules can be regarded as dipoles with different orientations. Several fluorescent molecules with arbitrary orientations at the same position can be regarded as a fluorophore. For a fluorophore, it has characteristics such as spatial position and intensity of different orientations, that is, spatial and angular distribution. Polarized fluorescence microscopy (PFM) is a powerful imaging technology in the biological field. Using fluorescent protein markers or specific proteins inside biological samples, biological samples can emit fluorescence of different intensities under the excitation of different polarized light. The intensity is closely related to the spatial and angular distribution of protein molecules, thus reflecting the specific properties of biological samples. Its application range extends from single-cell imaging to large tissue imaging. Polarized fluorescence microscopy has submicron spatial resolution, high contrast, and molecular orientation sensitivity. We can collect microscopic images under different polarization excitation modes for indirect observation, but in order to directly explore the structure and function of biological samples, it is necessary to restore the three-dimensional spatial and angular distribution of fluorescent molecules from these images; in addition, inherent blur and noise will reduce fluorescence data, which will affect researchers' understanding of real biological samples. However, as long as the forward projection process of the imaging system can be described, the sample information can be restored through a linear inverse problem solving algorithm to reconstruct the spatial and angular distribution of the sample.

[0003] At present, the three-dimensional spatial and angular distribution reconstruction algorithm of fluorescent molecules used in polarized fluorescence microscopy is mainly based on the least squares algorithm of singular value decomposition (SVD) [Richard W.Hendler, Richard I.Shrager.Deconvolutions based on singular value decomposition and the pseudoinverse: a guide for beginners.Journal of Biochemical and Biophysical Methods, Volume 28, Issue 1, 1994]. Although this method has been statistically proven to be the maximum likelihood estimator of data corrupted by uncorrelated Gaussian noise, the small eigenvalues ​​obtained by singular value decomposition will amplify the noise to an unacceptable level during the operation, which generally needs to be suppressed by adding a Tikhonov regularization term to the optimization problem [Fuhry, M., Reichel, L. A new Tikhonov regularization method. Numer Algorithm 59,433–445(2012)]; the problem is that it will introduce a bias to the final reconstruction result as a whole, which is manifested as a shift in a certain direction in the angular distribution and a reduction in the resolution in the spatial distribution. In addition, a lot of attempts are required to manually select a reasonable regularization parameter.

[0004] Therefore, how to improve the resolution and accuracy of the spatial and angular distribution of the reconstruction results while suppressing noise, and minimize the number of manual parameter adjustments while maintaining an acceptable computing cost is a hot topic in this field. Summary of the invention

[0005] In view of the above, the present invention provides a method for reconstructing the spatial and angular distribution of fluorescent molecules based on the generalized Richardson-Lucy algorithm, which can improve the spatial resolution and angular distribution accuracy of the reconstruction results while suppressing noise, and only requires a rough setting of the number of iterations in advance, without the need for multiple manual parameter adjustments. At the same time, the complex calculations of the angular distribution originally in the spherical domain are transferred to the simple spherical harmonic domain for implementation, thereby ensuring a reasonable computing cost.

[0006] A method for reconstructing the spatial and angular distribution of fluorescent molecules based on a generalized Richardson-Lucy algorithm comprises the following steps:

[0007] (1) In a polarized fluorescence microscope system, for each viewing angle, a fluorescently labeled biological sample is excited by using excitation light of several different polarization states, and data is collected through a detection objective lens to obtain biological sample image data corresponding to each polarization excitation mode at each viewing angle;

[0008] (2) obtaining the dipole point spread function corresponding to each polarization excitation mode at each viewing angle through simulation, then performing overall normalization on the dipole point spread function and calculating the back-projection operator with object space sensitive constants;

[0009] (3) preprocessing the collected biological sample image data;

[0010] (4) The Richardson-Lucy algorithm is introduced into the polarization state domain that characterizes the excitation mode and the spherical domain that characterizes the angular distribution, and the operations that should be performed in the spatial domain and spherical domain are converted to the frequency domain and spherical harmonic domain for implementation, thereby iteratively estimating the spatial angular distribution of the biological sample.

[0011] Furthermore, the number of viewing angles and the number of polarization excitation modes are single or multiple.

[0012] Furthermore, the biological sample image data collected in step (1) is a volume image, which is obtained by stacking two-dimensional slice images; the dipole point spread function corresponding to each viewing angle and polarization excitation mode has three spatial dimensions plus one spherical dimension, a total of four-dimensional matrix structure.

[0013] Furthermore, the imaging equation of the polarized fluorescence microscope system is as follows:

[0014]

[0015] Where: g v，p (r d ) represents the corresponding position r in the biological sample image data collected under the polarization excitation mode p at the viewing angle v d The gray level at Indicates that the unit concentration distance is r under the viewing angle v and polarization excitation mode p d -r o Towards The fluorescence molecular response is the dipole point spread function, Represents the biological sample at spatial position r o The angle direction is The concentration of fluorescent molecules, represents three-dimensional space, Represents spherical space.

[0016] Furthermore, the calculation expression of the back-projection operator in step (2) is as follows:

[0017]

[0018] in: It represents the unit concentration distance r and the direction under the polarization excitation mode p at the viewing angle v. The fluorescent molecular response, Indicates the unit concentration distance under the polarization excitation mode p at the viewing angle v is -r and the direction is The fluorescent molecular response, Corresponding to The back-projection operator is Represents three-dimensional space.

[0019] Furthermore, in step (4), for a certain viewing angle v, the spatial angle distribution of the biological sample is estimated iteratively by the following expression:

[0020]

[0021] Where: e k and e k+1 are the estimated results of the spatial angle distribution of biological samples after the kth and k+1th iterations, respectively, k is a natural number, h v and are the dipole point spread function and back projection operator at viewing angle v, respectively, i v is the preprocessed biological sample image data obtained under the viewing angle v, and They are the forward projection operator and the back projection operator respectively.

[0022] Furthermore, the forward projection operation And back projection operation The expression is as follows:

[0023]

[0024]

[0025] in: Indicates that the unit concentration distance under the viewing angle v is r d -r o Towards The fluorescent molecular response, Represents the estimated result e k The biological sample is at the spatial position r o The angle direction is The concentration of fluorescent molecules, Corresponding to The back-projection operator, i v，p (r d ) represents the corresponding position r in the preprocessed biological sample image data under the viewing angle v and polarization excitation mode pd The grayscale at .

[0026] Furthermore, in order to reduce the computational cost, the four operations involved in the iterative process of step (4), including forward projection, back projection, spherical domain multiplication, and spherical domain division, are rewritten as follows:

[0027]

[0028]

[0029]

[0030]

[0031] in: and denote Fourier transform and spherical harmonic transform respectively, and Corresponding representation and The subscripts lm, l′m′ and l″m″ represent the index numbers of the spherical harmonic components obtained after the spherical harmonic transformation. v，p is the preprocessed biological sample image data under the polarization excitation mode p at the viewing angle v, is the back-projection operator for polarization excitation mode p at viewing angle v, is the Gaunt coefficient.

[0032] Furthermore, the Gaunt coefficient The expression is as follows:

[0033]

[0034] in: is a spherical harmonic function, which means that the corresponding direction on the spherical domain is The mapping relationship between a certain point and the spherical harmonic component lm.

[0035] Furthermore, for the estimated initial value e0 of the spatial angular distribution of the biological sample, the same uniform angular distribution is set for each spatial position, that is,

[0036] The present invention introduces two dimensions, namely, excitation mode and angular distribution, into the traditional Richardson-Lucy algorithm, so that the spatial and angular distribution of fluorescent molecules can be better reconstructed through polarized fluorescence microscopy, without the need to introduce regularization terms that will cause estimation bias in order to suppress noise. In addition, the operations of the improved Richardson-Lucy algorithm itself are all based on the spherical domain. The present invention designs an operation process based on the spherical harmonic domain, which greatly reduces the amount of calculation of front and back projections and spherical domain multiplication and division.

[0037] In general, the present invention designs a generalized Richardson-Lucy algorithm for reconstructing the spatial and angular distribution of fluorescent molecules from polarized fluorescence microscopy data. At a reasonable computational cost, it better suppresses noise, improves the spatial resolution and angular distribution accuracy of the reconstruction results, and solves the problem of tedious multiple manual parameter adjustments in existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the flow of the method for reconstructing the spatial and angular distribution of fluorescent molecules based on the generalized Richardson-Lucy algorithm of the present invention.

[0039] FIG2(a) is a graph showing the change in the structural similarity between the spatial distribution of each iteration result and the true value when the present invention is used to reconstruct the simulated data after adding different levels of Poisson noise, and is also compared with the SVD-based least squares method using the optimal regularization term parameter.

[0040] FIG2( b ) is a curve diagram showing the change in the average cosine value of the angle difference between the peak direction of the angular distribution of each iterative result and the true value when the present invention is used to reconstruct the simulated data after adding different levels of Poisson noise, and is compared with the SVD-based least squares method using the optimal regularization term parameter.

[0041] Figure 3(a) is an angular distribution peak direction diagram of the three-dimensional reconstruction results of a partial area of ​​the polarized fluorescence microscopy data of a giant unilamellar vesicle (GUV) sample. The first row and the second row are the reconstruction results of 10 iterations of the present invention and the observation diagrams of the reconstruction results using the SVD-based least squares method at different viewing angles, respectively. The first to fourth columns correspond to four observation viewing angles.

[0042] Figure 3(b) is a spatial distribution diagram of the three-dimensional reconstruction results of a partial area of ​​the polarized fluorescence microscopy data of a giant unilamellar vesicle sample. The first and second rows are slice diagrams of the reconstruction results of 10 iterations of the present invention and the reconstruction results using the SVD-based least squares method at the z-axis position, respectively. The third row is a contour diagram of the reconstruction results of the two methods along the line position in the upper figure. The first to second columns correspond to two groups of slices. DETAILED DESCRIPTION

[0043] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0044] The excitation light path of the polarized fluorescence microscope emits a laser of a certain polarization state, irradiating the biological sample labeled with fluorescent protein or having fluorescent protein to make it emit fluorescence. The signal collection light path of the microscope collects fluorescence data, and the focal plane of the collection light path is continuously moved so that the signal of the entire biological sample is traversed. The images collected each time are stacked to form a three-dimensional volume image. For data in multi-viewing angle and multi-polarization excitation modes, the excitation direction of the fluorescence microscope is adjusted or the sample and the polarization mode of the excitation light are rotated, and multiple sets of volume data are collected to obtain biological sample data in multi-viewing angle and multi-polarization excitation modes.

[0045] like Figure 1 As shown, the method for reconstructing the spatial and angular distribution of fluorescent molecules based on the generalized Richardson-Lucy algorithm of the present invention comprises the following steps:

[0046] S1. Calculate the system matrix, including the dipole point spread function and the back-projection operator. Generate the dipole point spread function h for each viewing angle v and polarization excitation mode p through simulation. v，p , and calculate the back-projection operator with object-space sensitive constants

[0047]

[0048] Save them in the form of multidimensional matrices for subsequent data processing.

[0049] S2. Since the reference coordinate systems of the acquired images are inconsistent, preprocessing is first performed. Select a viewing angle and polarization excitation mode as a reference, and perform linear interpolation and rotation of a certain angle on the data obtained from other viewing angles and excitation modes so that they have corresponding coordinate systems. At this time, a roughly registered image is obtained. For more accurate image fusion, an intensity-based registration method is used to generate a registration matrix. The data of each viewing angle and excitation mode are multiplied by the corresponding registration matrix to obtain the registered multi-view and multi-polarization excitation mode image data.

[0050] S3. The generalized Richardson-Lucy algorithm that introduces the polarization state domain that characterizes the excitation mode and the spherical domain that characterizes the angular distribution is used for iterative calculation, and the corresponding reconstruction estimation result of the spatial angular distribution of the fluorescent molecules is obtained in each iteration:

[0051]

[0052] The operations originally defined in the spatial domain and spherical domain are converted to the frequency domain and spherical harmonic domain for calculation, and the front projection, back projection, spherical domain multiplication, and spherical domain division are transformed into:

[0053]

[0054]

[0055]

[0056]

[0057] The spatial and angular distribution of biological samples are estimated iteratively.

[0058] S4. Determine whether the iteration stop condition is met, that is, whether the expected number of iterations has been reached. If the condition is not met, execute step S3. If the condition is met, the iteration stops and the spatial angle distribution estimation result obtained by the last iteration is used as the final reconstruction result.

[0059] We take the reconstruction of data obtained under single-view and multiple polarization excitation modes as an example to introduce the generalized Richardson-Lucy algorithm that introduces the polarization state domain that characterizes the excitation mode and the spherical domain that characterizes the angle distribution. The traditional single-view Richardson-Lucy algorithm has the following iterative structure:

[0060] e0=i

[0061] for k = 0, 1, 2, ..., N

[0062]

[0063] end

[0064] The present invention introduces the polarization state domain characterizing the excitation mode and the spherical domain characterizing the angle distribution into the above formula, and proposes the following new iterative formula:

[0065] e0=1

[0066] for k = 0, 1, 2, ..., N

[0067]

[0068] end

[0069] The forward and backward projection operations are defined as:

[0070]

[0071]

[0072] Regarding the back-projection operator, the traditional Richardson-Lucy algorithm generates the point spread function by flipping it, but this introduces more dimensions. The domain spaces on both sides of the forward and backward projection processes are inconsistent, which will lead to rapid divergence of iterations. Therefore, an object space sensitive constant is introduced in the back-projection operator, and the final back-projection operator is as follows:

[0073]

[0074] Since in the calculation process, the spherical domain dimension Usually it needs to be discretized into thousands of directional values, which leads to a huge amount of calculation. Therefore, spherical harmonic transformation can be used to transfer to the spherical harmonic domain. In the spherical harmonic domain, usually only a few tens of lm values ​​are needed to represent the distribution that originally required thousands of values ​​in the spherical domain. In addition, the computational cost of doing products in the frequency domain is much lower than that of doing convolution operations in the spatial domain.

[0075] Therefore, the present invention rewrites the definition of each operator in the iteration structure as follows:

[0076]

[0077]

[0078]

[0079]

[0080] Similarly, for data obtained from multiple views with several polarization excitation modes at each view, the back-projection operator is:

[0081]

[0082] The additive iterative reconstruction formula is:

[0083] e0=1

[0084] for k = 0, 1, 2, ..., N

[0085]

[0086] …

[0087]

[0088]

[0089] end

[0090] The multiplication iterative reconstruction formula is:

[0091]

[0092] for k = 0, 1, 2, ..., N

[0093]

[0094] …

[0095]

[0096] end

[0097] In the following, we use simulated data to verify the effectiveness of the present invention. We randomly generate a phantom with a certain spatial and angular distribution, and use the dipole point spread function to perform forward projection to simulate the excitation and acquisition process of the polarized fluorescence microscope, and obtain a set of simulated data. After adding different levels of Poisson noise (SNR = 30dB and 5dB) to the simulated data, the present invention is used to reconstruct, and the results after each iteration are recorded, and compared with the SVD-based least squares method using the optimal regularization term parameter; Figure 2 (a) plots the gap between the spatial distribution of the reconstruction result and the true value, represented by the structural similarity index, and Figure 2 (b) plots the gap between the peak direction of the angular distribution of the reconstruction result and the true value, represented by the average cosine value of the angle difference; they together illustrate that the reconstruction result of the present invention is far better than the best result that can be obtained by manually adjusting the parameters of the SVD-based least squares method, especially the peak direction of the angular distribution of the reconstruction result, which has a very large accuracy improvement and is insensitive to noise.

[0098] In addition, we use experimental data from giant unilamellar vesicle samples to further verify the effectiveness of the present invention. Giant unilamellar vesicles are biological samples commonly used in biological polarized fluorescence imaging. According to current biological knowledge, the peak direction of the angular distribution of fluorescent molecules on the vesicle membrane is perpendicular to the tangent direction of the membrane surface. The data is collected by polarized dual-view inverted light sheet fluorescence microscopy (pol-diSPIM).

[0099] The reconstruction results of the fluorescent molecule spatial angular distribution reconstruction method based on the generalized Richardson-Lucy algorithm of the present invention are compared with those of the traditional least squares algorithm based on SVD. The number of iterations of the former is set to 10 times, and the regularization term parameter of the latter is set to 1 after many attempts. The two process the same data and display the observation diagrams of the angular distribution peak direction of the processing results at different viewing angles, as well as the slice diagrams of the spatial distribution of the processing results at different depths.

[0100] It can be seen intuitively from Figure 3(a) that in terms of the angular distribution of fluorescent molecules, compared with the results of the traditional least squares algorithm based on SVD, the reconstruction algorithm based on the generalized Richardson-Lucy algorithm of the present invention can obtain a more accurate angular distribution of sample fluorescent molecules that conforms to biological cognition. The specific improvement is that the peak direction of the angular distribution is almost perpendicular to the vesicle membrane, the overall change is more continuous, and there is a higher signal-to-noise and better discrimination in the region of interest.

[0101] It can be seen intuitively from Figure 3(b) that in terms of the spatial distribution of fluorescent molecules, compared with the results of the traditional least squares algorithm based on SVD, the reconstruction algorithm based on the generalized Richardson-Lucy algorithm of the present invention can obtain a higher resolution and noise spatial distribution of fluorescent molecules. The specific improvement is that the blurring degree of the spatial distribution is reduced, so that some inherent structures of the sample can be clearly distinguished, and the overall resolution and signal-to-noise ratio are improved.

[0102] The above description of the embodiments is to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made to the present invention by those skilled in the art based on the disclosure of the present invention should be within the scope of protection of the present invention.

Claims

1. A method for reconstructing the spatial and angular distribution of fluorescent molecules based on the generalized Richardson-Lucy algorithm, comprising the following steps: (1) In a polarized fluorescence microscope system, for each viewing angle, a fluorescently labeled biological sample is excited by using excitation light of several different polarization states, and data is collected through a detection objective lens to obtain biological sample image data corresponding to each polarization excitation mode at each viewing angle; The biological sample image data is a volume image, which is obtained by stacking two-dimensional slice images; the dipole point spread function corresponding to each viewing angle and polarization excitation mode has three spatial dimensions plus one spherical dimension, a total of four-dimensional matrix structure; (2) The dipole point spread function corresponding to each polarization excitation mode at each viewing angle is obtained by simulation, and then the dipole point spread function is overall normalized, and the back-projection operator with the object space sensitivity constant is calculated by the following formula; in: It represents the unit concentration distance r in the polarization excitation mode p under the viewing angle v. The fluorescent molecular response, Indicates the unit concentration distance under the polarization excitation mode p at the viewing angle v is -r and the direction is The fluorescent molecular response, Corresponding to The back-projection operator is Represents three-dimensional space; (3) preprocessing the collected biological sample image data; (4) The Richardson-Lucy algorithm is introduced into the polarization state domain representing the excitation mode and the spherical domain representing the angle distribution, and the operation that should be performed in the spatial domain and the spherical domain is converted to the frequency domain and the spherical harmonic domain for implementation; for a certain viewing angle v, the spatial angle distribution of the biological sample is iteratively estimated by the following expression; Where: e k and e k+1 are the estimated results of the spatial angle distribution of biological samples after the kth and k+1th iterations, respectively, k is a natural number, h v and are the dipole point spread function and back projection operator at viewing angle v, respectively, i v is the preprocessed biological sample image data obtained under the viewing angle v, ◇ and ο are the front projection operator and the back projection operator respectively.

2. The method for reconstructing the spatial and angular distribution of fluorescent molecules according to claim 1, characterized in that: The number of viewing angles and the number of polarization excitation modes are single or multiple.

3. The method for reconstructing the spatial and angular distribution of fluorescent molecules according to claim 1, characterized in that: The imaging equation of the polarized fluorescence microscope system is as follows: Where: g v,p (r d ) represents the corresponding position r in the biological sample image data collected under the polarization excitation mode p at the viewing angle v d The gray level at Indicates that the unit concentration distance is r under the viewing angle v and polarization excitation mode p d to r o The distance direction is The fluorescence molecular response is the dipole point spread function, Represents the biological sample at spatial position r o The angle direction is The concentration of fluorescent molecules, represents three-dimensional space, Represents spherical space.

4. The method for reconstructing the spatial and angular distribution of fluorescent molecules according to claim 1, characterized in that: Forward projection operation And back projection operation The expression is as follows: in: Indicates that the unit concentration distance under the viewing angle v is r d to r o The distance direction is The fluorescent molecular response, Represents the estimated result e k The biological sample is at the spatial position r o The angle direction is The concentration of fluorescent molecules, Corresponding to The back-projection operator, i v,p (r d ) represents the corresponding position r in the preprocessed biological sample image data under the viewing angle v and polarization excitation mode p d The grayscale at .

5. The method for reconstructing the spatial and angular distribution of fluorescent molecules according to claim 4, characterized in that: In order to reduce the computational cost, the four operations involved in the iterative process of step (4), including forward projection, back projection, spherical domain multiplication, and spherical domain division, are rewritten as follows: in: and denote Fourier transform and spherical harmonic transform respectively, and Corresponding representation and The subscripts lm, l′m′ and l″m″ represent the index numbers of the spherical harmonic components obtained after the spherical harmonic transformation. v,p is the preprocessed biological sample image data under the polarization excitation mode p at the viewing angle v, is the back-projection operator for polarization excitation mode p at viewing angle v, is the Gaunt coefficient, e0×e1 and They represent the spherical domain multiplication and spherical domain division operations between e0 and e0 respectively.

6. The method for reconstructing the spatial and angular distribution of fluorescent molecules according to claim 5, characterized in that: The Gaunt Coefficient The expression is as follows: in: is a spherical harmonic function, which means that the corresponding direction on the spherical domain is The mapping relationship between a certain point and the spherical harmonic component lm.

7. The method for reconstructing the spatial and angular distribution of fluorescent molecules according to claim 1, characterized in that: For the initial value e0 of the spatial angular distribution of the biological sample, the same uniform angular distribution is set for each spatial position, that is,

Images