Single particle diffusion quantitative feature prediction method and device, electronic equipment and storage medium

Abstract

The invention provides a single particle diffusion quantitative feature prediction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target input image corresponding to a target single particle; based on a pre-trained particle trajectory prediction model, according to the target input image, predicting to obtain a particle pseudo trajectory image; on the basis of the particle pseudo-trajectory image, diffusion quantitative characteristics of the target single particle are calculated; wherein the particle trajectory prediction model is obtained by performing training optimization according to a first training sample set formed by the single-particle motion blurred image and the corresponding particle motion trajectory image. According to the method, the actual movement path of the target single particle is predicted from the target input image through the particle trajectory prediction model, and the diffusion quantitative characteristic of the target single particle is calculated based on the actual movement path, so that the diffusion coefficient of a large number of single particles can be rapidly and accurately predicted in a high-mark-density environment, and the phototoxicity is remarkably reduced; diffusion characteristics of different biological particles can be effectively distinguished, and a powerful tool is provided for dynamic research of particles in a diffusion system and living cells.

Description

Technical Field

[0001] The present invention relates to the technical field of super-resolution imaging, and in particular to a method, device, electronic equipment and storage medium for predicting quantitative characteristics of single-particle diffusion. Background Art

[0002] Traditional imaging methods are limited by the optical diffraction limit and can only study the dynamic characteristics of group molecules at a low temporal and spatial resolution. Single-molecule localization microscopy (SMLM) is a super-resolution imaging technology that won one of the Nobel Prizes in Chemistry in 2014 and can image single fluorescent particles or fluorescent biomolecules.

[0003] With the development of fluorescence microscopy and new labeling techniques, people use SMLM to perform real-time imaging, positioning and tracking of single particles or single biological molecules in extracellular diffusion systems and living cells (single-particle tracking, SPT; single-molecule tracking, SMT; hereinafter referred to as SMT). Analysis of the obtained molecular trajectories can obtain measurement indicators such as molecular diffusion coefficient, binding fraction, diffusion angle distribution (anisotropy), etc. These indicators can be used to further infer the biological activity of the labeled protein and the intensity of interaction with other molecules. This technological innovation is becoming a powerful tool for understanding the single-molecule dynamics and functions of biological molecules in the life process of cells.

[0004] In the mainstream live cell single-molecule dynamics analysis that has been published so far, the general process is to locate the single molecule, connect the trajectory frame by frame, summarize the length of the molecular trajectory obtained, and fit various physical model formulas based on molecular diffusion to obtain parameters such as diffusion coefficient and binding percentage. After obtaining the original single-molecule fluorescence image, the single-molecule localization algorithm is first required to approximate the single-molecule fluorescence intensity distribution according to the approximate function of the point spread function (PSF) - a two-dimensional Gaussian function, and obtain the spatial coordinates corresponding to the maximum value of the function as the position of the molecule. Subsequently, statistical inference is used to connect the closest and most likely points in physical diffusion between the two frames to form a trajectory.

[0005] In order to ensure the accuracy of trajectory connection, it is generally necessary to control the output intensity of the photoactivated laser. The more molecules appear in each frame, the lower the accuracy of trajectory connection using the algorithm. In order to accurately obtain the trajectory, the molecular density needs to be maintained low throughout the SMT imaging process to avoid incorrect connections. The analysis of single-molecule trajectories often depends on the length of the molecular trajectory. The longer the molecular trajectory, the more accurate the kinetic parameters extracted from it. However, in order to make the subsequent kinetic analysis and model fitting statistically significant, it is required to obtain enough single-molecule trajectory data in the experiment. For this purpose, high-frame imaging is required to capture a sufficient number of single molecules. In this process, the effects of phototoxicity and cell movement on the experiment cannot be ignored.

[0006] For the currently more commonly used two-dimensional SMT imaging, it is difficult to obtain long trajectories at limited Z-axis resolution when the binding percentage of the molecule is low and the diffusion coefficient is fast. On the other hand, most trajectory-based analyses require a large number of trajectories (usually at least 100,000 trajectories) to obtain statistically reliable results. A common solution is to image many cells and pool the trajectories obtained from different cells. Although the kinetic parameters obtained after statistical modeling of a large number of single-molecule trajectories are accurate, the results are averages and have no spatial information, which limits the ability to use SMT to study the spatial dynamics of single-cell proteins.

[0007] Another limitation of trajectory-based analysis is the difficulty in measuring the dynamics of protein clustering. For example, RNA polymerase II (RNA Pol II, hereinafter referred to as Pol II) forms dynamic clusters with a radius close to or below the diffraction limit (the diffraction limit is about 200nm) in living cells. Since the spatial resolution of SMT using total internal reflection fluorescence microscopy in highly inclined light sheet (HILO) mode is currently about 30nm, it is inherently challenging to sparsely label the molecules within the clusters and track the trajectories to study the dynamic characteristics of the clusters.

[0008] At present, there are still limited methods to depict the spatial distribution of molecular diffusion in living cells. Some methods use the average displacement of the trajectory to represent the speed of molecular movement at each point in space. However, this method is limited by possible errors in the trajectory tracking process, including random flashing and photobleaching of fluorescent molecules, which lead to failure of trajectory connection; and the density of excited molecules in each frame cannot be too high, otherwise it is easy to cause incorrect connection. Therefore, this method requires the collection of long-term data, but long-term laser irradiation can easily cause cells to shift to a certain extent. Therefore, this method is only suitable for shooting proteins whose overall movement is slow and whose structure does not change greatly over time, such as chromatin histones.

[0009] There are also methods that directly use the tracked trajectory to analyze the spatial distribution of molecular diffusion, such as sptPALM. This method uses high-intensity laser to excite molecules, uses a high numerical aperture objective (model Olympus APO100XO-HR-SP, 1.65NA, while the numerical aperture NA value of the commonly used 100x objective is usually below 1.49) combined with total internal reflection illumination, tracks the trajectory of cell membrane proteins in the phototoxic COS7 cell line, and uses the MSD-Δt fitting method of each trajectory to obtain the diffusion coefficient to obtain the spatial distribution of molecular diffusion on the membrane surface.

[0010] On the one hand, this method relies on trajectory tracking, so there is a possibility of misconnection of molecules between each frame; on the other hand, this method relies on the MSD-Δt fitting method, which requires a long molecular trajectory to obtain an accurate estimation of the diffusion coefficient. Since cell membrane proteins mainly diffuse in two dimensions, they are suitable for obtaining long trajectories (>10 frames); while in other environments such as the nucleus, molecules diffuse in three dimensions, and the two-dimensional imaging method using HILO light illumination can only obtain a molecular trajectory length of 3 to 4 frames for stably bound molecules such as H2B, and the average molecular trajectory that can be obtained for other faster diffusing molecules is shorter, so it is difficult to use the MSD-Δt fitting method.

[0011] In addition, there is uPAINT (universal-points-accumulation-for-imaging-in-nanoscale-topography), which uses the non-covalent binding of ATTO-647N dye molecule ligand modified by divalent nickel ion-trisN-nitrilotriacetic acid (Ni-NTA) and membrane proteins with 6 histidine tags to achieve high-density labeling and tracking of membrane protein molecules, and finally uses the median value of the trajectory displacement length in a 200nm space to represent the speed of molecular movement. Although this method improves the molecular imaging density compared to sptPALM, it will reduce the accuracy of trajectory reconstruction. In addition, this method can only be used to label membrane proteins.

[0012] In addition, there is SMdM (Single-molecule displacement mapping), which arranges the order of stroboscopic exposure between two frames so that 1ms of stroboscopic exposure is distributed between two frames at an interval of 1ms, which can further shorten the time interval between two frames and track molecular motion faster. This method collects the trajectory displacement between two frames and obtains the estimated diffusion coefficient by fitting the distribution of all displacements within a fixed range (100×100 nm).

[0013] This method is useful for fast-moving molecules (diffusion coefficients above 10 μm2 / s) If there is no specific binding target fluorescent protein, the diffusion coefficient distribution in the cell has a good measurement effect. However, through simulation data, it is found that for the diffusion coefficient in the cell nucleus less than 10μm 2 / s transcriptional regulatory factors, when considering the positioning error of the fluorescent molecules in the imaging, will affect the detection sensitivity of this part of the movement speed, because at a time interval of 1ms, the diffusion coefficient is 10μm 2 The average displacement of molecules within 100 nm is within 200 nm. When there is a 30 nm error for each molecule at the beginning of the trajectory, the total error range of 60 nm will have a significant impact on the detection results.

[0014] Therefore, how to solve the problem that traditional particle trajectory tracking technology has low accuracy in predicting diffusion quantification characteristics in a high labeling density environment and cannot capture the spatial distribution of particles is an important issue that needs to be urgently solved in the field of super-resolution imaging. Summary of the invention

[0015] The present invention provides a single-particle diffusion quantification feature prediction method, device, electronic device and storage medium, which are used to solve the defects of traditional single-particle trajectory tracking technology, such as low diffusion quantification feature prediction accuracy in a high labeling density environment and inability to capture the spatial distribution of particles. The method can quickly and accurately predict the diffusion quantification features of a large number of single particles under high labeling density and significantly reduce phototoxicity.

[0016] On the one hand, the present invention provides a method for predicting the quantitative characteristics of diffusion of a single particle, comprising: obtaining a target input image corresponding to a target single particle; predicting a particle pseudo-trajectory image according to the target input image based on a pre-trained particle trajectory prediction model; and calculating the quantitative characteristics of diffusion of the target single particle based on the particle pseudo-trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

[0017] Furthermore, the method of obtaining a target input image corresponding to a target single particle includes: collecting an original single-particle motion blurred image of the target single particle; segmenting the original single-particle motion blurred image based on a pre-trained U-Net network to obtain a single-particle signal mask; pre-positioning the single-particle signal in the original single-particle motion blurred image, retaining a target single-particle signal mask containing only one positioning; filling the target single-particle signal mask with background and noise levels calculated using a median filter to obtain the target input image; wherein the U-Net network is obtained by training and optimizing a second training sample set consisting of a single-particle motion blurred image and its corresponding mask image.

[0018] Furthermore, the diffusion quantification characteristics of the target single particle include a diffusion coefficient and a diffusion direction; accordingly, the diffusion quantification characteristics of the target single particle are calculated based on the particle pseudo-trajectory image, including: determining a pseudo-trajectory area according to the particle pseudo-trajectory image; fitting the diffusion coefficient corresponding to the target single particle according to the quantitative relationship between the pseudo-trajectory area and the particle diffusion coefficient; and fitting the diffusion direction corresponding to the target single particle according to the density spatial distribution of the particle pseudo-trajectory image.

[0019] Furthermore, the particle trajectory prediction model is trained and optimized, specifically including: simulating a single-particle motion blurred image, and obtaining a particle motion trajectory image corresponding to the single-particle motion blurred image, so as to construct a first training sample set; using the single-particle motion blurred image as a model input, using the predicted pseudo-trajectory image as a model output, using the difference between the predicted pseudo-trajectory image and the particle motion trajectory image as a training loss, iteratively optimizing the particle trajectory prediction model, and obtaining a particle trajectory prediction model with converged training.

[0020] Furthermore, the simulated single-particle motion blurred image includes: simulating a two-dimensional particle trajectory and superimposing a Gaussian function on each point on the two-dimensional particle trajectory to obtain a motion blurred function; normalizing and pixelating the motion blurred function, and introducing Gaussian white noise and Poisson shooting noise to obtain a single-particle motion blurred image under different signal-to-noise ratios and background levels.

[0021] Furthermore, the diffusion coefficient corresponding to the target single particle is obtained by fitting, and then the method includes: when the diffusion coefficient of the target single particle is greater than a set threshold, performing centroid calculation on the particle pseudo trajectory image to obtain the location of the target single particle; when the diffusion coefficient of the target single particle is less than or equal to the set threshold, performing ellipsoid Gaussian fitting on the target input image to obtain the location of the target single particle.

[0022] Further, the location of the target single particle is obtained, and then the method includes: gridding the location of the target single particle to obtain a first grid containing the particle location, and a second grid that does not contain the particle location but is adjacent to the particle location; generating a particle density probability map based on the location of the target single particle; interpolating the second grid using the sum of Gaussian weights based on the particle density probability map and the first grid to obtain a diffusion coefficient corresponding to the second grid; obtaining a diffusion coefficient matrix based on the diffusion coefficient corresponding to the first grid and the diffusion coefficient corresponding to the second grid; locally smoothing the diffusion coefficient matrix to obtain a motility map, and displaying it using an HSV color map to complete image rendering.

[0023] In a second aspect, the present invention also provides a single-particle diffusion quantitative feature prediction device, comprising: a target input image acquisition module, used to acquire a target input image corresponding to a target single particle; a particle pseudo-trajectory image prediction module, used to predict a particle pseudo-trajectory image according to the target input image based on a pre-trained particle trajectory prediction model; a particle diffusion coefficient acquisition module, used to calculate the diffusion quantitative feature of the target single particle based on the particle pseudo-trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

[0024] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a single particle diffusion quantitative characteristic prediction method as described in any one of the above is implemented.

[0025] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for predicting the quantitative characteristics of single-particle diffusion.

[0026] The single particle diffusion quantification feature prediction method provided by the present invention obtains the target input image corresponding to the target single particle, and based on the pre-trained particle trajectory prediction model, predicts the particle pseudo trajectory image according to the target input image, and then calculates the diffusion quantification feature of the target single particle based on the particle pseudo trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing the first training sample set composed of the single particle motion blur image and its corresponding particle motion trajectory image. The method predicts the actual motion path of the target single particle from the target input image through the particle trajectory prediction model, and calculates the diffusion quantification feature of the target single particle based on this, which can quickly and accurately predict the diffusion quantification feature of a large number of single particles in a high labeling density environment, and significantly reduces the phototoxicity, making it suitable for at least 10 minutes of live cell imaging, and the diffusion quantification feature obtained by solving has a high dynamic range, effectively distinguishes the diffusion characteristics of different biological particles, and provides a powerful tool for the study of particle dynamics in diffusion systems and living cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is a schematic diagram of the flow of a method for predicting quantitative characteristics of single particle diffusion provided by an embodiment of the present invention.

[0029] Figure 2 It is a schematic diagram of the structure of the U-Net network provided by an embodiment of the present invention.

[0030] Figure 3 It is a schematic diagram of constructing a second training sample set of the single particle diffusion quantitative characteristic prediction method provided by an embodiment of the present invention.

[0031] Figure 4 It is a schematic diagram of constructing a first training sample set provided by an embodiment of the present invention.

[0032] Figure 5 Schematic diagram of training optimization of the particle trajectory prediction model provided by an embodiment of the present invention.

[0033] Figure 6 It is a schematic diagram of obtaining the diffusion coefficient of a target single particle provided by an embodiment of the present invention.

[0034] Figure 7 It is a schematic diagram of the fitting effect of the linear relationship between the pseudo-trajectory area and the particle diffusion coefficient provided by an embodiment of the present invention.

[0035] Figure 8 It is a schematic diagram of the positioning accuracy of different particle positioning algorithms provided by the embodiments of the present invention.

[0036] Fig. 9 It is a rendering schematic diagram of particle positioning and particle diffusion coefficient provided by an embodiment of the present invention.

[0037] Fig.10 It is a schematic diagram of the overall process of the method for predicting the quantitative characteristics of single-particle diffusion provided by an embodiment of the present invention.

[0038] Fig.11 It is a schematic diagram of the inference effect of the particle trajectory prediction model and the U-Net network provided in an embodiment of the present invention.

[0039] Fig.12 It is a schematic diagram of the image registration process provided by an embodiment of the present invention.

[0040] Fig.13 It is a schematic diagram of the structure of a single particle diffusion quantitative characteristic prediction device provided by an embodiment of the present invention.

[0041] Fig.14 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It is easy to understand that in order to solve the problem that the traditional single particle trajectory tracking technology has low diffusion coefficient prediction accuracy in a high labeling density environment and cannot capture the spatial distribution of particles, the present invention proposes a new single particle diffusion quantitative feature prediction method. Specifically, Figure 1 A schematic diagram of the flow of a method for predicting quantitative diffusion characteristics (eg, diffusion coefficient) of a single particle (eg, biological molecule) provided in an embodiment of the present invention is shown.

[0044] like Figure 1 As shown, the method includes steps S110-S130, and steps S110-S130 and related steps will be described in detail below.

[0045] S110, obtaining a target input image corresponding to a target single particle.

[0046] It is understandable that in order to make a single particle (such as a single biomolecule) visible under a microscope, it is first necessary to attach a fluorescent tag to the target single particle. These fluorescent tags can emit light when irradiated with a specific wavelength, making the particle visible. Common fluorescent labeling methods include antibody labeling, genetic engineering expression of fusion proteins with fluorescent proteins (such as GFP), etc.

[0047] Since single particles (target single particles) are usually very weak, a highly sensitive and high-resolution microscope system is required. Commonly used single-particle imaging microscopy systems include but are not limited to photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM).

[0048] In order to observe single particles (target single particles), the intensity and wavelength of the excitation light need to be precisely controlled to avoid exciting multiple adjacent fluorescent particles at the same time. In addition, a highly sensitive detector such as an EMCCD camera or APD (avalanche photodiode) is required to capture extremely weak fluorescence signals.

[0049] After all parameters are set, start acquiring images. Long exposure times or accumulation of multiple exposures are usually required to collect enough signal.

[0050] It should be noted that the collected image needs to be processed to obtain a clearer single particle image, that is, the target input image described in this embodiment. The processing here includes but is not limited to steps such as background subtraction and signal enhancement.

[0051] In this step, the target single particle can be a biological molecule (such as DNA, RNA, protein or other molecular structures within cells, especially protein), or a biological molecule such as a protein purified in a chemical solution, a chemically synthesized compound, a nanoparticle, etc., without specific limitation here.

[0052] The target input image is a processed single-particle motion blurred image / single-particle fluorescence image. Single-particle motion blurred image refers to the blurring phenomenon in the image caused by the rapid movement of a single particle during single-particle fluorescence imaging. This blurring usually occurs when tracking single particles in real time in living cells, because particles in cells are often in a state of constant motion.

[0053] On the basis of acquiring the target input image corresponding to the target single particle in step S110, further, step S120 is performed.

[0054] S120, based on a pre-trained particle trajectory prediction model, predict a particle pseudo trajectory image according to the target input image; wherein the particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

[0055] The particle trajectory prediction model is used to recover, predict or reconstruct the particle's motion trajectory (i.e., particle pseudo-trajectory image) from a blurred single particle image (target input image) to more accurately predict the behavior of a single particle of the target.

[0056] The particle trajectory prediction model is constructed based on a deep learning neural network. The deep learning neural network here may include a convolutional neural network, a recurrent neural network, or an architecture that combines the advantages of both, which is not specifically limited here.

[0057] In a specific embodiment, the particle trajectory prediction model includes a convolutional feature extraction layer, a maximum pooling layer, a bilinear upsampling layer, a normalization layer, and an activation function layer.

[0058] After determining the architecture of the particle trajectory prediction model, it is necessary to construct a first training sample set to train and optimize the particle trajectory prediction model. Specifically, during the training optimization, the single particle motion blurred image in the first training sample set is used as the model input, the predicted pseudo trajectory image is used as the model input, and the difference between the predicted pseudo trajectory image and the particle motion trajectory image is used as the training loss. The particle trajectory prediction model is iteratively optimized, thereby obtaining a trained particle trajectory prediction model.

[0059] The single particle motion blurred image may be real data obtained through experiments or data generated through simulation, which is not specifically limited here.

[0060] The particle motion trajectory image is the real motion trajectory image of biological particles, which can be obtained through high-precision imaging technology or manual annotation to serve as the "true value" for model training.

[0061] After the particle trajectory prediction model is trained, the target input image obtained in step S110 is input into the particle trajectory prediction model to obtain an output particle pseudo trajectory image of a target single particle.

[0062] Based on the particle pseudo-trajectory image predicted in step S120, step S130 is further performed.

[0063] S130, calculating and obtaining the diffusion quantitative characteristics of the target single particle based on the particle pseudo trajectory image.

[0064] Specifically, in order to simplify the problem, it can be assumed that there is less overlap in the particle trajectories. Therefore, when the particle pseudo-trajectory image is obtained, the area covered by the particle trajectory can be estimated, that is, the trajectory area. Then, according to the quantitative relationship between the pseudo-trajectory area and the particle diffusion coefficient, the diffusion coefficient of the target single particle is fitted. At the same time, according to the density spatial distribution of the particle pseudo-trajectory image, the diffusion direction corresponding to the target single particle can be fitted.

[0065] The quantitative relationship between the pseudo-track area and the particle diffusion coefficient is predetermined. When applied, the pseudo-track area can be substituted into the equation corresponding to the linear relationship. The diffusion coefficient and diffusion direction are both quantitative diffusion features.

[0066] The diffusion coefficient of a single target particle is a physical quantity that describes the random movement rate of a single target particle in a medium. It reflects the free diffusion ability of a single target particle without external forces and is represented by the symbol D. The unit of the diffusion coefficient is usually or , which represents the area covered by particle diffusion per unit time.

[0067] It is worth mentioning that the single particle diffusion quantitative characteristic prediction method provided in this embodiment has a high dynamic range within the common diffusion coefficient range of large biological particles, and can effectively distinguish the diffusion characteristics of different particles, which provides a powerful tool for the dynamic study of living cell particles.

[0068] In this embodiment, the target input image corresponding to the target single particle is obtained, and based on the pre-trained particle trajectory prediction model, a particle pseudo-trajectory image is predicted according to the target input image, and then the diffusion quantification feature of the target single particle is calculated based on the particle pseudo-trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing the first training sample set consisting of the single particle motion blurred image and its corresponding particle motion trajectory image. This method predicts the actual motion path of the target single particle from the target input image through the particle trajectory prediction model, and calculates the diffusion quantification feature of the target single particle based on this, which can quickly and accurately predict the diffusion quantification feature of a large number of single particles in a high labeling density environment, and significantly reduces the phototoxicity, making it suitable for at least 10 minutes of live cell imaging, and the diffusion coefficient obtained by solving has a high dynamic range, which effectively distinguishes the diffusion characteristics of different biological particles, and provides a powerful tool for the dynamic study of living cell particles.

[0069] On the basis of the above-mentioned embodiment, the process of acquiring the target input image will be described in detail below.

[0070] Obtaining a target input image corresponding to a target single particle includes: collecting an original single-particle motion blurred image of the target single particle; segmenting the original single-particle motion blurred image based on a pre-trained U-Net network to obtain a single-particle signal mask; pre-positioning the single-particle signal in the original single-particle motion blurred image, retaining a target single-particle signal mask containing only one positioning; filling the target single-particle signal mask with background and noise levels calculated using a median filter to obtain a target input image; wherein the U-Net network is obtained by training and optimizing a second training sample set consisting of the single-particle motion blurred image and its corresponding mask image.

[0071] It is understandable that, unlike typical single particle detection algorithms based on positioning, after capturing a single particle motion blurred image, it is necessary to segment the pixels containing a single particle. This embodiment uses a U-Net network to accomplish this task.

[0072] Figure 2 FIG. 1 shows a schematic diagram of the structure of the U-Net network provided by an embodiment of the present invention. Figure 2 As shown in Figure 1, the U-Net network consists of a contraction path (left side) and an expansion path (right side).

[0073] The contraction path follows the typical architecture of a convolutional network, which consists of two repeated applications of a 3×3 convolution (non-padded convolution), followed by a ReLU activation function and a 2×2 max pooling operation with a stride of 2 for downsampling. In each downsampling step, the number of feature channels is doubled.

[0074] Each step in the dilation path consists of upsampling the feature map, followed by a 2×2 convolution (“upconvolution”) that halves the number of feature channels, concatenation with the corresponding cropped feature map in the contraction path, and two 3×3 convolutions, each followed by a ReLU activation function. Cropping is necessary because each convolution loses boundary pixels.

[0075] In the last layer, a 1×1 convolution is used to map each 64-dimensional feature vector to the required number of classes (here only 1 class, corresponding to the pixel range of a single particle motion blurred image). The U-Net network has a total of 23 convolutional layers.

[0076] Before practical application, it is necessary to construct a second training sample set to train and optimize the U-Net network. Each training sample in the constructed second training sample set includes a single particle motion blurred image and its corresponding mask image. Specifically, Figure 3 A schematic diagram of constructing a second training sample set of the single particle diffusion quantitative characteristic prediction method provided by an embodiment of the present invention is shown.

[0077] like Figure 3 As shown, first, the simSPT algorithm is used to simulate the two-dimensional particle trajectory (2D particle trajectory), and a Gaussian function is superimposed on each point on the two-dimensional particle trajectory to obtain the motion modulus function Traj i,j .

[0078] The motion blur function is then normalized and pixelated to align with the actual camera pixel size (e.g., 110 nm), yielding the pixelated Traj i,j , then, to pixelation Traj i,j Gaussian white noise and Poisson shot noise are introduced to generate single particle motion blurred images at different signal-to-noise ratios and background levels.

[0079] At the same time, from pixelation Traj i,j The signal intensity area of ​​more than 95% is intercepted as a mask to obtain a mask image. Thus, according to the single particle motion blurred image and its corresponding mask image, the second training sample set can be constructed.

[0080] Then, the second training sample set is used to train and optimize the U-Net network. Specifically, during training, the single particle motion blurred image is used as the model input, the predicted mask image is used as the model input, and the difference between the predicted mask image and the (actual) mask image is used as the training loss, and the U-Net network is iteratively optimized, thereby obtaining a trained U-Net network.

[0081] It is worth mentioning that in order to increase the density of segmented particles during training, this embodiment places four single-particle motion blurred images in a 32×32 pixel wide image (pixel size 110 nanometers) and uses weights The cross entropy loss function is updated to force the U-Net network to learn the separation boundaries between single particle motion blurs.

[0082] Specifically, the weight graph of the U-Net network can be calculated as follows (1).

[0083] .

[0084] In formula (1), is a weight map to balance the class frequencies (i.e. the ratio of the number of pixels between instances with motion blur detection and instances without motion blur detection), is the distance to the nearest motion blur boundary, is the distance to the second closest motion blur boundary. In the experiment, we set Pixels and Pixel.

[0085] Therefore, in this embodiment, the maximum detection density that the U-Net network can achieve is about 0.5 motion blurred particle signal μm² / frame or 16.5 motion blurred particle signals μm² / s (equivalent to placing 4 motion blurs in a 32×32 pixel area with a pixel size of 110 nm), which is comparable to the localization density of classic live cell PALM (Photoactivated Localization Microscopy) and more than 12 times higher than the localization density of fast blinking single particle tracking.

[0086] In a specific embodiment, in order to train the U-Net network, 10320 pairs of single particle motion blurred images and mask images were simulated, with an exposure time of 30 milliseconds, covering 12 different diffusion coefficients D (ranging from 0.01μm² / s to 10μm² / s). At each diffusion coefficient D, single particle motion blurred images were randomly generated to simulate actual imaging data, with a signal-to-noise ratio between 19-35dB and a background level between 1000-3000. Dropout layers and random elastic deformations (rotation, translation, and flipping) were included in the data enhancement to improve the generalization ability of network training.

[0087] After the U-Net network is trained, it is used to segment pixels containing individual particles to finally obtain the target input image.

[0088] Specifically, first, an original single-particle motion blurred image of a target single particle is collected, which can be collected by TIRF (Total Internal Reflection Fluorescence Microscopy) or HILO (Highly Inclined Light Optical Sheet).

[0089] Then, the acquired original single-particle motion blurred image is input into the trained U-Net network to generate a single-particle signal mask covering the single-particle signal.

[0090] Then, in order to remove the mask covering multiple single-particle signals, ThunderSTORM is used to pre-position the single-particle signals in the original single-particle motion blurred image, and only the single-particle signal mask containing one positioning is retained, namely the target single-particle signal mask.

[0091] Next, a median filter is used to calculate the background and noise levels of each pixel in the target single particle signal mask within the previous and next 250 frames. Thus, the target single particle signal mask is filled with the calculated background and noise levels to obtain the target input image.

[0092] In this embodiment, the original single-particle motion blurred image of the target single particle is collected, and based on the pre-trained U-Net network, the original single-particle motion blurred image is segmented to obtain a single-particle signal mask, and then the single-particle signal in the original single-particle motion blurred image is pre-positioned, and the target single-particle signal mask containing only one positioning is retained, and the target single-particle signal mask is filled with the background and noise levels calculated by the median filter to obtain the target input image, so that the later particle trajectory prediction model can predict a particle pseudo-trajectory image with higher accuracy.

[0093] On the basis of the above-mentioned embodiments, the training optimization process of the particle trajectory prediction model will be described in detail below.

[0094] It is understandable that before using the particle trajectory prediction model to predict the particle pseudo-trajectory image, it is necessary to train and optimize the particle trajectory prediction model.

[0095] The particle trajectory prediction model is trained and optimized, specifically including: simulating a single-particle motion blurred image, and obtaining a particle motion trajectory image corresponding to the single-particle motion blurred image, so as to construct a first training sample set; using the single-particle motion blurred image as a model input, using the predicted pseudo-trajectory image as a model output, using the difference between the predicted pseudo-trajectory image and the particle motion trajectory image as a training loss, iteratively optimizing the particle trajectory prediction model, and obtaining a particle trajectory prediction model with training convergence.

[0096] Figure 4 FIG. 2 shows a schematic diagram of constructing a first training sample set provided by an embodiment of the present invention. Figure 4 As shown, first, the simSPT algorithm is used to simulate the two-dimensional particle trajectory (2D particle trajectory), and a Gaussian function is superimposed on each point on the two-dimensional particle trajectory to obtain the motion modulus function Traj i,j .

[0097] The motion blur function is then normalized and pixelated to align with the actual camera pixel size (e.g., 110 nm), yielding the pixelated Traj i,j , then, to pixelation Traj i,j Gaussian white noise and Poisson shooting noise are introduced to generate single particle motion blurred images at different signal-to-noise ratios and background levels. The size of the single particle motion blurred image is 32×32 pixels.

[0098] In order to pair the particle motion trajectories with the single particle motion blurred images for subsequent training, the Bresenham line algorithm is first applied to the simulated two-dimensional particle trajectories to generate a pixelated image with a width of 320 × 320 pixels, and then the pixelated image is convolved with a Gaussian filter of the same pixel width to generate the final trajectory image, i.e., the particle motion trajectory image.

[0099] Therefore, the first training sample set can be constructed based on the single particle motion blurred image and the particle motion trajectory image.

[0100] Then, the particle trajectory prediction model is iteratively optimized using the first training sample set. Specifically, Figure 5 A schematic diagram of training optimization of the particle trajectory prediction model provided by an embodiment of the present invention is shown.

[0101] according to Figure 5 It can be seen that during training, the noisy 32×32 pixel single-particle motion blurred image (pixel size 110 nanometers) is first enlarged to 320×320 pixels (pixel size 11 nanometers) using nearest neighbor interpolation as the input image for training.

[0102] The input image is encoded through three convolutional feature extraction layers (each convolutional feature extraction layer contains 3×3 convolution kernels, and the output channels are 32, 64, 128, and 512, respectively), followed by batch normalization, ReLU activation function, and maximum pooling layer (2×2, stride 2). Subsequently, it undergoes three bilinear upsampling layers (interpolation with 2x upsampling) and convolutional feature extraction layers (each layer has 3×3 convolution kernels, and the output channels are 128, 64, 32, followed by batch normalization and ReLU activation function) to upsample and decode features layer by layer. Finally, a 1×1 convolution layer without activation function is used to convert the 32-channel feature map into a single-channel density prediction image, thereby obtaining the predicted pseudo-trajectory image predicted by the particle trajectory prediction model from the single particle motion blurred image (image size 320×320, pixel size 11 nanometers).

[0103] Then, the difference between the predicted pseudo-trajectory image and the true trajectory image (i.e., the particle motion trajectory image in the first training sample set) is balanced by the sum of the mean absolute error loss function (L1 loss) and the mean square error loss function (MSE), and the weight parameters of the particle trajectory prediction model are continuously and iteratively optimized.

[0104] In a specific embodiment, the exposure time is 30 milliseconds, covering a wide distribution of protein diffusion coefficients in living cells (a total of 27 diffusion coefficients, exponentially increasing from 0.001 μm² / s to 31.62 μm² / s). At each diffusion coefficient D, 3000 single particle motion blurred images are randomly generated to simulate actual imaging data, with a signal-to-noise ratio between 19-35 dB and a background level between 1000-3000. The total number of image pairs (single particle motion blurred images and particle motion trajectory images) in the training sample set is 81,000. During network training, the sum of the mean absolute error and the mean squared error is used as the loss function to quantify and reduce the difference between the predicted pseudo-trajectory images and the particle motion trajectory images.

[0105] After training and optimizing the particle trajectory prediction model, in actual application, the acquired target input image is directly input into the particle trajectory prediction model to predict the corresponding accurate particle pseudo-trajectory image.

[0106] In this embodiment, the particle trajectory prediction model is optimized by training, and then based on the particle trajectory prediction model, a particle pseudo trajectory image is predicted according to the target input image, and then based on the particle pseudo trajectory image, the diffusion quantification characteristics of the target single particle are calculated; wherein the particle trajectory prediction model is obtained by training and optimizing the first training sample set consisting of a single particle motion blur image and its corresponding particle motion trajectory image. This method predicts the actual motion path of the target single particle from the target input image through the particle trajectory prediction model, and calculates the diffusion quantification characteristics of the target single particle based on this, which can quickly and accurately predict the diffusion quantification characteristics of a large number of single particles in a high labeling density environment, and significantly reduces phototoxicity, making it suitable for at least 10 minutes of live cell imaging, and the diffusion coefficient obtained by solving has a high dynamic range, which effectively distinguishes the diffusion characteristics of different biological particles, and provides a powerful tool for the dynamic study of living cell particles.

[0107] On the basis of the above-mentioned embodiment, the process of calculating the diffusion coefficient according to the particle pseudo-trajectory image will be described in detail below.

[0108] The quantitative diffusion characteristics of the target single particle include the diffusion coefficient and the diffusion direction; accordingly, based on the particle pseudo-trajectory image, the diffusion coefficient of the target single particle is calculated, including: determining the pseudo-trajectory area according to the particle pseudo-trajectory image; fitting the diffusion coefficient corresponding to the target single particle according to the quantitative relationship between the pseudo-trajectory area and the particle diffusion coefficient; fitting the diffusion direction corresponding to the target single particle according to the density spatial distribution of the particle pseudo-trajectory image.

[0109] It can be understood that D is estimated by using the particle pseudo-trajectory image, assuming that the target single particle undergoes Brownian motion and moves to the position after a time interval Δτ. The probability of finding the target single particle can be described by the following equation (2).

[0110] .

[0111] In formula (2), represents the displacement of the target single particle from the origin at time t, represents the probability density of the displacement, and D is the diffusion coefficient of the target single particle.

[0112] In single particle tracking, the trajectory displacement can be obtained and fitted to equation (2) to extract the diffusion coefficient D. However, the particle pseudo-trajectory image predicted by the particle trajectory prediction model is a spatial projection of the trajectory probability and has no temporal order. Therefore, we turn to deriving the diffusion coefficient D by the travel distance / trajectory distance of the target single particle instead of the displacement.

[0113] By integrating equation (2), the probability density of the target single particle jumping distance r per unit time can be obtained from equation (2), as shown in the following equation (3).

[0114] .

[0115] The total distance of the particle trajectory can be approximated as the sum of N discrete jump lengths with a time interval of t within the exposure time of the entire trajectory. The probability density of each jump length is determined by equation (3), assuming that each step is an independent and identically distributed event. Since the diffusion coefficient D should be related to the square of the total distance R of the particle movement, the total expectation R² can be written as follows equation (4).

[0116] .

[0117] After decomposing the total distance R of the particle movement into N consecutive jumps, equation (4) can be rewritten as follows: equation (5).

[0118] .

[0119] because , combined with equation (3), we can get . Combined , substituting into equation (5) we get the following equation (6).

[0120] .

[0121] In order to simplify the problem, this embodiment assumes that the particle motion trajectories themselves rarely overlap, and the particle pseudo-track image predicted by the particle trajectory prediction model can be regarded as a trajectory convolved with a narrow width w, which makes it possible to estimate the area covered by the particle pseudo-track image (pseudo-track area, abbreviated as PT area or pseudo-track area), as shown in the following formula (7).

[0122] .

[0123] In formula (7), Represents the total trajectory distance of a single target particle.

[0124] Since w represents the uncertainty of the particle trajectory prediction model when predicting the particle trajectory (a constant after training), in this embodiment, by replacing R in equation (6) with equation (7), a quantitative relationship between the square of the pseudo trajectory area and the particle diffusion coefficient D can be obtained. For details, see the following equation (8).

[0125] .

[0126] in, .

[0127] Figure 6 FIG. 2 shows a schematic diagram of obtaining the diffusion coefficient of a target single particle provided by an embodiment of the present invention. Figure 6 As shown in Figure 1, when calculating the diffusion coefficient of a target single particle, the numerical range of the particle pseudo-trajectory image is first linearly adjusted to 0 to 1, and the pseudo-trajectory area is defined as the pixel area exceeding 0.1. Then, the diffusion coefficient corresponding to the target single particle can be fitted using equation (8).

[0128] In a specific embodiment, 1700 single particle motion blurred images at high signal-to-noise ratio (35 dB) were simulated, with diffusion coefficients D ranging from 0.1 μm² / s to 31.62 μm² / s. Figure 7 The figure shows the fitting effect of the quantitative relationship between the pseudo-trajectory area and the particle diffusion coefficient provided by the embodiment of the present invention. Figure 7 It can be seen that by fitting the pseudo-trajectory area and the diffusion coefficient D using equation (8), an excellent determination coefficient (R 2 >0.9).

[0129] Therefore, equation (8) fitted at a high signal-to-noise ratio can be used to convert the particle pseudo-trajectory image predicted by the particle trajectory prediction model into the diffusion coefficient of the motion blurred image of a single particle.

[0130] In this embodiment, the pseudo-track area is determined according to the particle pseudo-track image, and the diffusion coefficient corresponding to the target single particle is fitted according to the quantitative relationship between the pseudo-track area and the particle diffusion coefficient. This method predicts the actual movement path of the target single particle from the target input image through the particle trajectory prediction model, and calculates the diffusion coefficient of the target single particle based on this. It can quickly and accurately predict the diffusion coefficient of a large number of single particles in a high labeling density environment, and significantly reduces phototoxicity, making it suitable for at least 10 minutes of live cell imaging. The diffusion coefficient obtained has a high dynamic range, which can effectively distinguish the diffusion characteristics of different biological particles, and provides a powerful tool for the dynamic study of live cell particles.

[0131] On the basis of the above-mentioned embodiments, the process of particle positioning will be further described in detail below.

[0132] The diffusion coefficient of the target single particle is calculated, and then the method includes: when the diffusion coefficient of the target single particle is greater than a set threshold, the centroid of the particle pseudo trajectory image is calculated to obtain the location of the target single particle; when the diffusion coefficient of the target single particle is less than or equal to the set threshold, the target input image is fitted with an ellipsoid Gaussian to obtain the location of the target single particle.

[0133] It can be understood that in this embodiment, a series of particle motion trajectories and corresponding single particle motion blurred images are simulated first, covering different diffusion coefficients D and signal-to-noise ratios. For details, see Figure 8 , Figure 8 The schematic diagram of the positioning accuracy of different particle positioning algorithms provided by the embodiments of the present invention is shown. Compared with using the elliptical Gaussian function to locate particles, when the diffusion coefficient D exceeds 1μm² / s, the centroid algorithm used on the particle pseudo-trajectory image shows higher positioning accuracy (the positioning accuracy is evaluated by comparing the deviation between the particle positioning obtained by the two methods and the geometric center of the actual particle motion trajectory).

[0134] It is noteworthy that for fast diffusing particles with diffusion coefficients D exceeding 10 μm² / s, the centroid algorithm achieves an accuracy of 20 nm, while the Gaussian function fit achieves 85 nm.

[0135] On the contrary, for slow-diffusing particles with a diffusion coefficient D lower than 1 μm² / s, using an elliptical Gaussian function to locate particles performs better than the centroid algorithm. Therefore, for downstream analysis, when the estimated diffusion coefficient D exceeds 1 μm² / s, this embodiment uses the weighted centroid of the particle pseudo-track image to locate the position of the particle, otherwise an elliptical Gaussian fit to the pre-processed original motion blurred image (target input image) is used.

[0136] In other words, the diffusion coefficient of the target single particle determines the method used to obtain the positioning of the target single particle.

[0137] Therefore, in this embodiment, the diffusion coefficient of the target single particle is compared with the set threshold. For the target single particle whose diffusion coefficient is greater than the set threshold, the center of mass is calculated according to the particle pseudo trajectory image predicted by the particle trajectory prediction model, and the position of the center of mass is used as the positioning of the target single particle; for the target single particle whose diffusion coefficient is less than or equal to the set threshold, the target input image is fitted with an ellipsoid Gaussian, and the position obtained by fitting is used as the positioning of the target single particle.

[0138] The threshold value can be set according to actual needs and is not specifically limited here. For example, in a specific embodiment, the threshold value is set to 1. .

[0139] In this embodiment, when the diffusion coefficient of the target single particle is greater than the set threshold, the centroid of the particle pseudo-trajectory image is calculated to obtain the location of the target single particle. When the diffusion coefficient of the target single particle is less than or equal to the set threshold, the target input image is fitted with an ellipsoid Gaussian to obtain the location of the target single particle. This method can simultaneously capture the spatial location of particles and the distribution of diffusion coefficients, can quickly and accurately predict the diffusion coefficients of a large number of single particles in a high labeling density environment, and significantly reduce phototoxicity, making it suitable for at least 10 minutes of live cell imaging. The diffusion coefficient obtained has a high dynamic range, effectively distinguishes the diffusion characteristics of different biological particles, and provides a powerful tool for the dynamic study of living cell particles.

[0140] On the basis of the above embodiments, the image rendering process will be described in detail below.

[0141] The positioning of the target single particle is obtained, and then the following steps are included: gridding the positioning of the target single particle to obtain a first grid containing the particle positioning, and a second grid that does not contain the particle positioning but is adjacent to the particle positioning; generating a particle density probability map based on the positioning of the target single particle; interpolating the second grid using the sum of Gaussian weights based on the particle density probability map and the first grid to obtain a diffusion coefficient corresponding to the second grid; obtaining a diffusion coefficient matrix based on the diffusion coefficient corresponding to the first grid and the diffusion coefficient corresponding to the second grid; locally smoothing the diffusion coefficient matrix to obtain a motility map, and displaying it using an HSV color map to complete image rendering.

[0142] According to the above embodiments, the location and diffusion coefficient D of the target single particle can be obtained. In order to intuitively show the spatial distribution and correlation between the spatial organization and diffusivity of the particles, this embodiment develops an image rendering algorithm that can generate a particle density probability map (PALM), a diffusion coefficient map (Mobility map), and a particle diffusion density map (MPALM) that integrates the particle density and diffusion coefficient in the same data set.

[0143] Specifically, first, a total of M pixels are placed in a square pixel grid of a given size. total The locations of the particles are gridded and summed to obtain I count . Select M total The positioning of each particle and the rendering pixel size are determined to achieve the required spatial resolution at the Nerquist sampling frequency.

[0144] Then, I count With the Gaussian kernel (σ x,y = mean localization error) to generate the PALM image I density , visualized using the hot-red colormap.

[0145] Then, M total The positioning of the particles is again similar to I count The grids are merged in the same way and the average diffusion coefficient is calculated for each merged grid. For the second grid (Draw) that does not contain particle positioning but is adjacent to the first grid (Draw) with particle positioning null ), using the Gaussian weighted sum for the second grid D null The specific interpolation formula can be found in the following formula (9).

[0146] .

[0147] In formula (9), is the diffusion coefficient corresponding to the second grid obtained by interpolation, and are the pixel coordinates of the first grid where the particles are located, and are the pixel coordinates of the second grid that is not located but adjacent to the first grid. The interpolated result matrix (i.e., the diffusion coefficient matrix) is labeled I D Then, the diffusion coefficient matrix I D Perform local smoothing to obtain the motion graph I smoothed D , the kinematic diagram can be seen in the following formula (10).

[0148] .

[0149] In order to integrate the information of particle localization density and diffusion coefficient D, this embodiment selects the HSV color map to generate the MPALM image. The value of the diffusion coefficient map is mapped to the hue channel (Hue). density Mapped to the Value channel, while the Saturation channel is set to 0 when the grain density is zero, and to 1 otherwise.

[0150] The image rendering results can be found in Fig. 9 , Fig. 9 A rendering schematic diagram of particle positioning and particle diffusion coefficient provided by an embodiment of the present invention is shown. Fig. 9 It includes particle density probability map (PALM), diffusion coefficient map (Mobility map) and particle diffusion density map (MPALM).

[0151] In this embodiment, by performing image rendering on the positioning and diffusion coefficient of a target single particle, a particle density probability map, a diffusion coefficient map, and a particle diffusion density map are obtained, which can more intuitively show the spatial distribution and correlation between the particle spatial organization and diffusivity.

[0152] In addition, taking the prediction of the diffusion coefficient and spatial localization of a single particle as an example, Fig.10 The figure shows the overall process diagram of the method for predicting the quantitative characteristics of single-particle diffusion provided by an embodiment of the present invention.

[0153] like Fig.10 As shown, firstly, the original single particle motion blurred image of the target single particle (i.e. Fig.10 The single particle motion blurred signal in the image is then segmented using the pre-trained U-Net network to obtain the single particle signal mask.

[0154] Next, ThunderSTORM is used to pre-position the single particle signal in the original single particle motion blurred image, and a mask of the target single particle signal containing only one positioning is retained.

[0155] Furthermore, the target single particle signal mask is filled with the background and noise levels calculated by the median filter to obtain the target input image.

[0156] Then, the target input image is input into the pre-trained particle trajectory prediction model to predict the output particle pseudo trajectory image.

[0157] Then, the pseudo-track area is calculated based on the particle pseudo-track image, and the diffusion coefficient corresponding to the target single particle is fitted according to the quantitative relationship between the pseudo-track area and the particle diffusion coefficient.

[0158] Finally, when the diffusion coefficient of the target single particle is greater than the set threshold, the centroid of the particle pseudo-trajectory image is calculated to obtain the location of the target single particle; when the diffusion coefficient of the target single particle is less than or equal to the set threshold, the target input image is fitted with an ellipsoid Gaussian to obtain the location of the target single particle.

[0159] Based on the above, the particle space distribution and diffusion coefficient distribution of the target single particle can be captured. It should also be noted that the above steps are described in detail in the above embodiments and will not be repeated here.

[0160] It is worth mentioning that, for the single-particle diffusion quantitative feature prediction method provided in an embodiment of the present invention, in order to verify the robustness of the method, a series of single-particle motion blurred images generated by particle motion trajectories with different diffusion coefficients under different signal-to-noise ratios were calculated and simulated, and then the pseudo-trajectory area obtained for the same particle motion trajectory under different signal-to-noise ratios was tested. The results showed very small fluctuations, indicating the robustness of the particle trajectory prediction model to noise.

[0161] At the same time, the robustness of the U-Net network segmentation under different signal-to-noise ratios was tested, and it was found that the area of ​​the single-particle signal mask generated by the same particle trajectory can also remain stable under different signal-to-noise ratios. Therefore, the particle trajectory prediction model and the U-Net network show strong robustness to different signal-to-noise ratios, which helps to mitigate the errors caused by signal-to-noise ratio fluctuations during imaging, because these errors may cause fluctuations in the estimated diffusion coefficient D.

[0162] The test results of the particle trajectory prediction model and the U-Net network can be found in Fig.11 , Fig.11 A schematic diagram of the inference effect of the particle trajectory prediction model and the U-Net network provided by an embodiment of the present invention is shown.

[0163] In some embodiments, the processes of cell sample preparation and image data acquisition are also described in detail.

[0164] The target protein (i.e., the target single particle) is first labeled with a suitable fluorescent tag, and HaloTag or SNAP-tag is connected to the N-terminus or C-terminus of the target protein. The target protein is then expressed in cells, and the cells are incubated and stained with a high quantum yield fluorescent dye (such as PA-JF646, HM-SiR, etc.) modified with the tag protein ligand. Through a total internal reflection fluorescence microscope, in total internal reflection mode or high-inclined light sheet mode, set an appropriate exposure time (such as 30ms) and shooting density (most adjacent particles do not overlap when observed by naked eye, or the particle density is less than 16.5 motion blurred particle signals / μm² / s), and capture a motion blurred image of a single particle.

[0165] In single-channel MPALM (particle diffusion density map) imaging, 2000 MPALM frames were taken continuously, and one frame of the whole image (widefield image not a single particle image) was connected as one cycle. This cycle was repeated 5 to 20 times depending on the cell viability and particle photobleaching. During each MPALM frame, the excitation light (642 nm) continuously illuminated the excitation dye within the 30 ms camera exposure time. When using a photoactivatable dye (e.g. PA-JF646), a 405 nm laser was pulsed within the 0.5 ms dead time of the camera to activate the photoactivatable dye. For each large-scale image, the excitation laser (560 nm) and exposure time were adjusted to achieve the desired signal-to-noise ratio.

[0166] In dual-channel MPALM-PALM imaging, 560 nm and 642 nm lasers are used simultaneously with a 30 ms camera exposure time. A pulsed 405 nm laser is used with a 0.5 ms camera dead time to activate the photoconvertible proteins in the PALM channel, such as mEosEM. Two cameras are used to detect the excitation light from both channels simultaneously. In addition to this acquisition mode, it is also possible to adjust to MPALM-MPALM dual-channel imaging as required.

[0167] In some embodiments, a detailed description is provided for the image registration process during dual-channel MPALM-PALM imaging.

[0168] For dual-color capture (e.g. dual-color MPALM-PALM), due to the different camera pixel sizes between the two channels, the image of one channel needs to be registered to match the coordinates of the image of the other channel. For details, see Fig.12 , Fig.12 A schematic diagram of the image registration process provided by an embodiment of the present invention is shown.

[0169] The target image was defined as the MPALM channel with a pixel size of 110 nm, while the moving image was defined as the PALM channel with a pixel size of 160 nm. The task was to find a geometric transformation (including rotation, scaling, and displacement) applied to the moving image such that the resulting image had the same spatial coordinates as the target image. Prior to imaging each cell sample, 100 nm TetraSpeck fluorescent microspheres (Invitrogen, T7279) were imaged and used to estimate the transformation matrix for the registration of the two-channel images.

[0170] Initially, the transformation matrix is ​​estimated using the phase correlation between the moving image and the target image. This estimation enables the transformation of the moving image into the initial registration image. Next, the microspheres are localized in the target image, the moving image, and the initial registration image using a two-dimensional Gaussian fitting localization.

[0171] The microsphere localization in the target image is matched with the microsphere localization in the initial registration image. Then, the microsphere localization in the initial registration image is inverted and matched with the moving image. This method facilitates matching the microsphere localization between the target image and the moving image using the localization of the initial registration image as an intermediary guide.

[0172] Subsequently, the paired microsphere locations are used as control points to update the estimate of the transformation matrix, which is then applied to align the two-channel image locations. When aligning two-channel fluorescent microsphere images, this method can achieve an average relative location distance of 10 nm for the same microsphere after image registration.

[0173] In some other embodiments, sample drift correction is described in detail.

[0174] For monochrome MPALM, the sample drift is estimated using the whole image captured between the MPALM images (see the acquisition of the first and second training sample sets above for details). Specifically, the peak of the cross-correlation between the whole images is calculated, and the deviation of the peak from the center of the image is used as an estimate of the sample drift. Then, the sample drift in the entire MPALM image sequence is interpolated using local linear regression, and the positioning of each MPALM frame is corrected to the first frame.

[0175] In single-channel MPALM imaging, sample drift is estimated by analyzing the whole image captured between MPALM frames (see the example of “Cell sample preparation and data acquisition” for details). Specifically, the peak of the cross-correlation between the whole images is calculated, and the deviation of this peak from the image center is used as an estimate of sample drift. Subsequently, the sample drift of the entire MPALM image sequence is interpolated by local linear regression. This corrects the positioning of each MPALM frame to the first frame.

[0176] In dual-channel MPALM-PALM imaging, the PALM image is first aligned to the MPALM image (see the "Image Registration" example for details). Then, the localization of the PALM channel is divided into an appropriate number of boxes, typically 2000 frames per box. The localization within each box is then used to reconstruct a single PALM image. The cross-correlation peak between the reconstructed PALM images is calculated, and the deviation of this peak from the image center is used as an estimate of the sample drift. Subsequently, the sample drift of the entire PALM image sequence is interpolated by local linear regression. Finally, the regression model is applied to correct the localization of each MPALM and PALM frame so that it is aligned with the first frame.

[0177] Corresponding to the single-particle diffusion quantitative characteristic prediction method described in the above embodiments, the present invention also provides a single-particle diffusion quantitative characteristic prediction device, specifically, Fig.13 The schematic diagram shows the structure of a single particle diffusion quantitative characteristic prediction device provided by an embodiment of the present invention.

[0178] like Fig.13 As shown, the device includes: a target input image acquisition module 1310, which is used to acquire a target input image corresponding to a target single particle; a particle pseudo-trajectory image prediction module 1320, which is used to predict a particle pseudo-trajectory image according to the target input image based on a pre-trained particle trajectory prediction model; a particle diffusion coefficient acquisition module 1330, which is used to calculate the diffusion quantitative characteristics of the target single particle based on the particle pseudo-trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

[0179] In this embodiment, the target input image corresponding to the target single particle is acquired by the target input image acquisition module 1310, and the particle pseudo-trajectory image prediction module 1320 predicts the particle pseudo-trajectory image based on the pre-trained particle trajectory prediction model according to the target input image, and then the particle diffusion quantification feature acquisition module 1330 calculates the diffusion quantification feature of the target single particle based on the particle pseudo-trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing the first training sample set composed of the single particle motion blur image and its corresponding particle motion trajectory image. The device predicts the actual motion path of the target single particle from the target input image through the particle trajectory prediction model, and calculates the diffusion quantification feature of the target single particle based on this, and can quickly and accurately predict the diffusion quantification feature of a large number of single particles in a high labeling density environment, and significantly reduces the phototoxicity, making it suitable for at least 10 minutes of live cell imaging, and the diffusion quantification feature obtained by solving has a high dynamic range, which effectively distinguishes the diffusion characteristics of different biological particles, and provides a powerful tool for the dynamic study of living cell particles.

[0180] Fig.14 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.14 As shown, the electronic device may include: a processor 1410, a communication interface 1420, a memory 1430 and a communication bus 1440, wherein the processor 1410, the communication interface 1420 and the memory 1430 communicate with each other through the communication bus 1440. The processor 1410 may call the logic instructions in the memory 1430 to execute a single particle diffusion quantization feature prediction method, the method comprising: obtaining a target input image corresponding to a target single particle; predicting a particle pseudo-trajectory image according to the target input image based on a pre-trained particle trajectory prediction model; calculating the diffusion quantization feature of the target single particle based on the particle pseudo-trajectory image; wherein the particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

[0181] In addition, the logic instructions in the above-mentioned memory 1430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0182] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the single particle diffusion quantization feature prediction method provided by the above-mentioned methods, the method comprising: obtaining a target input image corresponding to the target single particle; based on a pre-trained particle trajectory prediction model, predicting a particle pseudo-trajectory image according to the target input image; based on the particle pseudo-trajectory image, calculating the diffusion quantization feature of the target single particle; wherein the particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

[0183] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the quantitative characteristics of single particle diffusion, characterized in that: include: Obtain a target input image corresponding to a single target particle; Based on a pre-trained particle trajectory prediction model, a particle pseudo trajectory image is predicted according to the target input image; Based on the particle pseudo-trajectory image, a diffusion quantitative characteristic of the target single particle is calculated; The particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

2. The single particle diffusion quantitative characteristic prediction method according to claim 1, characterized in that: The step of obtaining a target input image corresponding to a target single particle includes: Collecting the original single-particle motion blurred image of the target single particle; Based on a pre-trained U-Net network, the original single-particle motion blurred image is segmented to obtain a single-particle signal mask; Pre-positioning the single particle signal in the original single particle motion blurred image, and retaining a target single particle signal mask containing only one positioning; Filling the target single particle signal mask using the background and noise levels calculated by the median filter to obtain the target input image; The U-Net network is obtained by training and optimizing a second training sample set consisting of a single-particle motion blurred image and its corresponding mask image.

3. The single particle diffusion quantitative characteristic prediction method according to claim 1, characterized in that: The diffusion quantitative characteristics of the target single particle include diffusion coefficient and diffusion direction; Accordingly, the step of calculating the diffusion quantification feature of the target single particle based on the particle pseudo trajectory image includes: Determining a pseudo track area according to the particle pseudo track image; According to the quantitative relationship between the pseudo-track area and the particle diffusion coefficient, the diffusion coefficient corresponding to the target single particle is obtained by fitting; According to the density spatial distribution of the particle pseudo trajectory image, the diffusion direction corresponding to the target single particle is obtained by fitting.

4. The single particle diffusion quantitative characteristic prediction method according to claim 1, characterized in that: Training and optimizing the particle trajectory prediction model, including: Simulating a single particle motion blurred image, and acquiring a particle motion trajectory image corresponding to the single particle motion blurred image, so as to construct a first training sample set; The single particle motion blurred image is used as the model input, the predicted pseudo trajectory image is used as the model output, and the difference between the predicted pseudo trajectory image and the particle motion trajectory image is used as the training loss. The particle trajectory prediction model is iteratively optimized to obtain a particle trajectory prediction model with training convergence.

5. The method for predicting the quantitative characteristics of single particle diffusion according to claim 4, characterized in that: The method of simulating a single particle motion blurred image comprises: Simulating a two-dimensional particle trajectory and superimposing a Gaussian function on each point on the two-dimensional particle trajectory to obtain a motion blur function; The motion blur function is normalized and pixelated, and Gaussian white noise and Poisson shooting noise are introduced to obtain single particle motion blurred images at different signal-to-noise ratios and background levels.

6. The method for predicting the quantitative characteristics of single particle diffusion according to claim 3, characterized in that: The diffusion coefficient corresponding to the target single particle is obtained by fitting, and then: When the diffusion coefficient of the target single particle is greater than a set threshold, the centroid of the particle pseudo trajectory image is calculated to obtain the location of the target single particle; When the diffusion coefficient of the target single particle is less than or equal to a set threshold, an ellipsoid Gaussian fitting is performed on the target input image to obtain the location of the target single particle.

7. The method for predicting the quantitative characteristics of single particle diffusion according to claim 6, characterized in that: The positioning of the target single particle is obtained, and then the following steps are included: Gridding the location of the target single particle to obtain a first grid including the location of the particle and a second grid not including the location of the particle but adjacent to the location of the particle; Based on the positioning of the target single particle, generating a particle density probability map; Based on the particle density probability map and the first grid, interpolating the second grid using the Gaussian weight sum to obtain a diffusion coefficient corresponding to the second grid; A diffusion coefficient matrix is ​​obtained according to the diffusion coefficient corresponding to the first grid and the diffusion coefficient corresponding to the second grid; The diffusion coefficient matrix is ​​locally smoothed to obtain a motility map, which is displayed using an HSV color map to complete image rendering.

8. A single particle diffusion quantitative characteristic prediction device, characterized in that: include: A target input image acquisition module is used to acquire a target input image corresponding to a single target particle; A particle pseudo-trajectory image prediction module is used to predict a particle pseudo-trajectory image according to the target input image based on a pre-trained particle trajectory prediction model; A particle diffusion quantitative feature acquisition module is used to calculate the diffusion coefficient of the target single particle based on the particle pseudo trajectory image; The particle trajectory prediction model is obtained by training and optimizing a first training sample set consisting of a single particle motion blurred image and its corresponding particle motion trajectory image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the single particle diffusion quantitative characteristic prediction method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the single particle diffusion quantitative characteristic prediction method according to any one of claims 1 to 7 is implemented.

Images