Flowing nanoparticle self-supervised image denoising and tracking analysis method

Abstract

The invention relates to a flowing nanoparticle self-supervised image denoising and tracking analysis method. The method comprises the following steps: randomly selecting a three-dimensional patch from a flowing nanoparticle fluorescence spectrum imaging video; inputting the three-dimensional patch into the trained flowing nanoparticle self-supervised image denoising model to realize flowing nanoparticle self-supervised image denoising; comprising the following steps: processing a center pixel of a three-dimensional patch, inputting the center pixel into a dynamic space blind spot convolutional neural network, and constructing a center mask structure from three angles; splicing other pixels of the three-dimensional patch and then inputting the spliced pixels into the space-time fusion convolutional neural network to integrate space-time information; the slices are spliced according to the original angle after being rotated and then combined with the remaining pixels, and a single-frame feature map is obtained after feature fusion; the self-supervised image denoising of the flowing nano-particles is realized by optimizing the loss of the single-frame feature map and the central frame of the input patch; the particle size of the nano-particles in the flowing state is obtained, and tracking analysis is achieved. According to the method, in-situ denoising of the data can be realized when clean data of high-quality nanoparticles is difficult to obtain.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing analysis, in particular to a flow nanoparticle self-supervised image denoising and tracking analysis method. BACKGROUND

[0002] With the progress of artificial intelligence technology, image denoising has developed from traditional mathematical methods to supervised deep learning methods, and then to recent self-supervised deep learning techniques. Traditional mathematical denoising methods rely on pre-defined filters and statistical techniques to smooth noise while trying to preserve the underlying signal. These methods usually assume specific noise characteristics, such as Gaussian noise, and require manual adjustment of parameters, often resulting in a trade-off between removing noise and preserving details. The advent of supervised deep learning marks a major breakthrough, using neural networks trained on large amounts of noisy data and clean data to learn complex mapping relationships, effectively removing noise while preserving complex details. However, this method has serious problems such as heavy reliance on large labeled data sets. To solve the above problems, self-supervised deep learning emerges as a powerful alternative. Self-supervised learning trains models by exploiting their inherent structure or redundancy without clean target data. Flow nanoparticle imaging signals are weak and the background is strong, making it urgent to develop a self-supervised in-situ denoising method.

[0003] Nanoparticle tracking analysis is a technique for characterizing the physical properties of nanoparticles and is widely used in the fields of nanomedicine and environmental science. This method plays a key role in measuring the size, concentration and distribution of nanoparticles in the sample, and is crucial for advancing applications such as drug delivery, diagnosis and materials science. Traditionally, nanoparticle tracking analysis relies on observing nanoparticles in a non-flowing state. However, this static method has limitations, especially when nanoparticles are part of a dynamic system (such as biological fluids, microfluidic devices), where their behavior is influenced by motion and flow. In this case, traditional nanoparticle tracking analysis techniques have limitations in reflecting the true characteristics of nanoparticles. Therefore, there is an urgent need to develop a tracking analysis method for flow nanoparticles. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a flow nanoparticle self-supervised image denoising and tracking analysis method.

[0005] The present application addresses the technical needs of nanoparticle spectral imaging and light scattering imaging analysis under flow conditions, providing a deep learning self-supervised image denoising method. This denoising method can achieve spectral image denoising of flow nanoparticles, promoting flow nanoparticle spectral imaging visualization and post-analysis. The present application also provides a flow nanoparticle tracking analysis method, which can achieve accurate particle size analysis of flow nanoparticles.

[0006] Terminology explanation: 1、DSBnet: dynamic-spatial blind-spot convolutional neural network 2、TSFnet: temporal-spatial fusion convolutional neural network 3、c-fNTA method: coordinate-system-correction-based flow nanoparticle tracking analysis

[0007] 4、GAM attention module: method proposed by Liu et al., see reference "Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions. arXiv preprint arXiv:2112.05561. 2021".

[0008] To achieve the above purpose, the application adopts the following technical solutions: The flow nanoparticle self-supervised image denoising and tracking analysis method comprises: Randomly selecting a three-dimensional patch from a flow nanoparticle fluorescence spectrum imaging video; The three-dimensional patch is input into a trained flow nanoparticle self-supervised image denoising model to realize flow nanoparticle self-supervised image denoising; wherein the flow nanoparticle self-supervised image denoising model comprises a dynamic-spatial blind-spot convolutional neural network DSBnet and a temporal-spatial fusion convolutional neural network TSFnet, comprising: The center pixel of the three-dimensional patch is sliced along the spatial direction xy and two time-varying process directions xt and yt, and after rotation, it is input into the DSBnet to construct a center mask structure from three angles; the remaining pixels of the three-dimensional patch are spliced and input into the TSFnet to integrate spatial and temporal information; The slice containing the center mask structure after the DSBnet is rotated and spliced according to the original angle, and then combined with the remaining pixels integrated by the TSFnet, the combined structure is subjected to feature fusion to obtain a single-frame feature map; The flow nanoparticle self-supervised image denoising is realized by optimizing the loss of the single-frame feature map and the center frame of the input patch; The particle size of the nanoparticle in a flowing state is acquired, and tracking analysis is realized.

[0009] According to the application, the remaining pixels of the three-dimensional patch are input into the TSFnet to integrate the space-time information after splicing. The TSFnet adopts the encoding-decoding structure of Unet, and the implementation process in the TSFnet includes an encoding path, a decoding path and a skip connection. In the encoding path, a GAM attention module is introduced, and the repeated structural unit convolution, the ReLU activation function and the pooling layer in the encoding structure of Unet are used to extract the local-to-global semantic information, gradually compress the spatial information of the image and reserve the key features. In the decoding path, the spatial resolution of the image is gradually restored through upsampling interpolation, and the shallow high-resolution features are combined through the skip connection. Then, the 1×1 convolution and the ReLU activation function are performed three times to finally output the feature map.

[0010] According to the application, the slices of the center pixel of the three-dimensional patch along the spatial direction xy and the two time-varying process directions xt and yt are rotated and input into the DSBnet to construct the center mask structure from three angles. The DSBnet includes three branches, and each branch includes a 3×3, 5×5 and 7×7 dilated convolution containing the center mask. The slices of the center pixel of the three-dimensional patch along the spatial direction xy and the two time-varying process directions xt and yt are rotated and input into the DSBnet to construct the center mask structure from three angles.

[0011] According to the application, the loss between the single-frame feature map and the center frame of the input patch is optimized to realize the self-supervised image denoising of the flowing nanoparticle. The loss function relationship between the acquired single-frame feature map and the center pixel of the input three-dimensional patch is established, as shown in the following formula: ; L1Loss is the mean absolute error, and MSELoss is the mean square error. The formula for L1Loss is as follows: ; in, For predicted values, The actual value; The formula for MSELoss is shown below: .

[0012] By using gradient updates and optimizing the loss of the center pixel of the single-frame feature map and the input 3D patch, self-supervised image denoising of flowing nanoparticles is achieved.

[0013] According to a preferred embodiment of the present invention, the particle size of nanoparticles in a flowing state is obtained by the c-fNTA method; comprising: The c-fNTA method rotates and corrects the spatial coordinate system by adjusting the overall particle movement direction. This new coordinate system separates the flow and Brownian motion from the mixed motion. In this new coordinate system, the y' direction represents pure Brownian motion, and the t direction represents time. The particle size of the nanoparticles in the flow state is obtained using the Stokes-Einstein equations. ; in, It is Boltzmann's constant. and Here, represents the fluid temperature and viscosity, respectively, and d represents the particle diameter; the diffusion coefficient D is calculated from the mean square displacement of the particle trajectory. ; Where N is the total number of trajectory points, Δt is the time interval, i is the index of the trajectory point, and y' i It is the coordinate value of the trajectory point in the y' direction, and then the diameter d of the particle is obtained, that is, the particle size of the nanoparticle.

[0014] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a method for self-supervised image denoising and tracking analysis of flowing nanoparticles.

[0015] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for self-supervised image denoising and tracking analysis of flowing nanoparticles.

[0016] The beneficial effects of this invention are as follows: 1. This invention uses a self-supervised deep learning method for denoising, replacing traditional mathematical methods and supervised learning denoising, and can achieve in-situ denoising of data when high-quality clean data cannot be obtained.

[0017] 2. This invention fully utilizes the spatial redundancy and dynamic process redundancy of spatiotemporal data, solving the problem of poor denoising effect caused by insufficient utilization of dynamic time dimension information in two-dimensional image denoising algorithms, and achieving high-quality denoising; image denoising can reach 13dB.

[0018] 4. Applications of spectral image denoising analysis, including: obtaining the fluorescence spectrum corresponding to nanoparticles after spectral image denoising; 5. This invention conducts size analysis of nanoparticles in a flow state, and can obtain the particle size distribution of nanoparticles in a flow state through a flow nanoparticle tracking analysis method with correction function without the need for complex system debugging and calibration.

[0019] 6. The nanoparticle tracking analysis results are good. This invention has been verified on 110 nm standard fluorescent nanoparticles, and the accurate particle size distribution can be obtained through size analysis in flow cytometry.

[0020] 7. This invention is applicable to the particle size analysis of other biomedical nanoparticles within the exosome particle size range and in other fields, and has broad applicability.

[0021] 8. The self-supervised image denoising method of the present invention can be widely applied to various types of video data with noise, such as moving video data with added noise of different scales and content movement speed. Attached Figure Description

[0022] Figure 1 This is a diagram of the DeepNano network framework, a self-supervised denoising network of this invention. Figure 2 This is a schematic diagram of the structure of the TSFnet of the present invention; Figure 3 This is a schematic diagram of the structure of the DSBnet of the present invention; Figure 4 This is a schematic diagram of representative frames of the mobile MNIST dataset video data before and after DeepNano denoising in this invention; Figure 5 This is a schematic diagram showing representative frames and spectral distribution curves of the spectral video data before and after DeepNano denoising in this invention; Figure 6 This is a schematic diagram illustrating the principle of the flow nanoparticle tracking and analysis method with correction function in this invention; Figure 7 This is a diagram showing the standard microsphere size distribution obtained using the flow nanoparticle tracking and analysis technique in this invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and the description of the embodiments.

[0024] Example 1 A self-supervised image denoising and tracking analysis method for flowing nanoparticles includes: Three-dimensional patches were randomly selected from the fluorescence spectroscopy imaging video of flowing nanoparticles; three-dimensional patches with dimensions of 61×61×61 (xyt) were randomly selected from the fluorescence spectroscopy imaging video of flowing nanoparticles, where xy direction is the spatial distribution of pixels and t represents time. The 3D patch is input into the trained self-supervised image denoising model for flowing nanoparticles to achieve self-supervised image denoising for flowing nanoparticles. The self-supervised image denoising model for flowing nanoparticles includes a dynamic spatial blind spot convolutional neural network (DSBnet) and a spatiotemporal fusion convolutional neural network (TSFnet), comprising: The central pixel (31, 31, 31) of the 3D patch is sliced ​​along the spatial direction xy and the two time-varying process directions xt and yt. After being rotated, it is input into DSBnet to construct the central mask structure from three angles. The remaining pixels of the 3D patch are stitched together and input into TSFnet to integrate the spatiotemporal information. The slices containing the central mask structure that have been processed by DSBnet are rotated and then stitched together at the original angle. They are then combined with the remaining pixels that have been processed by TSFnet to integrate spatiotemporal information. The combined structure is then fused to obtain a single-frame feature map. The combined structure has the same dimension as the input 3D patch. The combined structure is then fused to obtain features without changing the spatial size. The temporal dimension is compressed to a single frame, and finally a single-frame feature map is obtained. Self-supervised image denoising of flowing nanoparticles is achieved by optimizing the loss between the single-frame feature map and the center frame of the input patch; The particle size of nanoparticles in flowing state is obtained for tracking analysis.

[0025] Example 2 The difference between the self-supervised image denoising and tracking analysis method for flowing nanoparticles described in Example 1 and the method described in Example 1 is as follows: Figure 1 This is a diagram of the DeepNano network framework, a self-supervised denoising network of this invention. like Figure 1As shown, this deep learning-based self-supervised denoising algorithm for a single noisy video mainly comprises DSBnet for constructing a central mask structure and TSFnet for fusing spatiotemporal information. The network training process is self-supervised, with the input being a selected 3D patch (xyt) from the noisy experimental video. The center pixel of the patch is rotated along the xy, xt, and yt directions and then input into DSBnet, constructing a central mask structure from three angles. The remaining pixels are then concatenated and input into TSFnet to integrate spatiotemporal information. The mask-containing slices from DSBnet are rotated and concatenated at their original angles, then combined with the remaining pixels after TSFnet integration of spatiotemporal information. The combined structure has the same dimensions as the input patch, and this structure is fused to obtain a single-frame feature map. Video denoising is achieved by optimizing the loss between the single-frame feature map and the center frame of the input patch. The network input is a patch with a size of 61×61×61.

[0026] The remaining pixels of the 3D patch are stitched together and then input into TSFnet to integrate spatiotemporal information; for example... Figure 2 As shown, it includes: This invention provides a neural network model TSFnet based on the Unet architecture. Specifically, it introduces a GAM (Global Attention Mechanism) attention module into the traditional Unet architecture to enhance the model's ability to focus on key regions in the feature map, thereby improving the performance of multi-dimensional image information fusion. The specific implementation process of this network model is as follows: TSFnet adopts the Unet encoder-decoder structure. The implementation process in TSFnet includes the encoding path, the decoding path, and skip connections. In the encoding path, a GAM attention module is introduced to enhance the model's ability to perceive global features. Through repeated structural unit convolutions, ReLU activation functions, and pooling layers in the U-Net encoding structure, semantic information from local to global is extracted, gradually compressing the spatial information of the image while preserving key features to prepare for the subsequent decoding path. Simultaneously, the encoding path provides high-resolution features for U-Net's skip connections, aiding in the recovery of image details during the decoding process. Specifically, the network input consists of the remaining pixels of a 60×60×60 3D patch; after four consecutive 3×3 convolutions and ReLU activation functions, a GAM attention module, and 2×2 max pooling, a 3×3×128 feature map is output; the encoding path ends after one more 3×3 convolution and ReLU activation function and a GAM attention module. In the decoding path, the spatial resolution of the image is gradually restored through upsampling interpolation, and the ability to restore edges and details is improved by combining shallow high-resolution features through skip connections. Specifically, this includes: four consecutive upsampling interpolations, skip connections, 3×3 convolutions, and ReLU activation functions, resulting in a 60×60×16 feature map after upsampling. The subsequent three 1×1 convolutions and ReLU activation function output a feature map of size 60×60×60. In addition to the spatial information integration mentioned above, the network also incorporates temporal information.

[0027] The center pixel of the 3D patch is sliced ​​along the spatial direction xy and two time-varying process directions xt and yt, respectively, and then rotated before being input into DSBnet to construct the central mask structure from three angles; including: like Figure 3 As shown, this invention provides a network architecture, DSBnet, based on dilated convolutions with a self-constructed central mask structure. According to the network structure diagram shown in the attached figure, the network architecture progressively constructs a large receptive field and a central mask structure by concatenating the channel dimensions of the following convolutions: "3×3 dilated convolutions with dilation rates of 1, 2, 4, and 9 with a central mask," "5×5 dilated convolutions with dilation rates of 1, 3, and 6 with a central mask," "5×5 dilated convolutions with dilation rates of 1, 3, and 10 with a central mask," "5×5 dilated convolutions with dilation rates of 1, 3, and 14 with a central mask," and "7×7 dilated convolutions with dilation rates of 1, 4, and 8 with a central mask." Finally, a 1×1 convolution and a ReLU activation function are used to fuse the channel information, while the central mask structure is retained and does not participate in gradient propagation calculations.

[0028] Figure 3 In this context, r represents the dilation rate of the dilated convolution. By combining dilated convolutions with different dilation rates, the receptive field is effectively expanded while constructing the central mask structure, without changing the size of the feature map.

[0029] DSBnet consists of three branches, each containing a 3×3, 5×5, and 7×7 dilated convolution with a central mask, and each branch has a different dilation rate. The central pixel of the 3D patch is sliced ​​along the spatial direction xy and two time-varying process directions, namely xt and yt. After being rotated, it is calculated by dilation convolution of three branches and then connected through the channel dimension to obtain a large receptive field with a central mask. Finally, 1×1 convolution is used to fuse channel information to construct the central mask structure from three angles.

[0030] DSBnet is the core module for network denoising. It takes the spatiotemporal center pixel (31, 31, 31) of the 3D patch, rotated along the spatial xy direction and the two time-varying process directions xt and yt, and then inputs these slices into DSBnet. This allows for the construction of a mask structure from three perspectives: xy, xt, and yt. This mask structure alters the network's characteristics, preventing the output mask pixels from being directly obtained; instead, they are derived through inference from surrounding pixels. During inference, the mask structure is predicted using pixels from the other three perspectives outside the mask structure. Through this prediction-based process, the network eventually converges and achieves multi-view denoising of the video data.

[0031] The training set size of the self-supervised image denoising model for flowing nanoparticles is 1 video, the learning rate is 0.0001, and the batch size is 16. The training results can be used for in-situ denoising of the training videos.

[0032] Figure 4 This is a representative frame result of applying the DeepNano denoising algorithm of this invention to the denoised mobile MNIST video data. Figure 4 (a) shows the actual ground conditions. Figure 4 (b) contains noisy data; Figure 4 (c) shows the denoising result of BM3D; Figure 4 In the middle (d), the result of noise reduction by the method of the present invention is shown.

[0033] Moving MNIST video data is obtained by increasing the speed of movement of the digits in the MNIST dataset (moving 5 pixels per frame).

[0034] Unnoised MNIST video data was used as the original real images and served as the standard for evaluating the algorithm's denoising performance, but was not input into the network as a label.

[0035] The network was trained using a single moving MNIST video dataset with a large amount of Gaussian noise added.

[0036] The trained network performs in-situ denoising on mobile MNIST video data with a large amount of Gaussian noise added.

[0037] The amount of noise added is so great that it almost completely obscures the original MNIST digital signal.

[0038] After denoising using our proposed DeepNano, almost all noise can be removed, achieving good signal structure recovery even under extreme noise and high-speed conditions.

[0039] Meanwhile, the BM3D denoising algorithm will also be applied in this scenario for comparison purposes.

[0040] BM3D: A method proposed by Kostadin et al., see reference "Video denoising by sparse 3D transformdomain collaborative filtering. European signal processing conference. Vol. 149. No. 2. 2007". Among them, the noise removal effect of the BM3D denoising method is far inferior to that of our proposed method.

[0041] Figure 5 In this paper, the DeepNano denoising algorithm proposed in this invention is applied to streaming spectral video data. By using DeepNano for video denoising, weak spectral signals that were originally masked by noise can be revealed. Figure 5 In the middle (a), the experimental data includes noise; Figure 5 (b) shows the result after denoising using the method of the present invention; Figure 5 In Figure (a), (c) shows the intensity distribution corresponding to the red curve.

[0042] The flow cytometry experiment used 110 nm fluorescent microspheres as samples, with a fluorescence peak at 559 nm. After noise reduction using DeepNano, the basic spectral structure was restored, and it was basically consistent with the spectrum provided by the manufacturer.

[0043] Self-supervised image denoising of flowing nanoparticles is achieved by optimizing the loss between the single-frame feature map and the center frame of the input patch data; self-supervised learning is performed by establishing a loss function between the single-frame feature map output by the network and the center frame of the patch data input to the network, thus achieving self-supervised image denoising of flowing nanoparticles; including: The loss function relationship between the acquired single-frame feature map and the center pixel of the input 3D patch is established as follows: ; In deep learning, a loss function is used to measure the difference between the model's predictions and the true values. L1Loss and MSELoss are two widely used loss functions. L1Loss, or Mean Absolute Error, calculates the average of the absolute values ​​of the differences between the model's predictions and the true values. MSELoss, or Mean Squared Error, calculates the average of the squared differences between the model's predictions and the true values.

[0044] The formula for L1Loss is as follows: ; in, For predicted values, The actual value; The formula for MSELoss is shown below: .

[0045] By using gradient updates and optimizing the loss of the center pixel of the single-frame feature map and the input 3D patch, self-supervised image denoising of flowing nanoparticles is achieved.

[0046] The particle size of nanoparticles under flowing conditions was obtained using the c-fNTA method; such as Figure 6 As shown, it includes: In flow cytometry nanoparticle imaging, nanoparticle signal points exist in a superposition of flow and Brownian motion. Even minute changes in the position of the microfluidic channel during each experiment can alter the flow direction, and discontinuous stepping of the stepper motor in the syringe pump can lead to flow instability. To accurately extract Brownian motion from this superposition of unstable flow velocities and avoid complex manual calibration before each experiment, c-fNTA is proposed.

[0047] The c-fNTA method rotates and corrects the spatial coordinate system by analyzing the overall particle movement direction. In flow cytometry, all particles have velocities along the sample liquid flow direction. These velocities are unstable, but their direction is fixed. By statistically analyzing the motion directions of all particles, the flow direction of the sample liquid can be obtained. By rotating the coordinate system, the new spatial coordinate system separates the flow and Brownian motion from the mixed motion. In the new spatial coordinate system, the y' direction represents pure Brownian motion, independent of the flow, ensuring that Brownian motion in a single direction is not affected by unstable flow velocities. The t direction represents the time direction. Overall, the new coordinate system reflects the relationship between the one-dimensional Brownian motion of particles and time. The particle size of nanoparticles in the flow state is obtained using the Stokes-Einstein equations. ; in, It is Boltzmann's constant. and Here, represents the fluid temperature and viscosity, respectively, and d represents the particle diameter; the diffusion coefficient D is calculated from the mean square displacement of the particle trajectory. ; Where N is the total number of trajectory points, Δt is the time interval, i is the index of the trajectory point, and y' i It is the coordinate value of the trajectory point in the y' direction, and then the diameter d of the particle is obtained, that is, the particle size of the nanoparticle.

[0048] Figure 7 This invention demonstrates a method for obtaining the standard microsphere size distribution. The imaging data of the flowing nanoparticles is subtracted from the extracted background signal to obtain a background-free pattern.

[0049] Using c-fNTA for background-free flow cytometry nanoparticle imaging data correction analysis can yield unbiased nanoparticle size distribution.

[0050] In the experiment, 110 nm fluorescent microspheres were used for visualization detection and c-fNTA correction analysis. The obtained particle size distribution, after Gaussian fitting, had a mean of 113 nm, which is consistent with the particle size distribution provided by the manufacturer. This demonstrates the accuracy of the c-fNTA method.

[0051] This invention proposes a deep learning-based self-supervised image denoising algorithm using flowing nanoparticles. Based on the independent unpredictability of spatiotemporal data noise, it fully utilizes the spatial and dynamic redundancy of the signal to establish a mapping between noise and unbiased clean signal data. Specifically, for dynamic scenes, the spatial viewpoint contains its static structural information, while the dynamic viewpoint, orthogonal to the spatial viewpoint, contains its motion information. In streaming video data, the signal flows roughly in one direction (x-axis). Therefore, the algorithm selects a spatial viewpoint (xy), a dynamic viewpoint (xt), and a dynamic viewpoint (yt) orthogonal to both of these viewpoints, and masks the center pixel. It effectively utilizes the structural and motion correlations of the flowing signal to achieve statistical denoising during the masking process, without introducing artifacts or offsets. This statistical optimization problem does not require standard noise-free labels or massive training data; in-situ denoising can be performed using only video data obtained from a single experiment.

[0052] This invention designs a special network structure to strengthen the mapping relationship. In this network architecture, the network backbone constructs a spatial (xy) and two dynamic (xt, yt) perspectives around a central mask structure, and the remaining parts are fused and fed into branch networks to extract spatiotemporal information. To verify the network performance, this invention was extensively compared with mainstream denoising algorithms on simulated datasets. This invention simulates experimental data with different signal-to-noise ratios and flow velocities under flowing conditions by assigning different movement speeds and adding noise of different scales to the content in the MNIST (Modified National Institute of Standards and Technology) dataset. After comparative verification, the proposed self-supervised image denoising algorithm achieves the best denoising performance at different movement speeds and noise scales. Under low movement speed conditions, even with extreme noise pollution, the signal-to-noise ratio can still be improved by 16 dB. As the movement speed and noise scale gradually increase, DeepNano can improve the SNR by 13 dB under extreme noise and high-speed movement conditions.

[0053] This invention proposes a flow nanoparticle tracking and analysis method with correction functionality. This method involves rotating and correcting the original coordinate system. In the new coordinate system, the motion of a single particle in the x-direction is a superposition of flow and Brownian motion, while the motion in the y-direction is pure Brownian motion, independent of the flow, ensuring that subsequent analysis is unaffected by unstable flow velocities. This method accurately extracts Brownian motion from the superposition of motions with unstable flow velocities and avoids the complex manual calibration work before each experiment. By analyzing the extracted one-dimensional Brownian motion, the particle size distribution of nanoparticles can be effectively obtained in flow cytometry.

[0054] Example 3 A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the self-supervised image denoising and tracking analysis method for flowing nanoparticles described in Embodiment 1 or 2.

[0055] Example 4 A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the self-supervised image denoising and tracking analysis method for flowing nanoparticles described in Embodiment 1 or 2.

[0056] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for self-supervised image denoising and tracking analysis of flowing nanoparticles, characterized in that, include: Three-dimensional patches were randomly selected from the fluorescence spectral imaging video of flowing nanoparticles; The 3D patch is input into the trained self-supervised image denoising model for flowing nanoparticles to achieve self-supervised image denoising for flowing nanoparticles. The self-supervised image denoising model for flowing nanoparticles includes a dynamic spatial blind spot convolutional neural network (DSBnet) and a spatiotemporal fusion convolutional neural network (TSFnet), comprising: The central pixel of the 3D patch is sliced ​​along the spatial direction xy and the two time-varying process directions xt and yt, and after being rotated, it is input into DSBnet to construct the central mask structure from three angles; the remaining pixels of the 3D patch are stitched together and input into TSFnet to integrate spatiotemporal information. The slices containing the central mask structure that have passed through DSBnet are rotated and then stitched together at the original angle. They are then combined with the remaining pixels that have been integrated with spatiotemporal information by TSFnet. The combined structure is then fused to obtain a single-frame feature map. Self-supervised image denoising of flowing nanoparticles is achieved by optimizing the loss between the single-frame feature map and the center frame of the input patch; The particle size of nanoparticles in flowing state is obtained for tracking analysis.

2. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 1, characterized in that, The remaining pixels of the 3D patch are stitched together and then input into TSFnet to integrate spatiotemporal information; including: TSFnet adopts the Unet encoder-decoder structure. The implementation process in TSFnet includes the encoding path, the decoding path, and skip connections. In the encoding path, a GAM attention module is introduced. After repeated structural unit convolutions, ReLU activation functions, and pooling layers in the Unet encoding structure, local to global semantic information is extracted, gradually compressing the spatial information of the image while retaining key features. Specifically, the input consists of the remaining pixels of the 3D patch; after four consecutive 3×3 convolutions and ReLU activation functions, GAM attention module, and 2×2 max pooling, the feature map is output; after one more 3×3 convolution and ReLU activation function and one more GAM attention module, the encoding path ends. In the decoding path, the spatial resolution of the image is gradually restored through upsampling interpolation, and shallow high-resolution features are combined through skip connections. Specifically, the process includes four consecutive upsampling interpolations, skip connections, 3×3 convolutions, and ReLU activation functions, and the output feature map is upsampled. The final output feature map is then generated by three 1×1 convolutions and the ReLU activation function.

3. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 1, characterized in that, The central pixel of the 3D patch is sliced ​​along the spatial direction xy and the two time-varying process directions xt and yt. After being rotated, it is input into DSBnet to construct the central mask structure from three angles. include: DSBnet includes three branches, which respectively include 3×3, 5×5, and 7×7 dilated convolutions with a central mask. The dilated convolutions with the central mask have different dilation rates. The central pixel of the 3D patch is sliced ​​along the spatial direction xy and two time-varying process directions, namely xt and yt. After being rotated, it is calculated by dilation convolution of three branches and then connected through the channel dimension to obtain a large receptive field with a central mask. Finally, 1×1 convolution is used to fuse channel information to construct the central mask structure from three angles.

4. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 1, characterized in that, Self-supervised image denoising of flowing nanoparticles is achieved by optimizing the loss between the single-frame feature map and the center frame of the input patch; including: The loss function relationship between the acquired single-frame feature map and the center pixel of the input 3D patch is established as follows: ； L1Loss is the mean absolute error, and MSELoss is the mean square error. The formula for L1Loss is as follows: ； in, For predicted values, The actual value; The formula for MSELoss is shown below: ； in, For predicted values, This is the actual value.

5. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 4, characterized in that, By using gradient updates and optimizing the loss of the center pixel of the single-frame feature map and the input 3D patch, self-supervised image denoising of flowing nanoparticles is achieved.

6. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 1, characterized in that, The self-supervised image denoising and tracking analysis method for flowing nanoparticles is used to process various types of video data containing noise.

7. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 1, characterized in that, The particle size of nanoparticles in flowing conditions was obtained using the c-fNTA method; including: The c-fNTA method rotates and corrects the spatial coordinate system by adjusting the overall particle movement direction. This new coordinate system separates the flow and Brownian motion from the mixed motion. In this new coordinate system, the y' direction represents pure Brownian motion, and the t direction represents time. The particle size of the nanoparticles in the flow state is obtained using the Stokes-Einstein equations. ； in, It is Boltzmann's constant. and Here, represents the fluid temperature and viscosity, respectively, and d represents the particle diameter; the diffusion coefficient D is calculated from the mean square displacement of the particle trajectory. ； Where N is the total number of trajectory points, Δt is the time interval, i is the index of the trajectory point, and y' i It is the coordinate value of the trajectory point in the y' direction, and then the diameter d of the particle is obtained, that is, the particle size of the nanoparticle.

8. The method for self-supervised image denoising and tracking analysis of flowing nanoparticles according to claim 1, characterized in that, Brownian motion is extracted from the superposition motion of unstable flow velocities using the c-fNTA method.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the self-supervised image denoising and tracking analysis method for flowing nanoparticles as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the self-supervised image denoising and tracking analysis method for flowing nanoparticles as described in any one of claims 1-8.

Images