Flexible particle deformation detection method and system based on deep learning
Through deep learning point cloud matching and deformation modeling methods, the accuracy problem of flexible particle classification in Cryo-EM technology was solved, and efficient deformation prediction and classification under low signal-to-noise ratio conditions was achieved.
Patent Information
- Application Number
- CN202510523193.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-24
- Publication Date
- 2025-09-09
AI Technical Summary
When existing Cryo-EM technology processes flexible particles, the accuracy of traditional rigid matching methods decreases, and deep learning methods lack effective deformation modeling, resulting in inaccurate classification, especially under low signal-to-noise ratio conditions, it is difficult to accurately capture the tiny deformation of particles.
A deep learning-based point cloud matching and deformation modeling method is adopted, including image denoising, point cloud generation, KMP heuristic matching and variational autoencoder iterative prediction, to construct the deformation trajectory of particles. The KMP algorithm is used to improve the efficiency of point cloud matching, and the variational autoencoder is used to optimize deformation prediction.
High-precision flexible particle classification is achieved under low signal-to-noise ratio conditions, which improves the efficiency of point cloud matching and the robustness of deformation modeling, and is suitable for particle structure analysis in high-noise environments.
Smart Images
Figure CN120612690A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of biological computing and structural analysis technology, and specifically relates to a flexible particle deformation detection method and system based on deep learning. Background Art
[0002] Cryo-electron microscopy (Cryo-EM), a high-resolution three-dimensional imaging technique, has been widely used in the biomedical field to observe subcellular structures and macromolecular complexes. However, the automated analysis of cryo-EM data remains challenging due to the low signal-to-noise ratio (SNR) and flexible deformation of samples. In particular, during the two-dimensional classification of particles, traditional methods often assume that particles are in a limited number of discrete conformations, ignoring the continuous deformation properties of biological particles in real environments. This simplification can lead to classification errors, which in turn affect downstream tasks.
[0003] To address the problem of two-dimensional particle classification, existing research focuses on two types of methods: rigid alignment-based classification methods and deep learning-based particle classification. Rigid alignment-based classification methods, such as RELION and CryoSPARC, utilize maximum likelihood estimation (MLE) and reference image matching for classification. However, these methods assume that particles undergo only slight rigid rotations and translations and cannot effectively handle deformed particles. Deep learning-based particle classification methods, such as Topaz and DeepPicker, are primarily used for particle detection and are not optimized for flexible deformation modeling at the point cloud level. Therefore, they struggle to accurately capture subtle particle deformations.
[0004] Existing methods suffer from the following shortcomings. First, traditional rigid matching methods fail when faced with continuously deforming particles, resulting in reduced classification accuracy. Second, while deep learning methods can improve particle detection capabilities, existing research lacks effective deformation modeling strategies and cannot accurately estimate particle deformation trajectories. Furthermore, existing deformation modeling methods mostly rely on large-scale data training, which struggles to achieve good results in the high-noise, small-sample environment of cryo-EM.
[0005] This paper proposes a deep learning-based point cloud matching and deformation modeling method that effectively estimates the continuous deformation trajectory of particles, obtains the particle deformation variables, and eliminates deformed particles to achieve classification. Compared with existing methods, this paper proposes point cloud matching technology (KMP algorithm) and deformation prediction model, which can quickly capture the deformation process and converge to the entire template point cloud, thereby saving computation time and improving the integrity of information representation.
[0006] Currently, cryo-EM particle classification mainly relies on methods such as rigid matching and deep learning. However, these methods have many shortcomings when dealing with flexible particles. First, classification methods based on rigid matching, such as RELION and CryoSPARC, assume that particles only undergo rigid rotation and translation, and cannot accurately capture continuously deformed particles, resulting in reduced classification accuracy and affecting downstream tasks. Particle classification methods based on deep learning, such as Topaz and DeepPicker, mainly focus on particle detection, but lack targeted optimization in deformation modeling, resulting in unstable performance when classifying continuously deformed particles.
[0007] Another significant drawback of existing methods is the lack of effective modeling of particle deformation trajectories. The deformation trajectories of flexible particles in different states are crucial for classification, but traditional methods struggle to accurately estimate the continuous deformation process, resulting in blurred classification boundaries. Even some deep learning methods attempt to capture deformation through feature mapping. Summary of the Invention
[0008] In view of this, the purpose of the present invention is to provide a flexible particle deformation detection method and system based on deep learning, which can perform high-precision classification of flexible particles under low signal-to-noise ratio conditions, and at the same time construct a complete deformation trajectory. It can extract key point cloud information in Cryo-EM data, predict the deformation trajectory of particles, and obtain the deformation amount of particles to eliminate deformed particles to achieve the classification effect.
[0009] A method for predicting deformation of flexible particles based on deep learning, comprising:
[0010] The first step is to perform noise reduction on the Cryo-ET original image and then perform binarization to generate particle point cloud and template point cloud;
[0011] The second step is to match the particle point cloud with the template point cloud;
[0012] The third step is deformation prediction:
[0013] (1) Determine the template point cloud that matches each particle point cloud;
[0014] (2) Input the coordinates P(k) of the particle point cloud of the current iteration round into the encoder of the variational autoencoder VAE to obtain the encoding result, and then input the encoding result into the decoder of the variational autoencoder VAE to generate the predicted value of the particle point cloud displacement vector Δ(k);
[0015] (3) Use the predicted displacement vector to update the current particle point cloud position and obtain a new round of iteration result P(k+1):
[0016] P(k+1)=P(k)+Δ(k);
[0017] (4) Extract the corresponding template point cloud Q (determined by the index), calculate the Euclidean distance between the particle point cloud and the matching template point cloud as the loss function, and optimize the variational autoencoder VAE model;
[0018] (5) Judgment: If the iteration stop condition is reached, the position information of the current particle point cloud is output to realize the deformation prediction of the flexible particle; otherwise, return to step (2) to continue updating.
[0019] Preferably, in the first step, the noise reduction method includes: subjecting the Cryo-ET original image to Topaz, low-pass filtering, and NLM denoising respectively to obtain a noise-reduced image; and then weighted fusion of the three noise-reduced images to generate a final noise-reduced image.
[0020] Preferably, in the first step, the method of performing binarization processing to generate a particle point cloud includes:
[0021] First, the threshold method is used to binarize the denoised image to separate the particle area from the background. Then, the maximum connected area in the binary image is extracted as a feature to initialize the point cloud and generate a particle point cloud.
[0022] 4. The method for predicting deformation of flexible particles based on deep learning according to claim 2, wherein the second step comprises:
[0023] Calculate the point cloud distance matrix: Calculate the distance between each particle point cloud and all template point clouds to obtain an m×n distance matrix, where m is the number of particle point clouds and n is the number of template point clouds; each element in the i-th row of the matrix represents the distance between the i-th particle point cloud and each template point cloud;
[0024] Row heuristic matching and distance update: traverse each row of the distance matrix, find the column index corresponding to the minimum distance in the current row, and use this index as the matching result for the current particle;
[0025] Output matching result index: After completing the matching of all particle point clouds, an m×1 index list is finally output to record the template point cloud index corresponding to each particle point cloud.
[0026] Preferably, the second step further includes:
[0027] Set matching constraint parameters: set the maximum number of times each template point cloud is allowed to be matched, and define a distance penalty item; every time a template point cloud is matched, the distance value from it to other particle point clouds is added to the distance matrix as a penalty.
[0028] Preferably, in the second step, when performing heuristic matching and updating the distance, if the number of times the template point cloud is matched exceeds the upper limit, the second closest distance is found and so on.
[0029] Preferably, the iteration stopping condition is: the loss function value is lower than a set threshold, or the maximum number of iteration rounds is reached.
[0030] Preferably, the third step further includes deformation accuracy assessment: calculating the intersection over union (IoU) of the final particle point cloud and the template point cloud to quantify the degree of structural fit.
[0031] A flexible particle deformation prediction system based on deep learning, including an image feature extraction module, a point cloud matching module, and a deformation modeling module:
[0032] The image feature extraction module is used to perform the first step of denoising the Cryo-ET original image and then performing binarization processing to generate a particle point cloud and a template point cloud;
[0033] The point cloud matching module is used to perform the second step of matching the particle point cloud with the template point cloud;
[0034] The deformation modeling module is used to perform the third step, deformation prediction:
[0035] (1) Determine the template point cloud that matches each particle point cloud;
[0036] (2) Input the coordinates P(k) of the particle point cloud of the current iteration round into the encoder of the variational autoencoder VAE to obtain the encoding result, and then input the encoding result into the decoder of the variational autoencoder VAE to generate the predicted value of the particle point cloud displacement vector Δ(k);
[0037] (3) Use the predicted displacement vector to update the current particle point cloud position and obtain a new round of iteration result P(k+1):
[0038] P(k+1)=P(k)+Δ(k);
[0039] (4) Extract the corresponding template point cloud Q (determined by the index), calculate the Euclidean distance between the particle point cloud and the matching template point cloud as the loss function, and optimize the variational autoencoder VAE model;
[0040] (5) Judgment: If the iteration stop condition is reached, the position information of the current particle point cloud is output to realize the deformation prediction of the flexible particle; otherwise, return to step (2) to continue updating.
[0041] The present invention has the following beneficial effects:
[0042] The present invention is suitable for the classification of two-dimensional particles in cryo-electron microscopy (Cryo-EM) images. In response to the problems of low signal-to-noise ratio and complex particle deformation in Cryo-EM images, the system integrates graph feature extraction, point cloud matching module, and deformation modeling module to construct a set of automated processing procedures. First, the original image is subjected to noise reduction processing by methods such as Topaz, low-pass filtering and non-local mean filtering to improve structural distinguishability. Then, the particle outline is obtained by using binarization and connected area detection, and the corresponding two-dimensional point cloud expression is generated. In order to achieve efficient registration, the present invention proposes a KMP heuristic point cloud matching algorithm to effectively establish a corresponding relationship between the particle point cloud and the template point cloud. On the basis of matching, the continuous deformation trajectory of the simulated particles is predicted through multiple rounds of displacement to realize the detection of deformation values, and the deformed particles are eliminated to achieve the classification effect. While improving the efficiency of point cloud matching, the present invention improves the accuracy and robustness of flexible structure modeling, and is suitable for particle structure analysis tasks under high noise conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 This is a schematic diagram of the multi-method image denoising and fusion process of the present invention;
[0044] Figure 2 Schematic diagram of image binarization and initial point cloud extraction results;
[0045] Figure 3 Schematic diagram of the KMP heuristic point cloud matching process;
[0046] Figure 4 Schematic diagram of the particle deformation modeling process based on displacement iteration;
[0047] Figure 5 Schematic diagram of IoU convergence evaluation and classification result output. DETAILED DESCRIPTION
[0048] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0049] A method for predicting deformation of flexible particles based on deep learning of the present invention comprises:
[0050] A. Feature extraction:
[0051] Cryo-ET images usually have an extremely low signal-to-noise ratio (SNR) due to the limitation of electron dose and the imaging mechanism, and the background noise seriously interferes with the subsequent particle identification and feature extraction. Therefore, the present invention introduces a multi-method image denoising module in the first step of system design to enhance the structural details of the image and improve the accuracy of point cloud extraction. The present invention adopts three complementary image denoising methods: Topaz (an image restoration model based on deep learning), low-pass filtering (Low-passFiltering) and non-local mean filtering (NLM) for joint denoising. Among them, Topaz is a trained neural network model that can recover structural details from extremely low-noise images, has strong end-to-end noise reduction capabilities, and is suitable for enhancing particle boundaries. However, its feature enhancement process may cause some details to be lost. Low-pass filtering is a classic frequency domain filtering method that can effectively remove high-frequency noise in the image and retain the overall structural contour of the image, but it easily blurs the image edges. Non-local mean filtering is an image smoothing algorithm based on redundant information in similar regions. It can suppress random noise while preserving local details, but it may produce artifacts in high-noise areas and has high computational complexity. Considering the advantages of each of the three methods, this paper further designs a weighted fusion strategy to fuse the three noise reduction results at the pixel level to form a more robust image representation. The fusion formula is as follows:
[0052] I fused＝ αI Low-pass +βI Topaz +γI NLM ;
[0053] Among them, I Low-pass , I Topaz , I NLM Represent the images after low-pass filtering, Topaz and NLM processing respectively; I fused Represents the final fused image; α, β, and γ are the weighting coefficients of the three methods, satisfying α+β+γ=1. In practical applications, the weights can be adjusted according to the noise intensity of the dataset or prior experience. After completing image denoising and fusion, in order to extract the spatial structural information of the particles, the present invention performs binarization processing on the image and generates a point cloud. First, the threshold method is used to binarize the fused image to separate the particle area from the background. Subsequently, the maximum connected area in the binary image is extracted as a feature and then the point cloud is initialized, providing a unified data basis for subsequent point cloud matching and deformation modeling.
[0054] B.KMP heuristic point cloud matching algorithm:
[0055] To achieve efficient registration between flexible particle point clouds and template point clouds, this paper proposes a heuristic point cloud matching algorithm, called the KMP (KNN-Matching with Penalty) algorithm. While maintaining structural consistency, it obtains the template point cloud index corresponding to each particle point cloud. The specific process of the KMP algorithm includes the following stages:
[0056] (1) Calculate the point cloud distance matrix: Calculate the distance between each particle point cloud and all template point clouds to obtain an m×n distance matrix, where m is the number of particle point clouds and n is the number of template point clouds. Each element in the i-th row of the matrix represents the distance between the i-th particle point cloud and each template point cloud.
[0057] (2) Set matching constraint parameters: To prevent all particle point clouds from being matched to a template point cloud, set the maximum number of times each template point cloud is allowed to be matched, and define a distance penalty term. Whenever a template point cloud is matched, the distance value from it to other particle point clouds is added to the distance matrix as a penalty.
[0058] (3) Perform heuristic matching and update the distance: Traverse each row of the distance matrix (i.e., process each particle point cloud one by one), find the column index corresponding to the minimum distance in the current row (i.e., the template point cloud index), and use this index as the matching result for the current particle. If the number of matches exceeds the upper limit, find the second closest distance, and so on.
[0059] (4) Output matching result index: After completing the matching of all particle point clouds, an m×1 index list is finally output to record the template point cloud index corresponding to each particle point cloud.
[0060] C. Deformation prediction modeling:
[0061] After initially completing the matching between the particle point cloud and the template point cloud, the present invention further designs a model for calculating the deformation variable to simulate the dynamic evolution of particles during flexible deformation. The specific process is as follows:
[0062] (1) Preprocessing and index matching of particle point cloud and template point cloud: According to the pre-executed heuristic matching algorithm (such as KMP), the template point cloud index corresponding to each particle point cloud is determined to provide a matching target for subsequent deformation modeling.
[0063] (2) Obtaining the displacement: The coordinates P(k) of the particle point cloud of the current iteration round are input into the encoder of the variational autoencoder (VAE) to obtain the encoding result, and then the encoding result is input into the decoder of the variational autoencoder (VAE) to generate the particle point cloud displacement vector Δ(k), which represents the distance that each particle point cloud should move.
[0064] (3) Point cloud deformation update: Use the predicted displacement vector to update the current particle point cloud to obtain a new round of iteration result P(k+1). The specific calculation is as follows:
[0065] P(k+1)=P(k)+Δ(k)
[0066] (4) Computing the structural loss (Loss) with the corresponding template point cloud: extract the corresponding template point cloud Q (determined by the index), calculate the Euclidean distance between the particle point P(k+1) and Q as the loss function to optimize the variational autoencoder VAE model;
[0067] (5) Determine the convergence condition: If the loss function value is lower than the set threshold, or the maximum number of iterations is reached, stop the iteration and execute (6); otherwise, return to step (2) to continue updating.
[0068] (6) Deformation accuracy evaluation: The intersection over union (IoU) is calculated to quantify the degree of structural consistency between the final particle point cloud and the template point cloud. Both are projected into the image space and the intersection over union (IoU) is calculated as a convergence indicator:
[0069] IoU=|P∩T| / |P∪T|
[0070] Where P represents the image area occupied by the current particle point cloud, and T represents the template point cloud area. The higher the IoU value, the closer the structures of the two are, indicating that the model deformation prediction is more accurate.
[0071] The present invention provides a flexible particle deformation prediction system based on deep learning, which includes image feature extraction, point cloud matching module, and deformation modeling module;
[0072] The image denoising and fusion module includes a Topaz denoising submodule, a low-pass filtering submodule, a non-local mean filtering submodule and an image weighted fusion submodule;
[0073] The point cloud extraction module includes an image binarization submodule and a contour extraction submodule;
[0074] The point cloud matching module is a KMP heuristic matching algorithm module;
[0075] The deformation modeling module is a deformation prediction module based on iterative displacement updating;
[0076] The classification output module includes an IoU calculation module and a conformation clustering module.
[0077] The image denoising and fusion module, the core pre-processing module of the system, performs triple denoising on the input cryo-EM raw images and generates a fused image with clear structure and low noise through weighted fusion. The Topaz submodule restores image details using a deep learning model, the low-pass filtering submodule suppresses high-frequency noise, and the NLM submodule preserves local texture information. The weighted fusion submodule ultimately outputs the fused image. The point cloud extraction module binarizes the fused image and extracts particle features by connecting regions. It then generates a particle point cloud and a template point cloud, providing the foundational data for subsequent matching and modeling.
[0078] The KMP heuristic matching module gradually establishes the correspondence between the particle point cloud and the template point cloud through the nearest neighbor and maximum matching upper limit and penalty terms, improving the matching speed and accuracy while maintaining structural coherence.
[0079] The deformable modeling module uses an iterative displacement prediction strategy based on the initial point pair to update the point cloud position in each round. The formula is: P(k+1)=P(k)+Δ(k), where P(k) represents the point cloud position in the kth round and Δ(k) is the current predicted displacement. This process continues iteratively until the point cloud converges with the template or the preset number of rounds is reached.
[0080] The IoU calculation module is used to evaluate the degree of fit between the converged particle point cloud and the template point cloud to measure the modeling effect, and finally the deformation variable is obtained to determine whether the particle is deformed;
[0081] like Figure 1 As shown, it is the weighting of multiple particle denoising methods.
[0082] like Figure 2 As shown in the figure, it is the feature extraction process of particles, including denoising 1 for denoising, 2 for binarization and connected area extraction, and 3 for point cloud initialization.
[0083] like Figure 3 As shown in the KMP matching visualization interface, the blue points are template point clouds, the red points are particle point clouds, and the gray lines represent matching relationships.
[0084] like Figure 4 As shown in Figure 1, the alignment process of the particle point cloud and the template point cloud is demonstrated in each iteration of the deformable modeling process.
[0085] like Figure 5 As shown in the figure, the system output interface visualizes a as a particle image, b as the deformed particle detected by the model, c as the model predicts the deformation process of a particle, d as the model predicts the deformation area of the particle, where the darker the color, the greater the degree of deformation, and e as the IOU of the initial particle point cloud and the template point cloud and the IOU of the particle point cloud and the template point cloud when the model converges.
[0086] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for predicting deformation of flexible particles based on deep learning, characterized in that: include: The first step is to perform noise reduction on the Cryo-ET original image and then perform binarization to generate particle point cloud and template point cloud; The second step is to match the particle point cloud with the template point cloud; The third step is deformation prediction: (1) Determine the template point cloud that matches each particle point cloud; (2) Input the coordinates P(k) of the particle point cloud of the current iteration round into the encoder of the variational autoencoder VAE to obtain the encoding result, and then input the encoding result into the decoder of the variational autoencoder VAE to generate the predicted value of the particle point cloud displacement vector Δ(k); (3) Use the predicted displacement vector to update the current particle point cloud position and obtain a new round of iteration result P(k+1): P(k+1)=P(k)+Δ(k); (4) Extract the corresponding template point cloud Q (determined by the index), calculate the Euclidean distance between the particle point cloud and the matching template point cloud as the loss function, and optimize the variational autoencoder VAE model; (5) Judgment: If the iteration stop condition is reached, the position information of the current particle point cloud is output to realize the deformation prediction of the flexible particle; Otherwise, return to step (2) to continue updating.
2. The method for predicting deformation of flexible particles based on deep learning according to claim 1, characterized in that: In the first step, the noise reduction method includes: subjecting the Cryo-ET original image to Topaz, low-pass filtering, and NLM denoising respectively to obtain a denoised image; and then weighted fusion of the three denoised images to generate the final denoised image.
3. The method for predicting deformation of flexible particles based on deep learning according to claim 2, characterized in that: In the first step, binarization is performed to generate a particle point cloud using the following methods: First, the threshold method is used to binarize the denoised image to separate the particle area from the background. Then, the maximum connected area in the binary image is extracted as a feature to initialize the point cloud and generate a particle point cloud.
4. The method for predicting deformation of flexible particles based on deep learning according to claim 2, wherein: The second step includes: Calculate the point cloud distance matrix: Calculate the distance between each particle point cloud and all template point clouds to obtain an m×n distance matrix, where m is the number of particle point clouds and n is the number of template point clouds; each element in the i-th row of the matrix represents the distance between the i-th particle point cloud and each template point cloud; Row heuristic matching and distance update: traverse each row of the distance matrix, find the column index corresponding to the minimum distance in the current row, and use this index as the matching result for the current particle; Output matching result index: After completing the matching of all particle point clouds, an m×1 index list is finally output to record the template point cloud index corresponding to each particle point cloud.
5. The method for predicting deformation of flexible particles based on deep learning according to claim 4, characterized in that: The second step further includes: Set matching constraint parameters: set the maximum number of times each template point cloud is allowed to be matched, and define a distance penalty item; every time a template point cloud is matched, the distance value from it to other particle point clouds is added to the distance matrix as a penalty.
6. The method for predicting deformation of flexible particles based on deep learning according to claim 5, characterized in that: In the second step, when performing heuristic matching and updating the distance, if the number of times the template point cloud is matched exceeds the upper limit, the second closest distance is found and so on.
7. The method for predicting deformation of flexible particles based on deep learning according to claim 1, characterized in that: The iteration stopping condition is: the loss function value is lower than the set threshold, or the maximum number of iteration rounds is reached.
8. The method for predicting deformation of flexible particles based on deep learning according to claim 1, wherein: The third step also includes deformation accuracy evaluation: calculating the intersection over union (IoU) of the final particle point cloud and the template point cloud to quantify the degree of structural fit.
9. A flexible particle deformation prediction system based on deep learning, characterized in that: Including image feature extraction module, point cloud matching module, and deformation modeling module: The image feature extraction module is used to perform the first step of denoising the Cryo-ET original image and then performing binarization processing to generate a particle point cloud and a template point cloud; The point cloud matching module is used to perform the second step of matching the particle point cloud with the template point cloud; The deformation modeling module is used to perform the third step, deformation prediction: (1) Determine the template point cloud that matches each particle point cloud; (2) Input the coordinates P(k) of the particle point cloud of the current iteration round into the encoder of the variational autoencoder VAE to obtain the encoding result, and then input the encoding result into the decoder of the variational autoencoder VAE to generate the predicted value of the particle point cloud displacement vector Δ(k); (3) Use the predicted displacement vector to update the current particle point cloud position and obtain a new round of iteration result P(k+1): P(k+1)=P(k)+Δ(k); (4) Extract the corresponding template point cloud Q (determined by the index), calculate the Euclidean distance between the particle point cloud and the matching template point cloud as the loss function, and optimize the variational autoencoder VAE model; (5) Judgment: If the iteration stop condition is reached, the position information of the current particle point cloud is output to realize the deformation prediction of the flexible particle; Otherwise, return to step (2) to continue updating.