A mismatch removal method for cryo-electron tomography images

Abstract

The present invention discloses a method for removing mismatches from cryo-electron tomography images, comprising: S1, screening stable matches from a set of cryo-electron tomography image pairs, and extracting motion information between feature points as training samples; S2, inputting the training samples obtained in S1 into a pre-constructed nonlinear regression model, and fitting a recognition function for calculating the correct probability of all matches of the cryo-electron tomography image pairs through training; S3, based on the recognition function obtained through training fitting in S2, calculating the cryo-electron tomography image pairs to obtain an initial matching group; S4, repeating S1, S2, and S3 until an initial matching group for all cryo-electron tomography image pairs is obtained; and S5, based on all initial matching groups, completing the final mismatch removal using the RANSAC algorithm. This method utilizes the motion consistency between electron tomography projection image sequences to remove mismatches, effectively preserving the correct matches between images while improving the accuracy of alignment results.

Description

Technical Field

[0001] The present invention relates to the technical field of structural biology, and in particular to a method for removing mismatches from cryo-electron tomography images. Background Art

[0002] Cryo-electron microscopy combined with three-dimensional reconstruction technology has developed rapidly in the field of structural biology in recent years and is making important breakthroughs. Compared with traditional methods for determining the three-dimensional structure of protein molecules, such as X-ray crystallography and nuclear magnetic resonance spectroscopy, it has the following advantages:

[0003] 1. Maintain the activity and functional status of biological samples;

[0004] 2. No need to prepare crystals, especially suitable for determining the three-dimensional structure of macromolecules and their complexes that are difficult to crystallize;

[0005] 3. Combined with new electron microscopes, sample preparation robots and other equipment and technologies, the entire process of microscopic sample preparation, data collection, and three-dimensional reconstruction can be automated or semi-automated, laying the foundation for high-throughput and rapid analysis of the three-dimensional structure of macromolecules and their complexes.

[0006] The fundamental purpose of feature point registration is to establish a matching correspondence between feature point sets of two or more images. It is widely used in image registration, three-dimensional reconstruction, target positioning and recognition, etc. Due to the influence of factors such as lighting conditions, noise, geometric transformation, and spatial distortion, achieving completely accurate feature matching is a very challenging task.

[0007] Currently, the main feature operators used for image registration include SIFT, SURF, and ORB. These methods can achieve relatively accurate feature registration, but when it comes to cryo-electron tomography images, because each feature point is at the nanometer level or even higher, conventional methods are not well suited.

[0008] Patent document CN109949348SA discloses a method for removing mismatches based on superpixel motion statistics. The method includes extracting, describing, and matching features from two images to be matched; segmenting the images to be matched using an improved superpixel segmentation algorithm to obtain two superpixel label maps; and establishing a superpixel motion statistics model based on the superpixel label maps, which is used to automatically filter feature points for non-rigidly deformed image registration. This method eliminates mismatches of feature points for non-rigidly deformed image registration by optimizing the existing segmentation algorithm and introducing a motion statistics model. However, this method is not suitable for nanometer-scale image matching and is specific to non-rigidly deformed images.

[0009] Patent document CN113065566A discloses a method, system, and application for removing mismatches. The method involves extracting and matching SIFT and SURF feature points from a dataset, using RANSAC to identify inliers, and then filtering out 3D points from the point cloud using an AC-RANSAC adaptive threshold algorithm with modified parameters. This method can remove mismatches from images without requiring excessive parameter settings. However, it requires processing within a 3D point cloud and is not suitable for nanometer-scale image matching. Summary of the Invention

[0010] In order to solve the above problems, the present invention provides a mismatch removal method for cryo-electron tomography images. This method is based on the existing image feature mismatch removal method. For nano-level cryo-electron tomography images, mismatch removal is performed by increasing motion consistency. While ensuring correct matching between cryo-electron tomography images, the accuracy of the alignment results is improved, thereby obtaining a high-quality three-dimensional reconstructed model.

[0011] A method for removing mismatches from cryo-electron tomography images, comprising:

[0012] S1 screens stable matches from a set of cryo-electron tomography image pairs and extracts motion information between feature points as training samples;

[0013] S2 inputs the training samples obtained in S1 into a pre-built nonlinear regression model, and fits a recognition function for calculating the correct probability of all matches of the cryo-electron tomography images through training;

[0014] S3 calculates the recognition function obtained based on the training and fitting in S2 on the cryo-electron tomography image pair to obtain an initial matching group;

[0015] S4 repeats S1, S2, and S3 until an initial matching group of all cryo-electron tomography image pairs is obtained;

[0016] S5 completes the final mismatch removal work based on all initial matching groups through the RANSAC algorithm.

[0017] Preferably, the screening in S1 specifically involves first calculating the neighbor ratio between feature points, and then selecting feature point pairs with a ratio less than a threshold as stable matches.

[0018] Preferably, the threshold is 0.2-0.4, wherein the higher the threshold, the more stable matches there are, and the more time is needed for screening; conversely, the fewer stable matches there are, and it is more likely that there will be no stable matches.

[0019] Preferably, the motion information includes the coordinates of the matching point in the first cryo-electron tomography image and the relative motion offset of the matching point in the second cryo-electron tomography image that matches it.

[0020] Preferably, the training samples include motion information and a matrix corresponding to the motion information.

[0021] Preferably, the nonlinear regression model in S2 is a smoothing function evaluated based on motion consistency as a smoothing penalty term, optimized by a Huber loss function, and finally derived to obtain a recognition function.

[0022] Preferably, the initial matching group obtained in S3 is a match whose result is greater than a preset value after calculation by a recognition function.

[0023] Preferably, the preset value is 0.8-0.95, wherein the smaller the preset value is, the more matches there are in the initial matching group, but the accuracy is not high; conversely, the smaller the number of matches is, the higher the accuracy is, but some correct matches are easily removed.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) By introducing the motion consistency prior information between image sequences, the accuracy of mismatch removal between nanoscale images is improved.

[0026] (2) After removing mismatches between pairs of images, a second mismatch removal is performed on the entire image set using conventional methods to ensure the correctness of the matching between cryo-electron tomography images in high-noise conditions and improve the accuracy of the alignment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic flow chart of the mismatch removal method provided by the present invention;

[0028] Figure 2 This is the reprojection error map after removing mismatches using the SIFT method for the data in this embodiment;

[0029] Figure 3 This is the reprojection error map of the data in this embodiment after removing mismatches using the method provided by the present invention. DETAILED DESCRIPTION

[0030] A set of experimental data was set, with the projection angle range from -60° to 60° and the interval between each projection being 3°. A total of 41 cryo-electron tomography projection images were taken, each with a size of 5760×4092.

[0031] For this set of data, Figure 1 As shown, the projection image of the cryo-electron tomography is mismatched and removed, including:

[0032] S1 screens stable matches from a set of cryo-electron tomography image pairs, extracts motion information between feature points as training samples, first calculates the neighbor ratio between feature points, sets a threshold of 0.4, and then selects matching feature points with a neighbor ratio < the threshold as stable matches as the training set.

[0033] Data points in the training set:

[0034]

[0035] where p i is the motion information of the i-th matching pair, is the identification label, where label 1 represents that the match is correct, x i and y i Indicates the coordinates of the matching point in the first image, dx i and dy i Indicates the relative motion offset between this point and the matching point in the second image.

[0036] Among them, t1, t2, t3, and t4 represent the transformation of the neighborhood around the two feature points according to the direction and scale of the feature points:

[0037]

[0038] S2 inputs the training samples obtained in S1 into a pre-built nonlinear regression model, and fits the recognition function f(p):p→q for calculating the correct probability of all matches of the cryo-electron tomography image through training, where p is the motion information data point of the mismatched image to be removed, and q is the correct probability of the output.

[0039] The specific derivation process of the identification function:

[0040] Let f(p i )=φ(p i ),φ(p i ) is a smoothing function evaluated using motion consistency as a smoothing penalty term, which is as follows:

[0041]

[0042] Represents the function φ(p i ), Represents the Fourier transform of the Gaussian function, which achieves smoothing by penalizing high-frequency terms.

[0043] Known training set The smoothest function f(p i), according to the energy minimization principle, the above formula can be transformed into:

[0044]

[0045] Where E is energy and C(·) represents the Huber loss function:

[0046]

[0047] When ||z||>∈, the loss function becomes the mean absolute error, ensuring that the model updates parameters at a faster speed;

[0048] When ||z||≤∈, it becomes the mean square error and the gradient gradually decreases, which can ensure that the model obtains the global optimal value more accurately.

[0049] The first task of the above energy function is to make the estimated value of the function output obtained by regression as close as possible to the true observation value; the second task is to smooth the penalty term to make the obtained function as smooth as possible.

[0050] In order to calculate the function f(p i ), take the derivative of the energy function and set the reciprocal to zero:

[0051]

[0052] Where w(i) represents an N×1 dimensional vector, which serves as a placeholder for temporarily unknown variables, and N represents the number of data points used for function regression.

[0053] Arrange the above formula:

[0054]

[0055] Then after inverse Fourier transform, we can get:

[0056]

[0057] where g(p,p i ) represents the Gaussian radial basis function.

[0058] Then insert the equation with smooth penalty term into the above formula:

[0059] Ψ=w T Gw

[0060] Where w is the variable in the identification function, w T is the transposed matrix of w, and G is a symmetric matrix:

[0061]

[0062] Then φ(p i ) and E are brought into the energy function:

[0063]

[0064] Finally, the global minimum is obtained by gradient descent method:

[0065]

[0066] That is, the motion information of the corresponding feature points is brought in to obtain the corresponding correct probability.

[0067] S3 obtains the recognition function based on the training and fitting in S2, calculates the pair of cryo-electron tomography images, obtains the initial matching group, and sets the value to 0.9. When the match is found, the match is retained. S4 repeats S1, S2, and S3 until the initial matching group of all cryo-electron tomography image pairs is obtained;

[0068] S5 completes the final mismatch removal work based on all initial matching groups through the RANSAC algorithm, where the RANSAC algorithm is a common technical means, and the process will not be repeated here.

[0069] Comparison table of removal of mismatch results based on randomly sampled experimental data:

[0070]

[0071] As shown in the table, the inlier rate of the method provided by the present invention is higher than that of the traditional method, which indicates that the accuracy rate of removing false matches of the method is high.

[0072] like Figure 2 As shown in the figure, the reprojection error map is directly processed by the SIFT method to remove the mismatch; Figure 3 As shown in FIG, this is a reprojection error map after the mismatch removal method provided by the present invention removes the mismatch; the dotted line is the location of the average error. According to the comparison of the two error maps, it can be seen that the reprojection error distribution of the mismatch removal method provided by the present invention is more concentrated in the position with smaller errors than the SIFT method, and the average error is also smaller than the traditional SIFT algorithm. Therefore, it shows that the method provided by the present invention has certain advantages over the traditional SIFT algorithm.

Claims

1. A method for removing mismatches from cryo-electron tomography images, comprising: S1 screens stable matches from a set of cryo-electron tomography image pairs and extracts motion information between feature points as training samples. The training samples include motion information and a motion information corresponding matrix, which is expressed as follows: where p i is the motion information of the i-th matching pair, is the identification label, where label 1 represents that the match is correct, x i and y i Indicates the coordinates of the matching point in the first image, dx i and dy i Indicates the relative motion offset between the point and the matching point in the second image; Among them, t1, t2, t3, and t4 represent the transformation of the neighborhood around the two feature points according to the direction and scale of the feature points: S2 inputs the training samples obtained in S1 into a pre-built nonlinear regression model, and fits the recognition function for calculating the correct probability of all matches of the cryo-electron tomography images through training. The nonlinear regression model is a smoothing function evaluated based on motion consistency as a smoothing penalty term, optimized by the Huber loss function, and finally derived to obtain the recognition function, which is expressed as follows: That is, the motion information of the corresponding feature points is brought in to obtain the corresponding correct probability, where w(i) represents an N×1 dimensional vector, which serves as a placeholder for temporarily unknown variables, and N represents the number of data points used for function regression; S3 calculates the recognition function obtained based on the training and fitting in S2 on the cryo-electron tomography image pair to obtain an initial matching group; S4 repeats S1, S2, and S3 until an initial matching group of all cryo-electron tomography image pairs is obtained; S5 completes the final mismatch removal work based on all initial matching groups through the RANSAC algorithm.

2. The mismatch removal method according to claim 1, wherein: The screening in S1 specifically involves first calculating the neighbor ratio between feature points, and then selecting feature point pairs with a value smaller than a threshold as stable matches.

3. The mismatch removal method according to claim 2, wherein: The threshold value is 0.2 to 0.

4.

4. The mismatch removal method according to claim 1, wherein: The training sample includes motion information and a matrix corresponding to the motion information.

5. The mismatch removal method according to claim 1, wherein: The initial matching group obtained in S3 is obtained by calculating the recognition function and retaining the matching results greater than a preset value.

6. The mismatch removal method according to claim 5, wherein: The preset value is 0.8 to 0.95.

Images