Method for calibration of projection parameters in cryo-em image fiducial marker alignment

Abstract

The application provides a projection parameter calibration method in frozen electron microscope image fiducial mark alignment, and belongs to the technical field of electron microscope image processing. L 1 The property of norm and Laplace noise assumption are used to propose a new robust projection parameter calibration model to break through the defects of traditional robust methods, so that the influence of noise and outlier pollution in calibration point data can be better reduced.Based on the Laplace noise assumption, the application is more consistent with the noise and outlier distribution in the calibration point data; the numerical scheme based on the smoothing strategy can iteratively approximate the original problem, and can be more quickly and stably solved, so that the robust calibration of the projection parameter can be quickly and stably performed; the application avoids the problem of poor calibration effect caused by incomplete optimization of calibration point data or unreasonable weight distribution in traditional robust schemes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electron microscope image processing, in particular to a projection parameter calibration method in fiducial marker alignment of cryo-electron microscope images. BACKGROUND

[0002] In the three-dimensional reconstruction of cryo-electron microscope images, the fiducial marker alignment technology uses colloidal gold particles in the sample as calibration points to automatically align image sequences at different angles, and has been widely studied and applied.

[0003] As the last step of the fiducial marker alignment technology, the projection parameter calibration uses the calibration point position data measured by the previous step to optimize the projection parameters of each picture and the spatial point coordinates of each colloidal gold particle. However, due to inaccurate positioning of calibration points or matching errors, the measured calibration point data often contains noise and outliers, which will affect the parameter calibration to some extent. Therefore, how to perform robust parameter calibration to reduce the influence of noise and outliers in the calibration point data is an important research problem to improve the alignment accuracy of cryo-electron microscope images.

[0004] To solve the above problems, there are two common robust parameter calibration schemes. One scheme is to use probability statistical analysis or spatial geometric analysis techniques to iteratively optimize the calibration point data to remove noise and outliers. However, this method cannot perfectly remove the pollution in the data. Another scheme is to introduce a weighted iterative least squares technique, which assigns a weight value of 0 to 1 to each calibration point based on statistical analysis of the calibration point residual distribution and iteratively calibrates multiple times. However, this method may affect the final result due to unreasonable weight allocation. In addition, the mathematical models of the above two traditional robust calibration techniques are based on the assumption that the noise in the calibration point data is Gaussian distributed. However, in actual application, the positioning accuracy of the calibration points is mainly affected by the imaging quality of the point, and the noise distribution in the calibration point data should be more inclined to a part of the points with larger noise and the rest of the points with smaller noise, and the outliers caused by incorrect matching will increase this "two-end distribution" trend, so the more "extreme" Laplace distribution should be closer to the noise distribution in the calibration point data. Based on the Laplace assumption of the noise in the calibration point data, a bundle adjustment model based on L1 norm can be established for robust parameter calibration, but the existing numerical solution method has high running loss and is unstable, which cannot meet the application in the field of cryo-electron microscope image alignment. SUMMARY

[0005] The application aims to provide a projection parameter calibration method in frozen electron microscope image fiducial mark alignment, which uses a robust projection parameter calibration model to reduce the influence of noise and outliers in calibration point data on image alignment effect, overcomes the limitations of traditional robust models, and solves the problems of low efficiency and instability of existing solutions.

[0006] To achieve the above-mentioned purpose, the specific technical scheme adopted by the application is as follows:

[0007] A projection parameter calibration method in frozen electron microscope image fiducial mark alignment, comprising the following steps:

[0008] S1: taking the measured calibration point data and the initial estimation of the projection parameter as input;

[0009] S2: establishing a L1 norm binding adjustment model based on minimizing the re-projection residual, to obtain a non-convex non-smooth optimization problem;

[0010] S3: obtaining a smooth approximation problem based on a smoothing strategy;

[0011] S4: performing iterative numerical solution on the smooth approximation problem to obtain the final reconstruction result.

[0012] Further, in S1: the microscope takes an image sequence of a biological sample containing m colloidal gold particles from n different angles, defines q ij =(u ij ,v ij ) T is the calibration point of the jth colloidal gold particle in the ith image, C i is the projection parameter of the ith image, X j =(x j ,y j ,z j ) T is the spatial point coordinate of the jth colloidal gold particle, C=(C1,C2…C n ) T , X=(X1,X2…X m ) T , E=(C,X) T , p ij =(u i ′ j ,v i ′ j ) T =Proj(C i ,X j ) is the re-projection of the jth colloidal gold particle in the ith image.

[0013] Further, in the S2, a bundle adjustment model of L1 norm is constructed by using Laplace noise assumption and the property of L1 norm:

[0014]

[0015] where ||.||1 is L1 norm.

[0016] Further, in the S3, a smoothing approximation problem is derived:

[0017] Smoothing approximation: the non-smoothness in the bundle adjustment model of L1 norm comes from the absolute value function f(x)=|x|, and a smoothing function is selected, where τ is called smoothing parameter, and thus an approximate optimization problem of the bundle adjustment model of L1 norm is obtained:

[0018]

[0019] Smoothing parameter updating strategy: for the smoothing parameter τ in the approximate problem, a given initial value τ0 (the default value is 1) is given, and the value of τ is iteratively reduced, so that the approximate problem gradually approaches the original problem, and the following updating strategy is adopted: the updating time of the smoothing parameter is when the following inequality is satisfied:

[0020]

[0021] where represents the gradient at the point E, and the value of ω is 1000 by default, and the updating method of the smoothing parameter is:

[0022] τ k+1 = δτ k

[0023] where δ is 0.05 by default.

[0024] Further, in the S4, a numerical solution method:

[0025] S4-1: smoothing gradient method

[0026] For the approximate problem, a gradient method with Armijo line search is used for iterative numerical solution, which is described as follows: for the kth step iteration, the forward direction g k is the negative gradient direction of the current position E k The step size α k is the maximum value in the set {ρ 0 ,ρ 1 ,ρ 2 …} (ρ is 0.5 by default) that satisfies the following inequality:

[0027] ​

[0028] Then the update of the parameter in each iteration is:

[0029] E k+1 = E k + a k g k

[0030] S4-2: Z-score normalization

[0031] In order to improve the stability of the algorithm, Z-score normalization is used for data preprocessing, and inverse normalization is taken for data recovery after the end of iterative solution. The mean of the measured calibration point coordinate data is denoted as μ, and the standard deviation is denoted as σ. The normalization process is described as follows:

[0032]

[0033] The inverse normalization process is the reverse of the above process.

[0034] Compared with the prior art, the beneficial effects of the present application are:

[0035] Based on the properties of L1 norm and Laplace noise assumption, the present application proposes a new robust projection parameter calibration model to overcome the defects of traditional robust methods, so as to better reduce the influence of noise and outlier pollution in the calibration point data. Based on the Laplace noise assumption, the present application is more consistent with the noise and outlier distribution in the calibration point data; the numerical scheme based on the smoothing strategy can iteratively approximate the original problem, and can be solved more quickly and stably, so as to quickly and stably perform robust calibration of the projection parameter; the present application avoids the problem of poor calibration effect caused by incomplete optimization of calibration point data or unreasonable weight distribution in traditional robust schemes. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a projection parameter calibration diagram based on minimization of reprojection residual.

[0037] Figure 2 is a basic flowchart of the method described in the present application.

[0038] Figure 3 is a schematic diagram of the public data set used in Example 2. DETAILED DESCRIPTION

[0039] The technical solutions described in the present application will be further described and explained in conjunction with the embodiments.

[0040] Example 1

[0041] Projection parameter calibration is generally achieved by minimizing the residuals between the measured coordinates of the calibration points and the reprojected coordinates of the estimated spatial points, as shown in Figure 1 When measuring the calibration point data, noise is generated due to inaccurate positioning in the positioning stage, and outliers are generated due to incorrect matching in the matching stage, which affects the final alignment effect. The existing two robust solutions have their own limitations and are based on the Gaussian noise assumption which may not be appropriate.

[0042] The present application uses the properties of L1 norm and Laplace noise assumption to obtain a new robust calibration model, and designs a new numerical solution scheme based on the smoothing strategy, as shown in Figure 2 The specific model construction, smoothing approximation strategy and numerical solution method are as follows:

[0043] 1. L1 norm binding model construction:

[0044] 1.1 Basic concept: assuming that a microscope takes image sequences of a biological sample containing m colloidal gold particles from n different angles, define q ij =(u ij ,v ij ) T is the jth colloidal gold particle in the ith image, C i is the projection parameter of the ith image, X j =(x j ,y j ,z j ) T is the spatial point coordinate of the jth colloidal gold particle, C=(C1,C2…C n ) T , X=(X1,X2…X m ) T , E=(C,X) T , p ij =(u′ ij ,v′ ij ) T =Proj(C i ,X j ) is the re-projection of the jth colloidal gold particle in the ith image.

[0045] 1.2 Projection model construction: the present application is applicable to all projection models, and an example is given here to illustrate the calculation details of p ij . Assuming C i =(s i ,α i ,β i ,γ i ,t 1it 0i ) T Pi is the projection parameter of the ith image, where s represents the change of scale of the projected image, γ represents the in-plane rotation of the projected image, a represents the tilt angle of the rotation axis of the projected image in the imaging process, β represents the tilt angle of the rotation axis of the projected image in the imaging process, and t0 and t1 represent the translation variables of the projected image in the x direction and the y direction. Therefore, the orthogonal projection model can be represented as:

[0046]

[0047] wherein:

[0048]

[0049] 1.3 Reprojection residual construction: Generally, the projection parameter calibration is converted into a bundle adjustment model, which is described as a least squares problem of minimizing the reprojection residual as follows:

[0050]

[0051] where ψ(.) is the residual function, d(.) is the distance function, Φ ij is a mask representing whether the calibration point is visible or not, with a value of 0 or 1. Using the Laplace noise assumption and the properties of the L1 norm, a bundle adjustment model of the L1 norm is constructed as follows:

[0052]

[0053] where ||.||1 is the L1 norm.

[0054] 2 Smooth approximation problem derivation:

[0055] 2.1 Smooth approximation: The non-smoothness in the bundle adjustment model of the L1 norm comes from the absolute value function f(x) = |x|. We choose a smooth function to approximate it, where τ is called the smoothing parameter. Thus, an approximate optimization problem of the bundle adjustment model of the L1 norm is obtained as follows:

[0056]

[0057] 2.2 Smoothing parameter update strategy: For the smoothing parameter τ in the approximate problem, given an initial value τ0 (the default value is 1), its value is iteratively reduced, so that the approximate problem gradually approaches the original problem. The following update strategy is adopted. The update time of the smoothing parameter is when the following inequality is satisfied:

[0058]

[0059] where denotes The gradient at point E, ω value is 1000 by default. The smoothing parameter is updated as follows:

[0060] τ k+1 = δτ k

[0061] where δ is 0.05 by default.

[0062] 3 Numerical solution method:

[0063] 3.1 Smooth gradient method

[0064] For the approximation problem, the gradient method with Armijo line search is used for iterative numerical solution, described as follows: for the kth step iteration, the forward direction g k is the negative gradient direction of the current position E k . The step size α k is the maximum value in the set {ρ 0 ,ρ 1 ,ρ 2 …} (ρ is 0.5 by default) that satisfies the following inequality:

[0065]

[0066] Then the update of each iteration parameter is:

[0067] E k+1 = E k + α k g k .

[0068] 3.2 Z-score normalization

[0069] In order to improve the stability of the algorithm, Z-score normalization is used for data preprocessing, and inverse normalization is used for data recovery after iterative solution. Denote the mean of the measured calibration point coordinate data as μ and the standard deviation as σ. The normalization process is described as follows:

[0070]

[0071] The inverse normalization process is the reverse of the above process.

[0072] Example 2:

[0073] This example selects four sets of public data sets with different characteristics and different scales, V-1Gag data set, VEEV data set, Centriole data set and SNX3 data set, and compares them with general LM method and existing L1 interior point method. The images of the four sets of public data sets at 0° are shown in Figure 3 .

[0074] Table 1 Comparison results of different methods on different data sets

[0075]

[0076]

[0077] For each data set, the pre-marking processing is carried out by using the software Marker auto, including the marking point detection, the matching and the initial estimation of the projection parameters, and the initial estimation of the spatial point coordinates of the colloidal gold particles is obtained through the triangulation measurement. The experimental results are shown in Table 1. In terms of the method effect, the L1 residual error obtained by the smoothing method of the application is the smallest, which indicates that the highest precision result is obtained; in terms of the method efficiency, the time consumption of the smoothing method of the application is far lower than that of the interior point method, and the memory consumption is the lowest, which indicates the high efficiency and low loss of the smoothing method of the application, and the smoothing method of the application has the practical application prospect. The above results all indicate the high robustness and high efficiency of the L1 smoothing method of the application.

[0078] Finally, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and the person skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be properly combined to form other embodiments which can be understood by the person skilled in the art.

Claims

1. A method for calibrating projection parameters in alignment of fiducial markers in cryo-electron microscope images, characterized in that: The following steps are involved: S1: Takes the measured calibration point data and the initial estimate of the projection parameters as input; S2: Based on minimizing the reprojection residual, a bundle adjustment model with L1 norm is established, resulting in a non-convex and non-smooth optimization problem; S3: Based on the smoothing strategy, a smooth approximation problem is obtained; in S3, the smooth approximation problem is derived as follows: Smoothing approximation: The non-smoothness in the L1 norm bundle adjustment model comes from the absolute value function f(x) = |x|, and the smoothing function is selected It is approximated, where τ is called the smoothing parameter, thus obtaining an approximate optimization problem of the bundled adjustment model of the L1 norm: Using the following update strategy, the smoothing parameter is updated when the following inequality holds: in express The gradient at point E, ω value defaults to 1000, and the smoothing parameter is updated as follows: t k+1 =s k The default value of δ is 0.05; S4: Iteratively numerically solve the smooth approximation problem to obtain the final reconstruction result.

2. The projection parameter calibration method according to claim 1, wherein: In S1, a biological sample containing m colloidal gold particles is photographed from n different angles to obtain an image sequence, and q is defined as ij =(u ij ,v ij ) T is the calibration point obtained by imaging the j-th colloidal gold particle in the i-th image, C i is the projection parameter of the i-th image, X j =(x j ,y j ,z j ) T is the spatial coordinate of the jth colloidal gold particle, C=(C1, C2…C n ) T ,X=(X1,X2…X m ) T , E=(C,X) T , p ij =(u′ ij ,v′ ij ) T =Proj(C i ,X j ) is the reprojection of the j-th colloidal gold particle in the i-th image.

3. The projection parameter calibration method according to claim 1, wherein: In S2, the following L1 norm bundling adjustment model is constructed by using the Laplace noise assumption and the properties of the L1 norm: Among them, ‖.‖1 is the L1 norm, Φ ij A mask indicating whether the calibration point is visible, with a value of 0 or 1.

4. The projection parameter calibration method according to claim 1, wherein: In S4, the numerical solution method is: S4-1: Smooth Gradient Method For the approximation problem, the gradient method with Armijo line search is used for iterative numerical solution: for the kth iteration, the forward direction g k is the current position E k The negative gradient direction Step size α k For the set {ρ 0 ,ρ 1 ,ρ 2 …} (ρ defaults to 0.5) that satisfies the following inequality: Then the update of each iterative parameter is: E k+1 =E k +α k g k S4-2: Z-score normalization: Z-score normalization is used to preprocess the data. After the iterative solution is completed, inverse normalization is used to recover the data. The mean of the measured calibration point coordinate data is μ, and the standard deviation is σ. The normalization process is described as follows: The process of inverse normalization is the inverse of the above process.

Images