Projection parameter calibration method in alignment of image reference marks of cryoelectron microscope

Abstract

The invention provides a projection parameter calibration method in alignment of fiducial marks of an image of a cryoelectron microscope, and belongs to the technical field of image processing of electron microscopes. According to the method, a new robust projection parameter calibration model is provided based on the property of the L1 norm and the Laplace noise hypothesis to break through the defects of a traditional robust method, so that the influence of noise and outlier pollution in calibration point data can be better reduced. The method is based on a Laplacian noise hypothesis, and is more in line with noise and outlier distribution in calibration point data; the numerical scheme based on the smoothing strategy can iteratively approach to an original problem, and can more quickly and stably solve, so that robust calibration of projection parameters can be quickly and stably carried out; according to the method, the problem of poor calibration effect caused by incomplete calibration point data optimization or unreasonable weight distribution in a traditional robust scheme is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of electron microscope image processing, and in particular to a projection parameter calibration method in alignment of reference marks in cryo-electron microscope images. Background Art

[0002] In the three-dimensional reconstruction of cryo-electron microscopy images, the fiducial point alignment technology, which uses the colloidal gold particles in the sample as calibration points to automatically align image sequences at different angles, has been widely studied and applied.

[0003] As the last step of the reference point alignment technology, projection parameter calibration uses the calibration point position data measured in the previous step to optimize the projection parameters of each image and the spatial point coordinates of each colloidal gold particle. However, due to inaccurate positioning of the calibration points or matching errors, the measured calibration point data usually contains noise and outlier contamination, which will affect the effect of parameter calibration to a certain extent. Therefore, how to perform robust parameter calibration to reduce the impact of noise and outliers in the calibration point data is an important research issue to improve the accuracy of cryo-electron microscopy image alignment.

[0004] There are two common robust parameter calibration schemes for the above problems. One scheme is to use probability statistical analysis or spatial geometric analysis technology to iteratively optimize the calibration point data to remove noise and outliers, but such methods usually cannot perfectly remove the pollution in the data. Another scheme is to introduce weighted iterative least squares technology, assign a weight of 0 to 1 to each calibration point based on the statistical analysis of the residual distribution of the calibration points and iterate the calibration multiple times, but such methods may affect the final effect due to unreasonable weight distribution. In addition, the mathematical models of the above two traditional robust calibration techniques are established based on the assumption that the noise in the calibration point data is Gaussian distributed. However, in the actual application process, since the accuracy of calibration point positioning is mainly affected by the imaging quality of the point, the noise distribution in the calibration point data should be more inclined to a part of the point with large noise and the rest of the point with small noise, and the outliers caused by wrong matching will increase the trend of this "two-end distribution", so the more "extreme" Laplace distribution should be closer to the noise distribution in the calibration point data. Based on the Laplace assumption of calibration point data noise, a bundling adjustment model based on the L1 norm can be established for robust parameter calibration. However, the existing model numerical solution method has high running loss and is unstable, which cannot meet the application requirements in the field of cryo-electron microscopy image alignment. Summary of the invention

[0005] The purpose of the present invention is to provide a projection parameter calibration method for the alignment of reference markers in cryo-electron microscope images, and to use a robust projection parameter calibration model to reduce the influence of noise and outliers in the calibration point data on the image alignment effect, so as to overcome the limitations of traditional robust models and solve the problems of low efficiency and instability of existing solutions.

[0006] In order to achieve the above object, the specific technical solution adopted by the present invention is:

[0007] A method for calibrating projection parameters in alignment of fiducial markers in cryo-electron microscope images comprises the following steps:

[0008] S1: Take the measured calibration point data and the initial estimate of the projection parameters as input;

[0009] S2: Based on minimizing the reprojection residual, a bundle adjustment model with L1 norm is established, resulting in a non-convex and non-smooth optimization problem;

[0010] S3: Based on the smoothing strategy, a smooth approximation problem is obtained;

[0011] S4: Iteratively numerically solve the smooth approximation problem to obtain the final reconstruction result.

[0012] Furthermore, in S1: a microscope photographs a biological sample containing m colloidal gold particles from n different angles to obtain an image sequence, and defines q ij =(u ij ,v ij ) T is the calibration point obtained by imaging the jth colloidal gold particle in the i-th image, C i is the projection parameter of the i-th image, X j =(x j ,y j ,z j ) T is the spatial coordinate of the jth colloidal gold particle, C=(C1,C2…C n ) T , X=(X1,X2…X m ) T , E=(C,X) T , p ij =(u i ' j ,v i ' j ) T =Proj(C i ,X j ) is the reprojection of the j-th colloidal gold particle in the i-th image.

[0013] Furthermore, in S2, the following L1 norm bundling adjustment model is constructed by using the Laplace noise assumption and the properties of the L1 norm:

[0014]

[0015] Where ‖.‖1 is the L1 norm.

[0016] Furthermore, in S3, the smoothing approximation problem is derived as follows:

[0017] Smoothing approximation: The non-smoothness in the L1 norm bundle adjustment model comes from the absolute value function f(x) = |x|, and the smoothing function is selected It is approximated, where τ is called the smoothing parameter, thus obtaining an approximate optimization problem of the bundled adjustment model of the L1 norm:

[0018]

[0019] Smoothing parameter update strategy: For the smoothing parameter τ in the approximate problem, the initial value τ0 (the default value is 1) is given, and its value is iteratively reduced, so that the approximate problem gradually approaches the original problem. The following update strategy is adopted here; the smoothing parameter is updated when the following inequality holds:

[0020]

[0021] in express The gradient at point E, the ω value defaults to 1000, and the smoothing parameter is updated as follows:

[0022] τ k+1 =δτ k

[0023] The default value of δ is 0.05.

[0024] Furthermore, in S4, the numerical solution method is:

[0025] S4-1: Smooth Gradient Method

[0026] For the approximate problem, the gradient method with Armijo line search is used for iterative numerical solution, which is described as follows: For the kth iteration, the forward direction g k is the current position E k The negative gradient direction Step size α k For the set {ρ 0 ,ρ 1 ,ρ 2 …} (ρ defaults to 0.5) that satisfies the following inequality:

[0027]

[0028] Then the update of each iteration parameter is:

[0029] E k+1 =E k +α k g k

[0030] S4-2: Z-score normalization

[0031] In order to improve the stability of the algorithm, Z-score normalization is used to preprocess the data. After the iterative solution is completed, inverse normalization is used to restore the data. The mean value of the coordinate data of the calibration points measured is μ, and the standard deviation is σ. The normalization process is described as:

[0032]

[0033] The process of inverse normalization is the inverse of the above process.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention proposes a new robust projection parameter calibration model based on the properties of the L1 norm and the Laplace noise hypothesis to overcome the defects of the traditional robust method, so as to better reduce the influence of noise and outlier pollution in the calibration point data. The present invention is based on the Laplace noise hypothesis and is more consistent with the distribution of noise and outliers in the calibration point data; the numerical scheme based on the smoothing strategy can iteratively approximate the original problem and can solve it more quickly and stably, so as to quickly and stably perform robust calibration of projection parameters; the present invention avoids the problem of poor calibration effect caused by incomplete optimization of calibration point data or unreasonable weight distribution in traditional robust schemes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of projection parameter calibration based on minimizing the reprojection residual.

[0037] Figure 2 It is a basic flow chart of the method of the present invention.

[0038] Figure 3 Schematic diagram of the public data set used in Example 2. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further described below in conjunction with embodiments.

[0040] Example 1

[0041] Projection parameter calibration is generally achieved by minimizing the residual between the coordinates of the measured calibration points and the estimated reprojected coordinates of the spatial points, such as Figure 1 As shown in Figure 2. When measuring the calibration point data, noise will be generated in the positioning stage due to inaccurate positioning, and outliers will be generated in the matching stage due to incorrect matching. These noises and outliers will affect the final alignment effect. The two existing robust solutions have their own limitations and are based on Gaussian noise assumptions that may not be appropriate.

[0042] The present invention uses the properties of the L1 norm and the Laplace noise assumption to obtain a new robust calibration model, and designs a new numerical solution based on the smoothing strategy, such as Figure 2 As shown in the figure, the specific model construction, smoothing approximation strategy and numerical solution method are as follows:

[0043] 1L1 norm bundle adjustment model construction:

[0044] 1.1 Basic Concepts: Assume that a microscope takes a biological sample containing m colloidal gold particles from n different angles to obtain an image sequence. Define q ij =(u ij ,v ij ) T is the calibration point obtained by imaging the jth colloidal gold particle in the i-th image, C i is the projection parameter of the i-th image, X j =(x j ,y j ,z j ) T is the spatial coordinate of the jth colloidal gold particle, C=(C1,C2…C n ) T , X=(X1,X2…X m ) T , E=(C,X) T , p ij =(u′ ij ,v′ ij ) T =Proj(C i ,X j ) is the reprojection of the j-th colloidal gold particle in the i-th image.

[0045] 1.2 Projection Model Construction: The present invention is applicable to all projection models. Here is an example to illustrate the p ij Assume that C i =(s i ,α i ,β i ,γ i ,t 1i,t 0i ) T is the projection parameter of the i-th image, where s represents the change in the scale of the projected image, γ represents the intra-slice rotation of the projected image, α represents the tilt angle of the projected image around the rotation axis during the imaging process, β represents the tilt angle of the projected image around the rotation axis during the imaging process, t0 and t1 represent the translation variables of the projected image in the x-direction and y-direction, then, the orthogonal projection model can be expressed as:

[0046]

[0047] in:

[0048]

[0049] 1.3 Reprojection residual construction: Generally, the projection parameter calibration is transformed into a bundle adjustment model, which is described as the following least squares problem of minimizing the reprojection residual:

[0050]

[0051] Where ψ(.) is the residual function, d(.) is the distance function, Φ ij Is a mask indicating whether the calibration point is visible, and its value is 0 or 1. Using the Laplace noise assumption and the properties of the L1 norm, the following L1 norm bundling adjustment model is constructed:

[0052]

[0053] Where ‖.‖1 is the L1 norm.

[0054] 2 Derivation of smoothing approximation problem:

[0055] 2.1 Smoothing approximation: The non-smoothness in the L1-norm bundle adjustment model comes from the absolute value function f(x) = |x|. We use the smoothing function It is approximated, where τ is called the smoothing parameter. Thus, an approximate optimization problem of the bundled adjustment model of the L1 norm is obtained:

[0056]

[0057] 2.2 Smoothing parameter update strategy: For the smoothing parameter τ in the approximate problem, the initial value τ0 (the default value is 1) is given, and its value is iteratively reduced, so that the approximate problem gradually approaches the original problem. Here, the following update strategy is adopted. The smoothing parameter is updated when the following inequality holds:

[0058]

[0059] in express The gradient at point E, ω value defaults to 1000. The smoothing parameter is updated as follows:

[0060] τ k+1 =δτ k

[0061] The default value of δ is 0.05.

[0062] 3 Numerical solution method:

[0063] 3.1 Smooth Gradient Method

[0064] For the approximate problem, the gradient method with Armijo line search is used for iterative numerical solution, which is described as follows: For the kth iteration, the forward direction g k is the current position E k The negative gradient direction Step size α k For the set {ρ 0 ,ρ 1 ,ρ 2 …} (ρ defaults to 0.5) that satisfies the following inequality:

[0065]

[0066] Then the update of each iteration parameter is:

[0067] E k+1 =E k +α k g k .

[0068] 3.2 Z-score Normalization

[0069] In order to improve the stability of the algorithm, Z-score normalization is used to preprocess the data. After the iterative solution is completed, inverse normalization is used to restore the data. The mean value of the coordinate data of the calibration points measured is μ, and the standard deviation is σ. The normalization process is described as:

[0070]

[0071] The process of inverse normalization is the inverse of the above process.

[0072] Embodiment 2:

[0073] This example selects four public datasets with different characteristics and sizes, namely, the V-1Gag dataset, the VEEV dataset, the Centriole dataset, and the SNX3 dataset, and conducts comparative experiments with the general LM method and the existing L1 interior point method. Figure 3 shown.

[0074] Table 1 Comparison results of methods on different datasets

[0075]

[0076]

[0077] For each set of data sets, the software Markerauto was used for pre-calibration processing, including calibration point detection, matching, and initial estimation of projection parameters, and the initial estimate of the spatial point coordinates of the colloidal gold particles was obtained through triangulation. The experimental results are shown in Table 1. In terms of method effect, the L1 residuals obtained by the smoothing method of the present invention are all the smallest, indicating that it obtains the highest precision results; in terms of method efficiency, the time consumption of the smoothing method of the present invention is much lower than that of the interior point method, and the memory consumption is the lowest, indicating its high efficiency and low loss, and has practical application prospects. The above results all illustrate the high robustness and high efficiency of the L1 smoothing method of the present invention.

[0078] Finally, although this specification is described according to implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for calibrating projection parameters in alignment of fiducial markers in cryo-electron microscopy images, characterized in that: The following steps are involved: S1: Take the measured calibration point data and the initial estimate of the projection parameters as input; S2: Based on minimizing the reprojection residual, a bundle adjustment model with L1 norm is established, resulting in a non-convex and non-smooth optimization problem; S3: Based on the smoothing strategy, a smooth approximation problem is obtained; S4: Iteratively numerically solve the smooth approximation problem to obtain the final reconstruction result.

2. The projection parameter calibration method according to claim 1, characterized in that: In S1, a biological sample containing m colloidal gold particles is photographed from n different angles to obtain an image sequence, and q is defined as ij =(u ij ,v ij ) T is the calibration point obtained by imaging the jth colloidal gold particle in the i-th image, C i is the projection parameter of the i-th image, X j =(x j ,y j ,z j ) T is the spatial coordinate of the jth colloidal gold particle, C=(C1,C2…C n ) T , X=(X1,X2…X m ) T , E=(C,X) T , p ij =(u i ' j ,v i ' j ) T =Proj(C i ,X j ) is the reprojection of the j-th colloidal gold particle in the i-th image.

3. The projection parameter calibration method according to claim 1, characterized in that: In S2, the following L1 norm bundling model is constructed by using the Laplace noise assumption and the properties of the L1 norm: Where ‖.‖1 is the L1 norm.

4. The projection parameter calibration method according to claim 1, characterized in that: In S3, the smoothing approximation problem is derived as follows: Smoothing approximation: The non-smoothness in the L1 norm bundle adjustment model comes from the absolute value function f(x) = |x|, and the smoothing function is selected It is approximated, where τ is called the smoothing parameter, thus obtaining an approximate optimization problem of the bundled adjustment model of the L1 norm: Using the following update strategy, the smoothing parameter is updated when the following inequality holds: in express The gradient at point E, the ω value defaults to 1000, and the smoothing parameter is updated as follows: t k+1 =s k The default value of δ is 0.

05.

5. The projection parameter calibration method according to claim 1, characterized in that: In S4, the numerical solution method is: S4-1: Smooth Gradient Method For the approximate problem, an iterative numerical solution is performed using a gradient method with an Armijo line search: For the kth iteration, the forward direction g k is the current position E k The negative gradient direction Step size α k For the set {ρ 0 ,ρ 1 ,ρ 2 …} (ρ defaults to 0.5) that satisfies the following inequality: Then the update of each iteration parameter is: E k+1 =E k +α k g k S4-2: Z-score normalization: Z-score normalization is used to preprocess the data. After the iterative solution is completed, inverse normalization is used to recover the data. The mean value of the coordinate data of the calibration points obtained by measurement is μ, and the standard deviation is σ. The normalization process is described as: The process of inverse normalization is the inverse of the above process.

Images