Freeze electron tomography image denoising method based on self-supervised learning

Abstract

The invention relates to the technical field of biomedical image processing, and provides a frozen electron tomography image denoising method based on self-supervised learning, and the method comprises the following steps: S1, obtaining a tilted series projection image of frozen electron tomography, and carrying out the three-dimensional reconstruction of the tilted series projection image to obtain a noise image V (x); s2, carrying out two-dimensional denoising on the inclined series projection image to obtain a denoised inclined series projection image, and reconstructing the denoised inclined series projection image into a smooth image # imgabs0 # S3, respectively cutting V (x) and # imgabs1 # into a series of overlapped image blocks, and constructing a training data pair for the noise image blocks by using an average volume blind spot replacement strategy; s4, constructing a denoising model combining an anti-artifact pooling layer and a feature focusing attention module; and S5, inputting the training data pair and the corresponding smooth image block into the denoising model for training until the model converges. According to the method, the problems that the denoising effect is too smooth and artifacts exist in the prior art are solved, and the denoised image has the characteristics of high signal-to-noise ratio and clear details.

Description

Technical Field

[0001] The present invention relates to the technical field of biomedical image processing, and more specifically, to a cryo-electron tomography image denoising method based on self-supervised learning. Background Art

[0002] Cryo-Electron Tomography (Cryo-ET) is currently the only method that can visualize cells in situ in three-dimensional space with nanometer resolution. By rapidly freezing the sample and taking a series of two-dimensional images (called tilt series projection images) from different angles, Cryo-ET can reconstruct the three-dimensional structure, providing important structural information for fields such as biomedicine and drug design. In their native environment, proteins usually do not perform biological functions alone, but work together with various partners in complex networks. In order to fully understand the function of macromolecular protein complexes, structural analysis is crucial, and Cryo-ET is a powerful tool to achieve this goal. However, due to many limitations in Cryo-ET image capture, such as low electron dose shooting to reduce radiation damage and ice thickness, the image noise level is high and the contrast is low, which poses a great challenge to subsequent image analysis. Therefore, image denoising has become a basic and extremely important link in Cryo-ET image processing. Denoising can improve the signal-to-noise ratio (SNR) of Cryo-ET images, enhance the details and structural features of the image, and make subsequent data processing and analysis more accurate and reliable.

[0003] Traditional methods to improve the signal-to-noise ratio of cryo-ET images include various filtering techniques, such as Gaussian filtering, median filtering, and non-local mean filtering. These methods can remove noise to a certain extent, but often lead to the loss of image details. In addition, frequency domain filtering such as Fourier transform filtering and wavelet transform filtering are also widely used, but they also face the problem of detail loss. Among them, the block-matching and 4D filtering method (BM4D) is a more advanced one among the traditional methods. BM4D searches for similar blocks inside the image, combines these similar blocks into a four-dimensional data set, and then applies the hard thresholding technology in the four-dimensional transform domain to achieve denoising. Although BM4D performs well in denoising, there is still room for improvement in the denoising effect for cryo-ET images with extremely low signal-to-noise ratio, especially in the suppression of background noise.

[0004] With the development of neural networks in recent years, especially the rapid progress of deep learning, the performance of Cryo-ET image denoising methods based on artificial intelligence has surpassed traditional methods. Since supervised learning denoising methods rely on a large number of noisy and noise-free images, such noise-free images are difficult to obtain in Cryo-ET. In contrast, self-supervised learning does not require noise-free reference images, so it is more applicable in the field of Cryo-ET. In recent years, Noise2Noise (N2N) and Noise2Void (N2V) have become the two main branches of self-supervised denoising models. Specifically, N2N uses paired noisy images for training. Even if there is no clean reference image, it can still achieve denoising effects by learning the mapping from one noisy image to another. N2V is further simplified and only requires a single noisy image for training. It uses the redundancy and local similarity of the image itself to learn the intrinsic structural characteristics of the image by masking some pixels and predicting their values.

[0005] In 2019, Buchholz et al. first applied the Convolutional Neural Network (CNN) to cryo-EM image denoising. This method combined the idea of ​​N2N and proposed the Cryo-CARE network based on a two-layer U-Net structure. The network is trained with pairs of noisy images instead of pairs of noise and clean images, which increases the contrast of the image compared to traditional methods. However, the Cryo-CARE model is relatively simple, with limited feature extraction and recovery capabilities, and insufficient background noise removal. In 2020, Bepler et al. proposed the Topaz universal denoising model. Topaz uses a six-layer U-Net network and is trained using a dataset consisting of thousands of cryo-EM images collected under various imaging conditions. This universal model can denoise new datasets without additional training, which increases the generalization ability of the method and further improves the denoising effect. However, when faced with images with more complex noise, Topaz can easily cause the image to be too smooth and lose detail information. In 2021, Yang et al. proposed Sc-Net, which used sparsity constraints and the idea of ​​N2V to improve the quality of image restoration, but the denoised image had artifacts and the image was blurry overall. Among them, the generation of artifacts mainly stems from the downsampling process in image processing, especially when the image contains high-frequency components—that is, details or rapidly changing parts in the image. In the case of direct downsampling without proper processing, these high-frequency components may not be accurately presented in the reduced resolution. As a result, high-frequency components may be mixed with low-frequency components in a disorderly manner, generating erroneous low-frequency signals, resulting in aliasing. The direct consequence of the aliasing effect is image distortion and artifacts, which appear as unnatural patterns or edges. This distortion not only reduces the visual quality of the image, but also affects the effectiveness of subsequent processing and analysis based on these images.

[0006] Since the existing Cryo-ET image denoising methods produce denoised images that are too smooth, have artifacts, and blur important detail information, how to invent a method that can remove artifacts while retaining detail information is an urgent problem to be solved in this technical field. Summary of the invention

[0007] In order to solve the problem that the denoised image in the prior art is too smooth and has artifacts, the present invention proposes a cryo-electron tomography image denoising method based on self-supervised learning. The denoised image has the characteristics of high signal-to-noise ratio and clear details.

[0008] In order to achieve the above-mentioned purpose of the present invention, the technical scheme adopted is as follows:

[0009] A method for denoising cryo-electron tomography images based on self-supervised learning comprises the following steps: S1. Obtaining tilt series projection images of cryo-electron tomography, reconstructing them in three dimensions, and obtaining a noise image V(x); S2. Performing two-dimensional denoising on the tilt series projection images, obtaining denoised tilt series projection images, and reconstructing them into a smooth image. S3. V(x) and Crop them into a series of overlapping image blocks respectively, and use the average volume blind spot replacement strategy to construct training data pairs for the noisy image blocks; S4. Build a denoising model that combines the anti-artifact pooling layer and the feature focus attention module; S5. Input the training data pairs and the corresponding smoothed image blocks into the denoising model for training until the model converges.

[0010] Preferably, in step S1, the tilt series projection images are three-dimensionally reconstructed to obtain a noise image V(x), specifically, the original tilt series projection images are first aligned using IMOD software and a series of preprocessing operations are performed, and then reconstructed into a three-dimensional noise image.

[0011] Furthermore, in step S2, the tilted series of projection images are subjected to two-dimensional denoising and then reconstructed into a smooth image. Specifically, the Topaz 2D denoising pre-trained model is used to denoise each 2D image in the tilt series to obtain denoised tilt series projection images, and then the IMOD software is used to reconstruct these denoised images into smooth 3D images. Although the Topaz pre-trained model can reduce noise, it will also cause the loss of image details, making the reconstructed 3D Cryo-ET images appear smooth. This smooth image can provide the denoising network with prior knowledge about the image gradient.

[0012] Furthermore, in step S3, V(x) and The noise image and smooth image are cropped into a series of overlapping image blocks, and the training data pairs are constructed by the average volume blind spot replacement strategy. Specifically, the noise image and the smooth image are cropped into overlapping image blocks of size 128×128×128, and the enhancement operations are performed by rotation and mirroring. The average volume blind spot replacement strategy is as follows: in each training iteration, a group of voxels are randomly selected from the noise 3D image block V(x). For each selected voxel, its n-1 neighboring voxels are taken into account. Then, m n×n×n volume blocks are randomly selected from the local neighborhood of these voxels, and their average value is calculated. The corresponding position in the original image block is replaced with this average value to form the replaced volume block V′(x). The replaced volume block V′(x) is used as input, and together with the original input block V(x) as the target, it constitutes a training data pair in each iteration.

[0013] Furthermore, in the step S4, the denoising model combining the anti-artifact pooling layer and the feature focus attention module is constructed as follows: first, a nearest neighbor upsampling module is constructed, and then a U-Net type denoising network combining the anti-artifact pooling layer and the feature focus attention module is constructed, and finally, a nearest neighbor downsampling module is constructed. The input data passes through the nearest neighbor upsampling module and is then sent to the denoising network. The output of the denoising network is then subjected to nearest neighbor downsampling to obtain a denoised image of the same size as the input.

[0014] Furthermore, the basic convolutional block in the U-Net denoising network in this paper includes a Conv3D layer of size 3×3×3, a Batchnorm3D layer, and a LeakyReLU layer with k=0.1. The downsampling in the encoder uses an artifact-resistant pooling layer, and the upsampling in the decoder is the nearest neighbor upsampling. The corresponding layers of the encoder and decoder have skip connections, and a feature-focused attention module is inserted between each pair of skip connections to focus on important features.

[0015] Furthermore, the anti-artifact pooling layer in the U-Net type denoising network of this paper is specifically as follows: the anti-artifact pooling layer includes a three-dimensional maximum pooling layer, a deep convolution layer with a Gaussian blur kernel, and a trilinear interpolation downsampling layer. The three-dimensional maximum pooling layer is used to enhance the significant features in the feature map, while maintaining the original spatial size of the feature map by setting stride=1 and appropriate padding, in preparation for the subsequent Gaussian blur processing. The deep convolution layer applies a Gaussian blur kernel to smooth the feature map to reduce the high-frequency components of noise and details, thereby reducing aliasing artifacts. The trilinear interpolation downsampling layer is used to downsample the image and reduce the spatial size of the feature map. The anti-artifact pooling layer first strengthens the features through maximum pooling, then applies blur through deep convolution, and finally downsamples through trilinear interpolation, achieving the effect of reducing aliasing artifacts and retaining important details of the image during the downsampling process.

[0016] Furthermore, the feature-focused attention module in the U-Net denoising network in this paper is as follows: First, the skip connection of the corresponding layer of the input encoder and the features from the decoder are passed through a 1×1×1Conv3D layer and a BatchNorm3D layer respectively to adjust the number of channels of the feature map. Then, the two parts of the features are fused and nonlinearly transformed through a ReLU activation function. Subsequently, the activated features are passed through a 1×1×1Conv3D convolution layer and a BatchNorm3D layer again to make the number of output channels 1. Then, a Sigmoid activation function is used to limit the output to between 0 and 1. Finally, the output after Sigmoid activation is element-wise multiplied with the initial cross-connected input part to generate the final output. This process aims to adjust the strength of the features of the input cross-connected part through deeper features, enhance the expression of important features, and suppress unimportant features.

[0017] Furthermore, a combined loss function is constructed for the training of the denoising model. The combined loss function is:

[0018] l=λ1l str +λ2l grad +λ3l TV +λ4l reg

[0019] Among them, l str It is a structure-preserving loss function used to restore the structural information of the image; grad is the gradient-guided loss function, which is used to guide the network to smooth the image while denoising; l TV is the second-order total variation loss function, which is used to guide the network to retain more image details while denoising, making the denoised image more natural; reg It is a regularization term that constrains the first-order gradient of the network output image to prevent the model from overfitting the training data during training.

[0020] Furthermore, the structure-preserving loss function focuses on the voxels after blind spot replacement to preserve structural information while reducing noise. The calculation formula is:

[0021]

[0022] Where V i ′(x) is the image block obtained after blind spot replacement of the i-th image block, f θ (V i ′(x)) is the output of the network model, ⊙ represents element-by-element multiplication, and M is a three-dimensional mask indicating V i Whether the voxel (x, y, z) in ′(x) has been blind-spot replaced, 1 if replaced, 0 otherwise.

[0023] The gradient-guided loss function is specifically as follows: since the gradient value of the smooth area of ​​the image is very small, on the contrary, the gradient value of the texture or detail area of ​​the image is significantly increased, the gradient of the image can be used to indicate the smooth and texture detail areas, and the local smoothness of the image can be maintained by minimizing the difference between the gradient of the smooth image and the network output image.

[0024] This paper uses the Sobel operator as a gradient feature extractor, and the calculation formula is:

[0025]

[0026] in is the convolution operator, is the image block, G x and G y They are Sobel operators in two dimensions, as follows:

[0027]

[0028] Therefore, the gradient-guided loss function calculation formula is:

[0029]

[0030] The calculation formula of the second-order total variation loss function is:

[0031]

[0032] Where N is the total number of pixels in the image block, f θ (V i ′(x) is the output of the network model, G x ′ and G y ′ and G z ′ is the Laplace operator, specifically:

[0033]

[0034] The calculation formula of the regularization term is:

[0035]

[0036] Furthermore, the Adam optimizer was used to optimize the cryo-electron tomography image denoising model, where the hyperparameters of Adam were set to: β1 = 0.5, β2 = 0.999, and the initial learning rate was 0.0005. At the same time, the ReduceLROnPlateau learning rate scheduler was used to dynamically adjust the learning rate according to the performance of the model during training. In this way, the learning rate is continuously adjusted according to the performance of the model, thereby optimizing the parameters of the denoising model. When the loss function reaches the minimum, the training of the cryo-electron tomography image denoising model is completed.

[0037] The beneficial effects of the present invention are as follows:

[0038] The present invention discloses a method for denoising cryo-electron tomography images based on self-supervised learning. In view of the problems that the signal-to-noise ratio of cryo-electron tomography images is extremely low and there is no clean reference image, and the images denoised by the existing methods are too smooth and have artifacts, the present invention proposes a U-Net denoising model that combines an anti-artifact pooling layer and a feature-focused attention module, and combines the gradient-guided loss and the second-order total variation loss for network training. This method effectively removes noise while retaining the image detail area and reducing aliasing artifacts, thereby solving the above problems. The denoised image has the characteristics of high signal-to-noise ratio and clear details. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a process of a cryo-electron tomography image denoising method based on self-supervised learning according to the present invention;

[0040] Figure 2 A schematic diagram of the main process of training a cryo-electron tomography image denoising model of a cryo-electron tomography image denoising method based on self-supervised learning described in the present invention;

[0041] Figure 3 A schematic diagram of a denoising network structure of a cryo-electron tomography image denoising method based on self-supervised learning according to the present invention;

[0042] Figure 4 A schematic diagram of the structure of an anti-artifact pooling layer in a denoising network of a cryo-electron tomography image denoising method based on self-supervised learning described in the present invention;

[0043] Figure 5 A schematic diagram of the structure of a feature focusing attention module in a denoising network of a cryo-electron tomography image denoising method based on self-supervised learning according to the present invention;

[0044] Figure 6This is a slice diagram comparing the Centriole denoising results of the cryo-electron tomography image denoising method based on self-supervised learning described in the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below with reference to the accompanying drawings and specific implementation methods.

[0046] Example 1

[0047] like Figure 1 As shown, a cryo-electron tomography image denoising method based on self-supervised learning comprises the following steps: S1. Obtaining a tilt series projection image of cryo-electron tomography, reconstructing it in three dimensions, and obtaining a noise image V(x); S2. Performing two-dimensional denoising on the tilt series projection image, obtaining a denoised tilt series projection image, and reconstructing it into a smooth image. S3. V(x) and Crop them into a series of overlapping image blocks respectively, and use the average volume blind spot replacement strategy to construct training data pairs for the noisy image blocks; S4. Build a denoising model that combines the anti-artifact pooling layer and the feature focus attention module; S5. Input the training data pairs and the corresponding smoothed image blocks into the denoising model for training until the model converges.

[0048] Example 2

[0049] like Figure 2 As shown, a cryo-electron tomography image denoising method based on self-supervised learning comprises the following steps:

[0050] S1. Obtain tilted series projection images of cryo-electron tomography, perform three-dimensional reconstruction on them, and obtain a noise image V(x);

[0051] In the embodiment, the tilt series projection images are reconstructed into a noise image specifically by: firstly performing a series of preprocessing operations such as alignment on the tilt series projection images according to the IMOD three-dimensional reconstruction software tutorial, and then reconstructing the three-dimensional noise image.

[0052] S2. Perform two-dimensional denoising on the tilt series projection images to obtain the denoised tilt series projection images and reconstruct them into smooth images

[0053] In this embodiment, each two-dimensional image in the tilt series image is denoised using the Topaz two-dimensional denoising pre-trained model to obtain a denoised tilt series image, which is then reconstructed into a smooth three-dimensional image using IMOD software;

[0054] S3. V(x) and They are cropped into a series of overlapping image blocks, and the noise image blocks are replaced by the average volume blind spot strategy to construct training data pairs;

[0055] In this embodiment, V(x) and The 3D images were cropped into a series of overlapping image blocks of size 128×128×128, and then data enhancement was achieved through mirroring and rotation operations. The resulting original noisy image blocks were divided into training samples and validation samples in a ratio of 9:1. The average volume blind spot replacement strategy is as follows: in each training iteration, a group of voxels are randomly selected from the noisy image block V(x). The ratio selected in this paper is 0.1. For each selected voxel, its n-1 neighboring voxels are taken into account. Then, some volume blocks of size 3×3×3 are randomly selected from the local neighborhood of these voxels, and their average value is calculated. The average value of these volume blocks is used to replace the value of the corresponding position in the original image block to form a replaced volume block V′(x). The replaced volume block V′(x) is used as input, and together with the original input block V(x) as the target, it constitutes a training data pair in each iteration.

[0056] S4. Construct a denoising model combining an artifact-resistant pooling layer and a feature-focused attention module;

[0057] In this embodiment, a nearest neighbor upsampling module is first constructed, then a denoising network combining an anti-artifact pooling layer and a feature-focused attention module is constructed, and finally a nearest neighbor downsampling module is constructed. The denoising network is built on the basis of U-Net, and an anti-artifact pooling layer is introduced as a downsampling layer to retain important details and reduce aliasing artifacts. The feature-focused attention module is added to enable the denoising model to suppress irrelevant areas in the image while highlighting useful important features.

[0058] S5. Input the training data pairs and the corresponding smoothed images into the denoising model for training until the model converges.

[0059] In this embodiment, the loss function of the model training is a combined loss function, including a structure-preserving loss function, a gradient-guided loss function, a second-order total variation loss function, and a regularization term. During training, the structure-preserving loss function focuses on the voxels after blind spot replacement, which is used to retain structural information while reducing noise. The gradient-guided loss function is used to achieve smoothing of the output image. The second-order total variation loss function enables the network to retain image details while denoising, making the denoised image more natural. The regularization term is used to prevent overfitting.

[0060] In a specific embodiment, the cryo-electron tomography tilt series image dataset can be various Cryo-ET tilt series projection image datasets. This article takes the Centriole dataset as an example. It is a tilt series composed of 64 projection images, ranging from -61.0° to +65.0°, with an interval of 2°. The size of each tilt image is 1024×1024 pixels and is stored in the mrc file format. The three-dimensional image obtained after reconstruction is stored in the rec format.

[0061] In a specific embodiment, Figure 3 As shown in the figure, the denoising network consists of a four-layer U-Net encoder-decoder structure. The basic convolution block includes a Conv3D layer of size 3×3×3, a Batchnorm3D layer and a LeakyReLU layer with k=0.1. The padding of the convolution layer is set to 1, the anti-artifact pooling layer is used as downsampling, and the nearest neighbor interpolation is used as upsampling. The corresponding layers of the encoder and decoder have jump connections, and a feature focusing attention module is inserted between each pair of jump connections.

[0062] In a specific embodiment, Figure 4 As shown, the anti-artifact pooling layer includes a 3D maximum pooling layer, a deep convolution layer with a Gaussian blur kernel, and a trilinear interpolation downsampling layer, wherein the convolution kernel size of the 3D maximum pooling layer is 2, the stride is 1, the padding is 1, the size of the Gaussian blur kernel is 3×3, and the standard deviation is 1.0. The anti-artifact pooling layer first strengthens the features through maximum pooling, then applies blur through deep convolution, and finally downsamples through trilinear interpolation, thereby reducing aliasing artifacts and retaining important details of the image during the downsampling process.

[0063] In a specific embodiment, the feature-focused attention module is as follows: Figure 5 As shown in the figure, specifically: First, the skip connection of the corresponding layer of the input encoder and the features from the decoder are passed through a 1×1×1Conv3D layer and a BatchNorm3D layer respectively to adjust the number of channels of the feature map. Then, the two parts of the features are fused and nonlinearly transformed through a ReLU activation function. Subsequently, the activated features are passed through a 1×1×1Conv3D convolution layer and a BatchNorm3D layer again to make the number of output channels 1. Then, a Sigmoid activation function is used to limit the output to between 0 and 1. Finally, the output after Sigmoid activation is element-wise multiplied with the initial cross-connection input part to generate the final output. This process aims to adjust the strength of the features of the input cross-connection part, enhance the expression of important features, and suppress unimportant features.

[0064] In a specific embodiment, the loss function of the denoising model training is,

[0065] l=λ1l str +λ2l grad +λ3l TV +λ4l reg

[0066] Among them, l str is the structure preservation loss function, l grad is the gradient guided loss function, l TV is the second-order total variation loss function, l reg is the regularization term, λ1, λ2, λ3 and λ4 are the weights of the corresponding loss terms, which are set to 0.8, 0.1, 0.15 and 0.01 respectively in this paper, and their values ​​can be adjusted according to the characteristics of the dataset.

[0067] The structure-preserving loss function focuses on the voxels after blind spot replacement to preserve structural information while reducing noise. The calculation formula is as follows:

[0068]

[0069] Where V i ′(x) is the image block obtained after blind spot replacement of the i-th image block, f θ (V i ′(x)) is the output of the network model, ⊙ represents element-by-element multiplication, and M is a three-dimensional mask indicating V i Whether the voxel (x, y, z) in ′(x) has been blind-spot replaced, 1 if it has been replaced, 0 otherwise.

[0070] The gradient-guided loss function maintains the local smoothness of the image by minimizing the difference between the smoothed image and the gradient of the network output image. Using the Sobel operator as the gradient feature extractor, the calculation formula is as follows:

[0071]

[0072] in is the convolution operator, is the image block, G x and G y They are Sobel operators in two dimensions, as follows:

[0073]

[0074] Therefore, the calculation formula of the gradient-guided loss function is as follows:

[0075]

[0076] The second-order total variation loss function is used to retain the detail area of ​​the image, and its calculation formula is as follows:

[0077]

[0078] Where N is the total number of pixels in the image block, f θ (V i ′(x) is the output of the network model, G x ′ and G y ′ and G z ′ is the Laplace operator, which is as follows:

[0079]

[0080] The calculation formula of the regularization term is as follows:

[0081]

[0082] The network uses the Adam optimizer to optimize the cryo-electron tomography image denoising model, where the hyperparameters of the Adam optimizer are set to: β1 = 0.5, β2 = 0.999, and the initial learning rate is 0.0005. At the same time, the ReduceLROnPlateau learning rate scheduler is used to dynamically adjust the learning rate according to the performance of the model during training. In this way, the learning rate is continuously adjusted according to the performance of the model, thereby optimizing the parameters of the denoising model. When the loss function reaches the minimum, the training of the cryo-electron tomography image denoising model is completed.

[0083] The present invention aims at the problems that the signal-to-noise ratio of cryo-electron tomography images is extremely low, there is no clean reference image, and the images denoised by the existing methods are too smooth and have artifacts. A cryo-electron tomography image denoising method based on self-supervised learning is proposed; in this embodiment, Figure 6 As shown in the figure, the Centriole is denoised. It can be found that this method can better preserve the detail area, reduce artifacts, and achieve the best contrast while effectively denoising.

[0084] The present invention discloses a method for denoising cryo-electron tomography images based on self-supervised learning. Aiming at the problems that the signal-to-noise ratio of cryo-electron tomography images is extremely low and there is no clean reference image, and the images denoised by the existing methods are too smooth and have artifacts, a U-Net denoising model combining an anti-artifact pooling layer and a feature-focused attention module is proposed. At the same time, the gradient-guided loss function and the second-order total variation loss function are combined in the model training process, which can better retain the detail area while effectively removing the noise, thereby solving the problem that the images denoised by the existing technology are too smooth and have artifacts, and the denoised images have the characteristics of high signal-to-noise ratio and clear details.

[0085] Obviously, the above embodiments of the present invention are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A cryo-electron tomography image denoising method based on self-supervised learning, characterized in that: The following steps are involved: S1. Obtain tilted series projection images of cryo-electron tomography, perform three-dimensional reconstruction on them, and obtain a noise image V(x); S2. Perform two-dimensional denoising on the tilt series projection images to obtain the denoised tilt series projection images and reconstruct them into smooth images S3. V(x) and The images are cropped into a series of overlapping blocks, and the noise blocks are replaced by the average volume blind spot strategy to construct training data pairs; S4. Construct a denoising model combining an artifact-resistant pooling layer and a feature-focused attention module; S5. Input the training data pairs and the corresponding smoothed image patches into the denoising model for training until the model converges.

2. The method according to claim 1, characterized in that In the step S1, the tilt series projection images are three-dimensionally reconstructed to obtain a noise image, specifically: the tilt series projection images are reconstructed into a three-dimensional noise image using a three-dimensional reconstruction tool IMOD.

3. The method according to claim 1, characterized in that: In the step S2, two-dimensional denoising is performed on the tilted series projection images, specifically: each two-dimensional projection image is denoised using the Topaz two-dimensional denoising pre-trained model to obtain a denoised tilted series projection image.

4. The method according to claim 1, characterized in that: In the step S3, specifically: the noisy image and the smoothed image are respectively cropped into a series of image blocks of size 128×128×128, and enhanced by rotation and mirroring, and finally the noisy image blocks are constructed into training data pairs using the average volume blind spot replacement strategy.

5. The method according to claim 4, characterized in that The specific strategy of average volume blind spot replacement is as follows: in each training iteration, a group of voxels are randomly selected from the noisy 3D image block V(x). For each selected voxel, its n-1 neighboring voxels are taken into account. Then, m n×n×n volume blocks are randomly selected from the local neighborhood of these voxels, and their average value is calculated. The corresponding position in the original image block is replaced with this average value to form the replaced volume block V ′ (x), the volume V to be replaced ′ (x) as input, together with the original input block V(x) as the target, constitute a training data pair in each iteration.

6. The method according to claim 1, characterized in that In the step S4, a denoising model combining an anti-artifact pooling layer and a feature-focused attention module is constructed, specifically: first, a nearest neighbor upsampling module is constructed, then a U-Net denoising network combining an anti-artifact pooling layer and a feature-focused attention module is constructed, and finally, a nearest neighbor downsampling module is constructed. The data first passes through the nearest neighbor upsampling module, then enters the U-Net denoising network for processing, and finally outputs the data through the nearest neighbor downsampling module.

7. The method according to claim 6, characterized in that The denoising network consists of a four-layer U-Net encoder-decoder structure. The basic convolutional block includes a Conv3D layer of size 3×3×3, a Batchnorm3D layer and a LeakyReLU layer with k=0.

1. The padding of the convolutional layer is set to 1, the anti-artifact pooling layer is used as the downsampling layer, and the nearest neighbor interpolation is used as the upsampling layer. The corresponding layers of the encoder and decoder have jump connections, and a feature focusing attention module is inserted between each pair of jump connections.

8. The method according to claim 7, characterized in that The anti-artifact pooling layer includes a three-dimensional maximum pooling layer, a deep convolution layer with a Gaussian blur kernel, and a trilinear interpolation downsampling layer. The anti-artifact pooling layer first strengthens the features through maximum pooling, then smoothes the image through deep convolution, and finally downsamples through trilinear interpolation, thereby reducing artifacts and retaining important details of the image during the downsampling process; the feature-focused attention module contains three convolution blocks, a ReLu layer, and a Sigmoid layer, which enhances the expression of important features by adjusting the strength of the features of the cross-connected parts of the input.

9. The method according to claim 1, characterized in that: The loss function for denoising model training is: l=λ1l str +λ2l grad +λ3l TV +λ4l reg Among them, l str is the structure preservation loss function, l grad is the gradient guided loss function, l TV is the second-order total variation loss function, l reg is the regularization term, and λ1, λ2, λ3, and λ4 are the weights of the corresponding loss terms.

10. The method according to claim 9, characterized in that The structure-preserving loss function focuses on the voxels after blind spot replacement to preserve structural information while reducing noise. The calculation formula is as follows: Where V i ′ (x) is the image block obtained after blind spot replacement of the i-th image block, f θ (V i ′ (x)) is the output of the network model, ⊙ is represented by element-wise multiplication, and M is a three-dimensional mask used to indicate V i ′ Whether the voxel (x,y,z) in (x) has been blind-spot replaced. If so, it is 1; otherwise, it is 0.

11. The method according to claim 9, characterized in that The gradient-guided loss function maintains the local smoothness of the image by minimizing the difference between the gradient of the smoothed image and the network output image, while retaining the detail features. The Sobel operator is used as the gradient feature extractor. The calculation formula is as follows: in is the convolution operator, is the image block, G x and G y For the Sobel operator, the calculation formula of the gradient-guided loss function is as follows: in, is a smooth image block.

12. The method according to claim 9, characterized in that The second-order total variation loss function is used to retain the detail area of ​​the image and make the denoised image more natural. The calculation formula is as follows: Where N is the total number of pixels in the image block, f θ (V i ′ (x) is the output of the network model, G x ′ and G y ′ and G z ′ is the Laplace operator.

13. The method according to claim 9, characterized in that The regularization term is used to prevent overfitting, and the calculation formula is as follows: where f θ (V i ′ (x) is the output of the network model.

14. The method according to claim 1, characterized in that The denoising network uses the Adam optimizer to optimize the cryo-electron tomography image denoising model. At the same time, the ReduceLROnPlateau learning rate scheduler is used to dynamically adjust the learning rate according to the performance of the model during training, thereby optimizing the parameters of the denoising model. When the loss function tends to converge on both the training set and the validation set, that is, when the loss value no longer decreases significantly within several epochs, the training of the cryo-electron tomography image denoising model is completed.

Images