Parameter-sharing de-noising system for electronic tomography three-dimensional image

Abstract

The invention discloses a de-noising system for a parameter-shared electronic tomography three-dimensional image. The de-noising system comprises an image partitioning module, a position embedding module, a locality sensitive hash (LSH) module, a feature extraction module and an image de-noising module, the image partitioning module is used for partitioning the electronic tomography three-dimensional image into blocks with the same size; the position embedding module is used for generating an embedding vector for each block, generating a position code and adding the position code into the corresponding embedding vector; the LSH module is used for calculating the similarity between the embedded vectors and filtering the embedded vectors of which the similarity is lower than a set similarity threshold; the feature extraction module comprises an LSH module and an encoder, and the encoder is used for extracting the features of the filtered embedded vectors and reducing the resolution of the embedded vectors; and the image denoising module comprises an LSH module and a decoder, and the decoder is used for extracting the features of the filtered embedded vector and recovering the resolution of the embedded vector so as to realize denoising of the electronic tomography three-dimensional image.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a denoising system for electronic tomography three-dimensional images with shared parameters. Background Art

[0002] With the rapid development of nanotechnology and materials science, accurate characterization of atomic structure and electron distribution has become a key requirement in scientific research and industry. In recent years, with the continuous development of atomic-scale materials science and nanotechnology, the demand for three-dimensional accurate characterization of atomic structure and electron distribution has become increasingly urgent. Traditional techniques such as electron microscopy and atomic force microscopy are subject to many limitations in atomic-scale characterization, which highlights the urgent need for new technologies.

[0003] Electron tomography is an important method for electron three-dimensional imaging of nanomaterials, but its traditional characterization methods are limited in accuracy and efficiency. At present, the successful application of deep learning models in the processing and analysis of three-dimensional images has provided new ideas for improving the accuracy and efficiency of electron tomography. Among them, the design of a lightweight model for three-dimensional characterization of atomic electron tomography with shared parameters realizes high-precision and high-efficiency characterization of atomic structure and electron distribution. This model not only helps to deeply understand the structure and properties of nanomaterials, and provides important physical structure basis for the research and development of new nanomaterials and their performance improvement, but also has important significance for the research of nanodevices, nanomaterials and biomedicine, and is conducive to promoting the development and application of nanotechnology. In addition, the lightweight characteristics of the model also make it more efficient and convenient in practical applications, and it is expected to provide convenience for industrial production and scientific research experiments.

[0004] Although deep learning models offer the potential for improving the accuracy and efficiency of electron tomography, computational complexity and the number of parameters are challenges to their application. These models usually contain a large number of parameters and involve a large number of self-attention mechanisms and multi-head attention mechanisms, which results in a relatively large model with high computational complexity and high demand for computing resources, making it difficult to deploy on resource-constrained devices and perform real-time reasoning. Summary of the invention

[0005] The present invention proposes a parameter-sharing electronic tomography three-dimensional image denoising system, which solves the problem of the existing electronic three-dimensional imaging model having many parameters and high computational consumption.

[0006] In order to solve the above technical problems, the present invention provides a denoising system for electronic tomography three-dimensional images with shared parameters, comprising: an image segmentation module, a position embedding module, a local sensitive hashing (LSH) module, a feature extraction module and an image denoising module;

[0007] The image segmentation module is used to segment the electronic tomography three-dimensional image into blocks of the same size;

[0008] The position embedding module generates an embedding vector for each block, generates a position code for each embedding vector, and adds the position code to the corresponding embedding vector;

[0009] The local sensitive hash (LSH) module calculates the similarity between embedded vectors and filters embedded vectors whose similarity is lower than a set similarity threshold;

[0010] The feature extraction module includes an LSH module and an encoder, wherein the encoder gradually extracts features of the filtered embedding vector and reduces the resolution of the embedding vector through a series of convolutional layers and pooling layers, and the output features of each level of the encoder are directly connected to the corresponding level of the decoder through a skip connection;

[0011] The image denoising module comprises an LSH module and a decoder. The decoder gradually extracts the features of the filtered embedded vector and restores the resolution of the embedded vector through a series of upsampling layers and convolutional layers, thereby achieving denoising of the electronic tomography three-dimensional image.

[0012] Preferably, it also includes a feature description module, which is used to extract feature descriptors of key points in the embedded vector, including the following steps:

[0013] Step S1: using a Gaussian difference function to identify key points in the embedding vector;

[0014] Step S2: Calculate the gradient direction and gradient magnitude of the pixels in the neighborhood of the key point, and assign several directions to each key point;

[0015] Step S3: Divide the area of ​​each key point into 16 sub-areas of the same size, calculate the gradient histogram in 8 directions in each sub-area, form a vector with a length of 128, and use the vector as the descriptor of the key point.

[0016] Preferably, step S1 comprises the following steps:

[0017] Step S11: constructing a Gaussian pyramid of the embedding vector;

[0018] Step S12: randomly select a pixel point in a layer of the Gaussian pyramid, compare the pixel point with its 8 adjacent pixel points and 2×9 pixel points at corresponding positions of two adjacent embedding vectors in the same group, and if the pixel point is a local extreme value, take the pixel point as a key point;

[0019] Step S13: Repeat step S12 until each pixel point on each layer of the Gaussian pyramid is judged.

[0020] Preferably, the LSH module calculates the similarity between the embedded vectors through the following steps, and filters the embedded vectors whose similarity is lower than a set similarity threshold:

[0021] Step a: Define the number of hash buckets and the width of the hash buckets;

[0022] Step b: Construct a hash function family, use the hash function family to hash each embedded vector, and assign all embedded vectors to corresponding hash buckets according to the hash results;

[0023] Step c: introduce a query object q and hash the query vector q using the same hash function family;

[0024] Step d: retrieve the embedding vector with the same hash value as the query object q from the hash table to form a candidate set, calculate the similarity between the query vector q and each embedding vector in the candidate set, and delete the embedding vectors whose similarity is lower than the set similarity threshold;

[0025] Step e: Concatenate the remaining embedding vectors in the candidate set to obtain the filtered embedding vector.

[0026] Preferably, the expression of the hash function in the hash function family is:

[0027]

[0028] Where h(v) is the hash function; τ() means rounding down; a is a randomly generated vector of the same dimension as v; v is the embedding vector; b is a random real number; and w is the width of the hash bucket.

[0029] Preferably, the expression of the convolutional layer of the encoder is:

[0030]

[0031] In the formula, is the lth convolutional layer of the encoder; σ is the activation function ReLU; E l-1 is the output feature of the encoder layer l-1; W l 、b l are the weights and biases of the encoder’s l-th convolutional layer, respectively; * indicates a convolution operation;

[0032] The expression of the pooling layer of the encoder is:

[0033]

[0034] In the formula, is the lth pooling layer of the encoder; MaxPool() is the maximum pooling function; k and s are the size and step size of the pooling window, respectively.

[0035] Preferably, the expression of the upsampling layer of the decoder is:

[0036]

[0037] In the formula, is the lth upsampling layer of the decoder; D l+1 is the output feature of the decoder layer l+1; ↑ represents the upsampling operation; W l ', b l ' are the weights and biases of the upsampling layer of the decoder layer l respectively;

[0038] The expression of the convolutional layer of the decoder is:

[0039]

[0040] In the formula, is the lth convolutional layer of the decoder; σ is the activation function ReLU; * represents the convolution operation; CropOrResize is used to ensure that the two features of the jump connection have the same spatial size; E l-1 is the output feature of the encoder layer l-1; ""

[0041] W l is the corresponding layer weight; W l "、b l "respectively are the weights and biases of the lth convolutional layer of the decoder.

[0042] Preferably, the skip connection includes a concatenation operation or an addition operation, and the expression of the concatenation operation is:

[0043]

[0044] In the formula, is the fusion feature; concatenate is the concatenation operation; E i is the output feature of the encoder layer i; D i′ is the decoder with E i The output features of the corresponding level; axis is the dimension;

[0045] The expression of the addition operation is:

[0046]

[0047] Preferably, the expression for generating an embedding vector for each block is:

[0048] Embedi =f embedding (Patch i );

[0049] In the formula, Embed i is the embedding vector; f embedding Patch is an embedded function. i is the i-th block.

[0050] Preferably, the expression for generating the position encoding for each embedding vector is:

[0051]

[0052] In the formula, PE is the position code; pos is used to specify a position in the sequence; i is the dimension index of the position code; d model is the length of the embedding vector.

[0053] The benefits of the present invention include at least:

[0054] 1. By calculating the similarity between embedded vectors and filtering out vectors with similarity lower than the set threshold, unnecessary calculations can be reduced, focusing on blocks with high similarity and improving denoising efficiency;

[0055] 2. Reducing the resolution of the embedding vector can reduce the amount of computation required by the model during feature extraction. In the process of reducing the resolution, the model can focus on the key features in the image and ignore some unimportant details, which can improve the model's understanding of the image content. In the process of restoring the resolution of the embedding vector, the model can integrate local features and global context information to reconstruct a more complete and accurate image representation.

[0056] 3. By introducing skip connections and local sensitive hashing modules, the model can efficiently capture multi-scale information in the image and share parameters between different layers, reducing the number of model parameters and making the model more lightweight, making it easier to deploy on resource-constrained devices and perform real-time inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the system structure of an embodiment of the present invention;

[0058] Figure 2 The figure is a schematic diagram of the data processing flow of the local sensitive hash module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0060] In view of the shortcomings of the current atomic electron tomography 3D image denoising model, which has high computational complexity and huge number of parameters, the present invention proposes a parameter-sharing electron tomography 3D image denoising system. Based on the hierarchical design concept, the encoder and decoder of this system are composed of multiple levels, and the feature resolution of each level tends to gradually decrease or increase. At the same time, the introduction of cross-layer connection and local sensitive hashing modules can efficiently capture multi-scale information in the image and reduce computational complexity.

[0061] In order to achieve the above objectives, the embodiments of the present invention are designed and integrated with a local sensitive hashing LSH module in each layer of the encoder and decoder for processing high-dimensional data. The LSH module uses hashing technology to pre-filter out data objects with low similarity and unlikely to contribute to the final result, and only retains data objects with high similarity for subsequent processing. Subsequently, the actual distance or similarity metric calculation is performed on the screened candidate data set. This design ensures that samples that are similar in high-dimensional space can still maintain a high probability of similarity in low-dimensional space after LSH hashing, thereby significantly improving the efficiency and robustness of the model in representation extraction.

[0062] Specifically, Figure 1 As shown, a denoising system for a three-dimensional electron tomography image with shared parameters according to an embodiment of the present invention includes: an image segmentation module, a position embedding module, a special diagnosis description module, a local sensitive hashing (LSH) module, a feature extraction module and an image denoising module.

[0063] The image segmentation module is used to divide the electron tomography three-dimensional image into blocks of the same size.

[0064] Specifically, the input atomic electron tomography 3D image is a 3D tensor of shape [H, W, D, C], where H is the height of the electron tomography 3D image; W is the width of the electron tomography 3D image; D is the depth of the electron tomography 3D image, and C is the number of channels of the electron tomography 3D image. The input 3D tensor is divided into a series of patches, and the size of each patch is fixed P×P×P. After division, the set of patches can be represented by a 4D tensor with a shape of [N, P, P, P, C], where N is the number of patches. In the implementation process, in order to be compatible with the structure of Transformer, the 4D tensor will be further flattened or reshaped into a 2D tensor.

[0065] Assuming that each dimension of the 3D image is divisible by P, the expression for calculating the number of blocks N is:

[0066]

[0067] Where N is the total number of blocks; P is the side length of the block;

[0068] The position embedding module is used to generate an embedding vector for each block, generate a position encoding for each embedding vector, and add the position encoding to the corresponding embedding vector.

[0069] Specifically, a 1x1x1 convolutional layer or a fully connected layer is applied to generate an embedding vector for each patch. The length of this embedding vector is fixed and is denoted as E. This process can be expressed as:

[0070] Embed i =f embedding (Patch i );

[0071] In the formula, Embed i is the embedding vector; f embedding Patch is an embedded function. i is the i-th block.

[0072] Position coding is used to add position information to the embedded vector. The essence of position coding is a vector of the same length as the embedded vector. In the embodiment of the present invention, sine and cosine functions are used to generate position coding. For the block at the 2ith or 2i+1th position, the position coding can be expressed as:

[0073]

[0074] In the formula, PE is the position code; pos is used to specify a position in the sequence; i is the dimension index of the position code; d model is the length of the embedding vector.

[0075] Add the positional encoding to the embedding vector to get Eebed i +PE i , which is used as the new embedding vector.

[0076] This carefully designed encoding mechanism ensures that each position in the sequence will get a unique encoding vector. The length of these encoding vectors is consistent with the length of the embedding vector, thus ensuring the consistency and integrity of the information. This method not only takes into account the position information, but also captures the intrinsic characteristics of the data through the embedding vector.

[0077] The embedding vector containing the position information is then used as the input of the Transformer model. The Transformer model is able to use these embedding vectors with position information to understand the spatial relationship between different patches. In the Transformer model, the main operations include the self-attention mechanism (Self-Attention) and the feed-forward neural network (Feed-Forward Networks, FFN). The self-attention mechanism allows the model to consider the relationship between each patch and all other patches in the sequence, while the feed-forward neural network performs further nonlinear transformations on these relationships.

[0078] The self-attention mechanism captures the relationship between patches by calculating attention scores between them. These scores reflect the mutual importance between different patches:

[0079]

[0080] Q=XW q ,K=XW k ,V=XW v ;

[0081] Where Attention is the attention score; Q, K, and V are the query vector, key vector, and value vector of the self-attention mechanism respectively; Softmax is the activation function; X is the embedding vector containing position information; W q , W k , W v are respectively the learnable weight matrices; d k is the dimension of the key vector, which is used to scale the dot product attention scores to prevent the gradients from vanishing or exploding.

[0082] The feedforward neural network is usually placed after the self-attention mechanism to further transform the representation of each position. It usually includes two linear transformations and a ReLU activation function. The calculation formula of the feedforward neural network is as follows:

[0083] FFN(x)=ReLU(xW1+b1)W2+b2;

[0084] Where ReLU is the activation function; x is the output of the self-attention mechanism; W1 and W2 are learnable weight matrices; b1 and b2 are learnable bias terms.

[0085] The feature description module is used to extract the feature descriptors of key points in the embedding vector, including the following steps:

[0086] Step S1: Identify key points in the embedding vector using the Gaussian difference function, including the following steps:

[0087] Step S11: Perform Gaussian difference on the embedded vector to construct a Gaussian pyramid of the embedded vector:

[0088] DoG(x,y,σ)=G(x,y,kσ)-G(x,y,σ);

[0089] Where G(x, y, σ) is a two-dimensional Gaussian function; (x, y) is the coordinate of the pixel point; σ is the scale parameter; and k is the scaling factor between two adjacent scales.

[0090] Step S12: randomly select a pixel point in a layer of the Gaussian pyramid, compare the pixel point with its 8 adjacent pixel points and 2×9 pixel points at corresponding positions of two adjacent embedding vectors in the same group, and if the pixel point is a local extreme value, take the pixel point as a key point.

[0091] Step S13: Repeat step S12 until every pixel point on every layer of the Gaussian pyramid is judged.

[0092] Step S2: Calculate the gradient direction and gradient magnitude of the pixels in the neighborhood of the key point, and assign several directions to each key point, where the calculation formulas for the gradient magnitude and gradient direction are:

[0093]

[0094] Among them, m(x,y) is the gradient amplitude; θ(x,y) is the gradient direction; L(x,y) is the image grayscale value at the scale to which the key point belongs.

[0095] Step S3: Divide the area of ​​each key point into 16 sub-areas of the same size, calculate the gradient histogram in 8 directions in each sub-area, form a vector with a length of 128, and use the vector as the descriptor of the key point.

[0096] The local sensitive hashing (LSH) module is used to calculate the similarity between embedded vectors and filter the embedded vectors whose similarity is lower than the set similarity threshold, such as Figure 2 As shown, the following steps are included:

[0097] Step a: Define the number of hash buckets and the width of the hash buckets.

[0098] Step b: Construct a hash function family, use the hash function family to hash each embedded vector, and assign all embedded vectors to corresponding hash buckets according to the hash results.

[0099] Specifically, the expression of the hash function in the constructed hash function family is:

[0100]

[0101] Wherein, h(v) is a hash function; τ() represents rounding down; a is a randomly generated vector of the same dimension as v, whose elements obey the normal distribution; v is an embedding vector; b is a random real number used to increase the randomness of the hash function; w is a positive real number, defined as the width of the hash bucket. In order to meet the sensitivity condition of LSH, the embodiment of the present invention designs multiple such hash functions to form a hash function family H. Each hash function h∈H uses a different random vector a and random real number b.

[0102] Step c: Introduce a query object q and hash the query vector q using the same hash function family.

[0103] Step d: Retrieve the embedding vector with the same hash value as the query object q from the hash table to form a candidate set, calculate the similarity between the query vector q and each embedding vector in the candidate set, and delete the embedding vectors whose similarity is lower than the set similarity threshold.

[0104] Specifically, the embodiment of the present invention calculates the collision probability between the query object and all embedded vectors in the hash table, retrieves the embedded vector with the same hash value as the query object from the hash table, and forms a candidate set. In LSH, the collision probability refers to the probability that two similar objects are mapped to the same hash value by the same LSH function. Assuming that two points x and y are similar under the distance metric d, for any hash function h in an LSH function family H, the collision probability can be calculated by analyzing the properties of the LSH function family:

[0105] P(h(x)=h(y)|d(x,y)≤r1);

[0106] Where P is the collision probability; h(y) represents the hash value obtained by applying hash function h to the input object y; d(x,y) is the Euclidean distance between x and y; r1 is the set similarity threshold.

[0107] Step e: Concatenate the remaining embedding vectors in the candidate set to obtain the filtered embedding vector.

[0108] The feature extraction module includes an LSH module and an encoder. The encoder gradually extracts the features of the filtered embedding vector and reduces the resolution of the embedding vector through a series of convolutional layers and pooling layers.

[0109] The expression of the convolutional layer of the encoder is:

[0110]

[0111] In the formula, is the lth convolutional layer of the encoder; σ is the activation function ReLU; E l-1 is the output feature of the encoder layer l-1; W l 、b l are the weights and biases of the encoder’s l-th convolutional layer, respectively; * indicates a convolution operation;

[0112] The expression of the encoder's pooling layer is:

[0113]

[0114] In the formula, is the lth pooling layer of the encoder; MaxPool() is the maximum pooling function; k and s are the size and step size of the pooling window, respectively.

[0115] By introducing the LSH module into the feature extraction module, similar vectors in the original data space are mapped to similar buckets in the hash space, greatly reducing the number of comparisons. This design enables the model to better focus on important areas and features in the image, thereby improving the efficiency and robustness of representation extraction.

[0116] The image denoising module includes an LSH module and a decoder. The decoder gradually extracts the features of the filtered embedding vector and restores the resolution of the embedding vector through a series of upsampling layers and convolutional layers to achieve denoising of the electron tomography 3D image.

[0117] Similar to the encoding layer, the decoding layer also has multiple layers, each of which contains upsampling operations and multiple convolution blocks. In addition, in order to utilize the information in the encoder, a skip connection is set in the decoder to concatenate or add the features output by the encoder with the corresponding level in the decoder.

[0118] The skip connection includes a concatenation operation or an addition operation. To describe this process more accurately, we can use the mathematical representation of tensor operations. Assume that E i is the output feature of the i-th layer of the encoder, and its shape is (H i ,W i ,C i ), where H i and Wi is the height and width of the feature map, C i is the number of channels. Similarly, assuming D i′ is the decoder with E i The output feature of the corresponding level has the shape of (H i ,W i ,C i′ ). The two feature maps are merged in dimension, and the fused feature map F fusedi The shape is (H i ,W i ,C i +C i′ ).

[0119] The concatenation operation can be expressed as:

[0120] F fusedi =concatenate(E i ,D i′ ,axis=2);

[0121] In the formula, F fusedi is the fusion feature; concatenate is the concatenation operation; E i is the output feature of the encoder layer i; D i′ is the decoder with E i The output features of the corresponding layer; axis is the dimension, which actually stacks the two tensors on the third dimension, that is, the index starts from 0.

[0122] The addition operation is to add the element values ​​at corresponding positions to obtain new features. The addition operation can be expressed as:

[0123]

[0124] Before using the addition operation, the number of channels needs to be adjusted through a 1x1 convolution so that they match.

[0125] The upsampling layer is used to restore the spatial size of the feature map, and the expression is:

[0126]

[0127] In the formula, is the lth upsampling layer of the decoder; D l+1 is the output feature of the decoder layer l+1; ↑ represents the upsampling operation; W l ', b l ' are the weights and biases of the lth upsampling layer of the decoder respectively.

[0128] Similar to the encoder, the convolutional layer in the decoder is also used to extract features from the input, but here, the input also includes the skip connection from the corresponding level of the encoder. The expression of the convolutional layer of the decoder is:

[0129]

[0130] In the formula, is the lth convolutional layer of the decoder; σ is the activation function ReLU; * represents the convolution operation; CropOrResize is used to ensure that the two features of the jump connection have the same spatial size; E l-1 is the output feature of the encoder layer l-1; W l "" is the corresponding layer weight; W l "、b l "respectively are the weights and biases of the lth convolutional layer of the decoder.

[0131] In the embodiment of the present invention, the size of the number of samples batch_size for each iteration is set to 1, the initial number of iterations start_epoch is set to 0, the number of training rounds N_epoch is set to 20, and the voxel size block_size is set to 48. The training can be continued or terminated according to the performance of the model training process. The optimizer of the model uses Adam, the learning rate is set to 0.0002, and the exponential decay factors betas and weight decay weight_decay of the first and second moments of the gradient calculation are set to the default values. The model learnable parameters are initialized using a Gaussian distribution with an expectation of 0 and a standard deviation of 0.1, and the loss function uses the mean square error loss function.

[0132] The experiment of the embodiment of the present invention was carried out on a PC with an Intel Core i5-8300H CPU and 128G memory, and was implemented using the PyTorch 2.1.2 deep learning framework. The Amor and FCC public data sets generated by the method in the overview of the paper "Single-atom level determination of 3-dimensional surface atomic structure via neural network-assisted atomic electron tomography" by Yu Yang et al. were selected as evaluation objects, the network training parameters were set, and then the performance of the model algorithm was improved through experimental analysis. Table 1 is a comparison of experimental data of the system of the present invention with a deep learning model (DL), a convolutional neural network for image segmentation (Unet), an improved Unet (Unet++), a convolutional neural network for three-dimensional medical image segmentation (UNETR), and an improved convolutional neural network for three-dimensional medical image segmentation (UNETR++):

[0133] Table 1 Comparison of experimental data

[0134]

[0135] Among them, RMSE is the root mean square error, PSNR is the peak signal-to-noise ratio, and SSIM is the structural similarity index. All three are evaluation indicators of the model.

[0136] As can be seen from the table, the results of the system proposed in the present invention on all experimental data sets are the best overall. It performs best in terms of RMSE and PSNR, and is slightly worse than UNETR++ and on par with UNet++ in SSIM. The system of the present invention has advantages in segmenting complex structures and details, and thus performs well in RMSE and PSNR, two indicators related to pixel-level errors and image quality. It also effectively illustrates that by introducing the local sensitive hashing LSH module, the atomic electron tomography representation is reinforced at each level, which can further improve the segmentation performance of the model. The system of the present invention can effectively fuse feature information at different levels through its unique encoder-decoder structure. This multi-scale feature fusion strategy enables the model to understand the image content more comprehensively, thereby improving accuracy.

[0137] Table 2 is a comparison of the total training parameters of the system of the present invention and the current best comprehensive model UNETR++.

[0138] Table 2 Parameter comparison

[0139]

[0140] As can be seen from the table, the system proposed by the present invention has a significantly reduced number of parameters compared to UNETR++, which shows that the present invention not only has the best overall effect on the above evaluation indicators, but also effectively solves the shortcomings of high computational complexity and large number of parameters in the denoising model of atomic electron tomography three-dimensional images.

[0141] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. Only the preferred embodiments of the present invention are expressed. The description is more specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. As long as there is no contradiction in the combination of these technical features, they should be considered as the scope recorded in this specification.

[0142] It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these modifications and improvements all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A denoising system for three-dimensional electron tomography images with shared parameters, characterized in that: include: Image segmentation module, position embedding module, locality sensitive hashing (LSH) module, feature extraction module and image denoising module; The image segmentation module is used to segment the electronic tomography three-dimensional image into blocks of the same size; The position embedding module generates an embedding vector for each block, generates a position code for each embedding vector, and adds the position code to the corresponding embedding vector; The local sensitive hash (LSH) module calculates the similarity between embedded vectors and filters embedded vectors whose similarity is lower than a set similarity threshold; The feature extraction module includes an LSH module and an encoder, wherein the encoder gradually extracts features of the filtered embedding vector and reduces the resolution of the embedding vector through a series of convolutional layers and pooling layers, and the output features of each level of the encoder are directly connected to the corresponding level of the decoder through a skip connection; The image denoising module comprises an LSH module and a decoder. The decoder gradually extracts the features of the filtered embedded vector and restores the resolution of the embedded vector through a series of upsampling layers and convolutional layers, thereby achieving denoising of the electronic tomography three-dimensional image.

2. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: It also includes a feature description module, which is used to extract feature descriptors of key points in the embedded vector, including the following steps: Step S1: using a Gaussian difference function to identify key points in the embedding vector; Step S2: Calculate the gradient direction and gradient magnitude of the pixels in the neighborhood of the key point, and assign several directions to each key point; Step S3: Divide the area of ​​each key point into 16 sub-areas of the same size, calculate the gradient histogram in 8 directions in each sub-area, form a vector with a length of 128, and use the vector as the descriptor of the key point.

3. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 2, characterized in that: Step S1 includes the following steps: Step S11: constructing a Gaussian pyramid of the embedding vector; Step S12: randomly select a pixel point in a layer of the Gaussian pyramid, compare the pixel point with its 8 adjacent pixel points and 2×9 pixel points at corresponding positions of two adjacent embedding vectors in the same group, and if the pixel point is a local extreme value, take the pixel point as a key point; Step S13: Repeat step S12 until each pixel point on each layer of the Gaussian pyramid is judged.

4. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The LSH module calculates the similarity between embedded vectors through the following steps, filtering embedded vectors whose similarity is lower than the set similarity threshold: Step a: Define the number of hash buckets and the width of the hash buckets; Step b: Construct a hash function family, use the hash function family to hash each embedded vector, and assign all embedded vectors to corresponding hash buckets according to the hash results; Step c: introduce a query object q and hash the query vector q using the same hash function family; Step d: retrieve the embedding vector with the same hash value as the query object q from the hash table to form a candidate set, calculate the similarity between the query vector q and each embedding vector in the candidate set, and delete the embedding vectors whose similarity is lower than the set similarity threshold; Step e: Concatenate the remaining embedding vectors in the candidate set to obtain the filtered embedding vector.

5. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 4, characterized in that: The expression of the hash function in the hash function family is: Where h(v) is the hash function; τ() means rounding down; a is a randomly generated vector of the same dimension as v; v is the embedding vector; b is a random real number; and w is the width of the hash bucket.

6. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression of the convolutional layer of the encoder is: In the formula, is the lth convolutional layer of the encoder; σ is the activation function ReLU; E l-1 is the output feature of the encoder layer l-1; W l 、b l are the weights and biases of the encoder’s l-th convolutional layer, respectively; * indicates a convolution operation; The expression of the pooling layer of the encoder is: In the formula, is the lth pooling layer of the encoder; MaxPool() is the maximum pooling function; k and s are the size and step size of the pooling window, respectively.

7. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression of the upsampling layer of the decoder is: In the formula, is the lth upsampling layer of the decoder; D l+1 is the output feature of the decoder layer l+1; ↑ represents the upsampling operation; W l ', b l ' are the weights and biases of the upsampling layer of the decoder layer l respectively; The expression of the convolutional layer of the decoder is: In the formula, is the lth convolutional layer of the decoder; σ is the activation function ReLU; * represents the convolution operation; CropOrResize is used to ensure that the two features of the jump connection have the same spatial size; E l-1 is the output feature of the encoder layer l-1; W l "" is the corresponding layer weight; W l "、b l "respectively are the weights and biases of the lth convolutional layer of the decoder.

8. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The jump connection includes a concatenation operation or an addition operation, and the expression of the concatenation operation is: In the formula, For fusion features; concatenate is a concatenation operation; E i is the output feature of the encoder layer i; D i′ is the decoder with E i Output features of the corresponding level; axis is the dimension; The expression of the addition operation is:

9. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression for generating an embedding vector for each block is: Embed i =f embedding (Patch i ); In the formula, Embed i is the embedding vector; f embedding Patch is an embedded function; i is the i-th block.

10. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression for generating the position encoding for each embedding vector is: In the formula, PE is the position code; pos is used to specify a position in the sequence; i is the dimension index of the position code; d model is the length of the embedding vector.

Images