A denoising system for electron tomography three-dimensional images sharing parameters

Abstract

The application discloses a kind of shared parameter's electronic tomography three-dimensional image denoising system, comprising: image block module, position embedding module, local sensitive hash LSH module, feature extraction module and image denoising module;Image block module is used to divide electronic tomography three-dimensional image into the same size block;Position embedding module is used to generate an embedding vector for each block, and generates position coding and adds to corresponding embedding vector;LSH module is used to calculate the similarity between embedding vector, filter embedding vector with lower than set similarity threshold value similarity;Feature extraction module includes LSH module and encoder, and the encoder is used to extract the feature of filtered embedding vector and reduce the resolution of embedding vector;Image denoising module includes LSH module and decoder, and the decoder is used to extract the feature of filtered embedding vector and restore the resolution of embedding vector, realize the denoising of electronic tomography three-dimensional image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a denoising system for electron tomography three-dimensional images with shared parameters. BACKGROUND

[0002] With the rapid development of nanotechnology and materials science, accurate characterization of atomic structure and electron distribution has become a key requirement in scientific research and industrial fields. In recent years, with the continuous development of atomic-scale materials science and nanotechnology, the demand for three-dimensional accurate characterization of atomic structure and electron distribution has become increasingly urgent. Traditional electron microscopy and atomic force microscopy and other technologies have limitations in atomic-scale characterization, which highlights the urgent need for new technologies.

[0003] As an important electron three-dimensional imaging method for nanomaterials, the traditional characterization method of electron tomography has limitations in precision and efficiency. At present, the successful application of deep learning models in the processing and analysis of three-dimensional images provides a new way for the improvement of electron tomography precision and efficiency. Among them, the design of the lightweight model of atomic electron tomography three-dimensional characterization with shared parameters realizes high-precision and high-efficiency characterization of atomic structure and electron distribution. This model not only helps to deeply understand the structure and properties of nanomaterials, and provides important physical structure basis for the research and development of new nanomaterials and the improvement of their performance, but also has important significance for the research of nanodevices, nanomaterials and biomedical fields, and is conducive to the development and application of nanotechnology. In addition, the lightweight characteristics of this model make it more efficient and convenient in practical application, and it is expected to provide convenience for industrial production and scientific research.

[0004] Although deep learning models provide the possibility for the improvement of electron tomography precision and efficiency, the computational complexity and parameter quantity are challenges for its application. These models usually contain a large number of parameters, and involve a large number of self-attention mechanisms and multi-head attention mechanisms in the model, resulting in a large model, high computational complexity, and large demand for computing resources, which is not conducive to deployment and real-time inference on resource-constrained devices. SUMMARY

[0005] The present application provides a denoising system for electron tomography three-dimensional images with shared parameters, which solves the problem of too many parameters and high computational consumption of existing electron three-dimensional imaging models.

[0006] To solve the above technical problems, the present application provides a denoising system for electron tomography three-dimensional images with shared parameters, which includes an image blocking module, a position embedding module, a local sensitive hash (LSH) module, a feature extraction module, and an image denoising module.

[0007] The image block module divides the electron tomography three-dimensional image into blocks of the same size.

[0008] The position embedding module generates an embedding vector for each block, generates a position code for each embedding vector, and adds the position code to the corresponding embedding vector.

[0009] The local sensitive hash (LSH) module calculates the similarity between embedding vectors and filters embedding vectors with a similarity lower than a set similarity threshold.

[0010] The feature extraction module includes an LSH module and an encoder, which gradually extracts features of the filtered embedding vectors and reduces the resolution of the embedding vectors through a series of convolutional layers and pooling layers. The output features of each level of the encoder are directly connected to the corresponding level of the decoder through a skip connection.

[0011] The image denoising module includes an LSH module and a decoder, which gradually extracts features of the filtered embedding vectors and restores the resolution of the embedding vectors through a series of up-sampling layers and convolutional layers, achieving denoising of the electron tomography three-dimensional image.

[0012] Preferably, it further includes a feature description module for extracting feature descriptors of key points in the embedding vector, including the following steps:

[0013] Step S1: Identify key points in the embedding vector using a Gaussian difference function.

[0014] Step S2: Calculate the gradient direction and gradient amplitude of the pixels in the neighborhood of the key points, and assign several directions to each key point.

[0015] Step S3: Divide the domain of each key point into 16 sub-regions of the same size, calculate the gradient histogram of 8 directions in each sub-region, and form a vector of length 128, which is used as the descriptor of the key point.

[0016] Preferably, step S1 includes the following steps:

[0017] Step S11: Construct a Gaussian pyramid of the embedding vector.

[0018] Step S12: Randomly select a pixel point in one layer of the Gaussian pyramid, compare the pixel point with 8 adjacent pixel points and 2x9 pixel points corresponding to the positions of adjacent embedding vectors in the same group, and if the pixel point is a local extreme value, the pixel point is taken as a key point.

[0019] Step S13: repeat step S12 until the judgment is completed for each pixel point on each layer of the Gaussian pyramid.

[0020] Preferably, the LSH module calculates the similarity between the embedding vectors by the following steps, and filters the embedding vectors with a similarity lower than a set similarity threshold:

[0021] Step a: define the number of hash buckets and the width of the hash buckets;

[0022] Step b: construct a hash function family, hash each embedding vector using the hash function family, and distribute all embedding vectors into corresponding hash buckets according to the hash results;

[0023] Step c: introduce a query object q, hash the query vector q using the same hash function family;

[0024] Step d: retrieve embedding vectors with the same hash value as the query object q from the hash table to form a candidate set, calculate the similarity between the query vector q and each embedding vector in the candidate set, and delete embedding vectors with a similarity lower than a set similarity threshold;

[0025] Step e: splice the remaining embedding vectors in the candidate set to obtain filtered embedding vectors.

[0026] Preferably, the expression of the hash functions in the hash function family is:

[0027]

[0028] where h(v) is a hash function; τ() represents rounding down; a is a randomly generated vector with the same dimension as v; v is an embedding vector; b is a random real number; and w is the width of the hash bucket.

[0029] Preferably, the expression of the convolution layer of the encoder is:

[0030]

[0031] wherein, is the l-th layer of the convolution layer of the encoder; σ is an activation function ReLU; E l-1 is the output feature of the (l-1)-th layer of the encoder; W l , b l are the weight and bias of the l-th layer of the convolution layer of the encoder, respectively; * represents convolution operation.

[0032] The expression of the pooling layer of the encoder is:

[0033]

[0034] wherein, is the pooling layer of the ith layer of the encoder; MaxPool() is the max pooling function; k and s are the size and stride of the pooling window, respectively.

[0035] Preferably, the expression of the up-sampling layer of the decoder is:

[0036]

[0037] wherein, is the up-sampling layer of the ith layer of the decoder; D l+1 is the output feature of the ith+1 layer of the decoder; ↑ represents the up-sampling operation; W l are the weight and bias of the up-sampling layer of the ith layer of the decoder, respectively. l

[0038] The expression of the convolution layer of the decoder is:

[0039]

[0040] wherein, is the convolution layer of the ith layer of the decoder; σ is the activation function ReLU; * represents the convolution operation; CropOrResize is used to ensure that the two features of the skip connection have the same spatial size; E l-1 is the output feature of the ith-1 layer of the encoder.

[0041] W l is the weight of the corresponding layer; W l are the weight and bias of the convolution layer of the ith layer of the decoder, respectively. l

[0042] Preferably, the skip connection comprises a concatenation operation or an addition operation, and the expression of the concatenation operation is:

[0043]

[0044] wherein, is the fused feature; concatenate is the concatenation operation; E i is the output feature of the ith layer of the encoder; D i′ is the output feature of the corresponding layer in the decoder; axis is the dimension. i

[0045] The expression of the addition operation is:

[0046]

[0047] Preferably, the expression for generating an embedding vector for each block is:

[0048] Embed​​​i = f embedding (Patch i );

[0049] where Embed i is an embedding vector; f embedding is an embedding function; Patch i is the i-th patch.

[0050] Preferably, the expression for generating a positional encoding for each of the embedding vectors is:

[0051]

[0052] where PE is a positional encoding; pos is used to specify a certain position in the sequence; i is the dimension index of the positional encoding; d model is the length of the embedding vector.

[0053] The benefits of the present application include at least:

[0054] 1. By calculating the similarity between embedding vectors and filtering out vectors with similarity below a certain threshold, unnecessary calculations can be reduced, focusing on patches with high similarity, improving denoising efficiency;

[0055] 2. Reducing the resolution of the embedding vector can reduce the computational load of the model in the feature extraction process. In the process of reducing the resolution, the model can focus on the key features in the image and ignore some unimportant details, which can improve the understanding of the image content by the model. In the process of restoring the resolution of the embedding vector, the model can integrate local features and global context information to reconstruct a more complete and accurate image representation;

[0056] 3. By introducing a skip connection and a local sensitive hashing module, the model can efficiently capture multi-scale information in the image and share parameters between different levels, reducing the parameter amount of the model and making the model more lightweight, facilitating deployment and real-time inference on resource-constrained devices. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a system structure schematic diagram of an embodiment of the present application;

[0058] Figure 2 is a data processing flow schematic diagram of a local sensitive hashing module of an embodiment of the present application. DETAILED DESCRIPTION

[0059] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0060] The present application aims at the deficiency of high computational complexity and large parameter quantity in the current denoising model of atomic electron tomography three-dimensional images, and proposes a shared parameter denoising system for electron tomography three-dimensional images. Based on the layered design concept, the encoder and the decoder of the system are composed of multiple levels, and the feature resolution of each level gradually decreases or increases. At the same time, the cross-layer connection and the local sensitive hashing module are introduced, which can efficiently capture the multi-scale information in the image and reduce the computational complexity.

[0061] In order to achieve the above-mentioned target, the embodiments of the present application design and integrate a local sensitive hashing (LSH) module in each level of the encoder and the decoder, which is used to process high-dimensional data. The LSH module uses hashing technology to pre-filter out data objects with low similarity and low possibility to contribute to the final result, and only retains data objects with high similarity for subsequent processing. Then, the actual distance or similarity measurement calculation is performed on the selected candidate data set. This design ensures that similar samples in high-dimensional space can still maintain a high similarity probability in low-dimensional space after LSH hashing processing, thereby significantly improving the efficiency and robustness of the model in feature extraction.

[0062] Specifically, as shown in Figure 1 The denoising system for electron tomography three-dimensional images of the embodiments of the present application includes an image blocking module, a position embedding module, a feature description module, a local sensitive hashing (LSH) module, a feature extraction module and an image denoising module.

[0063] The image blocking module is used to divide the electron tomography three-dimensional image into blocks of the same size.

[0064] Specifically, the input atomic electron tomography three-dimensional image is a three-dimensional tensor with a shape of [H, W, D, C], where H is the height of the electron tomography three-dimensional image, W is the width of the electron tomography three-dimensional image, D is the depth of the electron tomography three-dimensional image, and C is the number of channels of the electron tomography three-dimensional image. The input three-dimensional tensor is divided into a series of patches, and the size of each patch is fixed P x P x P. After division, the set of patches can be represented by a four-dimensional tensor with a shape of [N, P, P, P, C], where N is the number of patches. In the implementation process, in order to be compatible with the structure of the Transformer, the four-dimensional tensor is further flattened or reshaped into a two-dimensional tensor.

[0065] Assuming that each dimension of the three-dimensional image can be divided by P, the expression for calculating the number of blocks N is:

[0066]

[0067] In the formula, N is the total number of blocks, and P is the side length of the block.

[0068] The position embedding module is used to generate an embedding vector for each block, generate a position code for each embedding vector, and add the position code to the corresponding embedding vector.

[0069] Specifically, a 1x1x1 convolutional layer or a fully connected layer is applied to generate an embedding vector for each patch, and the length of the embedding vector is fixed, denoted as E. This process can be represented as:

[0070] Embed i =f embedding (Patch i );

[0071] In the formula, Embed i is the embedding vector, f embedding is the embedding function, and Patch i is the i-th block.

[0072] The position code is added to the embedding vector to add position information to the embedding vector. The essence of the position code is a vector with the same length as the embedding vector. In the embodiment of the application, the sine and cosine functions are used to generate the position code. For the blocks at positions 2i or 2i+1, the position code can be represented as:

[0073]

[0074] In the formula, PE is the position code, pos is used to specify a position in the sequence, i is the dimension index of the position code, and d model is the length of the embedding vector.

[0075] The position encoding is added to the embedding vector to obtain Eebed i + PE i , which serves as the new embedding vector.

[0076] With this carefully designed encoding mechanism, it is ensured that each position in the sequence will get a unique encoding vector. The length of these encoding vectors is consistent with the length of the embedding vectors, thus ensuring the consistency and integrity of the information. This method not only considers the position information, but also captures the intrinsic characteristics of the data through the embedding vector.

[0077] The embedding vector containing position information is then input into the Transformer model. The Transformer model can understand the spatial relationship between different patches using these embedding vectors with position information. In the Transformer model, the main operations include Self-Attention and Feed-Forward Networks (FFN). Self-Attention allows the model to consider the relationship between each patch and all other patches in the sequence, while Feed-Forward Networks further transform these relationships in a nonlinear manner.

[0078] Self-Attention captures the relationship between patches by calculating attention scores between them. These scores reflect the mutual importance between different patches:

[0079]

[0080] Q = XW q , K = XW k , V = XW v ;

[0081] In the formula, Attention is the attention score; Q, K, and V are the query vector, key vector, and value vector of the self-attention mechanism, respectively; Softmax is the activation function; X is the embedding vector containing position information; W q , W k , and W v are learnable weight matrices; d k is the dimension of the key vector, used to scale the dot product attention score to prevent gradient vanishing or explosion.

[0082] Feed-Forward Networks are usually placed after Self-Attention to further transform the representation of each position. It usually includes two linear transformations and a ReLU activation function. The calculation formula of Feed-Forward Networks is as follows:

[0083] FFN(x) = ReLU(xW1 + b1)W2 + b2;

[0084] where ReLU is an activation function; x is the output of the self-attention mechanism; W1 and W2 are learnable weight matrices; and b1 and b2 are learnable bias terms.

[0085] The feature description module is configured to extract feature descriptors of the key points in the embedding vectors, including the following steps:

[0086] Step S1: using a Gaussian difference function to identify the key points in the embedding vectors, including the following steps:

[0087] Step S11: performing Gaussian difference on the embedding vectors to construct a Gaussian pyramid of the embedding vectors:

[0088] DoG(x, y, σ) = G(x, y, kσ) - G(x, y, σ);

[0089] where G(x, y, σ) is a two-dimensional Gaussian function; (x, y) is the coordinate of a pixel point; σ is a scale parameter; and k is a scale factor between two adjacent scales.

[0090] Step S12: randomly selecting a pixel point in one layer of the Gaussian pyramid, comparing the pixel point with 8 adjacent pixel points of the pixel point and 2x9 pixel points corresponding to the positions of the adjacent two embedding vectors in the same group, and taking the pixel point as a key point if the pixel point is a local extreme value.

[0091] Step S13: repeating step S12 until each pixel point on each layer of the Gaussian pyramid is judged.

[0092] Step S2: calculating the gradient direction and gradient amplitude of the pixels in the neighborhood of the key points, and assigning a number of directions to each key point, wherein the calculation formulas of the gradient amplitude and the gradient direction are respectively:

[0093]

[0094] where m(x, y) is the gradient amplitude; θ(x, y) is the gradient direction; and L(x, y) is the image gray value at the scale to which the key point belongs.

[0095] Step S3: dividing the domain of each key point into 16 sub-regions of the same size, calculating the gradient histogram of 8 directions in each sub-region, forming a vector with a length of 128, and taking the vector as the descriptor of the key point.

[0096] The local sensitive hashing (LSH) module is configured to calculate the similarity between the embedding vectors and filter the embedding vectors with a similarity lower than a set similarity threshold, such as:Figure 2 As shown, comprising the following steps:

[0097] Step a: define the number of hash buckets and the width of the hash buckets.

[0098] Step b: construct a family of hash functions, hash each embedding vector using the family of hash functions, and distribute all embedding vectors into corresponding hash buckets according to the hash results.

[0099] Specifically, the expression of the hash function in the constructed family of hash functions is:

[0100]

[0101] In the formula, h(v) is a hash function; τ() represents rounding down; a is a randomly generated vector of the same dimension as v, the elements of which follow a normal distribution; v is an embedding vector; b is a random real number used to increase the randomness of the hash function; w is a positive real number, which is defined as the width of the hash bucket. In order to meet the sensitivity condition of LSH, the present embodiment designs multiple such hash functions to form a family of hash functions H. Each hash function h∈H uses different random vectors a and random real numbers b.

[0102] Step c: introduce a query object q, and hash the query vector q using the same family of hash functions.

[0103] Step d: retrieve embedding vectors with the same hash value as the query object q from the hash table to form a candidate set, calculate the similarity between the query vector q and each embedding vector in the candidate set, and delete embedding vectors with a similarity lower than a set similarity threshold.

[0104] Specifically, the present embodiment retrieves embedding vectors with the same hash value as the query object from the hash table by calculating the collision probability of the query object and all embedding vectors in the hash table to form a candidate set. In LSH, the collision probability refers to the probability that two similar objects are mapped to the same hash value by the same LSH function. Assuming that two points x and y are similar under the distance metric d, the collision probability can be calculated by analyzing the properties of the LSH function family for any hash function h in the LSH function family H:

[0105] P(h(x)=h(y)|d(x,y)≤r1);

[0106] In the formula, P is the collision probability; h(y) represents the hash value obtained by applying the hash function h to the input object y; d(x,y) is the Euclidean distance between x and y; r1 is a set similarity threshold.

[0107] Step e: concatenate the remaining embedding vectors in the candidate set to obtain the filtered embedding vector.

[0108] The feature extraction module includes an LSH module and an encoder. The encoder gradually extracts features of the filtered embedding vector and reduces the resolution of the embedding vector through a series of convolutional layers and pooling layers.

[0109] The expression of the convolutional layer of the encoder is:

[0110]

[0111] In the formula, is the lth convolutional layer of the encoder; σ is the activation function ReLU; E l-1 is the output feature of the (l-1)th layer of the encoder; W l and b l are the weight and bias of the lth convolutional layer of the encoder, respectively; * represents the convolution operation;

[0112] The expression of the pooling layer of the encoder is:

[0113]

[0114] In the formula, is the lth pooling layer of the encoder; MaxPool() is the maximum pooling function; k and s are the size and step length of the pooling window, respectively.

[0115] By introducing the LSH module in the feature extraction module, similar vectors in the original data space are mapped to similar buckets in the hash space, greatly reducing the number of comparisons. This design enables the model to better focus on important regions and features in the image, thereby improving the efficiency and robustness of representation extraction.

[0116] The image denoising module includes an LSH module and a decoder. The decoder gradually extracts features of the filtered embedding vector and restores the resolution of the embedding vector through a series of upsampling layers and convolutional layers, achieving denoising of the electron tomography three-dimensional image.

[0117] Similar to the encoding layer, the decoding layer also has multiple layers, each of which contains an upsampling operation and multiple convolutional blocks. In addition, in order to utilize the information in the encoder, a skip connection is set in the decoder to concatenate or add the features output by the encoder with the corresponding layers in the decoder.

[0118] The skip connection includes a concatenation operation or an addition operation. In order to more accurately describe this process, a mathematical representation of tensor operation can be used. Assume that E i is the output feature of the i-th layer of the encoder, which has a shape of (H i , W i , C i ), where H i and Wi is the height and width of the feature map, C i is the number of channels. Similarly, assume D i′ is the output feature of the corresponding level in the decoder, which has a shape of (H i ,W i ,C i ). The two feature maps are merged in the channel dimension, and the fused feature map F i′ has a shape of (H fusedi ,W i ,C i ,C i +C i′ ).

[0119] The concatenation operation can be expressed as:

[0120] F fusedi = concatenate(E i ,D i′ ,axis = 2) ;

[0121] where F fusedi is the fused feature; concatenate is the concatenation operation; E i is the output feature of the i-th layer in the encoder; D i′ is the output feature of the corresponding level in the decoder as E i ; and axis is the dimension, which actually means stacking the two tensors in the third dimension, i.e., the index starts from 0.

[0122] The addition operation is to add the values of the elements in the corresponding positions to obtain a new feature, and the addition operation can be expressed as:

[0123]

[0124] Before using the addition operation, the number of channels needs to be adjusted by 1x1 convolution so that they match.

[0125] The up-sampling layer is used to restore the spatial size of the feature map, and the expression is:

[0126]

[0127] where is the up-sampling layer of the l-th layer in the decoder; D l+1 is the output feature of the l+1-th layer in the decoder; ↑ represents the up-sampling operation; W l ' and b l ' are the weight and bias of the up-sampling layer of the l-th layer in the decoder, respectively.

[0128] Similar to the encoder, the convolutional layers in the decoder are also used to extract features from the input, but here the input also includes the skip connections from the corresponding levels of the encoder. The expression of the convolutional layers of the decoder is:

[0129]

[0130] wherein, is the l-th layer of the convolutional layer of the decoder; σ is the activation function ReLU; * represents the convolution operation; CropOrResize is used to ensure that the two features of the skip connection have the same spatial size; E l-1 is the output feature of the (l-1)-th layer of the encoder; W l is the weight of the corresponding layer; W l , b l are the weight and bias of the l-th layer of the convolutional layer of the decoder, respectively.

[0131] In the embodiments of the present application, the size of the number of samples batch_size of each iteration is set to 1, the initial iteration number start_epoch is set to 0, the number of training rounds N_epoch is set to 20, and the voxel size block_size is set to 48. The training can be continued or terminated according to the performance during the model training process. The optimizer of the model uses Adam, the learning rate is set to 0.0002, and the exponential decay factors betas, weight decay, and the like for calculating the first and second moments of the gradient are all set to the default values. Among them, the model learnable parameters are initialized using a Gaussian distribution with an expectation of 0 and a standard deviation of 0.1, and the loss function uses the mean square error loss function.

[0132] The experiment of the embodiment of the present application is performed on a PC with an Intel Core i5-8300H CPU and 128G memory, and is implemented using a PyTorch 2.1.2 deep learning framework. The Amor and FCC public data sets generated by the method of YuYang et al. in the summary of the paper Single-atom level determination of 3-dimensional surface atomic structure via neural network-assisted atomic electron tomography by J. Lee et al. are selected as evaluation objects, network training parameters are set, and then the performance of the improved model algorithm is analyzed through experiments. Table 1 is a comparison of experimental data of the system and deep learning model (DL) of the present application, convolutional neural network (Unet) for image segmentation, improved Unet (Unet++), convolutional neural network (UNETR) for three-dimensional medical image segmentation, and improved convolutional neural network (UNETR++) for three-dimensional medical image segmentation:

[0133] Table 1 Comparison of experimental data

[0134]

[0135] Among them, RMSE is the root mean square error, PSNR is the peak signal-to-noise ratio, and SSIM is the structural similarity index, all of which are evaluation indexes of the model.

[0136] As can be seen from the table, the results of the system proposed by the present application on all data sets in the experiment are the best overall. It performs best in the two indicators of RMSE and PSNR, and is slightly worse than UNETR++ in SSIM, and is on a par with Unet++. The system of the present application has an advantage in segmenting complex structures and details, thereby performing excellently in the two indicators of RMSE and PSNR related to pixel-level error and image quality. At the same time, it also effectively shows that by introducing the local sensitive hash LSH module, the atomic electron tomography representation is strengthened in each level, which can further improve the segmentation performance of the model. The system of the present application can effectively fuse feature information of different levels through its unique encoder-decoder structure. This multi-scale feature fusion strategy enables the model to more comprehensively understand the image content, thereby improving the precision.

[0137] Table 2 is a comparison of the total training parameter amount of the system of the present application and the current best model UNETR++.

[0138] Table 2 Comparison of parameter amount

[0139]

[0140] As can be seen from the table, the parameter quantity of the system proposed in the application is greatly reduced compared with UNETR++. It is illustrated that the application not only has the best overall effect on the above evaluation indexes, but also effectively solves the problems of high calculation complexity and large parameter quantity in the denoising model of atomic electron tomography three-dimensional images.

[0141] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, only the preferred embodiments of the application are expressed, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent of the application. As long as the combinations of these technical features do not exist contradictory, they should be considered as the scope recorded in the specification.

[0142] It should be pointed out that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which are within the scope of the application. Therefore, the protection scope of the patent of the application should be subject to the appended claims.

Claims

1. A parameter-sharing 3D electron tomography image denoising system, characterized in that: include: Image segmentation module, position embedding module, locality sensitive hashing (LSH) module, feature extraction module, and image denoising module; The image segmentation module is configured to segment the three-dimensional electron tomography image into blocks of the same size. The position embedding module generates an embedding vector for each block, generates a position code for each embedding vector, and adds the position code to the corresponding embedding vector; The local sensitive hash (LSH) module calculates the similarity between embedded vectors and filters embedded vectors whose similarity is lower than a set similarity threshold; The feature extraction module includes an LSH module and an encoder. The encoder gradually extracts features of the filtered embedding vector and reduces the resolution of the embedding vector through a series of convolutional layers and pooling layers. The output features of each level of the encoder are directly connected to the corresponding level of the decoder through skip connections. The image denoising module includes an LSH module and a decoder. The decoder gradually extracts the features of the filtered embedding vector and restores the resolution of the embedding vector through a series of upsampling layers and convolutional layers to achieve denoising of the electronic tomography three-dimensional image.

2. The denoising system for three-dimensional electronic tomography images using shared parameters according to claim 1, characterized in that: The invention also includes a feature description module, wherein the feature description module is used to extract feature descriptors of key points in the embedding vector, including the following steps: Step S1: using a Gaussian difference function to identify key points in the embedding vector; Step S2: Calculate the gradient direction and gradient magnitude of the pixels in the neighborhood of the key point, and assign several directions to each key point; Step S3: Divide the area of ​​each key point into 16 sub-areas of equal size, calculate the gradient histogram of 8 directions in each sub-area, form a vector with a length of 128, and use the vector as the descriptor of the key point.

3. The denoising system for three-dimensional electronic tomography images using shared parameters according to claim 2, characterized in that: Step S1 includes the following steps: Step S11: constructing a Gaussian pyramid of the embedding vector; Step S12: randomly selecting a pixel point in a layer of the Gaussian pyramid, comparing the pixel point with its eight adjacent pixels and the 2×9 pixels at corresponding positions of two adjacent embedding vectors in the same group, and if the pixel point is a local extreme value, taking the pixel point as a key point; Step S13: Repeat step S12 until every pixel point on every layer of the Gaussian pyramid is judged.

4. The denoising system for three-dimensional electronic tomography images using shared parameters according to claim 1, characterized in that: The LSH module calculates the similarity between embedding vectors through the following steps, filtering out embedding vectors whose similarity is lower than the set similarity threshold: Step a: Define the number of hash buckets and the width of the hash buckets; Step b: Construct a hash function family, use the hash function family to hash each embedding vector, and assign all embedding vectors to corresponding hash buckets according to the hash results; Step c: introduce a query object q and hash the query object q using the same hash function family; Step d: Retrieve the embedding vector with the same hash value as the query object q from the hash table to form a candidate set. Calculate the similarity between the query object q and each embedding vector in the candidate set, and delete the embedding vectors whose similarity is lower than the set similarity threshold. Step e: Concatenate the remaining embedding vectors in the candidate set to obtain the filtered embedding vector.

5. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 4, characterized in that: The expression of the hash function in the hash function family is: Where h(v) is the hash function; τ() represents rounding down; a is a randomly generated vector of the same dimension as v; v is the embedding vector; b is a random real number; and w is the width of the hash bucket.

6. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression of the convolutional layer of the encoder is: Where, is the lth convolutional layer of the encoder; σ is the activation function ReLU; E l-1 is the output feature of the encoder layer l-1; W l 、b l are the weights and biases of the encoder's lth convolutional layer respectively; * denotes the convolution operation; The expression of the pooling layer of the encoder is: Where, is the lth pooling layer of the encoder; MaxPool() is the maximum pooling function; k and s are the size and step size of the pooling window respectively.

7. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression of the upsampling layer of the decoder is: Where, is the lth upsampling layer of the decoder; D l+1 is the output feature of the decoder layer l+1; ↑ represents the upsampling operation; W l '、b l ' are the weights and biases of the upsampling layer of the decoder layer l; The expression of the convolutional layer of the decoder is: Where, is the lth convolutional layer of the decoder; σ is the activation function ReLU; * represents the convolution operation; CropOrResize is used to ensure that the two features of the jump connection have the same spatial size; E l-1 is the output feature of the encoder layer l-1; W l "" is the corresponding layer weight; W l "、b l "respectively are the weights and biases of the lth convolutional layer of the decoder.

8. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The jump connection includes a splicing operation or an addition operation, and the expression of the splicing operation is: Where, It is a fusion feature; concatenate is a concatenation operation; E i is the output feature of the encoder layer i; D i′ For the decoder and E i Output features of the corresponding level; axis is the dimension; The expression of the addition operation is:

9. The parameter-sharing electron tomography 3D image denoising system according to claim 1, characterized in that: The expression for generating an embedding vector for each block is: Embed j =f embedding (Patch j ); In the formula, Embed j is the embedding vector of the jth block; f embedding For embedded functions; Patch j is the jth block.

10. The denoising system for three-dimensional electronic tomography images with shared parameters according to claim 1, characterized in that: The expression for generating the position encoding for each embedding vector is: Where PE is the position code; pos is used to specify a position in the sequence; f is the dimension index of the position code; d model is the length of the embedding vector.

Images