Three-dimensional particle reconstruction technology and system based on convolutional neural network

Abstract

The invention relates to a three-dimensional particle reconstruction technology and system based on a convolutional neural network, and the technology comprises the steps: setting particle attributes, randomly generating three-dimensional real particle distribution based on the set particle attributes, and building a two-dimensional projection array in combination with the three-dimensional real particle distribution. Generating three-dimensional reconstructed particle distribution according to the two-dimensional projection array through a spatial filtering multiplication algebraic number reconstruction technology, then creating a reconstruction model, dividing the three-dimensional reconstructed particle distribution into a training set and a test set according to a random proportion, inputting the training set into the reconstruction model, training the reconstruction model by adopting a gradient descent algorithm, and obtaining a reconstructed particle distribution model; according to the method, a CBAM module and an ASPP module are introduced on the basis of the reconstructed model, the feature extraction capability and the anti-noise capability of the reconstructed model are improved, in addition, a quality factor is introduced into a loss function, and the quality of the reconstructed model is improved. And the reconstruction precision of the reconstruction model is further improved.

Description

Technical Field

[0001] The present application relates to the field of flow visualization measurement technology, and in particular to a three-dimensional particle reconstruction technology and system based on convolutional neural networks. Background Art

[0002] Complex flow phenomena exist widely in nature and industry, and affect human daily life and industrial production in all aspects. From global atmospheric circulation to gas circulation in the respiratory system, from the surging of rivers, lakes and seas to the material transport of blood in the human body, understanding and exploring the generation mechanism and physical laws behind these flow phenomena is crucial for human survival and development.

[0003] Obtaining flow field distribution information through three-dimensional particle reconstruction is a key step in visual measurement. Traditional methods usually use limited two-dimensional projections to reconstruct particle distribution in three-dimensional space. However, this is an underdetermined problem and it is difficult to obtain an exact solution. In addition, the reconstruction results are often accompanied by the occurrence of false particles and particle elongation, which affects the subsequent flow field velocity measurement. Summary of the invention

[0004] Based on this, it is necessary to provide a three-dimensional particle reconstruction technology and system based on convolutional neural network to improve the flow field reconstruction accuracy and flow field velocity measurement effect.

[0005] On the one hand, the present application provides a three-dimensional particle reconstruction technology based on a convolutional neural network, comprising:

[0006] Setting particle attributes, randomly generating a three-dimensional real particle distribution based on the set particle attributes, creating a two-dimensional projection array in combination with the three-dimensional real particle distribution, and generating a three-dimensional reconstructed particle distribution based on the two-dimensional projection array through a spatial filtering multiplication algebraic number reconstruction technique;

[0007] Create a refactoring model;

[0008] The three-dimensional reconstructed particle distribution is divided into a training set and a test set in a random ratio, the training set is input into the reconstruction model, and the gradient descent algorithm is used to train the reconstruction model to obtain the trained reconstruction model;

[0009] Input the test set into the trained reconstruction model to determine whether the trained reconstruction model is qualified;

[0010] If the reconstructed model after training is qualified, the training is completed;

[0011] The three-dimensional reconstructed particle distribution is divided into a training set and a test set in a random ratio, the training set is input into a reconstruction model, and a gradient descent algorithm is used to train the reconstruction model to obtain a trained reconstruction model, including:

[0012] Introducing a CBAM module into a reconstruction model; the CBAM module includes a channel attention mechanism module and a spatial attention mechanism module;

[0013] Use the ASPP module to replace the transition layer of the reconstructed model;

[0014] Introduce the quality factor, combine the quality factor with the MSE loss function, and get the training loss function;

[0015] The training loss function is optimized through the gradient descent algorithm to obtain the reconstructed model after training.

[0016] On the other hand, the present application also provides a three-dimensional particle reconstruction system based on a convolutional neural network, and the three-dimensional particle reconstruction system based on a convolutional neural network includes:

[0017] A simulation distribution component, through which particle properties are set and particle distribution is simulated based on the particle properties;

[0018] A processing component, wherein the simulation distribution component is communicatively connected to the processing component, wherein the processing component includes a reconstruction model, and the aforementioned convolutional neural network-based three-dimensional particle reconstruction technology is executed through the processing component.

[0019] The present application relates to a three-dimensional particle reconstruction technology and system based on a convolutional neural network. By setting particle attributes, a three-dimensional real particle distribution is randomly generated based on the set particle attributes, a two-dimensional projection array is created in combination with the three-dimensional real particle distribution, and a three-dimensional reconstructed particle distribution is generated according to the two-dimensional projection array through a spatial filtering multiplication algebraic number reconstruction technology. After that, a reconstruction model is created, and the three-dimensional reconstructed particle distribution is divided into a training set and a test set in a random proportion. The training set is input into the reconstruction model, and a gradient descent algorithm is used to train the reconstruction model to obtain a trained reconstruction model. Finally, it is determined whether the trained reconstruction model is qualified. If the trained reconstruction model is qualified, the training is completed. The present application introduces a CBAM module and an ASPP module on the basis of the reconstruction model, which improves the feature extraction capability and noise resistance capability of the reconstruction model. In addition, a quality factor is introduced into the loss function, thereby further improving the reconstruction accuracy of the reconstruction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a process flow of a three-dimensional particle reconstruction technology based on a convolutional neural network provided in one embodiment of the present application.

[0021] Figure 2A schematic structural diagram of a three-dimensional particle reconstruction system based on a convolutional neural network provided in one embodiment of the present application.

[0022] Figure 3 A structural schematic diagram of an ASPP module of a three-dimensional particle reconstruction technology based on a convolutional neural network provided in one embodiment of the present application.

[0023] Figure 4 A flowchart of a CBAM module of a three-dimensional particle reconstruction technology based on a convolutional neural network is provided in one embodiment of the present application.

[0024] Figure 5 An embodiment of the present application provides a three-dimensional real particle distribution based on a three-dimensional particle reconstruction technology based on a convolutional neural network, a particle distribution after SF-MART reconstruction (two-dimensional projection without noise), and a particle distribution result diagram after further correction by a neural network.

[0025] Figure 6 A particle reconstruction accuracy curve diagram under different particle densities of a three-dimensional particle reconstruction technology based on a convolutional neural network provided in one embodiment of the present application.

[0026] Figure 7 An embodiment of the present application provides a three-dimensional real particle distribution based on a three-dimensional particle reconstruction technology of a convolutional neural network, a particle distribution after SF-MART reconstruction (two-dimensional projection with noise), and a particle distribution result diagram after further correction by a neural network.

[0027] Figure 8 A particle reconstruction accuracy curve under different noise intensities of a three-dimensional particle reconstruction technology based on a convolutional neural network provided in one embodiment of the present application.

[0028] Fig. 9 A structural schematic diagram of a reconstruction model of a three-dimensional particle reconstruction technology based on a convolutional neural network provided in one embodiment of the present application.

[0029] Reference numerals:

[0030] 100, simulation distribution component; 200, processing component; 201, reconstruction model. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] The present application provides a three-dimensional particle reconstruction technology and system based on convolutional neural network.

[0033] like Figure 1 As shown, in one embodiment of the present application, a three-dimensional particle reconstruction technology based on a convolutional neural network is provided, and the three-dimensional particle reconstruction technology based on a convolutional neural network includes:

[0034] S100, setting particle attributes, randomly generating a three-dimensional real particle distribution based on the set particle attributes, creating a two-dimensional projection array in combination with the three-dimensional real particle distribution, and generating a three-dimensional reconstructed particle distribution according to the two-dimensional projection array through a spatial filtering multiplication algebraic number reconstruction technology.

[0035] Optionally, after S100 and before S200, the method further includes:

[0036] K100, divides the 3D reconstructed particle distribution into multiple local particle fields with small resolution.

[0037] Since the artificial random particle field with a resolution of 256×256×128 is used as training and testing data when generating three-dimensional real particle distribution, but due to the limited memory of the graphics card in the actual simulation process, it is very challenging to perform 3D convolution on such a large 4D tensor on the GPU.

[0038] Therefore, in order to make the training affordable for computers, we chose to divide the 3D reconstructed particle distribution into 64 local particle fields with a resolution of 64×64×32 and feed them into the reconstruction model in batches, approximating it in a local mapping manner rather than a global mapping.

[0039] Step K100 reduces the size of the network's input and output, increases the amount of data used to train the reconstruction model, and expands the amount of data for small batch training, making the reconstructed model more reliable after training.

[0040] After the training is completed, you only need to splice the results of the reconstructed model output in sequence and perform loss with the three-dimensional real particle distribution to back-propagate and train the neural network.

[0041] S200, creating a reconstruction model.

[0042] Specifically, the reconstruction model is based on Unet3D.

[0043] like Fig. 9 As shown in the figure, the encoder is the left half of the U-shaped network, which includes the dark blue convolution module, the orange CBAM module, and the light pink downsampling module. The transition layer is the bottom part of the U-shaped network, which includes the dark blue convolution module and the purple ASPP module. The decoder is the right half of the U-shaped network, which includes the dark blue convolution module and the red upsampling module. In addition, the network also has a skip connection in the middle and a sigmoid activation layer at the end of the output.

[0044] S300, dividing the three-dimensional reconstructed particle distribution into a training set and a test set in a random ratio, inputting the training set into a reconstruction model, using a gradient descent algorithm to train the reconstruction model, and obtaining a trained reconstruction model.

[0045] Specifically, in the present application, the training set and the test set are divided in a ratio of 4:1, so that 80% of the data is used for training and 20% of the data is used for testing.

[0046] S400, inputting the test set into the trained reconstruction model to determine whether the trained reconstruction model is qualified.

[0047] S500: If the reconstructed model after training is qualified, the training is completed.

[0048] The three-dimensional reconstructed particle distribution is divided into a training set and a test set in a random ratio, the training set is input into a reconstruction model, and a gradient descent algorithm is used to train the reconstruction model to obtain a trained reconstruction model, including:

[0049] S310, introducing a CBAM module into the reconstruction model. The CBAM module includes a channel attention mechanism module and a spatial attention mechanism module.

[0050] S320, using the ASPP module to replace the transition layer of the reconstructed model.

[0051] S330, introducing a quality factor, combining the quality factor with the MSE loss function, and obtaining a training loss function.

[0052] S340, optimizing the training loss function through a gradient descent algorithm to obtain a reconstructed model after training.

[0053] In this embodiment, by setting particle properties, a three-dimensional real particle distribution is randomly generated based on the set particle properties, a two-dimensional projection array is created in combination with the three-dimensional real particle distribution, and a three-dimensional reconstructed particle distribution is generated according to the two-dimensional projection array through spatial filtering multiplication algebraic number reconstruction technology. Then, a reconstruction model is created, and the three-dimensional reconstructed particle distribution is divided into a training set and a test set in random proportions. The training set is input into the reconstruction model, and a gradient descent algorithm is used to train the reconstruction model to obtain a trained reconstruction model. Finally, it is determined whether the trained reconstruction model is qualified. If the trained reconstruction model is qualified, the training is completed. The present application introduces a CBAM module and an ASPP module on the basis of the reconstruction model, which improves the feature extraction capability and noise resistance capability of the reconstruction model. In addition, a quality factor is introduced into the loss function, thereby further improving the reconstruction accuracy of the reconstruction model.

[0054] In one embodiment of the present application, the S100 includes:

[0055] S110, setting particle attributes, including particle field voxel resolution, particle concentration, and particle diameter.

[0056] S120, performing simulation based on the set particle properties to randomly generate a three-dimensional real particle distribution.

[0057] Specifically, the simulation can be performed through Matlab.

[0058] S130, setting the virtual camera resolution, and generating a two-dimensional projection array in combination with the three-dimensional real particle distribution.

[0059] In this embodiment, in real experiments, particle images are taken by high-speed cameras, and the real positions of particles are not recorded. In addition, due to measurement errors and environmental interference in real experiments, the accuracy of the results cannot be guaranteed. In addition, the training of neural networks requires a large number of data samples. If these data are generated through real experiments, it often takes a lot of time and money, which is very costly.

[0060] Therefore, this application generates three-dimensional real particle distribution through simulation, so that the particle distribution used as training samples can meet the needs of deep learning and has diversity, so that the reconstruction model finally trained has better reliability.

[0061] In one embodiment of the present application, the S100 further includes:

[0062] S140, setting the total number of iterations K.

[0063] S141, obtain three-dimensional real particle distribution.

[0064] S142, reconstructing the projected particle distribution using formula 1, and continuously iterating and updating until a three-dimensional reconstructed particle distribution closest to the three-dimensional real particle distribution is obtained.

[0065]

[0066] Among them, E k (X j ,Y j ,Z j ) is the particle distribution of the kth iteration, X j is the abscissa of the particle distribution at the kth iteration, Y j is the ordinate of the particle distribution at the kth iteration, Z j is the vertical coordinate of the particle distribution of the kth iteration, k is the number of iterations, μ is the relaxation scalar parameter of the iteration, 0≤μ≤1, I(x i ,y i ) is the pixel (x i ,y i) The current measured pixel intensity distribution, x i is the horizontal coordinate of the pixel point on the two-dimensional projection, y i is the vertical coordinate of the pixel point on the two-dimensional projection, is the projection of the current spatial reconstruction target.

[0067] Specifically, since some data in the three-dimensional reconstructed particle distribution may contain noise, after each three-dimensional reconstructed particle distribution is obtained by Formula 1, it is necessary to perform Gaussian filtering on the three-dimensional reconstructed particle distribution to remove the noise that may be contained in the three-dimensional reconstructed particle distribution.

[0068] In this application, the reconstruction method of SF-MART-MyCNN is adopted, SF-MART-Unet3D is the reconstruction method of SF-MART combined with Unet3D, SF-MART-Unet is the reconstruction method of SF-MART combined with Unet, and SF-MART-10iterations is ten iterations of SF-MART.

[0069] S143, let the number of iterations k = k + 1, and determine whether the number of iterations k is equal to the total number of iterations K.

[0070] S144, if the number of iterations k is equal to the total number of iterations K, then the iteration is exited, and the particle distribution of the last K-th iteration is used as the three-dimensional reconstructed particle distribution.

[0071] S145, if the number of iterations k is less than the total number of iterations K, return to S142 until the number of iterations k is equal to the total number of iterations K.

[0072] In this embodiment, the currently measured pixel intensity distribution I(x i ,y i ) and the projection of the current spatial reconstruction target The ratio of represents the evaluation of the current projection error and is also the amplitude of an iterative correction. The spatially reconstructed intensity distribution will be continuously iterated based on the current projection error to approach a relatively optimal solution.

[0073] In one embodiment of the present application, the S310 includes:

[0074] S311, input the feature map into the channel attention module for pooling, obtain the channel attention weight matrix, and combine the channel attention weight matrix and the feature map to obtain a new feature map.

[0075] Specifically, Figure 4 As shown, the CBAM module is placed before each downsampling of the encoder.

[0076] S312, input the new feature map into the spatial attention mechanism module to obtain the spatial attention weight matrix, and combine the spatial attention weight matrix and the new feature map to obtain the final attention feature.

[0077] In this embodiment, in the feature extraction part of the encoder of the reconstruction model, the channel information and spatial information of the feature map are regarded as information with equal importance. However, doing so will ignore the importance of different feature channels and different spatial positions to the target to be segmented, resulting in inaccurate final segmentation results.

[0078] Therefore, the CBAM module is introduced. The CBAM module dynamically adjusts the weights of the feature maps through the channel attention mechanism and the spatial attention mechanism, so that the reconstruction model can focus on more important features, thereby improving the reconstruction performance of the reconstruction model.

[0079] In one embodiment of the present application, the S311 includes:

[0080] S3111, input the feature map into the channel attention mechanism module to perform global maximum pooling and global average pooling on the feature map to obtain a global maximum pooling feature map and a global average pooling feature map.

[0081] Specifically, global pooling is a commonly used dimensionality reduction method in convolutional neural networks, including global maximum pooling and global average pooling. These two operations can effectively extract channel information of feature maps, generate channel identifiers, and help improve the generalization ability of the model.

[0082] In this application, both global maximum pooling and global average pooling are performed on the channel dimension of the feature map.

[0083] S3112, respectively inputting the global maximum pooling feature map and the global average pooling feature map into the multi-layer perceptron for learning, and obtaining the global maximum pooling feature and the global average pooling feature respectively.

[0084] Specifically, a multilayer perceptron is a feedforward artificial neural network model: one or more hidden layers are introduced on the basis of a single-layer neural network, so that the neural network has multiple network layers, and the output of the hidden layer is transformed through an activation function.

[0085] S3113, add the global maximum pooling feature and the global average pooling feature element by element to obtain a sum feature map.

[0086] S3114, mapping the sum feature map is performed using Formula 2 to obtain a channel attention weight matrix.

[0087]

[0088] Among them, Mc(F) is the channel attention weight matrix, σ is the Sigmoid activation function, is the global maximum pooling feature, is the global average pooling feature, MLP represents multi-layer perceptron, W0 is the first weight matrix in MLP, W1 is the second weight matrix in MLP, and F is the input feature map.

[0089] Specifically, the first weight matrix W0 is used to connect the input layer and the hidden layer, and its dimension is the number of input features × the number of neurons in the hidden layer. The first weight matrix W0 maps the input data to the space of the hidden layer, which plays a role in preliminary feature extraction of the data.

[0090] The second weight matrix W1 is used to connect the hidden layer and the output layer, and its dimension is the number of neurons in the hidden layer × the number of neurons in the output layer. The activation value of the hidden layer is converted into the final output of the model through the second weight matrix W1, completing the mapping from features to decisions.

[0091] S3115, multiply the feature map by the channel attention weight matrix element by element to obtain a new feature map.

[0092] In this embodiment, feature information in the feature map is extracted through global maximum pooling and global average pooling. Global maximum pooling can highlight the most significant information of the feature map to reduce overfitting, while global average pooling can extract the overall information of the feature map to reduce the sensitivity of the reconstructed model to the feature position.

[0093] After completing the global maximum pooling and global average pooling of the feature map, the global maximum pooling feature map and the global average pooling feature map obtained by the global maximum pooling and the global average pooling feature map are respectively input into a shared multi-layer perceptron for learning, and the global maximum pooling features and the global average pooling features output by the multi-layer perceptron are added element by element, and finally a new feature map F' is obtained through the mapping processing of the Sigmoid activation function.

[0094] In one embodiment of the present application, the S312 includes:

[0095] S3121, input the new feature map into the spatial attention mechanism module to perform spatial pooling on the new feature map to obtain spatial pooling features.

[0096] Specifically, when performing spatial pooling, average pooling and maximum pooling are also included, so as to obtain spatial average pooling features and spatial maximum pooling features. Therefore, spatial pooling features include average pooling features and maximum pooling features.

[0097] S3122, splicing the spatial pooling features by channel to obtain a spliced ​​image.

[0098] S3123, performing a convolution operation on the stitched image to obtain a convolution image.

[0099] S3124, activate the convolution image through formula 3 to obtain the spatial attention weight matrix.

[0100] Ms(F')=σ(f 7×7 ([Avgpool(F'); Maxpool(F')])) Formula 3.

[0101] Among them, Ms(F') is the spatial attention weight matrix, σ is the Sigmoid activation function, and f 7×7 is the convolution operation, and F' is the new feature map.

[0102] S3125, multiply the spatial attention weight matrix by the new feature map element by element to obtain the final attention feature.

[0103] In this embodiment, the new feature map obtained by the channel attention mechanism module is used as the input of the spatial attention mechanism module, and the new feature map is pooled in the spatial dimension. The pooled new feature map is spliced ​​by channel, and then a 7×7 convolution operation is performed, and then the spatial attention weight matrix is ​​obtained through the Sigmoid activation function.

[0104] In one embodiment of the present application, the S320 includes:

[0105] S321, input the features extracted by the encoder into the ASPP module.

[0106] Specifically, the ASPP module is a dilated spatial convolutional pooling pyramid module.

[0107] S322, extracting feature information of different scales through the ASPP module to obtain feature maps containing information of different scales.

[0108] S323, concatenating the feature maps containing information of different scales, and fusing them through convolution to obtain a fused feature map.

[0109] S324, normalize and ReLU activate the fused feature map.

[0110] In this embodiment, the feature fusion between the encoder and decoder of the reconstruction model is performed only through a simple splicing operation, resulting in the reconstruction model being insufficient to fully fuse multi-scale features, especially for complex particle field segmentation tasks under high particle density conditions.

[0111] Therefore, this application replaces the simple convolution operation in the transition layer of the reconstruction model with the ASPP module, and effectively captures feature information of different scales through the multi-scale hole convolution of the ASPP module, thereby enhancing the expression ability of the context.

[0112] like Figure 3 As shown in the figure, the ASPP module contains a 1×1 convolution layer, three 3×3 dilated convolution layers with different dilation rates, a 1×1 convolution fusion layer, and an average pooling layer. The reason for selecting convolution kernels with different dilation rates is that convolution kernels with different dilation rates have different receptive field sizes.

[0113] After the final attention feature is input into the ASPP module, it passes through a 1×1 convolution layer to maintain the original scale characteristics of the final attention feature, and then passes through three 3×3 dilation layers with different expansion rates in sequence to extract feature information of different scales of the final attention feature. When passing through the dilation convolution, it can not only expand the receptive field, but also maintain the resolution of feature maps of different scales of the final attention feature, which helps to improve the ability of the reconstruction model for fine segmentation.

[0114] The feature information of different scales is then spliced ​​and passed through a 1×1 convolution fusion layer. Finally, it is normalized and ReLU activated to obtain a low-resolution but high-channel feature map, which contains high-level semantic information extracted after multiple convolutions and downsampling, and is used in the decoder to gradually restore the spatial resolution of the original image.

[0115] In one embodiment of the present application, the S330 includes:

[0116] S331, calculate the MSE loss function using Formula 4.

[0117]

[0118] Among them, L MSE is the MSE loss function, m b is the number of samples in each batch, n voxel is the sum of the voxels of the three-dimensional particle field, y ij is the j-th voxel intensity of the 3D real particle distribution of the i-th sample, is the j-th voxel intensity of the 3D reconstructed particle distribution of the ith sample.

[0119] Specifically, the mean square error (MSE) is a measure of the difference between the estimator and the estimated quantity.

[0120] S332, introduce the quality factor through formula 5.

[0121]

[0122] Among them, Q pr is the quality factor, m b is the number of samples in each batch, n voxel is the sum of the voxels of the three-dimensional particle field, y ij is the j-th voxel intensity of the 3D real particle distribution of the i-th sample, is the j-th voxel intensity of the 3D reconstructed particle distribution of the ith sample.

[0123] S333, combining the MSE loss function and the quality factor through formula 6 to obtain a new training loss function.

[0124] L=L MSE +α(1-Q pr ) Formula 6.

[0125] Among them, L is the new training loss function, L MSE is the MSE loss function, Q pr is the quality factor, and α is a hyperparameter that weighs the weight between the MSE loss function and the quality factor.

[0126] Specifically, in this embodiment, α=0.75.

[0127] In this embodiment, usually, when the reconstruction model performs a fitting task or an image generation task, the mean square error MSE is used as the loss function. In this application, it is the mean of the sum of squares of the pixel-by-pixel errors between the predicted output value of the reconstruction model and the true label value.

[0128] Due to the particularity of the particle reconstruction task, this application introduces a standard measurement indicator for the quality of three-dimensional reconstructed particle distribution that is unique to the field of experimental fluid mechanics, the quality factor, and combines the quality factor with the MSE loss function to obtain a new training loss function.

[0129] In order to reflect the impact of introducing quality factors and selecting hyperparameter values ​​on the training performance of the reconstruction model, 5 values ​​of α are selected in the interval [0, 1] at intervals of 0.25, and the reconstruction model is trained based on each value of α. After comparing different values ​​of α for verification, this application selects α=0.75 as the hyperparameter value of the new training loss function.

[0130] In one embodiment of the present application, the S340 includes:

[0131] S341, training the reconstruction model using Formula 7.

[0132]

[0133] Among them, m t is the first moment estimate of the gradient, v tis the second-order moment estimate of the gradient, β1 is m t The exponential decay rate of v t The exponential decay rate of is m t The bias-corrected estimate of Yes t is the bias-corrected estimate of , λ is the learning rate, is a constant.

[0134] In this embodiment, a gradient descent algorithm is used to improve the training speed of the reconstruction model.

[0135] like Figure 2 As shown, in one embodiment of the present application, a three-dimensional particle reconstruction system based on a convolutional neural network is provided, and the three-dimensional particle reconstruction system based on a convolutional neural network includes a simulation distribution component 100 and a processing component 200.

[0136] The particle attributes are set by the simulation distribution component 100, and the particle distribution is simulated based on the particle attributes. The simulation distribution component 100 is in communication with the processing component 200, and the processing component 200 includes a reconstruction model 201, and the three-dimensional particle reconstruction technology based on the convolutional neural network as described in the above embodiment is executed by the processing component 200.

[0137] In this embodiment, the simulation distribution component 100 is used to simulate the three-dimensional reconstructed particle distribution and generate a training set of the reconstruction model 201, and then the training set is input into the processing component 200. The processing component 200 executes the three-dimensional particle reconstruction technology based on the convolutional neural network as described in any of the aforementioned embodiments on the reconstruction model 201, thereby completing the training of the reconstruction model 201.

[0138] The reconstruction results have different performances under different particle densities. The reconstruction accuracy tends to decrease with the increase of particle density. Therefore, we generated six sets of data with particle concentrations of 0.05-0.3 and intervals of 0.05 to verify the performance of the present invention, and used the three-dimensional real particle distribution E g and three-dimensional reconstruction of particle distribution E SF-MART for comparison. Figure 5 The comparison results are shown for particle density ppp = 0.2, where Figure 5 (a) is the three-dimensional real particle distribution E g , Figure 5 (b) is the particle distribution E after reconstruction by SF-MART algorithm SF-MART , Figure 5 (c) is the reconstruction result of the present invention. It can be seen that the reconstruction result of the present invention is very close to the real three-dimensional particle distribution, which not only restores the elongated particles but also eliminates most of the false particles.

[0139] In order to quantitatively illustrate the improvement effect of the present invention on particle reconstruction accuracy, we use Unet3D, Unet and SF-MART for comparison. Since the Unet network can only process two-dimensional data, we first slice the three-dimensional particle field into two-dimensional data and then send it to the network during the training and testing process of Unet. In addition, the number of iterations of SF-MART is set to 10 times. Figure 6 The particle reconstruction accuracy curves of several methods at different particle densities are shown. From the figure, it can be seen that the reconstruction accuracy of the neural network designed by the present invention can reach about 99% when the particle density is 0.05-0.1. Although the reconstruction accuracy will decrease with the increase of particle density, it can still be maintained above 80%. When the particle density is 0.3, the reconstruction accuracy of SF-MART is only about 63%, while the reconstruction accuracy of the present invention reaches about 84%, an increase of about 28%.

[0140] Under real experimental conditions, various noises are often generated. In the experiment, we use Gaussian noise to uniformly describe these noises and add them to the two-dimensional projection of the virtual camera. Under the same particle density, Gaussian noise of different intensities has different effects on the reconstruction accuracy, and the particle reconstruction accuracy tends to decrease with the increase of noise intensity. Therefore, we limit the particle density and observe the performance of the present invention in particle reconstruction accuracy by continuously increasing the noise intensity. In the experiment, the noise intensity n takes a value of 0 to 0.3, with an interval of 0.05. Figure 7 Comparison results for particle density ppp = 0.15 and noise intensity n = 30% are shown.

[0141] in, Figure 7 (a) is the three-dimensional real particle distribution E g , Figure 7 (b) is the particle distribution E after reconstruction by SF-MART algorithm SF―MART , Figure 7 (c) is the reconstruction result of the present invention. It can be seen that in the case of medium particle density but high noise intensity, the reconstruction quality of SF-MART becomes very poor, the particles cannot even be completely reconstructed and are accompanied by many false particles. However, in the reconstruction result of the present invention, except for a small number of false particles, the particle shape has been basically completely reconstructed.

[0142] In order to illustrate the improvement of the particle reconstruction accuracy of the present invention for different noise intensities under a fixed particle density, we still choose the other three methods mentioned above for comparison. Figure 8The particle reconstruction accuracy curves of several methods under different noise intensities are shown. It can be seen from the figure that the particle reconstruction accuracy decreases with the increase of noise intensity, but the reconstruction accuracy of the present invention can still be maintained above 90%. When the noise intensity is 30%, the reconstruction accuracy of the SF-MART algorithm drops to about 67%, while the reconstruction accuracy of the present invention is about 93%, which is 38% higher than that of SF-MART, reflecting that the model has a strong ability to resist noise.

[0143] The technical features of the above-described embodiments may be arbitrarily combined, and the execution order of the method steps is not limited. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A three-dimensional particle reconstruction technology based on convolutional neural network, characterized in that: The three-dimensional particle reconstruction technology based on convolutional neural network includes: Setting particle attributes, randomly generating a three-dimensional real particle distribution based on the set particle attributes, creating a two-dimensional projection array based on the three-dimensional real particle distribution, and generating a three-dimensional reconstructed particle distribution based on the two-dimensional projection array through a spatial filtering multiplication algebraic reconstruction technique; Create a refactoring model; The three-dimensional reconstructed particle distribution is divided into a training set and a test set in a random ratio, the training set is input into the reconstruction model, and the gradient descent algorithm is used to train the reconstruction model to obtain the trained reconstruction model; Input the test set into the trained reconstruction model to determine whether the trained reconstruction model is qualified; If the reconstructed model after training is qualified, the training is completed; The three-dimensional reconstructed particle distribution is divided into a training set and a test set in a random ratio, the training set is input into a reconstruction model, and a gradient descent algorithm is used to train the reconstruction model to obtain a trained reconstruction model, including: Introducing a CBAM module into a reconstruction model; the CBAM module includes a channel attention mechanism module and a spatial attention mechanism module; Use the ASPP module to replace the transition layer of the reconstructed model; Introduce the quality factor, combine the quality factor with the MSE loss function, and get the training loss function; The training loss function is optimized through the gradient descent algorithm to obtain the reconstructed model after training.

2. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 1, characterized in that: The step of setting the particle attributes, randomly generating a three-dimensional real particle distribution based on the set particle attributes, and creating a two-dimensional projection array in combination with the three-dimensional real particle distribution includes: Setting particle properties; the particle properties include particle field voxel resolution, particle concentration, and particle diameter; Perform simulation based on the set particle properties and randomly generate three-dimensional real particle distribution; Set the virtual camera resolution and generate a 2D projection array based on the 3D real particle distribution.

3. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 2 is characterized in that: The method of generating a three-dimensional reconstructed particle distribution according to a two-dimensional projection array by using a spatial filtering multiplication algebraic number reconstruction technique includes: Set the total number of iterations K; Acquire a three-dimensional real particle distribution and perform a two-dimensional projection on the three-dimensional real particle distribution to obtain a projected particle distribution; The projected particle distribution is reconstructed by formula 1 and continuously updated iteratively until a three-dimensional reconstructed particle distribution closest to the three-dimensional real particle distribution is obtained; Among them, E k (X j ,Y j ,Z j ) is the particle distribution of the kth iteration, X j is the abscissa of the particle distribution at the kth iteration, Y j is the ordinate of the particle distribution at the kth iteration, Z j is the vertical coordinate of the particle distribution of the kth iteration, k is the number of iterations, μ is the relaxation scalar parameter of the iteration, 0≤μ≤1, I(x i ,y i ) is the pixel (x i ,y i ) The current measured pixel intensity distribution, x i is the horizontal coordinate of the pixel point on the two-dimensional projection, y i is the vertical coordinate of the pixel point on the two-dimensional projection, is the projection of the current spatial reconstruction target; Let the number of iterations k = k + 1, and determine whether the number of iterations k is equal to the total number of iterations K; If the number of iterations k is equal to the total number of iterations K, then the iteration is exited, and the particle distribution of the last K-th iteration is used as the three-dimensional reconstructed particle distribution; If the number of iterations k is less than the total number of iterations K, then return to reconstruct the projected particle distribution using Formula 1 until the number of iterations k is equal to the total number of iterations K.

4. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 3 is characterized in that: The introduction of the CBAM module into the reconstruction model comprises: The feature map is input into the channel attention module for pooling to obtain the channel attention weight matrix, and the channel attention weight matrix and the feature map are combined to obtain a new feature map; The new feature map is input into the spatial attention mechanism module to obtain the spatial attention weight matrix, and the final attention feature is obtained by combining the spatial attention weight matrix and the new feature map.

5. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 4 is characterized in that: The feature map is input into the channel attention module for pooling to obtain a channel attention weight matrix, and the channel attention weight matrix and the feature map are combined to obtain a new feature map, including: Input the feature map into the channel attention mechanism module to perform global maximum pooling and global average pooling on the feature map to obtain a global maximum pooling feature map and a global average pooling feature map; The global maximum pooling feature map and the global average pooling feature map are respectively input into the multi-layer perceptron for learning, and the global maximum pooling feature and the global average pooling feature are respectively obtained; Add the global maximum pooling feature and the global average pooling feature element by element to obtain the sum feature map; The sum feature map is mapped using formula 2 to obtain the channel attention weight matrix; Among them, Mc(F) is the channel attention weight matrix, σ is the Sigmoid activation function, is the global maximum pooling feature, is the global average pooling feature, MLP represents multi-layer perceptron, W0 is the first weight matrix in MLP, W1 is the second weight matrix in MLP, and F is the input feature map; Multiply the feature map by the channel attention weight matrix element by element to obtain a new feature map.

6. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 5, characterized in that: The new feature map is input into the spatial attention mechanism module to obtain the spatial attention weight matrix, and the final attention feature is obtained by combining the spatial attention weight matrix and the new feature map, including: Input the new feature map into the spatial attention mechanism module to perform spatial pooling on the new feature map to obtain spatial pooling features; The spatial pooling features are spliced ​​by channel to obtain a spliced ​​image; Perform a convolution operation on the spliced ​​image to obtain a convolution image; The convolution image is activated by formula 3 to obtain the spatial attention weight matrix; Ms(F')=σ(f 7×7 ([Avgpool(F'); Maxpool(F')])) Formula 3; Among them, Ms(F') is the spatial attention weight matrix, σ is the Sigmoid activation function, and f 7×7 is the convolution operation, and F' is the new feature map; Multiply the spatial attention weight matrix by the new feature map element-wise to obtain the final attention feature.

7. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 6, characterized in that: The method of replacing the transition layer of the reconstruction model with the ASPP module includes: Input the features extracted by the encoder into the ASPP module; The feature information of different scales is extracted through the ASPP module to obtain feature maps containing information of different scales; The feature maps containing information of different scales are concatenated and fused through convolution to obtain a fused feature map; Normalize and ReLU the fused feature map.

8. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 7, characterized in that: The introduction of the quality factor, combining the quality factor with the MSE loss function, obtains the training loss function, including: The MSE loss function is calculated using formula 4; Among them, L MSE is the MSE loss function, m b is the number of samples in each batch, n voxel is the sum of the voxels of the three-dimensional particle field, y ij is the j-th voxel intensity of the 3D real particle distribution of the i-th sample, is the j-th voxel intensity of the 3D reconstructed particle distribution of the i-th sample; The quality factor is introduced by formula 5; Among them, Q pr is the quality factor, m b is the number of samples in each batch, n voxel is the sum of the voxels of the three-dimensional particle field, y ij is the j-th voxel intensity of the 3D real particle distribution of the i-th sample, is the j-th voxel intensity of the 3D reconstructed particle distribution of the i-th sample; The new training loss function is obtained by combining the MSE loss function and the quality factor through formula 6; L=L MSE +α(1-Q pr ) Formula 6; Among them, L is the new training loss function, L MSE is the MSE loss function, Q pr is the quality factor, and α is a hyperparameter that weighs the weight between the MSE loss function and the quality factor.

9. The three-dimensional particle reconstruction technology based on convolutional neural network according to claim 8, characterized in that: The training loss function is optimized by the gradient descent algorithm to obtain a reconstructed model after training, including: The reconstruction model is trained using Formula 7; Among them, m t is the first moment estimate of the gradient, v t is the second-order moment estimate of the gradient, β1 is m t The exponential decay rate of v t The exponential decay rate of is m t The bias-corrected estimate of Yes t is the bias-corrected estimate of , λ is the learning rate, is a constant.

10. A three-dimensional particle reconstruction system based on convolutional neural network, characterized in that: The three-dimensional particle reconstruction system based on convolutional neural network includes: A simulation distribution component, through which particle properties are set and particle distribution is simulated based on the particle properties; A processing component, wherein the simulation distribution component is communicatively connected to the processing component, the processing component comprises a reconstruction model, and the three-dimensional particle reconstruction technology based on a convolutional neural network as described in any one of claims 1 to 9 is executed by the processing component.

Images