Method for three-dimensional reconstruction and random generation of sea sand particles based on deep learning diffusion model

Abstract

The application discloses a kind of sea sand particle three-dimensional reconstruction and random generation method based on deep learning diffusion model, including screening sea sand particle, random dispersion type sample preparation is carried out and micro-CT is scanned in layers;Utilize full convolutional neural network to carry out pixel-level classification to sea sand particle slice image, and obtain the slice data set of each sea sand particle different layer by semantic level image segmentation;Import deep learning diffusion model and train, generate new sea sand particle slice image;Then randomly stack new sea sand particle slice image, obtain each sea sand particle surface topography information discrete point cloud coordinates, utilize N spherical harmonics is three-dimensionally reconstructed, and a three-dimensionally reconstructed sea sand particle randomly generated is obtained, which is imported into discrete element software to analyze the stress contact state of real sea sand particles.The application adopts deep learning diffusion model to randomly generate and finely model sea sand particles, successfully solves the problem of randomly generating sea sand particles, and has certain accuracy and advancement.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of three-dimensional reconstruction and random generation of sea sand particles, and particularly relates to a three-dimensional reconstruction and random generation method of sea sand particles based on a deep learning diffusion model. BACKGROUND

[0002] At present, there are a large number of sea sands in nature, and more macroscopic and microscopic studies are carried out on common sands such as river sands in conventional research, but less studies are carried out on sands such as sea sands, and especially the microscopic morphology of sea sand particles is rarely involved. The true three-dimensional morphology reconstruction and random generation of sea sand particles is a relatively new research field.

[0003] The sea sand foundation is soft, and vehicles and other traffic vehicles are difficult to normally travel on the sea sand foundation. In order to explore the mechanical properties of sea sand, in addition to macroscopic tests, discrete element simulation methods are used in the research. The conventional method is to approximate sea sand particles as spheres, and the contact between particles is regarded as the contact between spheres. Although this method simulates the true foundation stress condition to some extent, it ignores the irregularity of the three-dimensional morphology of the true sea sand particles, which may cause errors in the discrete element simulation results to some extent. Therefore, it is necessary to reconstruct the three-dimensional morphology of sea sand particles, and to import the true sea sand particles into the discrete element software analysis, so as to make it more consistent with the true stress contact state of sea sand.

[0004] However, a large number of sea sand particles need to be imported for discrete element analysis, and in the case of limited sea sand particle samples, new methods need to be developed to achieve the purpose of generating a large number of natural sea sand particles, so that the discrete element simulation effect is more realistic. SUMMARY

[0005] The technical problem solved by the present application is that the three-dimensional reconstruction and random generation method of sea sand particles based on the deep learning diffusion model can effectively solve the limitation of the current true generation of a large number of sea sand particles.

[0006] The technical scheme is that the three-dimensional reconstruction and random generation method of sea sand particles based on the deep learning diffusion model comprises the following steps:

[0007] S1, sea sand particles meeting the requirements are selected, and a random dispersion type sample preparation method is used for micro-CT layer scanning to obtain sea sand particle slice images;

[0008] S2, a full convolutional neural network is used to classify the sea sand particle slice images at the pixel level, and slice data sets of different layers of each sea sand particle are obtained through semantic level image segmentation;

[0009] S3, import the slice data set of each sea sand particle in different layers into the deep learning diffusion model for training, use the "forward diffusion process" to gradually add Gaussian noise to each slice image of the sea sand particle in different layers until it becomes random noise; then use the "reverse generation process" to denoise each slice image of the sea sand particle in different layers to generate a new sea sand particle slice image;

[0010] S4, randomly stack the generated new sea sand particle slice image to obtain the discrete point cloud coordinates of the surface topography information of each sea sand particle, use N-order spherical harmonics for three-dimensional reconstruction to obtain a randomly generated three-dimensional reconstructed sea sand particle.

[0011] Preferably, the specific process of step S1 is to screen sea sand particles with impurity content less than 10%, stack the sea sand particles into several layers, and do not contact each other between any two particles, and then use resin that does not interfere with the CT scanning results to bond the particles, and then perform micro-CT layered scanning.

[0012] Preferably, the specific process of adding Gaussian noise in step S3 is: for the slice image original data X0~q(X0) of the sea sand particle in different layers, a total of T-step diffusion process, each step is to generate the sea sand particle slice image data set and the true data set data X t-1 Add Gaussian noise, specifically increase the noise as follows:

[0013]

[0014] In the formula, q represents the data distribution, q(X0) represents the original data distribution, q(X t |X t-1 ) represents the data distribution after adding Gaussian noise at t-1 step, N represents the standard Gaussian distribution, I represents the covariance matrix, β t represents a parameter given in advance to control the progress of adding noise, X t represents the data sample of the original data X0 after adding Gaussian noise at t step, X t-1 represents the data sample of the original data X0 after adding Gaussian noise at t-1 step, X T represents the final data sample of the original data X0 after adding Gaussian noise at T step.

[0015] Now adopt represents the variance taken at each step in the process of adding Gaussian noise to the original data of the sea sand particle slice image data set, between 0-1, when T is large enough, the final X T completely loses the original data X0~q(X0) of the generated sea sand particle slice image data set and becomes a random noise, and the entire diffusion process satisfies the Markov chain:

[0016]

[0017] In the formula, X 1:T Represents X1, X2, X3, ... X T ;

[0018] The diffusion process is directly based on the generated sea sand particle slice image dataset and the original data X0~q(X0) of the real dataset for any t step X t Sample X t ~q(X) t |X0), through the reparameter re-parameter technique With anti-repetition parameter techniques Guarantee the final X T For random noise, where α i =1-β t , With α i Both represent given parameters, and ε represents a given sampling result that is an independent random variable.

[0019] Preferably, the specific process of noise removal in step S3 is as follows: from a random noise X T Starting from ~N(0,Ι), gradually adjust the true distribution q(X) at each step. t-1 |X t Denoise the sea sand particle slice image and generate a new sea sand particle slice image that satisfies the Gaussian distribution parameterized by the neural network:

[0020]

[0021] p θ (X t-1 |X t )=N(X t-1 μ θ (X t ,t),∑ θ (X t ,t))

[0022] In the formula, p θ Let θ represent a Gaussian distribution, and μ be the training parameter. θ (X t ,t),∑ θ (X t ,t) represents the mean and variance, p θ (X t-1 |X t p(X) represents a parameterized Gaussian distribution; T ) represents a Gaussian distribution N(X) T ;0,Ι), which are all composed of mean and variance μ θ (Xt t),∑ θ (X t t) give.

[0023] Preferably, the deep learning diffusion model in step S3 adopts the variational lower bound as the loss function:

[0024]

[0025] In the formula, E is the total expectation, E q is the data sample expectation.

[0026] Beneficial effects: 1) The present application uses the diffusion model (DDPM) in deep learning to randomly generate and finely model sea sand particles, successfully solves the problem of random generation of sea sand particles, and has certain accuracy and advancement;

[0027] 2) The present application uses artificial intelligence deep learning algorithm to randomly generate sea sand particles, achieves real simulation, and achieves the purpose of generating a large number of natural sea sand particles in nature;

[0028] 3) The present application can randomly generate a large number of sea sand particles conforming to the three-dimensional morphology of real sea sand, successfully solves the error of using spherical analysis in discrete element analysis and the insufficient situation of sea sand particle samples imported into discrete element analysis software, and has great reference significance. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of the method of the present application;

[0030] Figure 2 is a schematic diagram of the deep learning diffusion model (DDPM) of the present application;

[0031] Figure 3 is a schematic diagram of the sea sand particle slice "forward diffusion process" and "reverse generation process" of the deep learning diffusion model (DDPM) of the present application;

[0032] Figure 4 is a CT slice image of real sea sand particles;

[0033] Figure 5 is a CT slice image of sea sand particles generated by the method of the present application;

[0034] Figure 6 is a three-dimensional point cloud diagram of the CT slice stacking of sea sand particles generated by the method of the present application;

[0035] Figure 7 is a sea sand particle diagram three-dimensionally reconstructed and randomly generated by the method of the present application. DETAILED DESCRIPTION

[0036] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments:

[0037] Example 1

[0038] like Figure 1 As shown, the method for 3D reconstruction and random generation of sea sand particles based on a deep learning diffusion model includes the following steps:

[0039] S1. Select sea sand particles with an impurity content of less than 10%. Use a random dispersion sampling method (separate sea sand particles into several layers, stacking them so that no two particles touch each other, and bonding the particles with resin that does not interfere with the CT scan results) for micro-CT layer-by-layer scanning to obtain slice images of the sea sand particles, such as... Figure 4 As shown.

[0040] S2. A fully convolutional neural network (FCN) is used to perform pixel-level classification of sea sand particle slice images. Semantic-level image segmentation is then used to obtain slice datasets for different layers of sea sand particles. Specifically, the FCN can accept input images of sea sand particle slices of any size. A deconvolutional layer upsamples the last convolutional layer to restore it to the same size as the input image, thus generating a prediction for each pixel while preserving the spatial information of the original input image. Finally, pixel-by-pixel classification is performed on the upsampled feature map, achieving semantic-level image segmentation of the sea sand particle slice images and obtaining slice datasets for different layers of sea sand particles. For slice images of different layers of sea sand particles, the FCN better preserves the spatial information of the original input image. Finally, pixel-by-pixel classification is performed on the upsampled feature map, achieving semantic-level image segmentation of the sea sand particle slice images and ensuring the stability of the sea sand particle slice image dataset input to the deep learning diffusion model (DDPM).

[0041] S3. Import the sliced ​​datasets of different layers of sea sand particles into a deep learning diffusion model (the principle diagram is shown below). Figure 2 Training is performed as shown. Figure 3 As shown, a "forward diffusion process" is used to gradually add Gaussian noise to the slice images of different layers of sea sand particles until they become random noise. For the original data X0 to q(X0) of the slice images of different layers of sea sand particles, there are a total of T diffusion steps. Each step involves comparing the sea sand particle slice image dataset generated in the previous step with the real dataset X. t-1 Add Gaussian noise, specifically as follows:

[0042]

[0043] In the formula, q represents the data distribution, q(X0) represents the original data distribution, and q(X) represents the original data distribution. t |X t-1The expression represents the data distribution after adding Gaussian noise at step t and step t-1, where N represents the standard Gaussian distribution, I represents the covariance matrix, and β... t X represents the parameter X that controls the rate at which noise is added, given in advance. t This represents the data sample after Gaussian noise is added to the original data X0 at step t. t-1 This represents the data sample of the original data X0 after Gaussian noise has been added at step t-1. T This represents the final data sample of the original data X0 after adding Gaussian noise in step T;

[0044] Currently adopted This represents the variance taken at each step in the process of adding Gaussian noise to the original data of the sea sand particle slice image dataset, ranging from 0 to 1. When T is sufficiently large, the final X is obtained. T The generated sea sand particle slice image dataset completely loses its original data X0~q(X0) from the real dataset and becomes random noise. The entire diffusion process satisfies a Markov chain:

[0045]

[0046] In the formula, X 1:T Represents X1, X2, X3, ... X T ;

[0047] The diffusion process is directly based on the generated sea sand particle slice image dataset and the original data X0~q(X0) of the real dataset for any t step X t Sample X t ~q(X) t |X0), through the reparameter re-parameter technique With anti-repetition parameter techniques Guarantee the final X T For random noise, where α i =1-β t , With α i Both represent given parameters, and ε represents independent random variables of the given sampling results;

[0048] Then, the "reverse generation process" is used to denoise the sea sand particle slice image, gradually removing noise from a random noise source until a sea sand particle slice image is generated. The "reverse generation process" applies the true distribution q(X) at each step. t-1 |X t ), from a random noise X T Starting from ~N(0,Ι), noise is gradually removed from the sea sand particle slice image to generate a sea sand particle slice image as shown below. Figure 5 As shown, the distribution satisfies the Gaussian distribution parameterization of the neural network:

[0049]

[0050] p θ (X t-1 |X t )=N(X t-1 ;μ θ (X t ,t),∑ θ (X t ,t))

[0051] where p θ represents a Gaussian distribution, and θ is a training parameter, and μ θ (X t ,t),∑ θ (X t ,t) represents the mean and variance, and p θ (X t-1 |X t ) represents a parameterized Gaussian distribution; and p(X T ) represents a Gaussian distribution N(X T ;0,Ι), which is given by the mean and variance μ θ (X t ,t),∑ θ (X t ,t);

[0052] The deep learning diffusion model (DDPM) uses the variational lower bound as a loss function:

[0053]

[0054] where E is the total expectation, and E q is the data sample expectation;

[0055] The deep learning diffusion model (DDPM) uses the “forward diffusion process” and “reverse generation process” to generate slice images of different layers of real sea sand particles, especially for sea sand particles.

[0056] S4, randomly stack the generated new sea sand particle slice images, as shown in Figure 6 , to obtain the discrete point cloud coordinates of the surface topography information of each sea sand particle, and use N-order spherical harmonics for three-dimensional reconstruction to obtain randomly generated three-dimensional reconstructed sea sand particles, as shown in Figure 7 ; import the randomly generated three-dimensional reconstructed sea sand particles into the discrete element software to analyze the stress contact state of real sea sand particles, wherein X = (x, y, z) can be represented as:

[0057]

[0058]

[0059]

[0060]

[0061] wherein x, y, z are the coordinate values of the one-to-one mapping of the three-dimensional closed surface to the unit sphere in the Cartesian coordinate system, θ is the polar angle of each discrete point on the surface of the particle, is the azimuth angle of each discrete point on the surface of the particle, and r is the polar radius of each discrete point on the surface of the particle.

[0062] The N-order spherical harmonic function of the particle surface point (x, y, z) is three-dimensionally reconstructed, and with the increase of the total order N of the spherical harmonic function, the reconstructed particle morphology is more and more similar to the real particle.

[0063]

[0064]

[0065]

[0066] wherein, is the x-coordinate, y-coordinate, and z-coordinate corresponding spherical harmonic coefficient; is the spherical harmonic function of order n and degree m, is the associated Legendre polynomial, wherein,

[0067]

[0068]

[0069] The coefficient amplitude of order n is:

[0070]

[0071]

[0072]

[0073] wherein, a xn is the coefficient amplitude under the order of the x-coordinate corresponding spherical harmonic coefficient, a yn is the coefficient amplitude under the order of the y-coordinate corresponding spherical harmonic coefficient, a zn is the coefficient amplitude under the order of the z-coordinate corresponding spherical harmonic coefficient.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for 3D reconstruction and random generation of sea sand particles based on a deep learning diffusion model, characterized in that, Includes the following steps: S1. Select sea sand particles that meet the requirements and use a random dispersion sample preparation method to perform micro-CT layered scanning to obtain sea sand particle slice images. The specific process is to select sea sand particles with impurity content of less than 10%, divide the sea sand particles into several layers and stack them, with no two particles in contact with each other, and bond the particles with resin that does not interfere with the CT scan results, and then perform micro-CT layered scanning. S2. Pixel-level classification of sea sand particle slice images is performed using a fully convolutional neural network, and slice datasets of different layers of sea sand particles are obtained through semantic-level image segmentation. S3. Import the slice datasets of different layers of sea sand particles into the deep learning diffusion model for training. Use the "forward diffusion process" to gradually add Gaussian noise to the slice images of different layers of sea sand particles until they become random noise. Then use the "reverse generation process" to remove noise from the slice images of different layers of sea sand particles and generate new slice images of sea sand particles. S4. Randomly stack the generated new sea sand particle slice images to obtain the discrete point cloud coordinates of the surface morphology information of each sea sand particle. Use the Nth order spherical harmonic function to perform three-dimensional reconstruction to obtain randomly generated three-dimensional reconstructed sea sand particles.

2. The method for three-dimensional reconstruction and random generation of sea sand particles based on a deep learning diffusion model according to claim 1, characterized in that, The specific process of adding Gaussian noise in step S3 is as follows: For the original data X0~q(X0) of the slice images of different layers of sea sand particles, there are a total of T diffusion steps. Each step involves processing the sea sand particle slice image dataset generated in the previous step and the real dataset X. t-1 Add Gaussian noise, specifically as follows: In the formula, q represents the data distribution, q(X0) represents the original data distribution, and q(X) represents the original data distribution. t |X t-1 The expression represents the data distribution after adding Gaussian noise at step t and step t-1, where N represents the standard Gaussian distribution, I represents the covariance matrix, and β... t X represents the parameter X that controls the rate at which noise is added, given in advance. t This represents the data sample after Gaussian noise is added to the original data X0 at step t. t-1 This represents the data sample of the original data X0 after Gaussian noise has been added at step t-1. T This represents the final data sample of the original data X0 after adding Gaussian noise in step T; Currently adopted This represents the variance taken at each step in the process of adding Gaussian noise to the original data of the sea sand particle slice image dataset, ranging from 0 to 1. When T is sufficiently large, the final X is obtained. T The generated sea sand particle slice image dataset and the original data X0~q(X0) of the real dataset are completely lost, becoming random noise. The entire diffusion process satisfies a Markov chain: In the formula, X 1:T Represents X1, X2, X3, ... X T ; The diffusion process is directly based on the generated sea sand particle slice image dataset and the original data X0~q(X0) of the real dataset for any t step X t Sample X t ~q(X) t |X0), through the reparameter re-parameter technique With anti-repetition parameter techniques Guarantee the final X T For random noise, where α i =1-β t , With α i Both represent given parameters, and ε represents a given sampling result that is an independent random variable.

3. The method for three-dimensional reconstruction and random generation of sea sand particles based on a deep learning diffusion model according to claim 2, characterized in that, The specific process of noise removal in step S3 is as follows: From a random noise X T Starting from ~N(0,Ι), gradually adjust the true distribution q(X) at each step. t-1 |X t Denoise the sea sand particle slice image and generate a new sea sand particle slice image that satisfies the Gaussian distribution parameterized by the neural network: p θ (X t-1 |X t )=Ν(X t-1 ;μ θ (X t ,t),∑ θ (X t ,t)) In the formula, p θ Let θ represent a Gaussian distribution, and μ be the training parameter. θ (X t ,t),∑ θ (X t ,t) represents the mean and variance, p θ (X t-1 |X t ) represents a parameterized Gaussian distribution, which consists of a mean and a variance μ. θ (X t ,t),∑ θ (X t ,t) is given; p(X) T ) represents a Gaussian distribution N(X) T ;0,Ι).

4. The method for three-dimensional reconstruction and random generation of sea sand particles based on a deep learning diffusion model according to claim 3, characterized in that: After step S3, the deep learning diffusion model uses a variational lower bound as the loss function: In the formula, E is the total expectation, E q This represents the expected value of the data sample.

Images