Overlapped particle segmentation and grading prediction technology based on generative adversarial network

Abstract

The invention belongs to the field of computational mechanics, and particularly relates to an overlapping particle segmentation and grading prediction technology based on a generative adversarial network. Generating a particle set model by using discrete element simulation software; constructing a training data set by storing particle set images in batches; capturing a mapping relation between the overlapped particle image and the scattered state image through Pix2Pix, and generating a corresponding scattered particle image based on the mapping relation; a deep learning segmentation model SAM is adopted to carry out particle segmentation on the image, and the contour boundary of particles is accurately extracted; by means of the OpenCV technology, geometrical characteristic parameters of the particles are extracted from the segmentation result, and a grading curve of the particles is drawn; and finally, comparing and analyzing the predicted grading curve and a known theoretical grading curve, evaluating the prediction precision and calculating an error. According to the method, particle segmentation and grading prediction can be efficiently and accurately realized, and particularly, excellent performance is shown when overlapped particles are treated.

Description

Technical Field

[0003] The present invention belongs to the field of computational mechanics, and in particular relates to an overlapping particle segmentation and grading prediction technology based on a generative adversarial network. Background Art

[0004] Granular materials are widely used in construction engineering, geological exploration, material manufacturing and other fields. Their mechanical properties and physical characteristics depend largely on the distribution, shape and arrangement of particles. Particle grading is an important indicator to describe the characteristics of particle size distribution, which directly affects the key properties of materials such as strength, stability, permeability and density. By studying the grading characteristics of particles, theoretical support can be provided for engineering design, material ratio optimization and the application of granular materials in complex mechanical behaviors. Therefore, the accurate measurement and quantitative analysis of particle grading not only has important scientific research value, but also has a far-reaching impact on the application of materials in actual engineering.

[0005] Traditional particle grading analysis methods mainly rely on experimental screening methods, optical microscopy, and X-ray tomography. However, these methods have certain limitations: experimental screening methods are usually inefficient and cannot cope with the complex characteristics of microscopic particles; image processing-based algorithms are often difficult to accurately segment and measure when dealing with irregular or overlapping particles. Specifically, when the number of particles is large, traditional grading analysis methods are laborious, inefficient, and perform poorly in the detection of tiny particles; and when particles overlap, conventional image processing algorithms find it difficult to accurately identify the independent contours of each particle, which affects the particle size measurement and grading accuracy.

[0006] In recent years, deep learning technology has made significant progress in the field of image segmentation, and has made even more progress in the application of particle segmentation and grading prediction. Although deep learning models can handle complex particle sets, their training process is highly dependent on data. When dealing with particle overlap, existing methods cannot provide effective solutions. Domestic and foreign researchers have proposed a variety of improved methods, such as three-dimensional particle modeling technology based on multi-view image reconstruction and automatic particle segmentation algorithms combined with machine learning. However, most of these studies focus on particle analysis in a single scene, lacking a systematic discussion on overlapping particle segmentation and grading prediction. In particular, when faced with highly complex particle sets, existing methods still have significant deficiencies in accuracy and efficiency. Therefore, it is urgent to develop a new method to further improve the automation level and accuracy of overlapping particle segmentation and grading prediction. Summary of the invention

[0007] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a technology for overlapping particle segmentation and gradation prediction based on a generative adversarial network. This method uses a generative adversarial network (GAN) model, especially a pix2pix architecture, and uses the mutual game between the generator and the discriminator to learn the feature mapping relationship between the overlapping particle image and its scattered image; finally, the extracted particle information is used to accurately predict the gradation characteristics of the particles, thereby providing reliable support for the optimization ratio and mechanical property analysis of granular materials in engineering applications. Through this technology, not only can the accuracy of particle segmentation be significantly improved, but also an efficient and accurate method for processing the analysis of complex particle groups is provided.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solution: an overlapping particle segmentation and grading prediction technology based on a generative adversarial network, comprising the following steps:

[0009] S1: According to the characteristics of overlapping granular materials, the gradation, porosity and inter-particle contact type and other parameters of the granular materials are preset in the discrete element simulation software to generate a particle collection sample M in a dispersed equilibrium state. During the generation process of the particle collection sample M, the granular materials gradually transition from a mutually overlapping state to a dispersed equilibrium state, and the finally generated particle collection sample M is in an equilibrium state.

[0010] Preferably, the particles in the particle set sample M generated in step S1 are standard round particles.

[0011] S2: The size and position information of all particles in the equilibrium state sample set M is exported from the discrete element simulation software and saved as a .txt file; then, Matplotlib is used in Visual Studio Code to reconstruct the particle model, calculate the gradation distribution information A1 of the original particles, and save it.

[0012] Preferably, in step S2, the center coordinates and radius information of all particles in the particle set sample M are obtained through the ball.pos.x(bp), ball.pos.y(bp) and ball.radius(bp) functions, and the data is exported as a .txt file.

[0013] S3: Save and output images of the generation process of the particle set sample M from overlapping to scattered in S1 at preset time intervals, wherein the image set of the particle set sample M in the overlapping state is D1, and the image set of the particle set sample M in the scattered state is D2.

[0014] Preferably, the preset time interval in step S3 is 700ms-1500ms, the image type is in .png format, and the generated image set D1 in the overlapping state and the image set D2 in the scattered state are stored separately.

[0015] S4: Filter and crop the images in D1 and D2 to make the images consistent in size, pair the overlapping particle image set D1 with the scattered particle image set D2 one by one to form a paired image set D3, and construct a training data set T1 and a test data set T2 in proportion from D3.

[0016] Preferably, in step S4, the training data set T1 retains n overlapping state images and scattered state images that are paired one by one in the generation process, wherein each generation process retains m overlapping state images and m scattered state images, where n≥20. The test data set T2 retains k overlapping state images and scattered state images that are paired one by one in the generation process, wherein each generation process retains 1 randomly selected overlapping state image and 1 scattered state image, where k≥10; the ratio of the number of images in the training data set T1 and the test data set T2 is 20:1-50:1.

[0017] S5: Use the pix2pix model in the generative adversarial network deep learning model to train the training data set T1 in S4, so that the model learns the image feature relationship of the particle set sample M from the overlapping state to the scattered state, and obtains the learning model H.

[0018] Preferably, the pix2pix adversarial generative network used in step S5 is mainly composed of two parts: a generator G and a discriminator D. The generator G is responsible for converting the input overlapping particle image into a scattered particle image, and generating an output similar to the target image by learning the mapping relationship between the input image and the target image; the discriminator D is used to judge the difference between the image generated by the generator G and the real image, and improve the output quality of the generator G through the adversarial training mechanism.

[0019] The loss function of the pix2pix adversarial generation network is generated by the generation loss L G and the discriminative loss L D The specific definitions are as follows: (a) Adversarial loss L cGAN :Adversarial loss is the core part of conditional generative adversarial network (cGAN). Generator G generates scattered particle images as realistic as possible by minimizing adversarial loss, while discriminator D distinguishes generated images from real images by maximizing adversarial loss. cGAN Calculated by formula (1),

[0020] L cGAN (G,D)=E x [-logD(x,G(x))] (1),

[0021] Where x represents the input overlapping particle image, G(x) represents the scattered particle image generated by the generator based on x, and D(x, G(x)) represents the probability that the discriminator judges that the input x and the generated image G(x) are real images;

[0022] (b) L1 loss: In order to improve the similarity between the generated image and the real image at the pixel level, L1 loss is introduced, which is calculated as follows:

[0023] L L1 (G) = E x,y [yG(x)1] (2),

[0024] Where y represents the real scattered particle image, yG(x)1 represents the absolute value error between the generated image G(x) and the real target image y;

[0025] (c) The total loss function L of the generator G : The total loss function L of the generator G is the adversarial loss L cGAN The weighted sum of the and L1 losses is calculated by formula (3):

[0026] L G =L cGAN (G,D)+λL L1 (G) (3),

[0027] Among them, λ is a hyperparameter used to balance the weights of adversarial loss and L1 loss;

[0028] (d) The loss function L of the discriminator D :

[0029] L D =E x,y [-logD(x,y)]+E x [-log(1-D(x,G(x)))] (4),

[0030] Where x represents the input overlapping particle image, y represents the real scattered particle image, G(x) represents the scattered particle image generated by the generator based on x, D(x,y) represents the probability that the discriminator judges that the input x and the real target y are real images, and D(x,G(x)) represents the probability that the discriminator judges that the input x and the generated image G(x) are real images.

[0031] S6: The weight file D4 is selected based on the generated image effect, and the weight file D4 is retrieved and run in the model test program to process the particle image D5 in the overlapping state in the test data set T2 to generate the particle image D6 in the scattered state.

[0032] S7: Formulate a judgment standard for the scattered state particle image, and judge whether the scattered state particle image D6 generated in S7 meets the judgment standard based on the judgment standard; if it meets the judgment standard, save the scattered state particle image D6; if it does not meet the judgment standard, return to step S5 to retrain the learning model H until the scattered state particle image D6 meets the judgment standard.

[0033] S8: In the Visual Studio Code code editor, the scattered particle image D6 that meets the judgment criteria in S7 is preprocessed by grayscale and binarization using the Python program environment to obtain a binarized particle image D7.

[0034] S9: Use the SegmentAnything Model image segmentation model to perform segmentation training on the binary particle image D7, segment out the boundary contour of each particle, and generate a segmented image D8.

[0035] Preferably, the architecture of the Segment Anything Model in step S9 includes three parts: an image encoder, a prompt encoder, and a mask decoder.

[0036] Among them: Image encoder: uses ViT (Vision Transformer) to convert the input image into a series of feature vectors; Hint encoder: encodes the hints provided by the user (such as points, boxes or text) into hint vectors through a multi-layer perceptron (MLP); Mask decoder: combines the image feature vector with the hint vector, and generates the mask probability of each pixel through another set of MLP to achieve accurate segmentation.

[0037] S10: In the Visual Studio Code code editor, the OpenCV machine learning library is used to extract contour feature information from the segmented image D8, and the extracted particle gradation distribution information A2 is saved.

[0038] Preferably, the method of extracting contour features of particles in the segmented image D5 in step S10 is implemented by the findCountours() function, minEnclosingCircle() function and circle() function in OpenCV. Among them: the indContours() function is used to detect the contours of all particles in the image D5 and return the point set of the contours;

[0039] The minEnclosingCircle() function is used to calculate the minimum circumscribed circle of each contour to help determine the circularity of the particle and its size information; the circle() function is used to draw the minimum circumscribed circle on the original image to intuitively display the boundary of the particle.

[0040] S11: Summarize the two sets of gradation information, namely, the gradation distribution information A1 of the original particles and the gradation distribution information A2 of the extracted particles, scale the two curves to the same coordinate scale system in the Visual Studio Code code editor, and draw a comparison image of the original gradation curve B1 of the particles and the predicted gradation curve B2 of the particles, compare the overlap of the two curves, and analyze the prediction error.

[0041] Preferably, in step S11, the method of scaling the two curves to the same coordinate scale system for drawing in the Visual Studio Code code editor is: linearly scaling the particle size data in the extracted particle grading distribution information A2, that is, the data on the horizontal axis, so that its particle size value and the particle size value of the grading distribution information A1 of the original particle are kept on the same scale, thereby facilitating comparative analysis.

[0042] The specific linear scaling method is: let the original radius of the particles in A2 be r, and the original range of the particle size data is [r min ,r max ], r is mapped to the new range [a, b], then the linear scaling is calculated as formula (5):

[0043]

[0044] Among them, r scaled is the data value after mapping, r max and r min are the minimum and maximum values ​​of the original data, a and b are the minimum and maximum values ​​of the target range, and r is the value of each particle size data in the original data.

[0045] S12: Formulate an error standard, and determine whether the generated particle prediction grading curve B2 meets the error standard requirements based on the error standard; if the error standard requirements are met, obtain the final conclusion; if the error standard requirements are not met, return to step S4 to restart the construction of the model training data set T1 and test data set T2 and retrain the learning model H until the generated particle prediction grading curve B2 meets the error standard requirements.

[0046] The beneficial effects of the present invention are:

[0047] (1) Innovative overlapping particle processing method: The present invention can effectively solve the problem of particle overlap by using the generative adversarial network pix2pix model, and generate particle images that are highly similar to the actual scattered state. Compared with traditional segmentation methods, the present invention can more accurately handle complex particle overlap, improve the accuracy of particle segmentation, and provide higher quality data support for particle grading analysis.

[0048] (2) Fully automated segmentation and grading prediction process: The present invention combines deep learning models and computer vision technologies, such as OpenCV and the Segment Anything Model (SAM), to achieve automatic processing and analysis of particle images. The entire process from data acquisition, image generation, particle segmentation to grading prediction is automated, which greatly improves work efficiency and reduces the need for manual intervention.

[0049] (3) High-precision particle segmentation technology: The SAM model used in the present invention is an advanced segmentation technology based on deep learning. By debugging the program code and modifying some parameters of the model, it can have very high recognition accuracy and can segment complex tasks that are difficult to complete with other similar segmentation models. In particular, when processing subtle features and complex situations in particle images, the SAM model can provide extremely accurate segmentation effects, thereby ensuring high-quality extraction of particle information and improving the accuracy of subsequent analysis and prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is the algorithm flow chart of the present invention;

[0051] Figure 2 It is an image of a particle collection sample M in a state of equilibrium generated by discrete element simulation software;

[0052] Figure 3 It is an image generated by discrete element simulation software, which is a pair of an overlapping particle image set D1 and a scattered particle image set D2;

[0053] Figure 4 The input image of the overlapping state of particles and the image of the scattered state generated by the pix2pix model;

[0054] Figure 5 It is an image of the dispersed particle collection generated by three forms: discrete element simulation software, python data reconstruction, and pix2pix learning model;

[0055] Figure 6 It is the particle set image generated by the pix2pix model and the binary particle set image;

[0056] Figure 7It is the particle collection image after SAM model segmentation;

[0057] Figure 8 It is the particle collection image after OpenCV contour extraction;

[0058] Fig. 9 It is a comparison chart between the original particle grading curve and the final predicted grading curve. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and implementation examples. It should be understood that the specific implementation examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0060] The method of the present invention is applied to the process of predicting the overall gradation distribution curve of the particle set by analyzing the overlapping images when the gradation distribution, porosity and other parameters of the particle set are known. Figure 1 As shown, the specific prediction process is as follows:

[0061] S1: According to the characteristics of overlapping granular materials, the gradation, porosity and inter-particle contact type and other parameters of the granular materials are preset in the discrete element simulation software to generate a particle collection sample M in a dispersed equilibrium state. During the generation process of the particle collection sample M, the granular materials gradually transition from a mutually overlapping state to a dispersed equilibrium state, and the finally generated particle collection sample M is in an equilibrium state.

[0062] First, program in the discrete element simulation software and run the sample preparation program P1 to generate the particle set sample M. Set the size of the particle set sample M to 100 mm × 100 mm, the particle shape to standard round particles, the contact type between particles to the linear elastic contact model, the elastic modulus of the material to 100 MPa, and the stiffness ratio to 1.2 × 10 -5 The minimum particle diameter is set to 5 mm, and the generated sample pattern is as follows Figure 2 shown.

[0063] The gradation of the particle set is represented by a discrete particle size and the corresponding cumulative volume fraction. The particle size distribution ratio is determined by the particle size interval and its corresponding volume fraction table. The porosity of the particle image of the training data set T1 is set to 0.4 and 0.3, the porosity of the particle image of the test data set T2 is set to 0.4, and the density of the particle is set to 2500kg / m 3 The damping coefficient is set to 0.7. The initial number of cycles is 1000, and the speed is adjusted every 10 runs to make the particles gradually tend to be stationary or reach a quasi-static state; the convergence criterion for the model operation is that the unbalanced force ratio is less than 10 -3When the system reaches equilibrium, the iteration stops.

[0064] Among them, the calculation formula of the unbalanced force ratio is formula (6):

[0065]

[0066] S2: The size and position information of all particles in the equilibrium state sample set M is exported from the discrete element simulation software and saved as a .txt file; then, Matplotlib is used in Visual Studio Code to reconstruct the particle model, calculate the gradation distribution information A1 of the original particles, and save it.

[0067] After the S1 sample preparation program P1 is finished, run the program P2 to export particle information, traverse the size and position information of all particles generated by the sample preparation program P1, and obtain the center coordinates and radius information of all particles through the ball.pos.x (bp), ball.pos.y (bp) and ball.radius (bp) functions, and export the center coordinates and radius information of the particle data as a .txt file; use Matplotlib in Vs code to reconstruct the particle model and reconstruct the particle image as shown in Figure 5 As shown, the gradation distribution information A1 of the original particles is calculated and A1 is stored in 1.xlsl.

[0068] S3: Save and output images of the generation process of the particle set sample M from overlapping to scattered in S1 at preset time intervals, wherein the image set of the particle set sample M in the overlapping state is D1, and the image set of the particle set sample M in the scattered state is D2.

[0069] Preferably, the preset time interval in step S3 is 700ms-1500ms, the image type is in .png format, and the generated image set D1 in the overlapping state and the image set D2 in the scattered state are stored separately.

[0070] Because the pix2pix network training requires a large amount of data sets and the manual screenshot task is huge, before running the S1 sample preparation program P1, it is necessary to set the timed automatic image saving under Tools-Options-Movie in the discrete element simulation software, set the saving time interval to 1000ms, the image type to png format, and the image size to 1024×768 pixels. After the settings are completed, running the sample preparation program P1 can generate a series of particle images, and the generated overlapping state image set D1 and scattered state image set D2 are stored separately.

[0071] S4: Filter and crop the images in D1 and D2 in S3 to make the images consistent in size, pair the overlapping particle image set D1 with the scattered particle image set D2 one by one to form a paired image set D3, and construct a training data set T1 and a test data set T2 in proportion from D3.

[0072] The images D1 and D2 stored in S3 are screened and processed. 40 images of particles in overlapping state and 40 images of particles in scattered state are retained in one simulation process, and the redundant images are deleted. The images after screening are cropped and the size of all images is modified to 725×700 pixels for subsequent model training. Figure 3 The image dataset shown.

[0073] In step S4, the training data set T1 in step S4 retains n overlapping state images and scattered state images that are paired one by one in the generation process, wherein each generation process retains m overlapping state images and m scattered state images, where n≥20. The test data set T2 retains k overlapping state images and scattered state images that are paired one by one in the generation process, wherein each generation process retains 1 randomly selected overlapping state image and 1 scattered state image, where k≥10; the ratio of the number of images in the training data set T1 and the test data set T2 is 20:1-50:1.

[0074] In this embodiment, the training data set T1 has 800 pairs of paired images, and each pair of 40 images is a simulated generation process of a particle set, of which 800 are overlapped and 800 are scattered, and the two are paired one by one; that is, T1 retains n=20 images of the generation process, and each generation process retains m=40 images of overlapping state and scattered state. The particle porosity in the first 400 pairs of images is set to 0.4, and the particle porosity in the last 400 pairs of images is set to 0.3. The test data set T2 has 20 pairs of paired images, and each pair of images is randomly selected from the 40 pairs of images of the simulated generation process of the above particle set. The particle porosity in the first 10 pairs of images is set to 0.4, and the particle porosity in the last 10 pairs of images is set to 0.3; that is, T2 retains k=20 images of the generation process, and each generation process extracts 1 image of overlapping state or scattered state. Then there are 1600 images in T1 and 40 images in T2, and the ratio between them is: 1600 / 40=40: 1. Obviously, in practical applications, the specific values ​​of m, n and k of images in T1 and T2 in this step can be modified according to experimental requirements.

[0075] S5: Use the pix2pix model in the generative adversarial network deep learning model to train the training data set T1 in S4, so that the model learns the image feature relationship of the particle set sample M from the overlapping state to the scattered state, and obtains the learning model H.

[0076] Place the images in the training dataset T1 in the specified file path, run the model training program (PIX1), and use the pix2pix model in the generative adversarial network (GAN) series to train the above paired datasets so that the pix2pix model can learn the image feature relationship of particles from overlapping state to scattered state.

[0077] The pix2pix adversarial generative network used in this example mainly consists of two parts: the generator G and the discriminator D. The generator G is responsible for converting the input overlapping particle image into a scattered particle image, and generates an output similar to the target image by learning the mapping relationship between the input image and the target image; the discriminator D is used to judge the difference between the image generated by the generator G and the real image, and improve the output quality of the generator G through the adversarial training mechanism. In this example, the network architecture of the generator G adopts the U-Net model framework, which consists of 6 downsampling convolutional layers and 5 upsampling deconvolutional layers; the discriminator D uses a convolutional neural network (CNN) to extract the features of the input image through a series of convolution operations.

[0078] The loss function of the pix2pix network in step S5 consists of the generation loss and the discrimination loss, which are specifically defined as follows:

[0079] (a) Adversarial loss: Adversarial loss is the core part of the conditional generative adversarial network (cGAN). The generator G generates a scattered particle image that is as realistic as possible by minimizing the adversarial loss, while the discriminator D distinguishes the generated image from the real image by maximizing the adversarial loss.

[0080] L cGAN (G,D)=E x [-logD(x,G(x))] (1),

[0081] Where x represents the input overlapping particle image, G(x) represents the scattered particle image generated by the generator based on x, and D(x, G(x)) represents the probability that the discriminator judges that the input x and the generated image G(x) are real images;

[0082] (b) L1 loss: In order to improve the similarity between the generated image and the real image at the pixel level, L1 loss is introduced, which is calculated as follows:

[0083] L L1 (G) = E x,y [yG(x)1] (2),

[0084] Where y represents the real scattered particle image, yG(x)1 represents the absolute value error between the generated image G(x) and the real target image y;

[0085] (c) The total loss function L of the generator G : The total loss function of the generator is the weighted sum of the adversarial loss and the L1 loss, and the formula is as follows:

[0086] L G =L cGAN (G,D)+λL L1 (G) (3),

[0087] Among them, λ is a hyperparameter used to balance the weight of λ adversarial loss and L1 loss.

[0088] In this model program, in order to balance the quality and diversity of images during training and help the generator better learn the structure and content of images, λ is not set to a fixed value, but a dynamic adjustment strategy is adopted. As the number of epoch training rounds increases, the size of λ is continuously adjusted. The specific program implementation is as follows:

[0089]

[0090] (d) The loss function L of the discriminator D :

[0091] L D =E x,y [-logD(x,y)]+E x [-log(1-D(x,G(x)))] (4),

[0092] Where x represents the input overlapping particle image, y represents the real scattered particle image, G(x) represents the scattered particle image generated by the generator based on x, D(x,y) represents the probability that the discriminator judges that the input x and the real target y are real images, and D(x,G(x)) represents the probability that the discriminator judges that the input x and the generated image G(x) are real images.

[0093] In step S5, in order to optimize the training efficiency, the Adam optimizer is introduced into the neural network to optimize the generator G and the discriminator D. The specific parameters are set as follows: the initial learning rate of the discriminator is set to 4×10 -4 , the initial learning rate of the generator is set to 4×10 -4 The exponential decay rates of the first-order moment estimation and the second-order moment estimation are set to 0.5 and 0.999 respectively, the total training batch epoch is set to 50 rounds, the batch size is set to 4, and the image output size is set to 512×512 pixels.

[0094] S6: After the model training is completed, the weight file D4 (.pth file) with better image generation effect is selected based on the image generation effect, and the weight file D4 is retrieved and run in the model testing program PIX2 to process the overlapping particle image D5 in the test data set T2 to generate a scattered particle image D6, as shown in FIG. Figure 4 and Figure 5 shown.

[0095] S7: Formulate a judgment standard for the scattered state particle image, and judge whether the scattered state particle image D6 generated in S7 meets the judgment standard based on the judgment standard; if it meets the judgment standard, save the scattered state particle image D6; if it does not meet the judgment standard, return to step S5 to retrain the learning model H until the scattered state particle image D6 meets the judgment standard.

[0096] S8: In the Visual Studio Code code editor, use the Python program environment to perform grayscale and binarization preprocessing on the scattered particle image D6 that meets the judgment criteria in S7 to obtain the binary particle image D7, as shown in Figure 6 shown.

[0097] The purpose of grayscale and binarization preprocessing of images is to reduce the data dimension and highlight the structural features of the image through grayscale to simplify subsequent processing; to clearly separate the target and background through binarization, remove noise interference, and strengthen the boundary information of particles, thereby providing more accurate input for the segmentation model and improving the accuracy and efficiency of segmentation.

[0098] S9: Run the segmentation program SA and use the Segment Anything Model (SAM) image segmentation model to perform segmentation training on the binary particle image D7, segment the boundary contour of each particle, and generate the segmented image D8, as shown in Figure 7 shown.

[0099] In step S9, some key parameter settings when using the SAM model are: the model weight file used is sam_vit_h_4b8939.pth, the deep learning framework is pytorch, the hardware processor is GPU, the image density (points_per_side) is set to 32, the mask transparency (alpha) is set to 0.3, and the contour boundary line width (contour_thickness) is 5.

[0100] The SAM model in this step is a general segmentation model based on deep learning, which can automatically extract the target area in the image guided by prompts (such as points, boxes, text). Compared with traditional segmentation models, the core advantage of SAM is that it does not need to be specially trained for each task or dataset, but directly uses the weights of the pre-trained model to complete the segmentation task, thus significantly saving time and computing resources.

[0101] S10: Use the OpenCV machine learning library in the Visual Studio Code editor to extract contour feature information from the segmented image D8, as shown below: Figure 8 The extracted image is shown, and the extracted particle grading distribution information A2 is saved and A2 is stored in 2.xlsl.

[0102] OpenCV is an open source computer vision library that provides a wealth of image processing functions, including image preprocessing, feature extraction, contour detection, image transformation, etc. Specifically, in step S10, the way to extract the contour features of the particles in the segmented image D5 is mainly implemented through the findCountours() function, minEnclosingCircle() function and circle() function in OpenCV. Among them: the findContours() function is used to detect all particle contours in the image D5 and return the point set of the contour; the minEnclosingCircle() function is used to calculate the minimum circumscribed circle of each contour to help determine the circularity of the particle and its size information; the circle() function is used to draw the minimum circumscribed circle on the original image to intuitively display the boundary of the particle.

[0103] S11: Summarize the two sets of gradation information, namely, the gradation distribution information A1 of the original particles and the gradation distribution information A2 of the extracted particles, scale the two curves to the same coordinate scale system in the Visual Studio Code code editor, and draw a comparison image of the original gradation curve B1 of the particles and the predicted gradation curve B2 of the particles, compare the overlap of the two curves, and analyze the prediction error.

[0104] Run the program (JP) in VS code, retrieve the two sets of gradation information in 1.xlsl and 2.xlsl, and scale the two curves to the same coordinate scale system for plotting, and obtain the comparison image of the original gradation curve B1 of the particles and the predicted gradation curve B2 of the particles, as shown in the figure below: Fig. 9 shown.

[0105] The purpose of scaling the two grading curves to the same coordinate scale system is that the grading information A1 and A2 of the two curves are measured in different ways. The grading information A1 of the original particles is generated and exported by discrete element software, which is the actual size of the particles generated by the software, while the extracted particle size distribution information A2 is the particle contour extracted by OpenCV, and then the size is calculated based on the image pixel distribution. Therefore, the horizontal axis dimensions between the two are inconsistent. A1 is based on the actual physical size, while A2 is a relative size calculation based on image pixels. In order to be able to compare the two grading curves in the same coordinate system, the horizontal axis (particle size) of the curve must be unified to the same dimension through linear scaling to ensure comparability between the two, so as to more accurately evaluate the particle grading predicted by the model.

[0106] Specifically, in step S11, the method for scaling the two curves to the same coordinate scale system for drawing in the Visual Studio Code code editor is as follows: linearly scaling the particle size data in the extracted particle gradation distribution information A2, i.e., the data on the horizontal axis, so that its particle size value and the particle size value of the gradation distribution information A1 of the original particle are kept on the same scale, thereby facilitating comparative analysis.

[0107] The specific linear scaling method is: suppose the original radius of the particle in the particle grading distribution information A2 is r, and the original range of the particle size data is [r min ,r max ], r will be mapped to the new range [a,b], then the linear scaling calculation is formula (5):

[0108]

[0109] Among them, r scaled is the data value after mapping, r min and r max are the minimum and maximum values ​​of the original data, a and b are the minimum and maximum values ​​of the target range, and r is the value of each particle size data in the original data.

[0110] S12: Formulate an error standard, and determine whether the generated particle prediction grading curve B2 meets the error standard requirements based on the error standard; if the error standard requirements are met, obtain the final conclusion; if the error standard requirements are not met, return to step S4 to restart the construction of the model training data set T1 and test data set T2 and retrain the learning model H until the generated particle prediction grading curve B2 meets the error standard requirements.

[0111] It has been verified through many experiments that this technology has high accuracy and reliability in predicting the gradation distribution of overlapping particles and has broad application prospects.

[0112] In the present invention, some abbreviations are used. Specifically, Visual Studio Code is referred to as VS code, and Segment Anything Model is referred to as SAM. The full names and abbreviations should be understood the same.

[0113] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A technology for overlapping particle segmentation and grading prediction based on generative adversarial networks, characterized by: The following steps are involved: S1: According to the characteristics of overlapping granular materials, the gradation, porosity and inter-particle contact type and other parameters of the granular materials are preset in the discrete element simulation software to generate a granular sample M in a dispersed equilibrium state. During the generation process of the granular sample M, the granular materials gradually transition from a mutually overlapping state to a dispersed equilibrium state, and the finally generated granular sample M is in an equilibrium state; S2: Export the size and position information of all particles in the equilibrium state sample M from the software and save it as a .txt file; then, use Matplotlib in VisualStudio Code to reconstruct the particle model, calculate the gradation distribution information A1 of the original particles, and save it; S3: saving and outputting images of the generation process of the particle set sample M from overlapping to spreading in S1 at a preset time interval, wherein the image set of the particle set sample M in the overlapping state is D1, and the image set of the particle set sample M in the spreading state is D2; S4: Screen and crop the images in D1 and D2 to make the images consistent in size, pair the overlapping particle image set D1 with the scattered particle image set D2 one by one to form a paired image set D3, and construct a training data set T1 and a test data set T2 from D3 in proportion; S5: Use the pix2pix model in the generative adversarial network deep learning model to train the training data set T1 in S4, so that the model learns the image feature relationship of the particle set sample M from the overlapping state to the scattered state, and obtains the learning model H; S6: The weight file D4 is selected based on the generated image effect, and the weight file D4 is retrieved and run in the model test program to process the particle image D5 in the overlapping state in the test data set T2 to generate the particle image D6 in the scattered state; S7: formulating a determination standard for the scattered particle image, and determining whether the scattered particle image D6 generated in S7 meets the determination standard according to the determination standard; if it meets the determination standard, the scattered particle image D6 is saved; if it does not meet the determination standard, returning to step S5 to retrain the learning model H until the scattered particle image D6 meets the determination standard; S8: Using the Python program environment in the Visual Studio Code code editor, grayscale and binarization preprocessing is performed on the scattered particle image D6 that meets the judgment criteria in S7 to obtain a binarized particle image D7; S9: Use the SegmentAnything Model image segmentation model to perform segmentation training on the binary particle image D7, segment the boundary contour of each particle, and generate the segmented image D8; S10: Using the OpenCV machine learning library in the VisualStudio Code code editor, extracting contour feature information from the segmented image D8, and saving the extracted particle gradation distribution information A2; S11: Summarize the two groups of gradation information, namely, the gradation distribution information A1 of the original particles and the gradation distribution information A2 of the extracted particles, scale the two curves to the same coordinate scale system in the Visual Studio Code editor, and draw a comparison image of the original gradation curve B1 of the particles and the predicted gradation curve B2 of the particles, compare the overlap of the two curves, and analyze the prediction error; S12: Formulate an error standard, and determine whether the generated particle prediction grading curve B2 meets the error standard requirements based on the error standard; if the error standard requirements are met, obtain the final conclusion; if the error standard requirements are not met, return to step S4 to restart the construction of the model training data set T1 and test data set T2 and retrain the learning model H until the generated particle prediction grading curve B2 meets the error standard requirements.

2. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: The particles in the particle set sample M generated in step S1 are standard round particles.

3. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: In step S2, the center coordinates and radius information of all particles in the particle set sample M are obtained through the ball.pos.x(bp), ball.pos.y(bp) and ball.radius(bp) functions, and the data is exported as a .txt file.

4. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: In step S3, the movie image export function of the discrete element simulation software is used to set a fixed time interval to save the particle state during the generation process. The preset time interval is 700ms-1500ms, and the image type is in .png format. A continuous image sequence from overlapping to scattered is obtained, and the generated overlapping state image set D1 and scattered state image set D2 are stored separately.

5. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: In step S4, n overlapping state images and scattered state images that are paired one by one in the generation process are retained in the training data set T1, wherein each generation process retains m overlapping state images and m scattered state images, where n≥20; In the test data set T2, k overlapping state images and scattered state images that are paired one by one in the generation process are retained, where each generation process retains one randomly selected overlapping state image and one scattered state image, where k ≥ 10; The ratio of the number of images in the training data set T1 and the test data set T2 is 30:1-20:

1.

6. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: The pix2pix adversarial generative network used in step S5 is mainly composed of two parts: a generator G and a discriminator D. The generator G is responsible for converting the input overlapping particle image into a scattered particle image, and generates an output similar to the target image by learning the mapping relationship between the input image and the target image; the discriminator D is used to judge the difference between the image generated by the generator G and the real image, and improve the output quality of the generator G through the adversarial training mechanism; the loss function of the pix2pix adversarial generative network is composed of the generation loss L G And the discriminative loss L D The specific definitions of the common components are as follows: (a) Adversarial loss L cGAN :Adversarial loss is the core part of conditional generative adversarial network (cGAN). Generator G generates scattered particle images as realistic as possible by minimizing adversarial loss, while discriminator D distinguishes generated images from real images by maximizing adversarial loss. cGAN Calculated by formula (1), L cGAN (G,D)=E x [-logD(x,G(x))] (1), Where x represents the input overlapping particle image, G(x) represents the scattered particle image generated by the generator based on x, and D(x, G(x)) represents the probability that the discriminator judges that the input x and the generated image G(x) are real images; (b) L1 loss: In order to improve the similarity between the generated image and the real image at the pixel level, L1 loss is introduced, which is calculated as follows: L L1 (G)=E x,y [||y-G(x)||1] (2), Where y represents the real scattered particle image, ||yG(x)||1 represents the absolute value error between the generated image G(x) and the real target image y; (c) The total loss function L of the generator G : The total loss function L of the generator G is the adversarial loss L cGAN The weighted sum of the and L1 losses is calculated by formula (3): L G =L cGAN (G,D)+λL L1 (G) (3), Among them, λ is a hyperparameter used to balance the weights of adversarial loss and L1 loss; (d) The loss function L of the discriminator D : L D =E x,y [-logD(x,y)]+E x [-log(1-D(x,G(x)))] (4), Where x represents the input overlapping particle image, y represents the real scattered particle image, G(x) represents the scattered particle image generated by the generator based on x, D(x,y) represents the probability that the discriminator judges that the input x and the real target y are real images, and D(x,G(x)) represents the probability that the discriminator judges that the input x and the generated image G(x) are real images.

7. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: The method of extracting the contour features of the particles in the segmented image D5 in step S10 is implemented by the findCountours() function, minEnclosingCircle() function and circle() function in OpenCV, where: The indContours() function is used to detect all particle contours in image D5 and return the point set of the contours; The minEnclosingCircle() function is used to calculate the minimum circumscribed circle of each contour to help determine the circularity of the particle and its size information; the circle() function is used to draw the minimum circumscribed circle on the original image to intuitively display the boundary of the particle.

8. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1 is characterized by: In step S11, the method for scaling the two curves to the same coordinate scale system in the Visual Studio Code editor is as follows: linearly scaling the particle size data in the extracted particle gradation distribution information A2, i.e., the data on the horizontal axis, so that the particle size value and the particle size value of the original particle gradation distribution information A1 are kept on the same scale, thereby facilitating comparative analysis; The specific linear scaling method is: let the original radius of the particles in A2 be r, and the original range of the particle size data is [r min ,r max ], r is mapped to the new range [a, b], then the linear scaling is calculated as formula (5): Among them, r scaled is the data value after mapping, r max and r min are the minimum and maximum values ​​of the original data, a and b are the minimum and maximum values ​​of the target range, and r is the value of each particle size data in the original data.

9. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 6, characterized in that: The network architecture of the generator G adopts the U-Net model framework, which consists of 6 downsampling convolutional layers and 5 upsampling deconvolutional layers; the discriminator D adopts a convolutional neural network to extract the features of the input image through convolution operations.

10. The overlapping particle segmentation and grading prediction technology based on generative adversarial network according to claim 1, characterized in that: The architecture of the SegmentAnythingModel model in step S9 consists of three parts: Image Encoder, Prompt Encoder and Mask Decoder. Among them: Image encoder: uses ViT (Vision Transformer) to convert the input image into a series of feature vectors; Hint encoder: encodes the hints provided by the user (such as points, boxes or text) into hint vectors through a multi-layer perceptron (MLP); Mask decoder: combines the image feature vector with the hint vector, and generates the mask probability of each pixel through another set of MLP to achieve accurate segmentation.

Images