A particle tracking method, device, electronic equipment and storage medium

Abstract

The present application relates to a kind of particle tracking method, device, electronic equipment and storage medium, particle tracking method includes: step S1, PointConv model is trained to extract particle morphology feature;Step S2, preliminary matching is carried out based on linear displacement assumption and anchor point set is constructed;Step S3, for the particle not participating in preliminary matching, combined with local layout vector and predicted probability, complete extended matching;Step S4, anchor point set is iteratively extended until whole system matching converges.This method has strong fault tolerance to broken and segmented error particles, with 100% matching accuracy, applicable to large-scale CT data in high strain stage, and provides reliable tool for microstructure evolution analysis of granular material.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of civil engineering, geotechnical engineering and soil reinforcement technology, in particular to a particle tracking method and device, electronic equipment and storage medium. BACKGROUND

[0002] Particle materials are widely present in nature and engineering practice, such as sand, rockfill and powder particles, etc. Their complex mechanical behavior is directly related to major engineering problems such as slope stability, road durability and dam safety. The micro behavior of particle materials involves the interaction between particles, including collision, sliding, accumulation, etc. In order to deeply understand the behavior characteristics of particle materials, researchers need to obtain the kinematic information of particles, such as displacement, rotation, etc.

[0003] Particle tracking technology is to obtain the micro information of particles by using photoelastic experiment, digital image correlation (DIV / DIV), three-dimensional CT scanning, etc. and to track the particles according to the micro information, so as to obtain the kinematic information of the particles. Among them, CT scanning has the advantages of non-destructive and high-resolution imaging, and can accurately reconstruct the three-dimensional space information of the geometry, size and relative position of the particles, and is widely used in particle tracking technology.

[0004] The traditional tracking method based on volume or morphological parameters is sensitive to segmentation errors, and when the particle displacement is large in the high strain stage, improper setting of the search window will lead to misjudgment, making it difficult to accurately track the broken or segmented particles. In addition, although the existing particle tracking based on neural network can effectively extract the morphological features of point cloud, due to the limitation of CT image resolution and the difference of feature vectors caused by particle crushing, it is easy to output false prediction with high confidence. The existing particle tracking method faces the problem of decreased matching accuracy caused by segmentation error and particle crushing when processing CT scanning data. SUMMARY

[0005] In view of the problems existing in the prior art, the purpose of the present application is to provide a particle tracking method, device, electronic equipment and storage medium, which can realize high-accuracy and high-robustness tracking of particles in CT data.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] A particle tracking method, comprising the following steps,

[0008] Step S1: segmenting the CT scanning body data of the particle medium sample to obtain particle label data, and extracting surface point cloud data describing the morphological features of the particles from the particle label data; constructing a PointConv neural network, and training the neural network with the surface point cloud data as input to generate a PointConv model capable of outputting predicted labels and predicted probabilities;

[0009] Step S2: Based on the overall deformation law of the sample, the linear displacement assumption is used to map the mass center of all particles in the deformation system to the reference system, and the search space is established based on the mapped mass center, thereby limiting the possible spatial position range of the particle in the reference system; if the mass center of the particle in the reference system is located in the search space of a particle in the deformation system, the particle in the reference system is defined as the candidate particle of the particle in the deformation system; the surface point cloud of each particle in the deformation system is sent into the PointConv model to obtain the predicted label and the corresponding predicted probability of the particle in the reference system, if the predicted label corresponds to the candidate particle of the particle in the reference system, and the predicted probability is not lower than the preset confidence threshold, a matching relationship is established between the particle in the deformation system and the particle in the reference system, and the two are taken as a pair of anchor particles, the particles satisfying the above conditions constitute an anchor set, and the particles not satisfying the above conditions are taken as matching particles to enter step S3 for further processing;

[0010] Step S3: For each matching particle, the particles in the reference system whose mass centers are located in the search space of the matching particle are selected as candidate particles, the relative mass center vector set with respect to the matching particle is constructed by selecting the adjacent anchor particles, and the relative mass center vector set with respect to the candidate particle is constructed by using the corresponding relationship in the anchor set, and the local layout difference between the candidate particle and the matching particle is calculated by using the two relative mass center vector sets; the predicted probability of the candidate particle is obtained, the local layout difference and the predicted probability are combined to construct an extended matching error, and the candidate particle with the smallest error is selected as the extended matching result;

[0011] Step S4: The particles successfully matched in step S3 are added to the anchor set, and step S3 is iteratively executed until the particle matching result converges, and the whole system particle tracking is completed.

[0012] Further, step S1 includes,

[0013] Step S1.1: The CT data is segmented to obtain particle label data, the Marching Cubes algorithm is used to generate particle surface point cloud, and the point normal vector is estimated to obtain 6-dimensional point cloud data composed of spatial coordinates and normal vectors;

[0014] Step S1.2: Using the 6-dimensional point cloud data obtained in step S1.1 as the object, the FPS algorithm is used to resample the particle surface point cloud to obtain a plurality of sub-point sets, and a training-test sample library is formed;

[0015] Step S1.3, constructing a PointConv neural network with the multiple groups of sub-point sets provided in step S1.2 as input, outputting particle prediction labels and corresponding prediction probabilities through multiple levels of Set Abstraction layers and fully connected layers, and connecting a Softmax function after the last fully connected layer to form a PointConv model capable of outputting prediction probabilities;

[0016] Step S1.4, training the PointConv model of step S1.3 using the PyTorch deep learning framework and the training-test sample library of step S1.2, optimizing the model parameters using an SGD optimizer and a negative log-likelihood loss function, and saving the model weights after training is complete.

[0017] Further, step S1.3 includes that the PointConv neural network model takes the multiple groups of sub-point sets obtained in step S1.2 as input data, and sequentially passes through multiple levels of Set Abstraction layers to realize local point cloud sampling, grouping, and feature aggregation; each level of Set Abstraction layer is composed of a sampling grouping layer and a PointConv convolutional layer, the sampling grouping layer samples local points and groups them in the neighborhood of the local points G Obtaining features of neighborhood points ; the PointConv convolutional layer generates high-dimensional features according to the following formula; the high-dimensional features are sent to the subsequent fully connected layer, and the particle prediction labels and prediction probabilities are obtained through the Softmax function,

[0018] wherein, represents the relative coordinates of the neighborhood points, is the weight matrix of the convolution, is the inverse density.

[0019] Further, step S2 includes,

[0020] Step S2.1, selecting the front and rear adjacent strain stages as the reference system and the deformation system respectively, and based on the overall deformation law of the sample, mapping the center of mass positions of the particles in the deformation system using the linear displacement assumption, the displacement of the particles at the loading end is consistent with the loading displacement, the positions of the particles at the fixed end remain fixed, and the displacements of the internal particles are linearly interpolated according to the height;

[0021] Step S2.2, establishing a search space with the mapped center of mass as the center of the sphere, which defines the possible spatial position range of the particles in the reference system;

[0022] Step S2.3, inputting the surface point cloud data of the particles in the deformation system into the PointConv model trained in step S1 to obtain the prediction labels and corresponding prediction probabilities of the particles in the reference system;

[0023] Step S2.4, if the predicted label corresponds to a reference system particle whose centroid is located in the search space and the predicted probability satisfies the confidence threshold, a matching relationship is established between the deformed system particle and the reference system particle corresponding to the predicted label, and both are added to the anchor set as a pair of anchor particles; otherwise, the deformed system particle is taken as a to-be-matched particle and proceeds to step S3.

[0024] Further, step S3 includes,

[0025] Step S3.1, for the to-be-matched particle p in step S2 whose prediction result does not meet the requirements, p select the adjacent anchor particles in the deformed system, find the matching objects of these anchor particles in the reference system through the corresponding relationship of the anchor set in step S2.4, and calculate the set of relative centroid vectors: ;

[0026] wherein represents the centroid coordinate vector of the anchor particle A i ( i =1, …, M ) in the deformed system, and the centroid coordinate vector of the to-be-matched particle p ;

[0027] Step S3.2, obtain the candidate particle set in the search space of the reference system corresponding to the to-be-matched particle p , take the centroid of each candidate particle k as the reference, and combine the reference system matching objects provided by the anchor set in step S3.1 to construct the corresponding set of relative centroid vectors: ;

[0028] wherein represents the centroid coordinate vector of the anchor particle A i ( i =1, …, M ) in the reference system, represents the centroid coordinate vector of the candidate particle k ;

[0029] Step S3.3, the local layout difference LD k between the candidate particle k and the to-be-matched particle is defined as the root mean square difference of the two sets of relative centroid vectors: ;

[0030] Step S3.4, calculate the local layout difference LD c ( ), to realize normalization, define the neighborhood feature error term of the particle as ; at the same time, combine the predicted probability of the candidate particle by the PointConv model P p ( k ), and calculate the matching error by integrating the particle shape feature error (1- P p ( k )) and the neighborhood feature error E k : ;

[0031] Select the candidate particle with the smallest error from each candidate particle as the matching result.

[0032] Further, step S4 includes,

[0033] Step S4.1, based on the successfully extended matching particle in step S3, extend the anchor point set in step S2, and take the newly matched particle as the anchor point for the construction of the relative centroid vector of the particle to be matched in the next round;

[0034] Step S4.2, repeat the local layout difference calculation and matching error evaluation of step S3 in each iteration until the matching result is stable.

[0035] A particle tracking device, comprising a model generation module, an initial matching module and an extended matching module:

[0036] The model generation module is used for segmenting the CT scan volume data to obtain particle label data, extracting surface point cloud data describing the particle shape feature from the particle label data, constructing a PointConv neural network, training the neural network with the surface point cloud data as input, and generating a PointConv model capable of outputting a predicted label and a predicted probability;

[0037] The initial matching module is used for obtaining anchor point particles: based on the overall deformation law of the sample, all particle centroids in the deformation system are mapped to a reference system by using a linear displacement assumption, and a search space is established based on the mapped centroids, so as to limit the possible spatial position range of the particles in the reference system; if the centroid of a particle in the reference system is located in the search space of a particle in a certain deformation system, the particle in the reference system is defined as a candidate particle of the particle in the deformation system; the surface point cloud of each particle in the deformation system is sent into a PointConv model to obtain a predicted label and a corresponding predicted probability of the particle in the reference system; if the reference system particle corresponding to the predicted label is a candidate particle of the particle, and the predicted probability is not lower than a preset confidence threshold, a matching relationship is established between the particle in the deformation system and the particle in the reference system, and the two particles are taken as a pair of anchor point particles; particles satisfying the above conditions form an anchor point set, and particles not satisfying the above conditions are taken as particles to be matched and enter an extended matching module for continuous processing.

[0038] The extended matching module is used for extended matching for the particles to be matched: for each particle to be matched, particles in the reference system with centroids located in the search space thereof are selected as candidate particles, adjacent anchor point particles are selected to construct a relative centroid vector set with respect to the particle to be matched, a relative centroid vector set with respect to the candidate particle is constructed in the reference system by using the corresponding relationship in the anchor point set, and the local layout difference between the candidate particle and the particle to be matched is calculated by using the two relative centroid vector sets; the predicted probability of the candidate particle is obtained, the local layout difference and the predicted probability are comprehensively considered to construct an extended matching error, and the candidate particle with the minimum error is selected as an extended matching result; the particles successfully matched by extension are added to the anchor point set, and the extended matching is iteratively performed until the particle matching result converges, and the particle tracking of the whole system is completed.

[0039] An electronic device includes a processor and a memory, the memory stores a computer program, and the processor implements the particle tracking method described above when executing the computer program.

[0040] A storage medium stores a computer program, and the computer program is executed by a processor to implement the particle tracking method described above.

[0041] Overall, the present application has the following advantages:

[0042] (1) High accuracy: experimental results show that the matching accuracy of the particles that are not broken and correctly segmented by the present method reaches 100%. Even in the face of segmented errors and broken particles, the present method can accurately associate them with the parent particles, which is significantly better than the traditional tracking method based on particle morphology.

[0043] (2) High robustness: By combining the morphological feature extraction of the neural network and the neighborhood feature constraint, the method can effectively deal with the segmentation error and particle breakage in the CT scan data. Even if there is a difference in resolution and noise influence at different strain stages, the method still significantly improves the accuracy of particle matching and the robustness of cross-stage matching.

[0044] (3) High efficiency matching: Based on the search space design of linear displacement assumption and multi-round iterative matching strategy, the computational complexity is reduced, and experiments show that the first round of matching can cover most of the particles, and 3-4 rounds of iteration can realize stable and efficient particle matching.

[0045] (4) Wide applicability: The method is suitable for large number of particle tracking at different strain stages, especially in high strain stage, it can accurately identify broken particles and segmentation error particles, and is suitable for large-scale CT data set containing tens of thousands of particles, providing reliable support for microstructure analysis of granular materials. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 PointConv neural network-based CT point cloud classification schematic diagram.

[0047] Figure 2 Particle tracking algorithm flowchart.

[0048] Figure 3 Anchor particle discrimination criterion schematic diagram.

[0049] Figure 4 Relative center of mass vector set construction schematic diagram.

[0050] Figure 5 Particle tracking device structure block diagram of the application.

[0051] Figure 6 Electronic device structure block diagram of the application. DETAILED DESCRIPTION

[0052] The application will be further described in detail below.

[0053] Example 1:

[0054] A particle tracking method based on PointConv neural network and particle neighborhood features, suitable for large number of particle tracking at different strain stages.

[0055] The CT scan sample input by the embodiment contains about 25000 particles, the test material is ISO standard sand in Fujian, China, provided by Xiamen ISO Standard Sand Co., Ltd., the particle size of the sand is 1-2mm, the uniformity coefficient C u =5.55, the curvature coefficient C c=1.59, and according to the Unified Soil Classification System (USCS), the sand belongs to poorly graded sand (SP). Triaxial undrained shear tests are performed on the sand sample and reach 15% strain. Data of 0-1% and 10-15% strain stages are taken respectively to apply the particle tracking method. The data includes broken particles and misclassified particles. The method can accurately identify and match particles. The implementation steps are as follows:

[0056] Step S1: segment the CT scan body data to obtain particle label data, and extract surface point cloud data describing the particle shape features from the particle label data; construct and train a PointConv neural network to generate a PointConv model that can output predicted labels and corresponding probabilities.

[0057] Step S2: based on the overall deformation law of the sample, use the linear displacement assumption to map the mass center of all particles in the deformation system to the reference system, and establish a search space based on the mapped mass center, thereby limiting the possible spatial position of the particle in the reference system to form a spatial position constraint; determine the candidate particles in the search space, and input the surface point cloud of the particle candidate particles into the PointConv model to obtain the predicted label and the corresponding predicted probability of the particle in the reference system; the candidate particles that meet the spatial position constraint and the predicted probability requirement are determined as anchor particles to construct an anchor set; the candidate particles that do not meet the prediction requirement are used as matched particles in the subsequent step; if the mass center of the reference system particle corresponding to the predicted label is located in the search space of the particle, and the predicted probability is not lower than the preset confidence threshold, a matching relationship is established between the particle in the deformation system and the reference particle, and they are used as a pair of anchor particles, the particles that meet the above conditions constitute an anchor set, and the particles that do not meet the conditions are used as matched particles in step S3 for further processing.

[0058] In the method, the deformation system refers to the particle system at the current strain stage, which is used to provide the original spatial position of the matched particle, and the mass center thereof needs to be mapped to the reference system after linear displacement assumption for comparison. The reference system refers to the particle system at the previous strain stage, which is used as the mapping reference and the search space origin, and all comparisons, error calculations and final matching results are based on the coordinates of the system.

[0059] Step S3: For each particle to be matched, select the particles in the reference system whose centroids are located in the search space thereof as candidate particles, select the neighboring anchor particles to construct a set of relative centroid vectors with respect to the particle to be matched, and simultaneously use the corresponding relationship in the anchor set to construct a set of relative centroid vectors with respect to the candidate particles in the reference system using the same set of anchor particles, and calculate the local layout difference between the candidate particles and the particle to be matched using the two sets of relative centroid vectors; obtain the prediction probability of the candidate particles, and construct an extended matching error by comprehensively considering the local layout difference and the prediction probability, and select the candidate particle with the smallest error as the extended matching result;

[0060] Step S4: Add the particles successfully matched in step S3 to the anchor set, iteratively perform step S3, gradually expand the matching range, and until the particle matching result converges, complete the particle tracking of the whole system.

[0061] Specifically, the implementation process of step S1 is as follows:

[0062] Step S1.1, as shown in Figure 1 , the input CT scan volume data is segmented to obtain particle label data, the Marching Cubes (isosurface extraction) algorithm is used to generate particle surface point cloud, and the point normal vector is estimated, each point in the obtained particle surface point cloud data has a corresponding 6-dimensional feature vector (including three-dimensional space coordinates and three-dimensional normal vector) to represent the shape feature of the particle.

[0063] Step S1.2, the point cloud data of each particle is sampled into 30 sub-point sets by FPS (Farthest Point Sampling) algorithm, each sub-point set contains 1200 sampling points, and the sub-point cloud data set is divided into a training set and a test set in a ratio of 0.85:0.15.

[0064] Step S1.3, a PointConv neural network is constructed, including multiple Set Abstraction layers (SA) and fully connected layers (FC), as shown in Figure 1 . The PointConv neural network learns features from the input training set, abstracts a single high-dimensional feature vector from the sub-point cloud data set (size 1200x6) representing the shape feature of the particle, and converts the high-dimensional feature vector representing the shape feature of the single particle into a probability distribution of all classes through the fully connected layer and the logarithmic softmax function, wherein the result with the highest probability is the predicted label. The SA layer is divided into two modules. The sampling grouping layer extracts local points through sampling and grouping operations, and obtains the features of the neighborhood points G in the neighborhood of the local points PointConv convolutional layer performs discrete convolution on the feature vector of a single local point to generate a high-dimensional feature vector. The specific convolution calculation formula is as follows:

[0065]

[0066] wherein represents the relative coordinates of the neighborhood points, is the weight matrix of convolution, which is different from the fixed kernel function in traditional convolution. The weight matrix is a dynamic kernel function based on the local geometric structure of the point cloud, which can be learned by a multi-layer perception (MLP) and shared among all local points. is the inverse density, which is used to dynamically adjust the feature contribution of different density regions. The weight of the sparse region (few points) is amplified to avoid feature dilution, and the weight of the dense region (many points) is suppressed to avoid feature redundancy. The inverse density weight is obtained by nonlinear transformation of the density calculated by the kernel density estimation (KDE) by the MLP.

[0067] Step S1.4, train the PointConv model using the PyTorch framework, use the SGD (stochastic gradient descent) optimizer (initial learning rate 0.01, momentum 0.9, weight decay 1e-4), reduce the learning rate by 0.7 times every 30 rounds, a total of 200 rounds, and use the negative log likelihood loss (NLLLoss) as the loss function. The correct rate of the training and test sets is used to determine the training result every round. After training is completed, the PointConv model can efficiently extract the morphological features of the particle surface point cloud and output the predicted label and predicted probability.

[0068] Specifically, step S2 is the first stage of initial matching, and the algorithm flow is as shown in Figure 2 , and the implementation process is as follows:

[0069] Step S2.1, as shown in Figure 3 , a search space is constructed. The adjacent strain stages (strain difference ) are selected as the reference system (sample height h 0) and the deformation system (sample height h 1), respectively. Based on the overall deformation law of the sample, the particle n in the deformation system (distance from the fixed end to the fixed end z 1) is mapped to the center position using linear displacement assumption. The displacement of the particles at the loading end (the top end of the sample where the load is applied) is consistent with the loading displacement, the position of the particles at the fixed end (the bottom end of the sample which is constrained to be stationary) remains fixed, and the displacement of the internal particles is linearly interpolated according to the height.

[0070] Step S2.2, a search space is constructed with the mapped centroid position as the center of the sphere, and the search radius is usually 2-4 times the average radius of the sand particles, which is adjusted according to the strain size. A reasonably sized search space will contain the correctly matched particles, which can be used as spatial position constraints for anchor particles in the first stage of initial matching, and also provide a candidate particle set for the second stage of extended matching.

[0071] Step S2.3, the PointConv model trained in step S1 is used to predict the surface point cloud data of the particles in the deformed system. Ten sub-point cloud data sets of each particle are input into the PointConv model to obtain the predicted label and the average prediction probability.

[0072] Step 2.4, for all the particles to be matched in the deformed system, the anchor particles are screened, which need to meet the following conditions: (1) the centroid corresponding to the predicted label meets the spatial position constraint (i.e. the centroid is located within the search space); (2) the average prediction probability meets a certain confidence threshold (in this embodiment, it is not less than 60%), to ensure the reliability of the prediction, as shown in Figure 3 The results of the embodiment show that, due to the influence of the resolution of the CT scan data at different strain stages and the label division results, the proportion of anchor particles in the first round of matching is 85.42% (0-1% strain) to 41.72% (10-15% strain).

[0073] Specifically, step S3 is the second stage of extended matching, and the algorithm flow is as shown in Figure 2 The implementation process is as follows: step S3.1, as shown in Figure 4 For the particles to be matched whose prediction results do not meet the two requirements in step S2, the relative centroid vector set is constructed by selecting the adjacent anchor particles. In the deformed system, the relative centroid vector set is calculated with the particle to be matched p as the center:

[0074] ;

[0075] Wherein represents the centroid coordinate vector of the anchor particle A i ( i =1, …, M ), and is the centroid coordinate vector of the particle to be matched p .

[0076] Step 3.2, as shown in Figure 4 , the candidate particles k are obtained within the search space of the reference system, and the corresponding relative centroid vector set is constructed:

[0077] ;

[0078] wherein represents the anchor particles in the deformed system A i i =1, …, M ) the centroid coordinate vector of the reference system, represents the candidate particle k .

[0079] Step 3.3, the local layout difference (LD k ) of the candidate particle and the particle to be matched is defined as the root mean square difference of the relative centroid vectors of the two:

[0080]

[0081] Step S3.4, calculate the local layout difference LD c ) for all candidate particles, in order to realize normalization, define the neighborhood feature error term of the particle k ; ; at the same time, combine the predicted probability of the candidate particle by the PointConv model P p k ), the matching error P p k ) is calculated by combining the particle shape feature error (1- E k :

[0082] Select the candidate particle with the smallest error among all candidate particles as the matching result.

[0083] Specifically, the implementation process of step S4 is as follows:

[0084] Step S4.1, based on the successfully matched particles in step S3, expand the anchor point set in step S2, and use the newly matched particles as anchor points for the construction of the relative centroid vector of the particle to be matched in the next round.

[0085] Step S4.2, repeat the LD calculation and error evaluation of step S3 in each iteration until the matching result is stable. Experiments show that usually 3-4 iterations can realize stable matching of the whole system.

[0086] Step S4.3, experimental results show that the matching accuracy of this method for unbroken and correctly segmented particles reaches 100%, and it has high fault tolerance for segmented errors and broken particles, and is suitable for high strain stages (such as 10-15% strain) and large-scale CT data sets containing tens of thousands of particles. ​​​​​​

[0087] The embodiment realizes particle tracking through the above steps, verifies the efficiency and robustness of the method in the high strain stage and large-scale data set, and is suitable for analysis of the shear band formation mechanism.

[0088] Embodiment 2:

[0089] As shown in Figure 5 A particle tracking device comprises:

[0090] The model generation module 501 is configured to segment CT scan volume data to obtain particle label data, extract surface point cloud data describing particle morphological features from the particle label data, construct a PointConv neural network, and train the neural network with the surface point cloud data as input to generate a PointConv model capable of outputting predicted labels and predicted probabilities.

[0091] The initial matching module 502 is configured to obtain anchor particles. Based on the overall deformation law of the sample, all particle centers in the deformed system are mapped to the reference system using a linear displacement assumption, and a search space is established based on the mapped particle centers, thereby limiting the possible spatial position range of the particles in the reference system. If the particle center in the reference system is located within the search space of a particle in the deformed system, the particle in the reference system is defined as a candidate particle of the particle in the deformed system. The surface point cloud of each particle in the deformed system is input into the PointConv model to obtain the predicted label and the corresponding predicted probability of the particle in the reference system. If the predicted label corresponds to a candidate particle in the reference system, and the predicted probability is not lower than a preset confidence threshold, a matching relationship is established between the particle in the deformed system and the particle in the reference system, and the two particles are taken as a pair of anchor particles. The particles satisfying the above conditions constitute an anchor set, and the particles not satisfying the above conditions are taken as particles to be matched and input into the extended matching module for further processing.

[0092] The extended matching module 503 is configured to, for each particle to be matched, select particles in the reference system whose centers are located within the search space of the particle to be matched as candidate particles, select adjacent anchor particles to construct a relative center vector set with respect to the particle to be matched, and use the corresponding relationship in the anchor set to construct a relative center vector set with respect to the candidate particles in the reference system using the same set of anchor particles, and calculate the local layout difference between the candidate particles and the particle to be matched using the two relative center vector sets. Obtain the predicted probability of the candidate particles, and construct an extended matching error by combining the local layout difference and the predicted probability, and select the candidate particle with the smallest error as the extended matching result. The successfully extended matched particles are added to the anchor set, and the extended matching is iteratively performed until the particle matching result converges, and the particle tracking of the whole system is completed.

[0093] The specific implementation of each module in this embodiment can be referred to the embodiment 1 described above, which will not be repeated here; it should be noted that the device provided in this embodiment is only used as an example for the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to the needs, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above.

[0094] Embodiment 3

[0095] As shown in Figure 6 An electronic device includes a processor 602, a memory, an input device 603, a display 604 and a network interface 605 connected through a system bus 601. Among them, the processor 602 is used to provide computing and control capability, the memory includes a non-volatile storage medium 606 and an internal memory 607, the non-volatile storage medium 606 stores an operating system, a computer program and a database, the internal memory 607 provides an environment for the running of the operating system and the computer program in the non-volatile storage medium 606, and the computer program is executed by the processor 602 to realize the above-mentioned particle tracking method.

[0096] Embodiment 4

[0097] A storage medium, the storage medium stores a computer program, and the computer program is executed by the processor to realize the above-mentioned particle tracking method.

[0098] The storage medium described in this embodiment can be a disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a U disk, a mobile hard disk and the like.

[0099] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, which are all included in the protection scope of the present application.

Claims

1. A particle tracking method, characterized by: The method comprises the following steps, Step S1: segmenting CT scan volume data of a granular medium sample to obtain particle label data, and extracting surface point cloud data describing particle morphological features therefrom; constructing a PointConv neural network, and training the neural network with the surface point cloud data as input to generate a PointConv model capable of outputting predicted labels and predicted probabilities; Step S2: based on the overall deformation law of the sample, using a linear displacement assumption to map all particle centers in the deformed system to a reference system, and establishing a search space with the mapped particle centers to limit the possible spatial position range of the particle in the reference system; If the particle center in the reference system is located within the search space of a particle in the deformed system, the particle in the reference system is defined as a candidate particle of the particle in the deformed system; the surface point cloud of each particle in the deformed system is input into the PointConv model to obtain the predicted label and the corresponding predicted probability of the particle in the reference system, if the predicted label corresponds to a candidate particle in the reference system, and the predicted probability is not lower than a preset confidence threshold, a matching relationship is established between the particle in the deformed system and the particle in the reference system, and the two are taken as a pair of anchor particles, the anchor particles satisfying the above conditions constitute an anchor set, and the particles not satisfying the above conditions are taken as matching particles and enter step S3 for further processing; Step S3: for each matching particle, selecting a particle in the reference system whose center is located within the search space as a candidate particle, selecting adjacent anchor particles to construct a relative center vector set with respect to the matching particle, and using the corresponding relationship in the anchor set to construct a relative center vector set with respect to the candidate particle in the reference system with the same set of anchor particles, and calculating the local layout difference between the candidate particle and the matching particle by using the two relative center vector sets; obtaining the predicted probability of the candidate particle, and constructing an extended matching error by comprehensively considering the local layout difference and the predicted probability, and selecting the candidate particle with the smallest error as the extended matching result; Step S4: adding the particles successfully matched in step S3 to the anchor set, and iteratively executing S3 until the particle matching result converges, and completing the whole system particle tracking.

2. The particle tracking method of claim 1, wherein: Step S1 comprises, Step S1.1: segmenting the CT scan volume data to obtain particle label data, using the Marching Cubes algorithm to generate particle surface point cloud, and estimating point normal vector to obtain 6-dimensional point cloud data composed of spatial coordinates and normal vectors; Step S1.2: using the 6-dimensional point cloud data obtained in step S1.1 as an object, using the FPS algorithm to resample the particle surface point cloud to obtain a plurality of sub-point sets, and forming a training-test sample library; Step S1.3: constructing a PointConv neural network with the plurality of sub-point sets provided in step S1.2 as input, outputting particle predicted labels and corresponding predicted probabilities through a plurality of Set Abstraction layers and a full connection layer, and connecting a Softmax function after the last full connection layer to form a PointConv model capable of outputting predicted probabilities; Step S1.4, training the PointConv model of step S1.3 using the PyTorch deep learning framework and the training-test sample library of step S1.2, using the SGD optimizer and negative log-likelihood loss function to optimize the model parameters, and saving the model weights after training is complete.

3. The particle tracking method of claim 2, wherein: Step S1.3 includes that the PointConv model takes the multiple groups of sub-point sets obtained in step S1.2 as input, and sequentially passes through multiple Set Abstraction layers to realize local point cloud sampling, grouping and feature aggregation. Each level of Set Abstraction layer is composed of a sampling grouping layer and a PointConv convolutional layer. The sampling grouping layer samples local points and obtains the features of the neighborhood points of the local points G obtaining the features of the neighborhood points The PointConv convolutional layer generates high-dimensional features according to the following formula, and the high-dimensional features are sent to the subsequent fully connected layer. The granular prediction label and prediction probability are obtained through the Softmax function. wherein, denotes the relative coordinates of the neighborhood points, is the weight matrix of the convolution, is the inverse density.

4. The particle tracking method of claim 1, wherein: Step S2 includes, Step S2.1, respectively selecting the adjacent strain stages before and after as the reference system and the deformation system, based on the overall deformation law of the sample, using linear displacement assumption to map the center of mass position of the particles in the deformation system, the displacement of the particles at the loading end is consistent with the loading displacement, the position of the particles at the fixed end remains fixed, and the displacement of the internal particles is linearly interpolated according to the height; Step S2.2, a search space is established with the mapped center of mass as the center of the sphere, which limits the possible spatial position range of the particles in the reference system; Step S2.3, input the surface point cloud data of the particles in the deformation system into the PointConv model trained in step S1 to obtain the predicted label and the corresponding prediction probability of the particles in the reference system; Step S2.4, if the center of mass of the reference system particle corresponding to the predicted label is located within the search space and the prediction probability meets the confidence threshold, a matching relationship is established between the particle in the deformation system and the reference system particle corresponding to the predicted label, and both are added to the anchor point set as a pair of anchor point particles; otherwise, the particle in the deformation system is taken as a particle to be matched and enters step S3 for further processing.

5. The particle tracking method of claim 4, wherein: Step S3 includes, Step S3.1, for the to-be-matched particles whose prediction results in step S2 do not meet the requirements p , in the deformed system, taking the to-be-matched particle p as the center, selecting adjacent anchor particles, finding the matching objects of these anchor particles in the reference system through the correspondence of the anchor set in step S2.4, and calculating the set of relative centroid vectors: ; wherein representing the anchor particles in the morphing system A i ( i =1,..., M ) the centroid coordinate vector of the particle, the particle to be matched p the centroid coordinate vector of the particle to be matched Step S3.2, in the particles to be matched p Obtain the candidate particle set within the search space of the corresponding reference system, and then select the candidate particles within it. k Using its centroid as a reference, and combining the reference system matching object provided by the anchor point set in step S3.1, construct the corresponding relative centroid vector set: ; wherein represents an anchor particle in the morphing system A i ( i = 1,..., M ) the centroid coordinate vector of the candidate particle represents an anchor particle in the morphing system k the centroid coordinate vector of the candidate particle Step S3.3, candidate particles k Local layout difference LD with the particle to be matched k defined as the root mean square difference of the two sets of relative centroid vectors: ; Step S3.

4. Calculate the local layout difference LD for all candidate particles c ( ) To achieve normalization, define the neighborhood feature error term of the particle k as ; meanwhile, combine the predicted probability of the candidate particle by the PointConv model P p ( k ) , and calculate the matching error by integrating the particle shape feature error and the neighborhood feature error E k : , (1- P p ( k )) is the particle shape feature error; Selecting the candidate particle with the smallest error as the matching result from the candidate particles.

6. The particle tracking method of claim 1, wherein: Step S4 includes, Step S4.1, based on the successfully expanded matched particles in step S3, expanding the anchor point set in step S2, taking the newly matched particles as anchors for the construction of the relative center of mass vector of the particles to be matched in the next round; Step S4.2, repeating the local layout difference calculation and matching error evaluation of step S3 in each iteration until the matching result is stable.

7. A particle tracking device, characterized by The particle tracking method of any one of claims 1-6, comprising a model generation module, an initial matching module and an extended matching module: The model generation module is used for segmenting the CT scan volume data to obtain particle label data, and extracting surface point cloud data describing the morphological characteristics of the particles; a PointConv neural network is constructed, and the neural network is trained with the surface point cloud data as input to generate a PointConv model that can output predicted labels and prediction probabilities; The initial matching module is used to obtain anchor point particles: based on the overall deformation law of the sample, the center of mass of all particles in the deformation system is mapped to the reference system using linear displacement assumption, and a search space is established with the mapped center of mass, thereby limiting the possible spatial position range of the particles in the reference system; If the centroid of a particle in the reference system is located in the search space of a particle in the deformed system, the particle in the reference system is defined as a candidate particle of the particle in the deformed system; the surface point cloud of each particle in the deformed system is sent into the PointConv model to obtain a predicted label and a corresponding predicted probability of the particle in the reference system, if the predicted label corresponds to a candidate particle of the particle in the reference system and the predicted probability is not lower than a preset confidence threshold, a matching relationship is established between the particle in the deformed system and the particle in the reference system, and the two particles are taken as a pair of anchor particles, the particles satisfying the above conditions constitute an anchor set, and the particles not satisfying the above conditions are taken as particles to be matched and enter an extended matching module for further processing; The extended matching module is configured to perform extended matching on the particles to be matched: for each particle to be matched, a particle in the reference system whose centroid is located in the search space of the particle to be matched is selected as a candidate particle, adjacent anchor particles are selected to construct a set of relative centroid vectors of the candidate particle, and the set of relative centroid vectors of the candidate particle is constructed in the reference system using the same set of anchor particles, and a local layout difference between the candidate particle and the particle to be matched is calculated using the two sets of relative centroid vectors; a predicted probability of the candidate particle is obtained, an extended matching error is constructed by combining the local layout difference and the predicted probability, and the candidate particle with the smallest error is selected as an extended matching result; The particles successfully matched by the extended matching are added to the anchor set, and the extended matching is iteratively performed until the particle matching result converges, and the particle tracking of the whole system is completed.

8. An electronic device, comprising: The particle tracking method includes a processor and a memory, the memory stores a computer program, and the processor executes the computer program to implement the particle tracking method of any one of claims 1-6.

9. A storage medium, characterized by The particle tracking method is implemented by executing the computer program stored in the storage medium by the processor.

Images