Calcareous sand unit test broken particle tracking and matching method based on digital image

Abstract

The invention relates to a calcareous sand unit test broken particle tracking and matching method based on a digital image, which comprises the following steps: S1, taking a calcareous sand unit test picture, and making the taken test picture into a two-dimensional picture with three-dimensional information; s2, performing instance segmentation by using the SAM; s3, the BoT-SORT algorithm is improved; s4, tracking an instance segmentation result in the step S2 by using the BoT-SORT algorithm improved in the step S3; s5, using a PIV algorithm to track the part except for the instance segmentation in the S2; s6, performing feature point extraction on the instance segmentation tracking result in the step S4 by using an SURF, and performing matching on the extracted feature points by using a KNN matching algorithm to obtain a particle crushing matching result; and S7, making a final result graph. According to the method, non-uniform grading particle tracking and crushing matching can be realized through the CNN without training, and the method has extremely high robustness and generalization ability.

Description

Technical Field

[0001] The invention relates to the technical field of digital image processing technology, and in particular to a method for tracking and matching crushed particles in a calcareous sand unit test based on digital images. Background Art

[0002] Calcareous sand is a special type of marine sediment, mainly composed of the remains of marine organisms such as foraminifera, corals and shells. It has the characteristics of complex mineral composition, irregular particle morphology and porous microstructure. These characteristics cause calcareous sand to show significant brittle characteristics under the action of external forces. With the continuous development and utilization of marine resources by humans, more and more engineering projects are being built in island and reef areas, making the widely distributed calcareous sand a new filler for foundations. The strength and deformation characteristics of calcareous sand directly affect the safety of infrastructure. Under high stress conditions, calcareous sand particles are prone to significant crushing, thereby changing the physical and mechanical properties of the foundation, leading to instability or even destruction of the superstructure. Therefore, it is of great significance to study the mechanical properties and particle crushing behavior of calcareous sand.

[0003] At present, the research on calcareous sand is often carried out through unit tests, which reveal the physical and mechanical properties of calcareous sand by analyzing the stress-strain law under different external forces. Although the physical and mechanical properties of calcareous sand can be obtained by unit tests, due to its invisibility, it is impossible to directly observe the displacement and crushing process of particles through images, resulting in the understanding of the displacement and crushing behavior of calcareous sand particles under different stresses is still unclear.

[0004] Convolutional neural network (CNN) is a deep learning model that performs well in tasks such as image and video recognition, classification and segmentation. It is widely used in image recognition, video analysis and medical image processing, and has advantages such as feature extraction and spatial hierarchical structure. When performing target recognition tasks, traditional convolutional neural networks rely on a large number of labeled data sets to train the model so that it can accurately identify and classify different targets. The lack of data sets directly affects the effect of model training, resulting in a decrease in the generalization ability of the model and a decrease in the model recognition accuracy. However, when it comes to uncommon targets such as calcareous sand, this process faces significant challenges. Due to the low frequency of these targets in the natural environment, the relevant image data collected is relatively limited, which makes it difficult to build a comprehensive and diverse dataset. At the same time, it takes a lot of time and resources to collect, clean and annotate data to produce a large number of datasets. Therefore, the training and recognition efficiency of traditional CNNs is greatly limited when dealing with uncommon targets in calcareous sand unit tests.

[0005] CNN performs well in the field of target recognition. However, when faced with calcareous sand with dense targets and uneven gradation distribution, the traditional CNN architecture often has difficulty adapting to such complex situations with dense targets and small size differences, resulting in reduced recognition performance. At the same time, when the particle size of calcareous sand particles is smaller than a certain range, CNN cannot recognize and segment the particles, resulting in the loss of particle displacement information, which hinders the study of the mechanical behavior of calcareous sand particles.

[0006] In view of the above problems, it is urgent to propose a digital image tracking and broken particle matching method suitable for calcareous sand unit test. Summary of the invention

[0007] In view of the shortcomings of the background technology, the technical problem to be solved by the present invention is to provide a method for tracking and matching crushed particles in a calcareous sand unit test based on digital images. The method can realize uneven gradation particle tracking and crushing matching through a CNN without training, and has extremely high robustness and generalization ability.

[0008] The present invention is completed by adopting the following technical scheme: a method for tracking and matching broken particles in a calcareous sand unit test based on digital images, the steps are as follows:

[0009] S1. Conducting a calcareous sand unit test and taking pictures of the calcareous sand unit test, and making the taken test pictures into two-dimensional pictures with three-dimensional information;

[0010] S2. Use SAM to perform instance segmentation on the two-dimensional image converted in S1;

[0011] S3. Improve the BoT-SORT algorithm to make it more suitable for tracking irregular particles;

[0012] S4. Use the improved BoT-SORT algorithm in S3 to track the instance segmentation results in S2 to obtain the displacement data of large-size calcareous sand particles;

[0013] S5. Use the PIV algorithm to track the parts other than the instance segmentation in S2 to obtain the displacement data of small-size calcareous sand particles;

[0014] S6. Use SURF to extract feature points from the instance segmentation and tracking results in S4, and use the KNN matching algorithm to match the extracted feature points to obtain particle crushing matching results;

[0015] S7. The instance segmentation result in S2, the large-size calcareous sand displacement data obtained in S4, the small-size calcareous sand displacement data obtained in S5, and the particle crushing matching result in S6 are made into a final result map.

[0016] Further, S1 includes the following specific steps:

[0017] S11. Conduct a unit test on calcareous sand, during which multiple CMOS high-speed cameras are used to capture the displacement and breakage of particles;

[0018] S12. Use Meshroom to convert multiple test images taken at the same time in S11 into obj files;

[0019] S13. Use Tutte to read the obj file, extract the top surface and face information, and then calculate the convex combination weight of each vertex based on the angle between adjacent vertices. Finally, based on the Floater weight or uniform weight, establish a linear equation to solve the coordinates of the internal vertices to obtain a two-dimensional image with three-dimensional information.

[0020] Furthermore, in S2, the two-dimensional image in S1 is input into the CNN model, and the Segment Anything Model (SAM) is used to perform instance segmentation on the coarse-grained soil particles in the two-dimensional image converted in S1 to obtain the surface feature information of the larger particles of calcareous sand and complete particle recognition.

[0021] Further, S3 includes the following specific steps:

[0022] S31. Add the calculation of Feret diameter to the BoT-SORT algorithm to calculate the particle size of each particle instance segmentation result in S2;

[0023] S32. Add the Feret diameter part to the Kalman filter in the BoT-SORT algorithm.

[0024] Furthermore, in S4, the instance segmentation result in S2 is put into the BoT-SORT algorithm. The BoT-SORT algorithm tracks the instance segmentation result of each frame to obtain the displacement of all large-size coarse-grained soil particles during the entire test process.

[0025] Further, S5 includes the following specific steps:

[0026] S51. Divide the area outside the instance segmentation in S2 in the image into a PIV tracking area;

[0027] S52. Gridding the PIV tracking area in S51;

[0028] S53. Performing cross-correlation operations between grids between adjacent frames by cross-correlation algorithm, and calculating the cross-correlation coefficient between adjacent frames;

[0029] S54. A mutual correlation coefficient threshold is set. When the mutual correlation coefficient is higher than the threshold, the grids are successfully matched, and the displacement of particles with smaller particle sizes is obtained through the displacement between the grids.

[0030] Further, S6 includes the following specific steps:

[0031] S61. Use SURF to extract feature points from the instance segmentation and tracking results in S4;

[0032] S62. Use the KNN matching algorithm to match the feature points of the broken particles extracted in S61 with the feature points of the unbroken particles in the previous frame, find out which original particle the broken particles are generated by, and obtain the particle breaking matching result.

[0033] Furthermore, in S7, the data were plotted on a picture using Python for display, and the tracking result graph covered the boundary coordinates of each large-size particle and its displacement vector, as well as the overall displacement of small particles.

[0034] Furthermore, the calcareous sand unit test in S11 uses a TPU transparent film instead of the rubber film of the traditional calcareous sand unit test.

[0035] Furthermore, it includes a calcareous sand unit test module, a particle recognition module, a particle tracking module, a particle crushing matching module and a result display module, the particle recognition module includes SAM recognition, the particle tracking module includes a BoT-SORT tracking module and a PIV tracking module, the particle crushing matching module includes SURF feature point extraction and KNN feature point matching, and the result display module draws out the uneven gradation tracking results and the calcareous sand particle crushing matching results through the displacement data.

[0036] Beneficial effects of the present invention:

[0037] 1. Transparent test. Because the rubber film is opaque and the test process cannot be observed during the traditional triaxial test, the TPU transparent film can replace the traditional rubber film to provide good visual clarity and protection, allowing the subsequent CMOS high-speed camera to take clear photos.

[0038] 2. Surface recognition. Since the specimen of the triaxial test is cylindrical, the test image taken is a surface image. By using Tutte's embedding algorithm to read the obj file, the three-dimensional point image of the obj format file is projected into a two-dimensional image with three-dimensional information. That is, the three-dimensional surface of the test image is unfolded into a two-dimensional image. Each pixel in the generated two-dimensional image has three-dimensional point information, and surface recognition is performed through the two-dimensional plane image.

[0039] 3. Full particle size recognition: first use CNN to identify large particle size particles, and then use PIV to assist in capturing smaller particles that are difficult for CNN to identify.

[0040] 4. When using SAM to perform instance segmentation on images, there is no need to conduct additional training on the model in advance to achieve good segmentation results.

[0041] 5. Uneven-graded particle tracking and fragmentation matching are achieved through untrained CNN, which has extremely high robustness and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a general flow chart of the method for tracking and matching crushed particles in a calcareous sand unit test based on digital images;

[0043] Figure 2 This is a KNN feature point matching effect diagram in S5 of the broken particle tracking and matching method of calcareous sand unit test based on digital image. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0045] Reference Figure 1-2 As shown, the present invention provides a digital image-based calcareous sand unit test crushed particle tracking and matching method, including a calcareous sand unit test module, a particle recognition module, a particle tracking module, a particle crushing matching module and a result display module. The calcareous sand unit test module includes taking photos during the calcareous sand unit test and performing image processing, the particle recognition module includes SAM recognition, the particle tracking module includes a BoT-SORT tracking module and a PIV tracking module, the particle crushing matching module includes SURF feature point extraction and KNN feature point matching, and the result display module draws out the uneven gradation tracking results and the calcareous sand particle crushing matching results through displacement data.

[0046] The specific steps of the above-mentioned digital image-based calcareous sand unit test crushed particle tracking and matching method are as follows:

[0047] S1. Conduct a conventional triaxial unit test on calcareous sand and take pictures of the calcareous sand unit test, and make the taken test pictures into two-dimensional pictures with three-dimensional information. Specifically, the steps include:

[0048] S11. During the calcareous sand unit test, multiple CMOS high-speed cameras were used to capture the displacement and crushing of particles. In particular, the calcareous sand unit test used a TPU transparent film instead of the rubber film used in the traditional calcareous sand unit test. The TPU transparent film has the characteristics of high transparency, good flexibility, strong wear resistance, and a wide temperature resistance range, and can provide good visual clarity and protection effects, so that subsequent CMOS high-speed cameras can capture clear photos. In particular, because the specimens of the triaxial test are cylindrical, the photos taken are curved surface images.

[0049] S12. Use Meshroom to convert multiple test images taken at the same time in S11 into obj files through 3D Coat.

[0050] S13. Use Tutte's embedding algorithm to read the obj file and project the three-dimensional point image of the obj format file into a two-dimensional image with three-dimensional information, that is, unfold the three-dimensional surface of the test image into a two-dimensional plane image with three-dimensional information.

[0051] Specifically, first input the obj format file into Tutte's embedding algorithm to find the boundary vertices and fix them on the two-dimensional plane, extract the top surface and patch information, and map these vertices to a unit circle through geometric methods; then calculate the convex combination weight of each vertex based on the angle between adjacent vertices; finally, based on the floater weight or uniform weight, establish a linear equation to solve the coordinates of the internal vertices, and finally convert the processed image into a jpg file to obtain a two-dimensional image with three-dimensional information. The two-dimensional image with three-dimensional information means that each pixel in the generated two-dimensional image has three-dimensional point information.

[0052] The calculation formula for Floater weight is as follows (1)(2):

[0053]

[0054] Among them, weights i is the weight of vertex i, θ i is the angle between vertex i and its neighbors. Then, the contribution of each neighbor vertex to the current vertex can be calculated as follows:

[0055]

[0056] Among them, weights ij is the weight of the jth neighbor of vertex i, B ij is the position of the jth neighbor of vertex i.

[0057] The calculation formula for uniform weight is as follows (3):

[0058]

[0059] Among them, A i is the position of vertex i, B ij is the position of the jth neighbor of vertex i, and n is the number of neighbors.

[0060] These weights are used to construct a linear system of equations, which are then solved to find the 2D coordinates of the internal vertices. This process involves constructing a linear system Ax=b, where A is the weight matrix, x is the vector of unknowns (the coordinates of the internal vertices), and b is the known boundary condition vector. Solving this linear system yields the coordinates of the internal vertices.

[0061] S2. Input the two-dimensional image in S1 into the CNN model and use SAM to perform instance segmentation on the two-dimensional image in S1.

[0062] The Segment Anything Model (SAM) was used to perform instance segmentation on the coarse-grained soil particles in the two-dimensional image converted in S1 to obtain the surface feature information of the larger calcareous sand particles.

[0063] Specifically, SAM adopts a modular architecture and is mainly composed of three parts: image encoder, prompt encoder and mask decoder. The image encoder is based on the visual Transformer (ViT) and is responsible for extracting high-dimensional feature representations of the input image to capture global and local information. The prompt encoder receives user prompts in various forms and converts these prompts into a format that the model can understand to guide the segmentation process. The mask decoder combines the features extracted by the image encoder and the information provided by the prompt encoder to generate an accurate segmentation mask to determine the boundaries of the target objects in the image. Through this collaborative work, SAM can achieve efficient, flexible and accurate image segmentation in various complex scenarios, with good generalization ability and real-time processing performance. And when using SAM to perform instance segmentation on images, there is no need to conduct additional training on the model in advance.

[0064] Instance segmentation refers to an advanced image segmentation technology that, based on giving the category label of each pixel like semantic segmentation, further distinguishes the pixel areas of different individuals in the same category, that is, generates a unique mask for each independent object. Instance segmentation not only identifies objects of each category in the image, but also needs to clarify the precise position and outline of each independent individual, so that multiple similar objects can be finely distinguished in the image.

[0065] S3. Improve the BoT-SORT algorithm to make it more suitable for tracking irregular particles. The specific steps are as follows:

[0066] S31. Add the calculation of Feret diameter to the BoT-SORT algorithm to calculate the particle size of each particle instance segmentation result in S2;

[0067] S32. Add the Feret diameter part to the Kalman filter in the BoT-SORT algorithm.

[0068] Specifically, Feret diameter is a parameter widely used to describe the geometric size of particles and objects. It determines the size characteristics by measuring the maximum distance between the edges of objects in multiple directions. First, select several measurement directions, then draw two tangents parallel to the direction in each direction, and measure the vertical distance between the two tangents, which is the Feret diameter in that direction. By traversing all directions of 360 degrees, statistical indicators such as the maximum Feret diameter and the minimum Feret diameter can be obtained, thereby comprehensively evaluating the shape and size distribution of the object.

[0069] Specifically, the BoT-SORT algorithm predicts the trajectory of the recognition box based on the 8-dimensional Kalman filter. After applying the Kalman filter to predict the particle trajectory, the predicted trajectory is first preliminarily matched with the recognition box with high confidence. Subsequently, the successfully matched pairs are screened and output based on the surface texture features. For the trajectories that failed to successfully match in the first match, they are further matched with the recognition box with low confidence. Finally, the trajectories that have not been matched will be directly eliminated.

[0070] S4. Large particle size tracking.

[0071] The improved BoT-SORT algorithm in S3 is used to track the instance segmentation results in S2 to obtain the displacement data of large-size calcareous sand particles. Specifically, the instance segmentation results in S2 are put into the BoT-SORT algorithm in S3. The BoT-SORT algorithm tracks the instance segmentation results of each frame to obtain the displacement of all large-size coarse-grained soil particles during the entire test.

[0072] S5. Small particle tracking.

[0073] Use the PIV algorithm to track the parts outside the instance segmentation in S2 to obtain the displacement data of small-size calcareous sand. The specific steps are as follows:

[0074] S51. Divide the area outside the instance segmentation result range of S2 in the image into a PIV tracking area.

[0075] S52. Divide the PIV tracking area in S51 into multiple tracking grids, mark the IDs, and then calculate the grayscale distribution map in each grid.

[0076] S53. Performing cross-correlation operations between grids between adjacent frames by cross-correlation algorithm, and calculating the cross-correlation coefficient between adjacent frames;

[0077] S54. A mutual correlation coefficient threshold is set according to actual conditions. When the mutual correlation coefficient is higher than the threshold, the grids are matched successfully, and the displacement of the particles with smaller particle sizes is obtained through the displacement between the grids. Specifically, after the match is successful, the grid will inherit the ID of the previous grid, and the position of each grid in different frames is calculated through the ID, so as to obtain the real-time displacement of each grid, and then obtain the overall displacement of the small particles.

[0078] S6. Particle crushing matching.

[0079] SURF is used to extract feature points from the instance segmentation and tracking results in S4, and the KNN matching algorithm is used to match the extracted feature points to obtain the particle crushing matching results. The specific steps include the following:

[0080] S61. Use SURF to extract feature points from the instance segmentation and tracking results in S4. SURF detects feature points in an image through the Hessian matrix, and obtains a 64-dimensional feature vector based on its grayscale gradient. It has high robustness and can maintain good matching effects under rotation, scale changes, and illumination changes. Specifically, SURF detects feature points in an image through the Hessian matrix, and obtains a 64-dimensional feature vector based on its grayscale gradient. It has high robustness and can maintain good matching effects under rotation, scale changes, and illumination changes.

[0081] S62. Use the KNN matching algorithm to match the feature points of the broken particles extracted in S61 with the feature points of the unbroken particles in the previous frame, find out which original particle the broken particles are generated by, and obtain the particle breaking matching result. Specifically, the KNN algorithm calculates the correlation between the 64-dimensional feature points detected by SURF through the Euclidean distance, then selects the nearest K training samples as neighbors, and finally matches the feature points through classification and regression.

[0082] S7. Results presentation.

[0083] The instance segmentation results in S2, the large-size calcareous sand displacement data obtained in S4, the small-size calcareous sand displacement data obtained in S5, and the particle crushing matching results in S6 are made into the final result map.

[0084] The data is plotted on a picture using Python. The tracking result graph covers the boundary coordinates and displacement vector of each large-size particle, as well as the overall displacement of small particles.

[0085] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for tracking and matching broken particles in a calcareous sand unit test based on digital images, characterized by: The steps are as follows, S1. Conducting a calcareous sand unit test and taking pictures of the calcareous sand unit test, and making the taken test pictures into two-dimensional pictures with three-dimensional information; S2. Use SAM to perform instance segmentation on the two-dimensional image converted in S1; S3. Improve the BoT-SORT algorithm; S4. Use the improved BoT-SORT algorithm in S3 to track the instance segmentation results in S2 to obtain the displacement data of large-size calcareous sand particles; S5. Use the PIV algorithm to track the parts other than the instance segmentation in S2 to obtain the displacement data of small-size calcareous sand particles; S6. Use SURF to extract feature points from the instance segmentation and tracking results in S4, and use the KNN matching algorithm to match the extracted feature points to obtain particle crushing matching results; S7. The instance segmentation result in S2, the large-size calcareous sand displacement data obtained in S4, the small-size calcareous sand displacement data obtained in S5, and the particle crushing matching result in S6 are made into a final result map.

2. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1 is characterized by: S1 includes the following specific steps: S11. Conduct a unit test on calcareous sand, during which multiple CMOS high-speed cameras are used to capture the displacement and breakage of particles; S12. Use Meshroom to convert multiple test images taken at the same time in S11 into obj files; S13. Use Tutte to read the obj file, extract the top surface and face information, and then calculate the convex combination weight of each vertex based on the angle between adjacent vertices. Finally, based on the Floater weight or uniform weight, establish a linear equation to solve the coordinates of the internal vertices to obtain a two-dimensional image with three-dimensional information.

3. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1, characterized in that: S2 In the CNN model, the two-dimensional image in S1 is input into the CNN model, and the Segment Anything Model (SAM) is used to perform instance segmentation on the coarse-grained soil particles in the two-dimensional image converted in S1 to obtain the surface feature information of the larger particles of calcareous sand.

4. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1 is characterized by: S3 includes the following specific steps: S31. Add the calculation of Feret diameter to the BoT-SORT algorithm to calculate the particle size of each particle instance segmentation result in S2; S32. Add the Feret diameter part to the Kalman filter in the BoT-SORT algorithm.

5. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1 is characterized by: In S4, the instance segmentation result in S2 is put into the BoT-SORT algorithm. The BoT-SORT algorithm tracks the instance segmentation result of each frame to obtain the displacement of all large-size coarse-grained soil particles during the entire test process.

6. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 5 is characterized by: S5 includes the following specific steps: S51. Divide the area outside the instance segmentation in S2 in the image into a PIV tracking area; S52. Gridding the PIV tracking area in S51; S53. Performing cross-correlation operations between grids between adjacent frames by cross-correlation algorithm, and calculating the cross-correlation coefficient between adjacent frames; S54. A mutual correlation coefficient threshold is set. When the mutual correlation coefficient is higher than the threshold, the grids are successfully matched, and the displacement of particles with smaller particle sizes is obtained through the displacement between the grids.

7. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1 is characterized by: S6 includes the following specific steps: S61. Use SURF to extract feature points from the instance segmentation and tracking results in S4; S62. Use the KNN matching algorithm to match the feature points of the broken particles extracted in S61 with the feature points of the unbroken particles in the previous frame to obtain a particle breaking matching result.

8. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1 is characterized by: In S7, the data are plotted on a picture using Python for display. The tracking result graph includes the boundary coordinates of each large-size particle and its displacement vector, as well as the overall displacement of small particles.

9. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 2 is characterized by: In the S11 calcareous sand unit test, a TPU transparent film was used to replace the rubber film used in the traditional calcareous sand unit test.

10. The method for tracking and matching broken particles in a calcareous sand unit test based on digital images according to claim 1 is characterized by: It includes a calcareous sand unit test module, a particle recognition module, a particle tracking module, a particle crushing matching module and a result display module. The particle recognition module includes SAM recognition, the particle tracking module includes a BoT-SORT tracking module and a PIV tracking module, the particle crushing matching module includes SURF feature point extraction and KNN feature point matching, and the result display module draws out the uneven gradation tracking results and the calcareous sand particle crushing matching results through displacement data.

Images