Particle tracking method for coarse-grained soil crushing based on YOLOv8 network model

Abstract

The present invention relates to a coarse-grained soil crushing particle tracking method based on a YOLOv8 network model, comprising S1. making an initial data set; S2. expanding the data set; S3. performing image processing on the image of the data set after data enhancement; S4. training to complete a segmentation model; S5. performing instance segmentation of an experimental image; S6. extracting feature points of the experimental image using SURF; S7. matching the instance segmentation area in S5 with the feature points extracted by SURF in S6, and matching the feature points to obtain a tracking result of feature point matching; S8. performing multi-target tracking to obtain a tracking result obtained based on YOLOv8; S9. comparing the tracking result of S7 with the tracking result of S8, and outputting the tracking result. The method realizes high-precision tracking of crushed particles in a coarse-grained soil crushing test.

Description

Technical Field

[0001] The present invention relates to the technical field of coarse-grained soil crushing particle tracking, and in particular to a coarse-grained soil crushing particle tracking method based on a YOLOv8 network model. Background Art

[0002] Coarse-grained soil is a common special soil, and its physical and mechanical properties have attracted much attention from scholars at home and abroad. Coarse-grained soil contains a large number of brittle rock particles. According to statistics, the tensile strength of brittle rock materials is much smaller than the compressive strength and shear strength. Therefore, coarse-grained soil particles are more prone to tensile failure. Existing research mainly reveals the particle crushing mechanism of coarse-grained soil through unit body tests and other methods, but such methods cannot observe the changes in particle morphology and the displacement of broken particles, resulting in unclear displacement rules of broken particles.

[0003] Instance segmentation is one of the classic visual tasks. It has the characteristics of semantic segmentation, which requires pixel-level classification, and also has some characteristics of target detection, which requires locating different instances and segmenting them. Common instance segmentation neural networks include Mask-RCNN, PANNet, Deep Mask, YOLOv8, etc. YOLOv8 is the latest version of the YOLO series of target detection algorithms. YOLOv8 inherits the advantages of the YOLO series of algorithms. Based on the above problems and the characteristics of YOLOv8, a coarse-grained soil crushing particle tracking method based on the YOLOv8 network model is designed.

[0004] A lot of optimization and innovation have been carried out on this basis, which makes it efficient and real-time, and makes it possible to realize particle tracking of coarse-grained soil crushing. Summary of the invention

[0005] In view of the shortcomings of the background technology, the technical problem to be solved by the present invention is to provide a coarse-grained soil crushing particle tracking method based on the YOLOv8 network model. The method improves the speed and accuracy of coarse-grained soil segmentation, reduces the error rate of tracking, and improves the tracking accuracy.

[0006] The present invention is completed by adopting the following technical scheme: a coarse-grained soil crushing particle tracking method based on the YOLOv8 network model, the steps are as follows:

[0007] S1. Photograph and obtain the original dataset of coarse-grained soil crushed particles, perform labeling processing, and make it into an initial dataset;

[0008] S2. Perform data augmentation on the images in the initial data set to expand the data set;

[0009] S3. performing image processing on the images of the data set after data augmentation;

[0010] S4. The image processed dataset is converted into the YOLOv8 input tag format and trained to complete the segmentation model;

[0011] S5. Input the experimental image into the trained segmentation model to perform instance segmentation of the experimental image;

[0012] S6. Use SURF to extract feature points of the experimental image;

[0013] S7. Correspond the instance segmentation area in S5 to the feature points extracted by SURF in S6, and match the feature points to obtain a tracking result of feature point matching;

[0014] S8. Input the coarse-grained soil crushing experiment video into the MOT algorithm of YOLOv8 to perform multi-target tracking and obtain the tracking result based on YOLOv8;

[0015] S9. Compare the tracking results of S7 and S8, delete the inferior tracking tracks, retain the matching pairs with the same tracks, and output the tracking results.

[0016] Furthermore, the original dataset of crushed particles of coarse-grained soil is obtained in S1, and the AI ​​labeling function in CVAT is used to label the pictures in the video, and the poor-quality labels are manually deleted to produce what is called the initial dataset.

[0017] Furthermore, in S2, the images of the initial data set obtained in S1 are rotated 90°, 180°, and 270° clockwise and flipped left and right and up and down to increase the number of images and expand the data set.

[0018] Furthermore, in S3, after data enhancement and expansion, the data set is subjected to noise reduction processing by Gaussian filtering, and an opening operation of first corrosion and then expansion is performed, so that the edges of the marked images in the data set are smoother, and the image processing is completed.

[0019] Furthermore, in S4, the labeled file output by CVAT is in png format, and then the labeled png file is converted into a json file through code, and then the json file is converted into a YOLOv8 input labeling format, and the original image and the labeled file name are corresponded to each other to form a YOLOv8 instance segmentation input file, and then trained through the YOLOv8m-seg model to complete the segmentation model.

[0020] Furthermore, in S5, a video of a coarse-grained soil crushing experiment is filmed, the video of the coarse-grained soil crushing experiment is converted into a plurality of experimental pictures, and the experimental pictures are input into a trained segmentation model to segment the experimental pictures and obtain segmentation results.

[0021] Furthermore, before extracting SURF feature points in S6, a ROI region is set for the experimental image, and feature points are identified for the image in the ROI region to reduce the number of irrelevant feature points, while improving the contrast of the image and increasing the number of feature points extracted from the ROI region.

[0022] Furthermore, in S7, the instance segmentation result of the experimental image in S5 is used as a cluster of feature points. The feature points in the same segmentation area belong to the same cluster. The feature points extracted by SURF in S6 are matched. If the feature points in a cluster in the previous frame match the feature points in multiple different clusters in this frame, it is determined that the coarse-grained soil particles are broken, and their breakage conditions are analyzed.

[0023] Furthermore, the YOLOv8m-seg model parameters include that the loss function in the model adopts IOU, and an attention mechanism is added. During model training, 300 rounds of training are selected, and the batch size is 4.

[0024] The effects of the present invention are as follows:

[0025] (1) The present invention uses YOLOv8 to perform instance segmentation on coarse-grained soil. Due to the high efficiency of YOLOv8 instance segmentation, the speed and accuracy of segmentation are greatly improved.

[0026] (2) In the SURF feature point matching process, conventional methods cannot accurately cluster the feature points. This paper performs instance segmentation on the original image and regards each instance segmentation area as a cluster of feature points, so that the feature points can be accurately clustered.

[0027] (3) The SURF feature point matching method and multi-target detection method were used to track coarse-grained soil, which reduced the tracking error rate and improved the tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of a coarse-grained soil crushing particle tracking method based on the YOLOv8 network model;

[0029] Figure 2 is a schematic diagram of the original data set enhancement of the present invention;

[0030] Figure 3 It is the training and test labeling graph and YOLOv8 labeling file of the present invention. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0032] Reference Figure 1-3 As shown, the present invention provides a coarse-grained soil crushing particle tracking method based on the YOLOv8 network model, and the steps are as follows:

[0033] S1. Photograph and obtain the original dataset of coarse-grained soil crushed particles, perform labeling processing, and make it into the initial dataset.

[0034] Specifically, different coarse-grained soils are placed on a white tray, and a CMOS camera is used to collect the original data set of coarse-grained soil. For example, 100 pictures of coarse-grained soil are taken as the original data set. Then, according to the density of the particles, they are divided into four situations: sparse, relatively sparse, relatively dense, and dense. The picture ratios of the four situations are 3:2:2:1. The AI ​​marking function in CVAT is used to mark the original data of coarse-grained soil. Then, the inferior marks are manually screened out, deleted, and manually re-marked to create the initial data set.

[0035] S2. Perform data augmentation on the images in the initial dataset to expand the dataset.

[0036] Specifically, the images of the initial data set obtained in S1 are rotated 90°, 180°, 270° clockwise and flipped left and right and up and down to increase the number of images and expand the data set. For example, 100 images in the original data set are enhanced to 800 images, and the enhanced data set is put into the training set, test set, and validation set in a ratio of 7:2:1.

[0037] S3. Perform image processing on the images of the dataset after data augmentation.

[0038] Specifically, after data augmentation and expansion, the dataset is subjected to denoising by Gaussian filtering, and an opening operation of first erosion and then dilation is performed to improve the edges of the labeled images in the dataset, thus completing the image processing. The opening operation of first erosion and then dilation can make the intervals between particles more obvious, which is helpful for model learning and subsequent segmentation.

[0039] S4. The image-processed dataset is converted into the YOLOv8 input tag format, and the original image and the tag file name are mapped to each other to form a YOLOv8 instance segmentation input file, which is then trained through the YOLOv8m-seg model to complete the segmentation model.

[0040] Since CVAT cannot be directly exported to the YOLOv8 instance segmentation dataset format after annotation, it is necessary to first convert the CVAT output tag file to png format, then convert the tag png file to json file through code, and then convert the json file to YOLOv8 input tag format.

[0041] The YOLOv8m-seg model parameters include that the loss function in the model adopts IOU, and the attention mechanism is added. When training the model, 300 rounds of training are selected, the batch size is 4, and the identification type is set to 1. The training speed of YOLOv8m-seg is fast, and the segmentation accuracy meets the experimental requirements. YOLOv8m-seg is a better model in the recognition of coarse-grained soil. Due to the high efficiency of YOLOv8 instance segmentation, the speed and accuracy of segmentation are greatly improved.

[0042] S5. Input the experimental image into the trained segmentation model, perform instance segmentation of the experimental image, and obtain the segmentation result.

[0043] Specifically, a coarse-grained soil crushing experiment video is shot and converted into multiple experimental pictures. The experimental pictures are input into the trained segmentation model, each coarse-grained soil experimental picture is segmented, and the segmentation results are obtained.

[0044] S6. Use the SURF computer vision algorithm to extract feature points from the coarse-grained soil experimental images.

[0045] Before extracting SURF feature points, set an ROI area for the experimental image and perform feature point recognition on the image within the ROI area to reduce the number of irrelevant feature points, improve the contrast of the image, increase the number of feature points extracted from the ROI area, and then perform SURF feature point extraction.

[0046] S7. Match the instance segmentation area in S5 with the feature points extracted by SURF in S6, use SURF to match the feature points in the two frames, obtain the crushing and displacement of the coarse-grained soil, and obtain the tracking result of the feature point matching.

[0047] Specifically, the instance segmentation results of the experimental image in S5 are used as clusters of feature points. Feature points in the same segmentation area belong to the same cluster, and the identified feature points are classified and marked, and the marks in the same cluster have the same number. Then the feature points extracted by SURF in S6 are matched, and after the matching is completed, the marks of the previous frame are transferred to this frame. If the feature points in a cluster of the previous frame match the feature points in multiple different clusters of this frame, it is determined that the coarse-grained soil particles are broken, and the breakage is analyzed. The particles with fewer feature points after breaking are renumbered, and the particles with more feature points after breaking retain the original numbers. By performing instance segmentation on the original image and treating each instance segmentation area as a cluster of feature points, the feature points can be accurately clustered, which solves the problem that conventional methods cannot accurately cluster feature points.

[0048] In the process of matching feature points using SURF, the pixel displacement in the image between the two frames before and after the matching point is calculated using the Euclidean distance, and a threshold is set. When the Euclidean distance exceeds the threshold, the matching pair is deleted to reduce the probability of false matching.

[0049] S8. Input the coarse-grained soil crushing experiment video into the MOT algorithm of YOLOv8, perform multi-target tracking, obtain the crushing and displacement of the coarse-grained soil, and obtain the tracking result based on YOLOv8;

[0050] S9. Compare the tracking results of S7 and S8, that is, compare the crushing and displacement of coarse-grained soil obtained by S7 and S8. If the tracking results of the two methods are the same, the tracking results are directly output. If the tracking results of the two methods are different, the particles with too large differences in the recognition results of the two recognition methods are deleted, that is, the inferior tracking tracks are deleted, and the matching pairs with the same tracks are retained, and the tracking results are output. Since multi-target tracking often results in incorrect tracking, analyzing the crushing and displacement of particles by two different methods will improve the accuracy of recognition.

[0051] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A coarse-grained soil crushing particle tracking method based on the YOLOv8 network model, characterized by: The steps are as follows, S1. Photograph and obtain the original dataset of coarse-grained soil crushed particles, perform labeling processing, and make it into an initial dataset; S2. Perform data augmentation on the images in the initial data set to expand the data set; S3. performing image processing on the images of the data set after data augmentation; S4. The image processed dataset is converted into the YOLOv8 input tag format and trained to complete the segmentation model; S5. Input the experimental image into the trained segmentation model to perform instance segmentation of the experimental image; S6. Use SURF to extract feature points of the experimental image; S7. Correspond the instance segmentation area in S5 with the feature points extracted by SURF in S6, and match the feature points to obtain the tracking result of feature point matching. The instance segmentation results of the experimental image in S5 are used as clusters of feature points. Feature points in the same segmented area belong to the same cluster. The identified feature points are classified and marked. The marks in the same cluster have the same number. They are then matched with the feature points extracted by SURF in S6. After the matching is completed, the marks of the previous frame are transferred to this frame. If the feature points in a cluster of the previous frame are matched to the feature points in multiple different clusters of this frame, it is determined that the coarse-grained soil particles are broken, and their breakage conditions are analyzed. S8. Input the coarse-grained soil crushing experiment video into the MOT algorithm of YOLOv8 to perform multi-target tracking and obtain the tracking result based on YOLOv8; S9. Compare the tracking results of S7 and S8, delete the inferior tracking tracks, retain the matching pairs with the same tracks, and output the tracking results.

2. The coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 1 is characterized by: In S1, the original dataset of coarse-grained soil crushing particles is obtained, and the AI ​​labeling function in CVAT is used to label the pictures in the video, and the poor-quality labels are manually deleted to produce the initial dataset.

3. A coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 1 or 2, characterized in that: S2 In the method, the images of the initial data set obtained in S1 are rotated 90°, 180°, 270° clockwise and flipped left and right and up and down to increase the number of images and expand the data set.

4. The coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 3 is characterized in that: S3 In the data augmentation and expansion, the dataset is subjected to denoising by Gaussian filtering, and an opening operation of first corrosion and then expansion is performed to make the edges of the marked images in the dataset smoother, thus completing the image processing.

5. A coarse-grained soil broken particle tracking method based on the YOLOv8 network model according to claim 4, characterized in that: in S4, since the marked file output by CVAT is in png format, it is necessary to first convert the output marked file format into the YOLOv8 input file format, and make the original image and the marked file name correspond to each other to form a YOLOv8 instance segmentation input file, and then train it through the YOLOv8m-seg model to complete the segmentation model.

6. The coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 5 is characterized in that: S5 In the process, a coarse-grained soil crushing experiment video is shot, and the coarse-grained soil crushing experiment video is converted into multiple coarse-grained soil experiment pictures. The experimental pictures are input into the trained segmentation model, the experimental pictures are segmented, and the segmentation results are obtained.

7. The coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 4, 5 or 6, characterized in that: S6 Before extracting SURF feature points, a ROI area is set for the experimental image, and feature points are identified in the image within the ROI area to reduce the number of irrelevant feature points, improve the contrast of the image, and increase the number of feature points extracted from the ROI area.

8. The coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 5 is characterized by: The YOLOv8m-seg model parameters include that the loss function in the model adopts IOU, and an attention mechanism is added. During model training, 300 rounds of training are selected and the batch size is 4.

9. The coarse-grained soil crushing particle tracking method based on the YOLOv8 network model according to claim 5 is characterized in that: The label file output by CVAT is in png format, and then the label png file is converted into json file through code, and then the json file is converted into YOLOv8 input label format.

Images