A particle tracking method for coarse-grained soil crushing based on improved YOLOv8
By improving the YOLOv8 and strongsort algorithms, combining data enhancement and attention mechanisms, the efficiency and accuracy problems of coarse-grained soil particle crushing identification and tracking are solved, and real-time observation and accurate tracking of coarse-grained soil particle crushing are achieved.
Patent Information
- Application Number
- CN202410559837.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-08
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2044-05-08
AI Technical Summary
The existing image processing technology takes a long time and has low accuracy when identifying and segmenting coarse-grained soil, and cannot accurately and efficiently observe the particle crushing process, resulting in unclear particle crushing mechanism and evolution laws.
Using the improved YOLOv8 coarse soil crushing particle tracking method, the video frame image is obtained through the difference method, data labeling and enhancement, the YOLOv8 model and strongsort algorithm are optimized, attention mechanism and crushing tracking mechanism are added, and multi-objective tracking and morphology are achieved. Feature matching, output motion trajectory and particle displacement.
Real-time observation of the crushing displacement and morphology of coarse soil particles is achieved, which improves model identification accuracy and tracking accuracy, and reduces data preparation time.
Smart Images
Figure CN118470062B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of coarse-grained soil crushing particle tracking, and in particular to a coarse-grained soil crushing particle tracking method based on improved YOLOv8. Background Art
[0002] Coarse-grained soil is a general term for gravel soil and sand and gravel, and refers to soil with a high content of particles with a diameter greater than 0.1 mm. Coarse-grained soil contains a large number of brittle rock particles, which are easily broken under the action of external forces, resulting in changes in mechanical properties and affecting the safe operation of structures. Therefore, revealing the particle crushing mechanism and its development law is the key to further exploring the changes in the physical and mechanical properties of coarse-grained soil. Although image processing technology can be used to explore the crushing mechanism, the existing image processing technology has the disadvantages of long time consumption, low precision, and complex debugging when identifying and segmenting coarse-grained soil. It is impossible to accurately and efficiently observe the particle crushing during the crushing process of coarse-grained soil, resulting in unclear particle crushing evolution law and insufficient understanding of the particle crushing mechanism.
[0003] Target tracking technology has become a research hotspot in the field of computer vision. Target tracking technology can identify and track monitoring targets in images or videos, and has been widely used in engineering practice. Common target detection neural network models include R-CNN series, YOLO series, SSD, RetinaNet, EfficientDet, MTCNN, etc. YOLOv8 is a neural network model in the YOLO series, and YOLOv8 inherits the advantages of the YOLO series algorithms. In response to the above problems, this application proposes a coarse-grained soil crushing particle tracking method based on improved YOLOv8. Summary of the invention
[0004] In view of the shortcomings of the background technology, the technical problem to be solved by the present invention is to provide a coarse-grained soil crushing particle tracking method based on improved YOLOv8. The method realizes real-time observation of coarse-grained soil particle crushing displacement and morphology.
[0005] The present invention is completed by adopting the following technical scheme: a coarse-grained soil crushing detection method based on improved YOLOv8, the steps are as follows:
[0006] S1. Obtain the video of the coarse-grained soil crushing experiment;
[0007] S2. Use the difference method to find multiple frames of different images in the coarse-grained soil crushing particle video as the original data set, mark the original data set, and then perform data enhancement before and after marking;
[0008] S3. Convert the format of the labeled and data-augmented images in S2;
[0009] S4. Create a dataset by converting the converted files in S3 and the unlabeled images after data enhancement in S2;
[0010] S5. Optimize the YOLOv8 model, change the built-in loss function in YOLOv8 to the EIOU loss function, and add an attention mechanism in the YOLOv8 model;
[0011] S6. Input the prepared data set into the optimized YOLOv8 model for training to obtain a trained prediction model;
[0012] S7. Optimize the strongsort algorithm, optimize the shape of the Kalman filter prediction box in the strongsort, and then perform multi-target tracking, and add a fragmented tracking mechanism in the multi-target tracking process;
[0013] S8. Perform multi-target tracking on the experimental video, obtain tracking data, and finally output the motion trajectory;
[0014] S9. The obtained tracking data is converted from pixel coordinates to actual coordinates by converting the motion trajectory coordinate axes obtained in S8, and the actual particle displacement and broken particle situation of the coarse-grained soil are calculated.
[0015] Furthermore, in S2, a picture is selected every few frames in the captured experimental video as the original data set through the difference method, and the images of different frames are marked using the AI marking function in CVAT. The images of the original data set and the marked images are rotated 90°, 180°, 270°, and flipped left and right and up and down, and other data enhancement operations are performed.
[0016] Furthermore, in S3, the data set after the data augmentation operation is formatted, and the CVAT output format is converted to the YOLOv8 input format.
[0017] Furthermore, in S4, the unlabeled images after data enhancement obtained in S2 and the labeled files converted in S3 are divided into three parts: training set, test set, and validation set in a ratio of 7:2:1.
[0018] Furthermore, in S5, a CBAM attention mechanism module is added above the SPPF module in the backbone part of YOLOv8, and the YOLOv8 model optimization is completed.
[0019] Furthermore, in S6, the data set prepared in S4 is input into the YOLOv8 training model optimized in S5 for training to obtain a trained prediction model.
[0020] Furthermore, in S7, the Kalman filter in strongsort is optimized, and the rectangular recognition frame of the Kalman filter is changed to a polygonal recognition frame for multi-target tracking.
[0021] Furthermore, in S7, the matching process in strongsort is added with fragmentation matching, and the fragmentation particles of coarse-grained soil are traced and matched by obtaining morphological features, and then the Euclidean distance is added as the threshold.
[0022] Furthermore, in S8, the image segmentation is first performed using the training model in S6, and then the coarse-grained soil crushing particle video is multi-target tracked using the strongsort model modified in S7, and finally the motion trajectory is output, and the tracking result is output as a file in the form of data for storage.
[0023] Furthermore, the image features of coarse-grained soil are obtained through strongsort, and the appearance features of the surface of coarse-grained soil particles are extracted, and then matched through the image features in the next frame of image. If the coarse-grained soil particles are broken, the broken coarse-grained soil can be traced through the image features, and a threshold is added according to the Euclidean distance to improve the accuracy of tracing. Each time the coarse-grained soil is broken, its ID increases by one dimension.
[0024] The effects of the present invention are as follows:
[0025] (1) The present invention uses the images in the experiment as the training set, which can reduce a lot of time for making the data set and improve the segmentation accuracy of the model for the experimental images.
[0026] (2) The present invention optimizes the YOLOv8 model to improve the model's recognition of coarse-grained soil.
[0027] (3) The present invention optimizes the Kalman filter in strongsort, changes the rectangular frame of the Kalman filter into a polygonal frame, adds morphological features to each identified coarse-grained image, and improves the accuracy of the Kalman filter prediction trajectory.
[0028] (4) The present invention traces the broken coarse-grained soil by extracting image features in strongsort, thereby achieving the purpose of tracking the broken particles of the coarse-grained soil. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a flow chart of a coarse-grained soil crushing detection method based on improved YOLOv8;
[0030] Figure 2 It is a schematic diagram of the original data set enhancement of the coarse-grained soil crushing detection method based on the improved YOLOv8;
[0031] Figure 3 It is a schematic diagram of the dataset folder of the coarse-grained soil crushing detection method based on the improved YOLOv8;
[0032] Figure 4 It is a Kalman filter polygonal block diagram of strongsort in the coarse-grained soil crushing detection method based on improved YOLOv8; DETAILED DESCRIPTION
[0033] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.
[0034] Reference Figure 1-4 As shown, the present invention provides a coarse-grained soil crushing detection method based on improved YOLOv8, the steps are as follows:
[0035] S1. Obtain video of coarse-grained soil crushing particles.
[0036] The coarse-grained soil particles were placed in an acrylic box for a confined compression test, and the experimental video was shot through a transparent observation window. Specifically, a CMOS high-speed camera was used to shoot a video of the coarse-grained soil crushing particles through the transparent observation window, and the shooting frame rate was 10fps.
[0037] S2. Use the difference method to find multiple frames of different images in the coarse-grained soil crushing particle video as the original data set, mark the original data set, and then perform data enhancement before and after marking.
[0038] Specifically, first use the difference method to select a picture every few frames in the captured experimental video as the original data set (for example, select one picture every 5 frames to make a data set). Use the AI marking function in CVAT to mark the images in different frames, and manually select the poor quality annotations for deletion. Then perform data enhancement operations such as rotating the images of the original data set and the marked images by 90°, 180°, 270°, and flipping them left and right and up and down to increase the number of images to achieve the purpose of expanding the data set. Use CVAT to annotate images. The AI annotation function in CVAT can greatly reduce the annotation time to achieve the purpose of reducing time costs.
[0039] S3. Convert the format of the images that have been labeled and data augmented in S2, convert the CVAT output format to the YOLOv8 input format, and obtain the converted labeled file. Specifically, convert the labeled data in the augmented data set from the png format exported by CVAT to the json format through the code, and then convert the json format to the YOLOv8 instance segmentation input txt file format.
[0040] S4. Create a dataset by converting the converted files in S3 and the data-augmented but unlabeled images in S2.
[0041] Specifically, the original images (unlabeled) after data enhancement in S2 and the txt labeled files processed by S3 are divided into three parts: training set (train), test set (test), and validation set (valid) in a ratio of 7:2:1. The data sets in the three folders are used for training, testing, and validation respectively. The train, test, and valid folders are composed of images and labels folders respectively. The experimental images are placed in the images folder, and the labeled txt files are placed in the labels folder.
[0042] S5. Optimize the YOLOv8 model, change the built-in loss function in YOLOv8 to the EIOU loss function, and add an attention mechanism in the YOLOv8 model.
[0043] Specifically, first change the CIOU loss function built into YOLOv8 to the EIOU loss function (add the EIOU module in the metrics.py file and set it to True). EIOU separates the aspect ratio of the true box and the predicted box based on CIOU, and calculates the length and width of the true box and the predicted box respectively, solving the problem that the length and width of the true box and the predicted box cannot be increased or decreased at the same time when using CIOU as the loss function, thereby improving the recognition accuracy of coarse-grained soil. The specific calculation method is as follows:
[0044]
[0045]
[0046] Where b is the center point of the prediction box; b gt is the center point of the real box; w is the width of the predicted box; h is the height of the predicted box; w gt is the width of the real frame; h gt is the height of the real frame; ρ represents the Euclidean distance between two points; w c The width of the smallest outer frame covering the two frames; h c is the height of the minimum bounding box covering the two boxes; A is the predicted box; B is the real box.
[0047] Then add the CBAM attention mechanism module above the SPPF module in the backbone part of YOLOv8, and the YOLOv8 model optimization is completed. The CBAM attention mechanism module improves the perception ability of the model through the channel attention mechanism and the spatial attention mechanism to solve the problem of recognition accuracy. The optimized YOLOv8 model is named YOLOv8m-seg-CBAM.
[0048] S6. Input the data set prepared in S4 into the optimized YOLOv8 model (YOLOv8m-seg-CBAM) in S5 for training to obtain the trained prediction model.
[0049] S7. Optimize the strongsort algorithm. First, optimize the shape of the Kalman filter prediction box in the strongsort, then perform multi-target tracking, and add a fragmented tracking mechanism during the multi-target tracking process.
[0050] Specifically, the Kalman filter in strongsort is optimized first, and the rectangular recognition frame of the Kalman filter is changed to a polygonal recognition frame for multi-target tracking. Since the real frame and prediction frame of the Kalman filter in strongsort are both rectangular frames, which do not conform to the morphological characteristics of coarse-grained soil, the real frame and prediction frame of the Kalman filter in strongsort are changed to polygons, and morphological features are added to each identified coarse-grained map, which is more in line with the morphological characteristics of coarse-grained soil, and the accuracy of the Kalman filter prediction trajectory is improved.
[0051] Then, in the matching process (multi-target tracking process) of strongsort, broken matching is added. The broken particles of coarse-grained soil are traced and matched by obtaining morphological features. Then, the Euclidean distance is added as the threshold. The matching accuracy is improved by adding the threshold through the Euclidean distance, thus achieving the purpose of tracking the broken particles of coarse-grained soil.
[0052] Among them, the morphological features are image features of coarse-grained soil obtained by strongsort, the appearance features of the surface of coarse-grained soil particles are extracted, and then matched through the image features in the next frame of image. The specific matching method includes cross-correlation operation on the appearance features of the surface of coarse-grained soil particles in the previous frame and this frame, and the operation method is as follows:
[0053]
[0054] The variables I and I′ are the appearance features of the two images, and for each (x, y) value, a cross-correlation function is generated.
[0055] If coarse-grained soil particles are broken, the broken coarse-grained soil can be traced through image features, and a threshold can be added based on the Euclidean distance to improve the accuracy of tracing. Each time the coarse-grained soil is broken, its ID adds a dimension, and the Euclidean distance is added as the threshold to analyze the distribution of broken coarse-grained soil particles.
[0056] S8. Perform multi-target tracking on the experimental video, obtain tracking data, and finally output the motion trajectory;
[0057] Specifically, the image is first segmented using the training model in S6, and then the strongsort model modified by S7 is used to track multiple targets in the coarse-grained soil crushing particle video, and finally the motion trajectory is output, and the tracking result is output as a file in the form of data for storage.
[0058] S9. The obtained tracking data is converted from pixel coordinates to actual coordinates by converting the motion trajectory coordinate axis obtained in S8, and the actual particle displacement and broken particle situation of the coarse-grained soil are calculated to achieve real-time observation of the crushed displacement and morphology of coarse-grained soil particles.
[0059] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A coarse-grained soil crushing detection method based on improved YOLOv8, characterized by: The steps are as follows, S1. Obtain the video of the coarse-grained soil crushing experiment; S2. Use the difference method to find multiple frames of different images in the coarse-grained soil crushing particle video as the original data set, mark the original data set, and then perform data enhancement after marking; S3. Convert the format of the labeled and data-augmented images in S2; S4. Create a dataset by converting the converted files in S3 and the unlabeled images after data enhancement in S2; S5. Optimize the YOLOv8 model, change the built-in loss function in YOLOv8 to the EIOU loss function, and add an attention mechanism to the YOLOv8 model; S6. Input the prepared data set into the optimized YOLOv8 model for training to obtain a trained prediction model; S7. Optimize the strongsort algorithm, optimize the shape of the Kalman filter prediction box in the strongsort, and then perform multi-target tracking. Add a fragmented tracking mechanism in the multi-target tracking process. Among them, the shape of the Kalman filter prediction box in strongsort is optimized by changing the rectangular recognition box of the Kalman filter to a polygonal recognition box; S8. Perform multi-target tracking on the experimental video, obtain tracking data, and finally output the motion trajectory; S9. The obtained tracking data is converted from pixel coordinates to actual coordinates by converting the motion trajectory coordinate axes obtained in S8, and the actual particle displacement and broken particle situation of the coarse-grained soil are calculated.
2. According to the coarse-grained soil crushing detection method based on improved YOLOv8 according to claim 1, it is characterized in that: in S2, a picture is selected every few frames in the captured experimental video as the original data set by the difference method, and the images of different frames are marked using the AI marking function in CVAT, and the images of the original data set and the marked images are rotated 90°, 180°, 270°, and flipped left and right and up and down, and other data enhancement operations are performed.
3. A coarse-grained soil crushing detection method based on improved YOLOv8 according to claim 1 or 2, characterized in that: in S3, the data set after the data enhancement operation is format converted, and the CVAT output format is converted to the YOLOv8 input format.
4. According to the coarse-grained soil crushing detection method based on improved YOLOv8 described in claim 1, it is characterized in that: in S4, the label file converted in the format in S3 is divided into three parts: training set, test set, and verification set in a ratio of 7:2:
1.
5. The coarse-grained soil crushing detection method based on improved YOLOv8 according to claim 1 is characterized in that: S5 In the figure, add the CBAM attention mechanism module above the SPPF module in the backbone part of YOLOv8, and the YOLOv8 model optimization is completed.
6. A coarse-grained soil crushing detection method based on improved YOLOv8 according to claim 1, characterized in that: in S6, the data set prepared in S4 is input into the optimized YOLOv8 training model in S5 for training to obtain a trained prediction model.
7. A coarse-grained soil fragmentation detection method based on improved YOLOv8 according to claim 1 or 2 or 4 or 5, characterized in that: in S7, fragmentation matching is added to the matching process in strongsort, the coarse-grained soil fragmentation particles are traced and matched by obtaining morphological features, and then the Euclidean distance is added as a threshold.
8. The coarse-grained soil crushing detection method based on improved YOLOv8 according to claim 7 is characterized by: In S8, the image is first segmented using the training model in S6, and then the coarse-grained soil crushing particle video is tracked using the modified strongsort model in S7. Finally, the motion trajectory is output, and the tracking result is output as a file in the form of data for storage.
9. The coarse-grained soil crushing detection method based on improved YOLOv8 according to claim 7 is characterized by: The image features of coarse-grained soil are obtained through strongsort, and the appearance features of the surface of coarse-grained soil particles are extracted. Then, they are matched through the image features in the next frame of image. If the coarse-grained soil particles are broken, the broken coarse-grained soil can be traced through the image features, and a threshold is added according to the Euclidean distance to improve the accuracy of tracing. Each time the coarse-grained soil is broken, its ID increases by one dimension.