Real-time rockfall monitoring and tracking method based on improved YOLOv8 model

Abstract

The invention discloses a rockfall real-time monitoring and tracking method based on an improved YOLOv8 model. The method comprises the following steps in a model training stage: collecting stone data; the structure of the YOLOv8 model is improved, images in the stone data set are preprocessed, and a stone recognition model is trained based on the improved YOLOv8 model; the rockfall identification and tracking stage comprises the following steps: receiving mountain and highway video information collected by a front end in real time, identifying stones in the video in real time, and recording position information of the stones; judging whether the stone is falling stone or not according to the change condition of the stone position, and calculating impact force; and judging whether to give an alarm and record according to whether the rockfall stays on the road or the value of the impact force of the rockfall. According to the method, the detection head of the YOLOv8 model is improved, and the stone image data set is collected to train the stone recognition model, so that a rock falling event possibly occurring in a mountain or slope area can be effectively monitored, measures can be taken in time to protect personnel and property safety, and reliable support is provided for disaster early warning and risk management.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition in computer vision, and in particular to a real-time rockfall monitoring and tracking method based on an improved YOLOv8 model. Background Art

[0002] Rockfall recognition technology is a method that uses computer vision and machine learning technologies to automatically identify rocks. By analyzing the shape, color, texture and other characteristics of the rocks, they can be classified into different types or their properties can be judged. This technology has applications in many fields, such as geological disasters, architecture, archeology and other fields, and has great potential. At present, most of the data sets of deep learning-based rock recognition technology are rocks with certain characteristics. They only support the classification of rocks, and do not support the identification of rocks in the wild and the judgment of the movement of rocks. In addition, the current stone recognition technology has very limited recognition capabilities for small target rocks, and the recognition accuracy is low, and it cannot effectively detect sudden rockfalls and alarm.

[0003] At present, the monitoring and early warning methods for rockfall disasters are mostly high-precision displacement monitoring methods. Such methods have high deployment costs and limited monitoring range, so they cannot be used on a large scale. The advantages of using the real-time monitoring and tracking method of rockfall based on the improved YOLOv8 model for early warning are: ① The cost is relatively low, the deployment method is relatively simple, and it can be deployed on a large scale at sites where rockfall disasters frequently occur; ② The monitoring position can be flexibly adjusted by increasing or decreasing the number of deployed cameras; ③ The stone recognition technology based on deep learning is used to achieve real-time monitoring around the clock to avoid false alarms and missed alarms. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a real-time monitoring and tracking method for rockfall based on an improved YOLOv8 model. By improving the detection head of the YOLOv8 model and using a self-made image data set containing a large number of stones to train a stone recognition model, it is possible to monitor rockfall disasters that may occur in mountainous or slope areas, so as to take timely measures to protect the safety of people and property. The accuracy of small target stone recognition can be greatly improved, and the movement trajectory of stones can be monitored, thereby improving the success rate of rockfall disaster warning and reducing the loss of property and life.

[0005] In order to achieve the purpose of the present invention, the present invention provides a real-time monitoring and tracking method for falling rocks based on an improved YOLOv8 model, which is divided into a recognition model training stage and a falling rock recognition and tracking stage.

[0006] The model training phase includes the following steps:

[0007] Collect stone data, expand the data, use Roboflow to annotate the data set, and divide the annotated data set into training set, validation set, and test set;

[0008] Improve the structure of the YOLOv8 model and add a detection head structure to enhance the rockfall detection capability;

[0009] Preprocess the images in the stone dataset and train the stone recognition model based on the improved YOLOv8 model;

[0010] The rockfall identification and tracking phase includes the following steps:

[0011] Receive the video information of the mountain and the road collected by the front end in real time, identify the stones in the video in real time and record the location information of the stones;

[0012] Determine whether the stone is a falling rock based on the change in the stone's position and calculate the impact force;

[0013] Depending on whether the rockfall stays on the road or the value of the rockfall impact force, it is determined whether to send an alarm message to the early warning release platform and record it.

[0014] The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned real-time monitoring and tracking method for falling rocks based on the improved YOLOv8 model is implemented.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] (1) The real-time rockfall monitoring and tracking method of the present invention supports the identification of rocks in the wild and the determination of the movement of rocks;

[0017] (2) The real-time rockfall monitoring and tracking method of the present invention improves the YOLOv8 model structure and effectively improves the recognition ability of small target rocks;

[0018] (3) The real-time monitoring and tracking method for falling rocks of the present invention utilizes deep learning-based rock recognition technology to achieve real-time monitoring around the clock, reducing deployment costs and difficulty while avoiding false alarms and missed reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 It is the overall flow chart of the present invention.

[0021] Figure 2 This is a workflow diagram of the real-time rockfall monitoring and tracking method proposed in the present invention.

[0022] Figure 3 This is a sample training flow chart proposed by the present invention.

[0023] Figure 4 It is a flow chart of rockfall identification processing of the present invention.

[0024] Figure 5 It is a real-time early warning flow chart of the present invention. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] like Figure 2 As shown, a real-time rockfall monitoring and tracking method based on an improved YOLOv8 model is divided into a recognition model training stage and a rockfall recognition and tracking stage;

[0027] The model training phase includes the following steps:

[0028] Step 1: Collect stone data, expand the data using data augmentation technology, annotate the data set using Roboflow, and divide the annotated data set into training set, validation set, and test set according to the ratio of 8:1:1;

[0029] Use high-definition night vision cameras to collect stone sample data covering different scenes and angles within the monitoring range, and use data enhancement technology to expand the data. Specifically, the scenes include lawns, sandy land, roads, hills, etc., and environmental factors include different angles, directions, lighting conditions, etc. In addition, among the collected stone samples, the detected objects have at least one similar feature, including but not limited to similar shapes, similar sides, relative sizes, rotation angles, tilts, lighting, etc., to ensure the diversity of stone samples in natural scenes. Data enhancement techniques include: randomly rotating the image by an angle, randomly zooming in and out of the image, randomly adding noise to the image, randomly changing the saturation of the image, and randomly changing the hue of the image. Then use Roboflow to annotate the dataset, and divide the annotated dataset into training set, validation set, and test set according to 8:1:1.

[0030] Step 2: Improve the YOLOv8 model, add a detection head structure, and perform rockfall detection;

[0031] Improve the YOLOv8 model by adding a small object detection head to detect small objects smaller than 4 pixels x 4 pixels, thus enhancing its ability to recognize small objects.

[0032] At the same time, the structure of the YOLOv8 model in the prior art is adjusted, and a C2f (Cross Stage Feature Fution) module with 128 channels is added to the head part of the YOLOv8 model as the output layer P2. The corresponding detection image size is 160 pixels × 160 pixels, which is used to detect targets larger than 4 pixels × 4 pixels.

[0033] Step 3: Preprocess the images in the stone dataset and train the stone recognition model based on the improved YOLOv8 model;

[0034] like Figure 2 and Figure 3 As shown in the figure, the specific steps for preprocessing the images in the stone dataset and training the stone recognition model are as follows:

[0035] Step S31: Adjust the image size and scale the image to a size of 640 pixels×640 pixels specified by the improved YOLOv8-based stone recognition model.

[0036] Step S32: normalize the image by dividing the pixel value by 255 so that the pixel range is scaled from 0 to 255 to between 0 and 1.

[0037] Step S33: Expand the image dimension by adding an additional batch size dimension to the first dimension of the image so that it is expanded from the original three channels to four channels.

[0038] Step S34: convert the image data type into the input format BGR and tensor format (batch_size, channel, height, width) required by the improved YOLOv8-based stone recognition model, where batch_size represents the batch size, channel represents the number of image channels, and height and width represent the height and width of the image.

[0039] Step S35: training the stone model to obtain an optimal stone recognition model;

[0040] Based on the improved YOLOv8 model, the stone recognition model is trained. First, the training parameters are set:

[0041] The important parameters of training are: epochs (iteration rounds) is set to 300, patience (early stopping tolerance) is 50, imgsz (image size) is 640, workers (kernels) is 4, optimizer is AdamW, lr0 (initial learning rate) is 0.005, lrf (final learning rate) is 0.01, momentum is 0.9, weight_decay is 0.0005.

[0042] Next, the training process is iteratively optimized until the maximum number of training iterations is reached, the entire training process is completed, and the optimal stone recognition model is obtained.

[0043] The rockfall identification and tracking phase includes the following steps:

[0044] Step 4: Receive the video information of the mountain and the road collected by the front end in real time, identify the stones in the video in real time and record the location information of the stones.

[0045] According to the stone recognition model trained in step 3, the stones in the video signals of the mountain and the road are identified in real time, and the location information of the stones is recorded. The specific steps are as follows:

[0046] S41: receiving in real time the video information of the mountain and the road collected by the front end of the high-definition network night vision camera, and performing corresponding video preprocessing on it;

[0047] Select a certain frame rate or interval to extract video frames to reduce the amount of data and reduce the computational cost; then adjust the image size, normalize, expand the dimension and convert the data type of the extracted video frames according to the image preprocessing method; finally, perform optical flow estimation on continuous video frames to obtain motion information between frames. The processed video frames are used as the input of the stone recognition model for target detection.

[0048] S42: using the trained stone recognition model to recognize stones in the video in real time;

[0049] The trained stone recognition model uses a screening algorithm (non-maximum suppression) to remove the recognition box of each video frame and select the best candidate box to remove redundant detection results and ensure that each stone target is only detected once. Then the screened recognition box is drawn on the video frame, and the category, confidence and spatial location information are displayed in the upper left corner of the recognition box to visualize the detection results. Finally, the video frame with the drawn recognition box is output and displayed in real time.

[0050] S43: After the model recognizes the stone, it records the position of each stone in each frame of the video (x i ,y i), that is, the horizontal coordinate x of the identification box of the i-th stone in each video frame in the canvas i With the vertical coordinate y i Pixel value, saved in the database;

[0051] Step 5: Determine whether the stone is a falling rock based on the change in the stone's position, and calculate the impact force P;

[0052] S5: judging whether the stone is a fallen rock according to the change degree of the position information of the identified stone in each video frame. The specific judgment method is as follows:

[0053] S511: Get the current stone position information (x i1 ,y i1 ), get the horizontal coordinate x of the identification box of the i-th stone in the current video frame in the canvas i1 With the vertical coordinate y i1 The pixel value is used as the coordinates of the stone.

[0054] S512: Search the database for the latest position information of the i-th stone, that is, the position of the stone in each frame of the video saved in S43 (x i ,y i ) is the latest piece of information, represented by (x i2 ,y i2 ), compare the current stone coordinates with the latest recorded position information (x i2 ,y i2 ) to obtain Δx=|x i1 -x i2 | and Δy = |y i1 -y i2 |, if both differences are less than the set threshold ε, it means that the stone has not moved. If any difference is greater than ε, it means that the stone is moving and can be judged as a fallen rock. If the latest position information of the i-th stone cannot be found in the database, it means that the stone is recognized for the first time.

[0055] S513: Set the position of the current video frame (x i1 ,y i1 ) is written into the database as the latest position information of the i-th stone.

[0056] S52: Calculate the impact force P of the falling rock. The specific calculation steps are:

[0057] S521: The identified stone is cut out from the background through the segmentation function of the stone recognition model based on the improved YOLOv8. The pixel area S occupied by the stone in the canvas can be obtained according to the x and y coordinate points of the stone frame in the canvas. Then the volume of the fallen rock V = σπS is approximately estimated, where σ is a coefficient, which is positively correlated with the area S and the distance between the camera and the stone.

[0058] S522: Calculate the mass of the falling rock m=ρV according to the density of the falling rock ρ.

[0059] S523: Estimate the speed of the falling rock based on the position of the rock in two adjacent video frames and the distance between the camera and the rock Where Δx represents the displacement change of the stone on the x-axis of the canvas between two adjacent video frames, Δy represents the displacement change of the stone on the y-axis of the canvas between two adjacent video frames, p is a coefficient positively correlated with the distance between the camera and the stone, and Δt is the time interval between two adjacent video frames.

[0060] S524: Calculate the impact force of the falling rock P = μm·v, where μ is a constant, m is the mass of the falling rock, and u is the speed of the falling rock.

[0061] Step 6: According to whether the rockfall stays on the road or the value of its impact force, determine whether to send an alarm message to the early warning release platform and record it;

[0062] like Figure 5 As shown, if the speed of the rockfall v is 0, that is, the impact force P is 0, it is determined whether it stays in the highway area; if it stays on the highway, the warning information is issued to the warning release platform software;

[0063] If the speed v of the falling rock is not 0, the need for an early warning is determined based on the magnitude of its impact force P; if its impact force P is greater than the set threshold, an early warning message is issued to the early warning release platform software.

[0064] In this embodiment, the real-time rockfall monitoring and tracking method designed by the present invention uses the improved YOLO v 8 The recognition model is combined with the rockfall discrimination method and the impact force calculation method to achieve real-time monitoring of rockfall disaster-prone areas around the clock, and effectively improve the recognition ability of small target rocks, thereby improving the accuracy of rockfall recognition and reducing the possibility of false alarms and missed alarms. Compared with the high-precision displacement monitoring method for rockfall disasters, the cost of this embodiment is relatively low, the deployment method is relatively simple, and it can be deployed on a large scale at sites where rockfall disasters frequently occur. In addition, the monitoring direction can be flexibly adjusted by increasing or decreasing the number of deployed video monitoring devices.

[0065] The above is a detailed description of the specific implementation examples of the present invention. It should be noted that the present invention is not limited to the above specific implementation examples, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essential content of the present invention.

Claims

1. A real-time rockfall monitoring and tracking method based on an improved YOLOv8 model, characterized in that: It includes the recognition model training stage and the rockfall recognition and tracking stage. The model training stage includes the following steps: Collect stone data, expand the data, use Roboflow to annotate the data set, and divide the annotated data set into training set, validation set, and test set; Improve the structure of the YOLOv8 model and add a detection head structure to enhance the rockfall detection capability; Preprocess the images in the stone dataset and train the stone recognition model based on the improved YOLOv8 model; The rockfall identification and tracking phase includes the following steps: Receive the video information of the mountain and the road collected by the front end in real time, identify the stones in the video in real time and record the location information of the stones; Determine whether the stone is a falling rock based on the change in the stone's position and calculate the impact force; Depending on whether the rockfall stays on the road or the value of the rockfall impact force, it is determined whether to send an alarm message to the early warning release platform and record it.

2. The rockfall real-time monitoring and tracking method according to claim 1, characterized in that: In the steps of collecting stone data, expanding the data, annotating the data set using Roboflow, and dividing the annotated data set into a training set, a validation set, and a test set, the stone data are stone samples within the monitoring range, covering stone conditions in different scenes and angles, including different light exposure, different backgrounds, different postures, and whether there are shadows.

3. The method for real-time rockfall monitoring and tracking based on the improved YOLOv8 model according to claim 1, characterized in that: The improvement method for the YOLOv8 model is as follows: adding a detection head, adding a C2f module with 128 channels in the head part of the YOLOv8 model as the output layer P2, and the corresponding detection image is 160 pixels × 160 pixels, which is used to detect targets larger than 4 pixels × 4 pixels.

4. The rockfall real-time monitoring and tracking method according to claim 1, characterized in that: The specific steps of preprocessing the images in the stone data set and training the stone recognition model are as follows: Step S31: adjusting the image size, scaling the image to adjust it to a size specified by the stone recognition model based on the improved YOLOv8; Step S32: normalizing the image; Step S33: Expand the image dimension by adding an additional batch size dimension to the first dimension of the image, so that it is expanded from the original three channels to four channels; Step S34: converting the image data type into the BGR and tensor formats required by the stone recognition model based on the improved YOLOv8; Step S35: training the stone model to obtain the optimal stone model.

5. The rockfall real-time monitoring and tracking method according to claim 1, characterized in that: The specific steps of receiving the video information of the mountain and the road collected by the front end in real time, identifying the stones in the video in real time and recording the location information of the stones are as follows: Receive the video information of mountains and roads collected by the front end of the high-definition network night vision camera in real time, and perform corresponding video preprocessing; Use the trained stone recognition model to identify stones in the video in real time; After the model recognizes the stones, it records the position of each stone in each frame of the video.

6. The rockfall real-time monitoring and tracking method according to claim 1, characterized in that: In the step of judging whether a stone is a falling rock according to the change of the stone position and calculating the impact force, the specific steps of judging whether a stone is a falling rock according to the change of the stone position are as follows: Get the location information and the horizontal coordinate x of the identification frame of the i-th stone in the canvas i1 With the vertical coordinate y i1 The pixel value is used as the coordinates of the stone; Compare the coordinates of the current stone with the latest recorded position (x i2 ,y i2 ) to obtain Δx=|x i1 -x i2 | and Δy=|y i1 -y i2 |, if both differences are less than the set threshold ε, it means that the stone has not moved; if any difference is greater than ε, it means that the stone is moving and is judged as a rockfall; Record the current position (x i1 ,y i1 ), written into the database to replace the latest information of the i-th stone (x i2 ,y i2 ).

7. The rockfall real-time monitoring and tracking method according to claim 1, characterized in that: In the step of judging whether a stone is a falling rock according to the change of the position of the stone and calculating the impact force, the calculation of the falling rock impact force is: P = μm·v, Where P is the impact force, μ is a constant, m is the mass of the falling rock, and v is the velocity of the falling rock.

8. The rockfall real-time monitoring and tracking method according to claim 7, characterized in that: According to the changing position of the stone in two adjacent video frames and the distance between the camera and the stone, the speed of the falling stone is calculated: Where Δx represents the displacement change of the stone on the x-axis of the canvas between two adjacent video frames, Δy represents the displacement change of the stone on the y-axis of the canvas between two adjacent video frames, p is a coefficient positively correlated with the distance between the camera and the stone, and Δt is the time interval between two adjacent video frames.

9. The rockfall real-time monitoring and tracking method according to claim 1, characterized in that: In the step of determining whether to send an alarm message to the early warning release platform and record the message according to whether the rockfall stays on the road or the value of its impact force, if the speed of the rockfall v is 0, that is, the impact force P is 0, then it is further determined whether it stays in the road area; if it stays on the road, then the early warning message is released to the early warning release platform software; if the speed of the rockfall v is not 0, then it is determined whether an early warning is needed according to the magnitude of its impact force P; If the impact force P is greater than the set threshold, the warning information will be issued to the warning release platform software.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images