Rockfall detection method and system based on YOLOv9 improved algorithm

Abstract

The invention relates to a rockfall detection method and system based on a YOLOv9 improved algorithm, and belongs to the technical field of image processing. The invention aims to solve the problems of rockfall form diversity and complex environment which are difficult to identify. According to the method, firstly, image data of a rockfall area are obtained, then an improved YOLOv9 algorithm is used for carrying out target detection on an image, and improvement comprises the step of introducing an effective channel attention ECA mechanism and a Wise-IoU loss function. And finally, the detection result of the rockfall target is output to a monitoring platform or other equipment, and tracking or early warning is carried out. The method can effectively improve the precision and efficiency of rockfall detection, has good generalization performance, and can be applied to the fields of slope rockfall monitoring, traffic early warning and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and relates to a rockfall detection method and system based on an improved YOLOv9 algorithm. Background Art

[0002] Some cities are built on the hillsides. The geological conditions in the hillside areas are unstable and are susceptible to erosion by the sun and rain, resulting in collapse, landslides and rockfalls. Rockfall is one of the common natural geological disasters. It occurs when rocks or soil fall off, roll, bounce or fly off the steep slopes due to natural forces (such as gravity, earthquakes, rain erosion, etc.) or human activities (such as excavation, blasting, etc.). The impact of slope rockfall is wide-ranging. Mainly from several aspects, rockfall may injure or kill people passing through the disaster area, especially on slopes near roads, railways or residential areas. Rockfall may directly damage houses, vehicles, roads, bridges, power lines and other structures, and may even cause serious accidents such as train derailments or vehicle accidents. Rockfall may also block traffic roads, causing traffic interruptions, affecting people's daily travel and transportation of goods. In addition, traffic interruptions and property losses will bring economic losses. In addition, the cleanup, repair and prevention work after the disaster also requires a lot of funds. At the same time, rockfall may also destroy vegetation, cause soil erosion, change watershed characteristics, affect local ecosystems and many other adverse effects.

[0003] Traditional rockfall monitoring methods mainly rely on manual inspections, fixed-point video and sensor monitoring, but they have limited monitoring range and low efficiency. Technologies such as LiDAR also have problems such as high costs. However, with the development of computer vision technology and camera equipment, the accuracy of computer-generated videos is constantly improving. The application of new camera equipment such as high-definition cameras, panoramic cameras, and drones has made it easier and more popular to obtain high-quality video data. These video data not only have higher resolution, but also have richer information, providing a more reliable data basis for monitoring and analyzing rockfalls.

[0004] Nowadays, whether it is the slopes near railways, highways, mines, or slopes in places with large traffic such as residential areas and parks built near hillsides, once rockfall occurs, it is easy to cause injuries. When rockfall occurs, people often cannot find it in time, but the automatic detection, identification and alarm method can notify people to take shelter in time, and the timely detection after rockfall also greatly helps emergency personnel to deal with it in time.

[0005] The traditional rockfall detection system detects whether an object has moved by using the frame difference method, and then determines whether it is a rockfall based on the geometric features of the moving object. Its detection speed is slow, the detection accuracy is not high, and it is more prone to false predictions and missed reports. Thanks to the rise of deep learning, a series of deep learning-based target detection algorithms have been proposed and widely used in multiple fields including slope rockfall detection. Algorithms such as the R-CNN series (including Fast R-CNN, Faster R-CNN, etc.), SSD, and YOLO series. This paper mainly improves the latest YOLO algorithm to achieve real-time detection of rockfall areas and improve the accuracy of detection. Through rockfall detection, timely discovery of rockfall disasters can significantly reduce casualties and property losses, and is of great significance for timely remediation after rockfall occurs.

[0006] There are several main methods for rockfall monitoring at present: physical detection methods, remote sensing technology and computer vision target detection technology.

[0007] Physical detection methods include various sensors, strain gauges and crack meters: these devices measure the deformation of rocks or soil. When the internal stress of rocks or soil exceeds a certain threshold, it may cause the development of cracks. By continuously measuring the opening, length and deformation rate of the cracks, the changing trend of the slope can be monitored. Inclinometers and displacement meters: these devices are used to measure the inclination and displacement of the slope. When the slope moves, these changes can be detected by the instrument and a signal warning is sent. Physical detection can directly measure the physical changes of the rock or slope, and the data is intuitive and reliable. For the processing of sensor data, there are now wireless system designs based on LoRa, but it also has the problem that severe weather conditions may affect the stability and accuracy of the sensor. The equipment needs to be inspected and maintained regularly, and may face high operating costs.

[0008] Remote sensing technology relies on the properties of electromagnetic waves, and obtains information by sensing electromagnetic waves reflected or radiated by objects. It can use aerial photography, satellite images, and sensors carried by unmanned aerial vehicles (UAVs). For example, it can focus on collecting detailed terrain data through different remote sensing technologies (i.e., drones and satellite images) to conduct rockfall research. Or the extracted discontinuous set features can be used for local-scale rockfall susceptibility assessment. Remote sensing technology can cover large areas, but the spatial resolution is low and will be affected by cloud cover, etc.

[0009] The ground-based synthetic aperture radar SAR (GB-SAR) system has become a well-known remote sensing technology that can provide 2D radar images and estimate sub-millimeter displacements up to several kilometers. GB-SAR can provide very high spatial resolution and is suitable for long-term monitoring of deformations in a specific area, such as landslides, slope collapses, and other geological changes. It is particularly suitable for continuous or periodic monitoring of slopes, which is crucial for early detection of cracks and small deformations that may cause rockfalls or landslides, but the precursor time of sudden rockfall events is short and the collapse speed is fast. Since the GB-SAR system is sensitive to slow movement of terrain targets but cannot handle fast-moving targets, it has certain limitations in handling rockfalls.

[0010] Lidar (Light Detection and Ranging), a type of terrestrial laser scanner (TLS), has been widely used to study building structures and geological phenomena such as glacier movement, landslides, earthquakes, and rockfalls. Lidar is popular because it can provide high-resolution data. For example, Lidar can be used to detect changes in building structures by comparing two point clouds of a building obtained at two different times. For geological research, 3D point clouds obtained from Lidar can be analyzed using clustering algorithms to study rock glacier landforms. Similar techniques have been applied to detect rockfall events in multi-temporal 3D point clouds. A deep neural network for 3D point cloud learning has been developed based on the advanced PointNet++ architecture. And a real-time rockfall monitoring system built using Doppler radar. Lidar can provide high-resolution terrain data and can detect small terrain changes, but Lidar and TLS are expensive when applied over large areas, and are not very effective during heavy rains that cause a lot of light refraction. In general, there are problems of high cost and sensitivity to weather conditions.

[0011] As early as 2001, Viola and Jones proposed the most influential target detection algorithm in the field of target detection, which can process target detection in real time and has a high detection rate, and has been successfully applied to face detection. With the major breakthrough of deep convolutional neural network in image classification tasks in 2012, it has a more complex network structure and has more powerful feature learning and feature expression capabilities compared with traditional machine learning methods. Since its introduction, the convolutional neural network model trained with deep learning algorithms has achieved remarkable results in multiple large-scale recognition tasks in the field of computer vision. Many scholars have begun to use Deep CNN to replace shallow classifiers to solve target detection problems, which has also led to the improvement of target detection accuracy. Subsequent target detection algorithms such as Faster R-CNN, YOLO and SSD have been launched, and have achieved good results in target detection tasks such as faces, vehicles, and pedestrians.

[0012] Initially, deep learning-based object detection adopted a two-stage strategy, which explicitly divided the detection process into two stages: candidate region selection and target region judgment. The first stage is to generate region proposals, that is, to extract regions that may contain targets from the input image. This step usually uses algorithms such as selective search or region proposal networks. The second stage uses the region proposals generated in the first stage to extract features through convolutional neural networks (CNNs), and performs classification and bounding box regression to accurately locate the object. Its detection accuracy is high, but the detection speed is slow. In 2016, one-stage object detection methods represented by YOLOv1 appeared. This type of method does not require the extraction of candidate areas in advance, but directly predicts the category probability and position of the target object. The first-stage algorithm usually divides the image into grids, each grid is responsible for predicting several candidate boxes, and directly regresses and classifies the position coordinates of these candidate boxes and the probability of their categories, which greatly reduces the consumption of computing resources and improves the detection speed. It achieves better performance in situations where real-time or near real-time detection is required.

[0013] Thanks to the development of computer vision technology and camera equipment, the accuracy of computer-generated videos has been continuously improved. In addition to various data, the staff's demand for actual video materials on site has been growing. Detection methods based on computer vision have begun to emerge. There have been some studies on rockfall detection based on computer vision. VT Bickel et al. used convolutional neural networks (CNN) to detect rocksfall on Mars in pictures. Chung YL et al. combined the Mask R-CNN algorithm with the Retinex image enhancement algorithm to improve the resolution and brightness of low-resolution images, thereby emphasizing objects and making it easier for neural networks to learn, thereby improving the accuracy of detecting rockfall targets. Liu Linya et al. also realized real-time detection of rockfall on slope roads based on the YOLOv3 algorithm. Liu Zixue et al. introduced the feature pyramid network (FPN) and expanded the original three feature maps of different scales into four feature maps to improve YOLOv3 for the detection of rockfall on road slopes. To address the problems of high false positives in complex dynamic environments and low accuracy in small target recognition, Hu Hao et al. integrated a hybrid attention mechanism and improved the feature pyramid structure of the YOLOX algorithm into a bidirectional feature pyramid network to achieve the detection of falling rocks on railways. To address the issues of recognition accuracy and real-time detection, Chen Ken et al. proposed an improved spatial pyramid pooling structure for the YOLOX algorithm and optimized the loss function to CIoU to address the problems of the loss function.

[0014] (1) Some fallen rocks are small targets: The fallen rocks in the rockfall area are of different sizes and shapes. In terms of the size of the fallen rocks, there are currently two definitions of small targets in target detection: one is a target with a size less than one-tenth of the original image size, and the other is a target with less than 32×32 pixels. For the detection of fallen rocks on slopes, in video surveillance, distant targets usually present the characteristics of small targets. Since small targets have fewer pixels and less obvious features, their detection rate is low and the false alarm rate is high compared with large targets. In addition, the sizes of fallen rocks are diverse. Some fallen rocks may be small targets or small and medium targets. Small targets may only occupy a few pixels on the feature map, which limits the amount of information that can be used for classification and positioning. For fallen rocks of different shapes, the feature extractor must be able to extract features that can represent various shapes and sizes. The different sizes of fallen rocks increase the difficulty of falling rock detection.

[0015] (2) Complex environment and difficult to identify: The environment where the rocks are located is complex. Natural factors such as changing lighting, shadows, weather conditions or vegetation will interfere with the detection of slope rocks. Due to the small size of the rocks or obstruction, key information may not be captured. In addition, the rockfall target may be close to the road surface or the slope environment. If the rockfall is made of stone, or the color of the rockfall is similar to the color of the road, it will be difficult to distinguish between the rockfall and the environmental background, which will increase the difficulty of detection and reduce the quality of the sample. Summary of the invention

[0016] In view of this, the object of the present invention is to provide a rockfall detection method and system based on the improved YOLOv9 algorithm.

[0017] In order to achieve the above object, the present invention provides the following technical solutions:

[0018] A rockfall detection method based on an improved YOLOv9 algorithm comprises the following steps:

[0019] Step 1: Obtain image data of the rockfall area;

[0020] Step 2: Use the YOLOv9 algorithm to detect rockfall targets in image data, including:

[0021] The network structure containing programmed gradient information (PGI) is used for feature extraction to enhance the model's feature learning ability for rockfall targets.

[0022] The effective channel attention (ECA) mechanism is introduced between the RepNCSPELAN4 layer after shallow feature extraction and the convolutional layer after Concat to improve the model's ability to detect small target falling rocks;

[0023] Further optimize channel features on detection heads of different scales to enhance the model's ability to detect falling rocks of different shapes;

[0024] The Wise-IoU loss function is used to optimize the target detection results to improve the model's ability to detect falling rocks in complex environments that are difficult to identify, and to enhance the generalization performance of the model.

[0025] Furthermore, in step 2, the YOLOv9 algorithm further includes:

[0026] Use CSPNet blocks and RepConv to replace the original ELAN blocks to optimize the downsampling module;

[0027] Optimization is performed at the prediction layer to fine-tune the Anchor-Free point prediction head.

[0028] Furthermore, in the step 2, the Wise-IoU loss function is the Wise-IoUv3 version.

[0029] Furthermore, in step 1, the rockfall area image data is obtained by:

[0030] Using image acquisition equipment to collect data;

[0031] Obtained through network download or dataset sharing.

[0032] Furthermore, in step 2, the target detection result output includes:

[0033] Types of rockfall targets;

[0034] Location information of rockfall targets;

[0035] Confidence level of rockfall target.

[0036] Furthermore, the rockfall detection method further comprises the following steps:

[0037] Step 3: Track or warn the rockfall target based on the target detection results;

[0038] Step 4: Output the detection results of the rockfall target to the monitoring platform.

[0039] A rockfall detection system based on the rockfall detection method comprises:

[0040] An image acquisition module, used to obtain image data of the rockfall area;

[0041] A target detection module, used to detect rockfall targets in the image data, wherein the target detection module adopts the rockfall detection method;

[0042] A result output module is used to output the detection results of the rockfall target to the monitoring platform;

[0043] The image acquisition module acquires image data of the rockfall area in real time through the image acquisition device; the acquired image data is sent to the target detection module for processing; the target detection module uses the improved YOLOv9 algorithm to detect rockfall targets in the image data; the algorithm analyzes the image, identifies the rockfall targets in the image, and extracts the category, location information and confidence features of the rockfall targets; the target detection module outputs the detection results of the rockfall targets to the result output module; the result output module outputs the detection results in a certain form.

[0044] Furthermore, the image acquisition module includes a camera.

[0045] Furthermore, the result output module includes a display screen.

[0046] The beneficial effects of the present invention are:

[0047] (1) By introducing YOLOv9, ECA attention mechanism and Wise-IoU loss function, the problems of diverse rockfall shapes, different sizes and complex environment that are difficult to identify are effectively solved, thereby improving the accuracy of rockfall detection and reducing the false alarm and missed alarm rates.

[0048] (2) The use of the Wise-IoU loss function enables the model to better handle rockfall images with complex environments and similar backgrounds, enhancing the generalization ability of the model and enabling it to maintain good detection performance in different scenarios.

[0049] (3) The YOLOv9 algorithm itself has the characteristic of fast speed. Combined with the lightweight ECA attention mechanism, the present invention can achieve real-time or near real-time rockfall detection while ensuring detection accuracy, meeting the needs of practical applications.

[0050] (4) Compared with traditional physical detection methods and remote sensing technology, the rockfall detection method based on computer vision of the present invention has the advantages of low cost, easy deployment and maintenance, and is more suitable for large-scale promotion and application.

[0051] (5) The present invention can detect rockfall disasters in a timely manner and issue early warnings, which helps to take emergency measures in a timely manner and ensure the safety of personnel.

[0052] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0054] Figure 1 This is a schematic diagram of the ECA (Efficient Channel Attention) module;

[0055] Figure 2 This is the YOLOv9 network architecture diagram;

[0056] Figure 3 Schematic diagram of the Stable Diffusion diffusion model. DETAILED DESCRIPTION

[0057] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0058] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0059] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0060] 1 Research objectives

[0061] The YOLO algorithm is very popular in application scenarios that require real-time or near-real-time detection due to its high speed and relatively good detection accuracy. In addition, the YOLO series versions have fast iterations and better detection accuracy. Today, the latest version YOLOv9 can better meet the performance requirements of real-time detection of falling rocks. Current target detection methods have achieved high-precision detection of large and medium-sized targets. However, for small targets, due to factors such as the small number of pixels and features that can be used for the target in the image, the detection performance of small targets is usually unsatisfactory. This paper mainly focuses on the problems of the diversity of falling rock sizes and the difficulty in identifying falling rocks in complex backgrounds in rockfall detection, and makes improvements based on YOLOv9 to achieve real-time monitoring of falling rock targets.

[0062] 2 Main research contents

[0063] (1) Target detection: The detection of falling rocks is a target detection problem, so it is necessary to select a suitable target detection algorithm to achieve the detection task. Common target detection algorithms include methods based on traditional machine learning (such as Haar cascade, HOG+SVM, etc.) and methods based on deep learning (such as YOLO, Faster R-CNN, SSD, etc.). In today's target detection applications, methods based on deep learning are more optimal. Deep learning models automatically learn image features through multi-layer networks, while traditional methods rely on manually designed features. The latter often cannot capture all important variables and complexities of the data. They have stronger feature extraction capabilities and have higher accuracy and better generalization capabilities in complex target detection. This paper adopts the YOLO algorithm with a single-stage strategy to take into account both real-time and accuracy. This paper selects the YOLOv9 algorithm for improvement, aiming to improve the traditional YOLO algorithm in the detection of falling rocks with different sizes and complex backgrounds.

[0064] (2) Small rockfalls are difficult to detect: To address the problem of small rockfalls being difficult to detect, an attention mechanism can be introduced to allow the network to focus on the target area and obtain more detailed information. For target detection tasks, the addition of an attention module to the network can improve the representation ability of the network model and effectively reduce the interference of invalid targets, thereby improving the detection effect of the focused target and achieving the purpose of improving the overall detection effect of the network model. Since small rockfalls occupy a small space on the feature map, using a lightweight ECA attention mechanism to strengthen the feature channels related to them may help improve the model's sensitivity and recognition ability to these small targets, and achieve better feature extraction for small targets and rockfalls of different shapes while ensuring its real-time performance.

[0065] (3) Complex environment and difficult to identify: For falling rocks that are difficult to identify due to complex environments, geometric metrics such as distance and aspect ratio will increase the penalty for low-quality examples, thereby reducing the generalization performance of the model. The CIoU used in the original model has such a problem. We can improve the generalization ability of the model and enhance the detection ability of falling rocks that are difficult to identify by replacing the original CIoU with Wise-IoU.

[0066] 2.1 Key issues to be addressed

[0067] Diversity of rockfall shapes: Due to the scale differences between the distance of the camera equipment and the rockfall targets themselves, improvements are made to the problem of low detection accuracy for small rockfall targets due to the small number of pixels and features available for the target in the image and the different shapes of the rocks.

[0068] The rockfall environment is complex and difficult to identify: rockfall may be similar to the environmental background, and may be affected by natural factors such as changing lighting, shadows, weather conditions or vegetation obstruction, resulting in some rockfalls being difficult to identify and some samples may be of low quality. This also requires improvements on how to accurately identify and judge rockfall targets.

[0069] 3.2 Research methods adopted

[0070] Based on the questions raised in 2.1, this paper adopts YOLOv9, a target detection algorithm that takes into account both detection accuracy and real-time performance, and conducts research and improvements based on the actual situation of rockfall detection. The improved algorithm can be applied to real-time rockfall detection and has improved detection accuracy.

[0071] 2.2.1 Object Detection

[0072] YOLOv9 adds two new improvements on the previous YOLO. It introduces Programmable Gradient Information (PGI), a new concept designed to solve the problem of information loss in deep networks. Traditional object detection networks often lose details that are crucial to the final prediction when transmitting deep information. PGI technology can ensure that the network maintains complete input information during the learning process, thereby obtaining more reliable gradient information and improving the accuracy of weight updates. This innovation significantly improves the accuracy of object detection and makes real-time high-precision object detection possible. PGI optimizes the training effect by adjusting the propagation of gradient information at different semantic levels. Its innovation lies in the introduction of auxiliary reversible branches, which does not require additional computational costs. PGI mainly consists of three parts, namely (1) main branch, (2) auxiliary reversible branch, and (3) multi-level auxiliary information. PGI's reasoning process only uses the main branch, so no additional reasoning cost is required. The auxiliary reversible branch is to deal with the problems caused by the deepening of the neural network. The deepening of the network will cause information bottlenecks, resulting in the inability of the loss function to generate reliable gradients; multi-level auxiliary information is designed to deal with the error accumulation problem caused by deep supervision, especially the architecture of multiple prediction branches and lightweight models.

[0073] The network architecture of YOLOv9 is based on YOLOv7 AF. It uses CSPNet blocks and the planned RepConv to replace the original ELAN blocks, optimizes the downsampling module, and makes adjustments in the prediction layer to optimize the Anchor-Free point prediction head. For the auxiliary loss part of PGI, the auxiliary head setting of YOLOv7 is fully adopted. Thus, a generalized efficient layer aggregation network (GELAN) is designed that takes into account light weight, inference speed and accuracy. GELAN optimizes the network structure through gradient path planning and uses traditional convolution operators to achieve parameter utilization efficiency that exceeds the current most advanced methods (including methods based on deep convolution). This design not only improves the performance of the model, but also ensures the efficiency of the model.

[0074] 2.2.2 Attention Mechanism

[0075] Attention mechanism is a data processing method in machine learning, which is widely used in various types of tasks such as natural language processing, image processing and speech recognition. Thanks to the introduction of attention mechanism, the target area where the network focuses can obtain more detailed information. Attention mechanism can help the model focus on the features related to rockfall detection and ignore background noise and irrelevant information. By increasing the focusing ability of the model, the YOLO detector can more clearly identify smaller or differently shaped rocks. Attention mechanism is generally divided into channel attention and spatial attention. For channel attention mechanism, since the feature map generated by the convolution layer is a collection of multiple channels, each channel can be regarded as a feature detector to detect certain specific patterns or structures. However, not all feature channels are equally important. Some channels may be more critical to the current task, while other channels may contain less useful information. By introducing the channel attention mechanism, the network can learn to assign different weights to each channel and prioritize more helpful features. Spatial attention is to determine the weight relationship between different pixels in the spatial neighborhood, increase the weight of pixels in the key area, and let the algorithm pay more attention to the research area we need and reduce the weight of unnecessary areas. In practice, spatial attention is usually implemented by generating a spatial weight map, which modulates the original feature map by learning the importance of each spatial position of the input feature map. As an example, a simple spatial attention module can extract spatial features from the input feature map through operations such as convolution, maximum pooling, and average pooling, and then obtain the spatial weight map through certain operations (such as convolution or fully connected layers), and finally apply the weight map to the original feature map by element-by-element multiplication to enhance or suppress certain areas.

[0076] In the detection of falling rocks, the sizes, shapes, colors, etc. of falling rocks are different, and the locations of falling rocks are random. To improve the above problems, a channel attention mechanism is adopted. This paper introduces ECA-Net (Effective Channel Attention), such as Figure 1 shown.

[0077] ECA is actually an improved version of SENET. SENET calculates the weights of network channels through two nonlinear fully connected layers and an activation layer, while ECA-Net interacts local information across channels to improve the utilization of feature information. This structure uses fast one-dimensional convolution with a kernel size of k to calculate channel weights. The value range of k is affected by the channel dimension C. In addition, it removes the fully connected layer in the original SENET and replaces it with a 1*1 convolution kernel for processing, making the model parameters smaller and more lightweight. The reason is that convolution has good cross-channel information capture capabilities, so it is unnecessary to capture information from all channels. Therefore, the fully connected layer is cancelled and replaced with a 1*1 convolution. First, the input feature map is globally averaged pooled, and the two-dimensional spatial features of each channel are compressed into a value to obtain a global description of each channel:

[0078]

[0079] Among them, z c represents the result of global average pooling of channel c; H represents the height of the feature map; W represents the width; i represents the row in the feature map (height direction); j represents the column in the feature map (width direction); X c (i,j) represents the pixel value (feature value) of the i-th row and j-th column of the feature map on the c-th channel;

[0080] ECA generates channel weights by performing fast 1D convolutions of size k. Given the channel dimension C, the convolution kernel size k can be adaptively determined as follows.

[0081]

[0082] Among them, b represents the offset; ψ(C) represents a function mapping, the input is the number of channels C, and the output is the convolution kernel size k adaptively calculated based on C; γ represents the scaling factor; || odd It means converting the result to the nearest odd value;

[0083] Then use the sigmoid activation function to map the weight vector ω to the interval [0,1], is the sigmoid function, w c is the attention value of the cth channel. Finally, each channel of the input feature map is multiplied by the corresponding attention weight. here, is the weighted c-th channel, w c is the attention weight of this channel.

[0084] The network architecture of yolov9 is as follows Figure 2 As shown in the figure, this paper adds ECA after RepNCSPELAN4 (layers 3 and 5) after shallow feature extraction. This is the shallow part of the network, which extracts low-level features such as local edges and textures. After adding ECA, the ability to capture the preliminary features of small targets can be enhanced. Between the 15th and 16th layers of the convolutional layer after Concat, it is used to optimize the fusion effect of shallow features and deep features, and retain more edge information of small targets. Finally, ECA is added after the 31st, 34th, and 37th convolutional layers to further optimize channel features on detection heads of different scales, making the network more sensitive to small target detection, as shown in Figure 1. Figure 2 shown.

[0085] 2.2.3 Loss Function Improvement

[0086] In view of the complex background of falling rocks, falling rocks may be difficult to identify, and the quality of samples varies. Because the loss function can evaluate the quality of model training results, the training results are usually used to adjust the network weights to speed up the convergence of the model. Therefore, the loss function will directly affect the effect of target detection. It is hoped that by adjusting the loss function, the focus on samples that are easy to identify will be reduced, but for samples that are too difficult, an excessively high contribution to the loss function will not be given, so as to improve the detection ability of falling rocks and ensure its generalization. The loss function used by YOLOv9 is CIoU loss, which takes into account the three geometric parameters of overlapping area, center point distance and aspect ratio. Although CIoU takes into account the distance, overlapping area, center point offset and aspect ratio of bounding box regression, it avoids the situation where the intersection and union ratio cannot be distinguished when the center points of DIoU coincide, and has a better convergence rate for bounding box regression. However, it does not take into account the direction of mismatch between the true box and the predicted box. It only reflects the difference in aspect ratio, rather than the actual relationship between the width and height of the predicted box and the width and height of the true box. In addition, poor quality regression samples have a relatively large impact on the regression loss, and samples with relatively good regression quality are difficult to further optimize. The improved ELoU then separates the influencing factors of the aspect ratio of the predicted box and the true box based on the penalty term of CIoU, and calculates the length and width of the predicted box and the true box respectively to solve the problems of CIoU. Focal Loss designs a monotonic focusing mechanism for cross entropy, which effectively reduces the contribution of simple examples to the loss value. This enables the model to focus on difficult examples. Based on this idea, there was an improved study and the Focal-EIoU method was proposed, but its focusing mechanism is static and does not fully tap the potential of the non-monotonic focusing mechanism. A new dynamic non-monotonic focusing mechanism, namely Wise-IoU (WIoU), is proposed. Constructing the monotonic focusing coefficient and This mechanism also uses "outlier degree β" instead of traditional IoU to evaluate the quality of anchor boxes, and introduces a wise gradient gain allocation strategy. This strategy reduces the competitiveness of high-quality anchor boxes while reducing the harmful gradients generated by low-quality examples. This allows the final WIoU to focus more on processing anchor boxes of ordinary quality, thereby improving the performance of the overall detector. Wise-IoU has three versions: v1, v2, and v3. The formula is as follows.

[0087]

[0088] in, represents the Wise-IoU v1 loss function; Represents a dynamic weight factor, which adjusts the weight of the IoU loss according to the center point offset of the anchor box; represents the basic IoU loss;

[0089] L WIoUv2 represents the Wise-IoU v2 loss function; γ represents a hyperparameter; Represents the dynamically adjusted IoU loss mean;

[0090] represents the dynamic weight factor; x represents the horizontal coordinate of the center point of the predicted bounding box; x gt represents the horizontal coordinate of the center point of the real bounding box; y represents the vertical coordinate of the center point of the predicted bounding box; gt Represents the ordinate coordinate of the center point of the real bounding box; Represents the square of the width of the ground-truth bounding box; represents the square of the height of the ground-truth bounding box;

[0091] β represents the degree of outlier; Represents the dynamic mean of IoU loss;

[0092] represents the Wise-IoU v3 loss function; r represents the dynamic gradient gain factor; δ represents the peak position of the control gradient gain; α represents the degree of influence of the control outlier β; α β-δ Represents α raised to the β-δ power.

[0093] WIoUv3 is used to optimize CIoU to solve the problem that for fallen rocks that are difficult to identify due to complex environments, geometric metrics such as distance and aspect ratio will increase the penalty for low-quality examples, thereby reducing the generalization performance of the model, and the detection accuracy of samples with similar backgrounds is poor.

[0094] 2.2.4 Dataset Expansion

[0095] For the data set, this paper adopts the method of self-built data set. The data set is composed of data collected from the Internet and the real world. For the training data, the collected rockfall pictures are enhanced by Mixup and Mosaic data enhancement methods. The specific data enhancement methods include: splicing pictures, randomly rotating pictures by an angle, randomly enlarging and reducing the size of pictures, randomly adding noise to pictures, translating pictures horizontally or vertically, randomly changing the saturation of sample pictures, randomly changing the hue of pictures, etc., and then using labelimg to annotate the data. In view of the problem that the data set may be difficult to collect or the amount of data is insufficient in the application, consider using Stable Diffusion to generate a part of the images as the training set. Stable Diffusion is an image generation model based on diffusion. The diffusion process is like Figure 3 As shown in the figure, it simulates the reverse process of image noise addition, that is, the process of denoising. Noise addition is a process from right to left as shown in the figure below, called the reverse diffusion process, and denoising is a process from left to right, called the forward diffusion process. It generates a new image by applying the diffusion process in the latent space. This model does not operate directly in the pixel space, but first compresses the image into the latent space, and then gradually removes the noise in the latent space until an image close to the text description is generated. This method can increase the diversity of samples to a certain extent.

[0096] 3.3.5 Evaluation indicators

[0097] Evaluation index The detection accuracy (AP) index is often used in target detection algorithms. This index is determined by the detection accuracy (precision) and recall (recall). The improved model in this paper is compared with R-CNN, SSD, YOLOv8, YOLOv9 and other models. In addition, considering the application in real-time monitoring of slope rockfall, the inference time is used as a reference index. Figure 3 The prediction results are divided into true positive samples (TP), false positive samples (FP), false negative samples (false negative, FN), true negative samples (true negative, TN). The accuracy rate represents the precision rate of the model, and the recall rate represents the recall rate of the model. The two need to be combined to more comprehensively evaluate the detection ability of the model. The average precision is measured by the mean detection accuracy. In its PR curve, the area enclosed by the PR curve and the coordinate axis is equal to the AP value. The mAP can be obtained by taking the average AP value of all categories. Generally, mAP is used to evaluate the detection performance of the entire target detection network model. The calculation method is as follows:

[0098]

[0099] Among them, Recall represents the recall rate; R(u) represents the positive sample predicted by the model for sample u; T(u) represents the true positive sample in sample u; u represents a single sample in the data set; U represents the set of all samples in the data set;

[0100] Precision means accuracy; A AP It represents the average precision, which measures the average accuracy of the model under different recall rates; P(R) means that the precision P is a function of the recall rate R; R represents the recall rate.

[0101] The present invention provides a specific implementation of a rockfall detection method and system based on an improved YOLOv9 algorithm, comprising the following steps:

[0102] (1) Image data acquisition:

[0103] Use high-definition cameras or cameras mounted on drones to continuously monitor the rockfall area and collect image data. Camera parameters can be adjusted according to the actual scene, such as focal length, exposure time, etc., to ensure image quality.

[0104] The acquired image data is stored in a server or local computer and is ready for subsequent rockfall detection.

[0105] (2) Model training:

[0106] Dataset preparation: Build a dataset containing rockfall images, where the rockfall images can be downloaded from the Internet, collected in real time, or generated using image generation models such as Stable Diffusion. The dataset needs to contain rockfall images of various shapes, sizes, and environments to improve the generalization ability of the model.

[0107] Data augmentation: Perform data augmentation on the dataset, such as random cropping, rotation, scaling, adding noise, etc., to increase the diversity of the dataset and improve the robustness of the model.

[0108] Model selection and improvement: Select the YOLOv9 algorithm as the basic model and make the following improvements:

[0109] PGI network structure: The network structure of the YOLOv9 algorithm is replaced with a network structure containing PGI, such as YOLOv7AF, to improve the model's ability to learn the characteristics of falling rocks.

[0110] ECA attention mechanism: The ECA attention mechanism is introduced between the RepNCSPELAN4 layer and the convolutional layer after Concat to improve the model's ability to detect small target rockfalls. For example, the ECA module is added after the RepNCSPELAN4 layer to enhance the ability to capture the initial features of small targets; the ECA module is added between the 15th and 16th layers of the convolutional layer after Concat to optimize the fusion effect of shallow features and deep features and retain more edge information of small targets; the ECA module is added after the 31st, 34th and 37th convolutional layers to enhance the model's ability to detect rockfalls of different scales.

[0111] Wise-IoU loss function: The CIoU loss function used in the YOLOv9 algorithm is replaced with the Wise-IoUv3 loss function to improve the model's ability to detect falling rocks in complex environments that are difficult to identify, and to enhance the generalization performance of the model.

[0112] Model training: Use the labeled data set to train the improved YOLOv9 model, and adjust the model parameters according to indicators such as loss function and accuracy during training until the model achieves satisfactory performance.

[0113] (3) Rockfall detection:

[0114] The collected image data of the rockfall area is input into the trained YOLOv9 model for rockfall detection.

[0115] The model will output detection results such as the category, location information, and confidence level of the rockfall target.

[0116] A confidence threshold can be set, such as 0.9, and detection results with confidence levels lower than the threshold are considered false detections and filtered.

[0117] (4) Result output and application:

[0118] The rockfall detection result is displayed on a display screen or other output devices, for example, information such as the location, category, and confidence level of the rockfall target is displayed.

[0119] The detection results can be output to the monitoring platform for recording, analysis and early warning.

[0120] The detection results can be output to other devices, such as sensors, alarm systems, etc., and corresponding measures can be taken according to the detection results, such as triggering alarms, initiating emergency measures, etc.

[0121] (5) System deployment and maintenance:

[0122] Deploy the rockfall detection system in the rockfall area that needs to be monitored, and perform necessary configuration and maintenance.

[0123] Regularly check the system's operating status and update model parameters to ensure system performance and accuracy.

[0124] For example:

[0125] There is a risk of rockfall on the slope of a certain highway, which needs to be monitored in real time.

[0126] Deploy a rockfall detection system based on the improved YOLOv9 algorithm, including high-definition cameras, servers and monitoring platforms.

[0127] The system collects slope image data in real time and uses the trained YOLOv9 model for rockfall detection.

[0128] When a rockfall is detected, the system will trigger an alarm and send the alarm information to the monitoring platform and relevant departments.

[0129] Relevant departments can take timely measures based on the alarm information, such as closing roads and evacuating people, to avoid accidents.

[0130] Depending on the size of the falling rocks, the long distance of the camera may cause some of the falling rocks to appear as small targets. The lightweight ECA attention mechanism is used to achieve better feature extraction of small targets, rocks of different shapes, etc. while ensuring its real-time performance.

[0131] According to the complex background of falling rocks, Wise-IoU is used to replace the original CIoU, which can improve the detection ability of samples with complex environment and difficult to identify, and at the same time improve the generalization ability of the model.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A rockfall detection method based on an improved YOLOv9 algorithm, characterized in that: The following steps are involved: Step 1: Obtain image data of the rockfall area; Step 2: Use the YOLOv9 algorithm to detect rockfall targets in image data, including: The network structure containing programmed gradient information (PGI) is used for feature extraction to enhance the model's feature learning ability for rockfall targets. The effective channel attention (ECA) mechanism is introduced between the RepNCSPELAN4 layer after shallow feature extraction and the convolutional layer after Concat to improve the model's ability to detect small target falling rocks; Further optimize channel features on detection heads of different scales to enhance the model's ability to detect falling rocks of different shapes; The Wise-IoU loss function is used to optimize the target detection results to improve the model's ability to detect falling rocks in complex environments that are difficult to identify, and to enhance the generalization performance of the model.

2. The rockfall detection method based on the improved YOLOv9 algorithm according to claim 1, characterized in that: In step 2, the YOLOv9 algorithm further includes: Use CSPNet blocks and RepConv to replace the original ELAN blocks to optimize the downsampling module; Optimization is performed at the prediction layer to fine-tune the Anchor-Free point prediction head.

3. The rockfall detection method based on the improved YOLOv9 algorithm according to claim 1, characterized in that: In the step 2, the Wise-IoU loss function is the Wise-IoUv3 version.

4. The rockfall detection method based on the improved YOLOv9 algorithm according to claim 1, characterized in that: In step 1, the rockfall area image data is obtained by: Using image acquisition equipment to collect data; Obtained through network download or dataset sharing. Generate images using artificial intelligence.

5. The rockfall detection method based on the improved YOLOv9 algorithm according to claim 1, characterized in that: In the step 2, the target detection result output includes: Types of rockfall targets; Location information of rockfall targets; Confidence level of rockfall target.

6. The rockfall detection method based on the improved YOLOv9 algorithm according to claim 1, characterized in that: The rockfall detection method further comprises the following steps: Step 3: Track or warn the rockfall target based on the target detection results; Step 4: Output the detection results of the rockfall target to the monitoring platform.

7. A rockfall detection system based on the rockfall detection method according to any one of claims 1 to 6, characterized in that: include: An image acquisition module, used to obtain image data of the rockfall area; A target detection module, used to detect rockfall targets in the image data, wherein the target detection module adopts the rockfall detection method; A result output module is used to output the detection results of the rockfall target to the monitoring platform; The image acquisition module acquires image data of the rockfall area in real time through an image acquisition device; the acquired image data is sent to the target detection module for processing; The target detection module uses the improved YOLOv9 algorithm to detect rockfall targets in image data; the algorithm analyzes the image, identifies the rockfall targets in the image, and extracts the characteristics of the rockfall target’s category, location information, and confidence level; The target detection module outputs the detection result of the rockfall target to the result output module; The result output module outputs the test results in a certain form.

8. The rockfall detection system according to claim 7, characterized in that: The image acquisition module includes a camera.

9. The rockfall detection system according to claim 7, characterized in that: The result output module includes a display screen.

Images