Tunnel face blast hole identification method based on YOLOv8

Abstract

The invention discloses a tunnel face blast hole identification method based on YOLOv8, relates to the technical field of tunnel engineering and computer vision, and provides the tunnel face blast hole identification method based on YOLOv8. The method comprises the following main implementation steps: S1, preparing a data set; s2, data annotation; s3, data enhancement; s4, model training: performing model training based on a DCN-YOLOv8-NMS model and the enhanced data set; s5, blast hole identification: inputting the tunnel face image into the trained blast hole identification model, wherein the model can automatically identify blast hole position information in the image; and S6, outputting a result. The problem that a traditional blast hole recognition method is low in efficiency and accuracy can be solved.

Description

Technical Field

[0001] The present invention relates to the technical fields of tunnel engineering and computer vision, and particularly to a method for identifying blast holes on a tunnel face based on YOLOv8. Background Art

[0002] With the development of the economy and the improvement of technical level, the quantity and scale of tunnels and underground projects in China have increased significantly. Usually, corresponding construction methods are selected according to geological conditions. The drill and blast method is widely used in engineering construction due to its economic and high-efficiency advantages. However, at the present stage, blasting construction lacks professional automated equipment and often relies on manual operation. Professional operators need to manually load detonators and explosives into blast holes, which not only limits the construction efficiency but also poses a potential threat to construction quality and safety. In view of the characteristics of the tunnel environment and the characteristics around blast holes, it is necessary to develop an automated, intelligent, and intrinsically safe explosive automatic filling device for tunnels. The primary task of explosive automatic filling is to achieve accurate identification of blast hole positions. Object detection technology plays a crucial role in this process. It can identify objects in images and provide the category and position information of the objects. Existing object detection technologies can be divided into two types: traditional recognition methods and deep learning-based recognition methods.

[0003] Traditional recognition methods manually extract the features of pictures through methods such as Scale-invariant feature transform (SIFT) or Local Binary Patterns (LBP), then input them into classifiers such as support vector machines for classification, and finally filter redundant detection frames through methods such as Non-Maximum Suppression (NMS) and Weighted Box Clustering (WBC) to improve the detection accuracy. Traditional recognition methods rely on manual design for feature extraction and classification, with slow recognition speed and low detection accuracy. Especially in the candidate region generation stage, the calculation of a large number of candidate regions consumes a large amount of computing power. This will lead to insufficient adaptability in complex environments, especially in special scenarios such as tunnels, where light conditions and other interference factors will seriously affect the recognition effect. Although traditional object detection algorithms still have application value in certain specific scenarios, with the development of deep learning technology, the object detection field has shifted to deep learning-based methods, and these methods have been greatly improved in terms of detection speed and accuracy.

[0004] The target detection of blast holes on the tunnel face faces challenges such as complex backgrounds, small sizes of blast hole targets, and large differences in images under different surrounding rock and lighting conditions. The blast hole target detection based on deep learning faces the following problems: (1) The tunnel environment is complex, and the presence of equipment such as lining brackets interferes with the detection of blast hole targets, making it difficult to accurately locate and detect blast hole targets; (2) The blast hole targets are relatively small compared to the entire tunnel face, occupying fewer pixels in the tunnel face image, and it is difficult to extract target feature information. There may be a phenomenon of missed detection during detection; (3) The blast hole targets vary greatly under different lighting and surrounding rock conditions, and may appear in various shapes such as circular, oval, and irregular; (4) There may be low-quality labeled data. Due to the presence of shadows, etc., the integrity of the blast hole targets is weak, and incorrect labeling of the targets or incomplete labeled target structures may occur during manual labeling, resulting in poor detection effects and the above challenges. Summary of the Invention

[0005] In view of the above problems, the present invention aims to provide a blast hole recognition method for tunnel faces based on YOLOv8, aiming to solve the problems of low efficiency and low accuracy of traditional blast hole recognition methods.

[0006] To achieve the above object, the technical solutions adopted by the present invention are as follows:

[0007] A blast hole recognition method for tunnel faces based on YOLOv8, characterized by comprising the following steps:

[0008] S1: Collect multiple blast hole images on the tunnel face to obtain a blast hole image dataset;

[0009] S2: Label the positions of the blast holes in the blast hole image dataset in S1 to obtain a labeled dataset;

[0010] S3: Perform data augmentation processing on the labeled dataset in S2 to obtain an augmented dataset;

[0011] S4: Train the improved YOLOv8 model based on the augmented dataset in S3 to obtain a blast hole recognition model;

[0012] S5: Input the tunnel face image to be recognized into the blast hole recognition model in S4 to obtain a recognition result.

[0013] Further, in S3, horizontal flipping and brightness adjustment are used to perform data augmentation processing on the labeled dataset in S2.

[0014] Further, the improved YOLOv8 model in S4 is obtained by improving the Backbone structure and Head structure in the YOLOv8 model.

[0015] Furthermore, the improvement of the Backbone structure in the YOLOv8 model is achieved by replacing the C2f module in the Backbone structure with the DCNv3 module.

[0016] Furthermore, the improvement of the Head structure in the YOLOv8 model is obtained by adding an NMS model for re-screening the blast holes after the detect module in the Head structure.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] (1) The improved YOLOv8 model provided by the present invention has a gradually increasing accuracy during the iteration process. During the entire training process, the fluctuation amplitude of the model accuracy curve is small. Specifically, the improved YOLOv8 model gradually stabilizes after 35 iterations, and the accuracy is increased by 0.3; and the recall rate is significantly higher than that of the YOLOv8 model during the iteration process, indicating that the improved YOLOv8 model provided by the present invention has the advantages of stable training results and better robustness. Therefore, the improved YOLOv8 model has higher accuracy and more accurate recognition results in blast hole recognition compared with the original model.

[0019] (2) The design of replacing the C2f module with the DCNv3 module in the present invention can enhance the modeling ability of the YOLOv8 model for target deformation in the blast hole recognition task; specifically: the DCNv3 module is a deformable convolution, which is no longer limited to regular grid positions, but can freely sample on the input feature map as needed. Thereby enhancing the modeling ability of the model for target deformation; that is to say, the standard convolution samples according to the regular grid positions during the convolution operation, while the deformable convolution realizes irregular sampling by introducing an offset, so it has stronger generalization ability in terms of shape transformation (scale, aspect ratio, rotation, etc.).

[0020] (3) Through the joint design of the detect module and the NMS model in the present invention, the false blast holes in the tunnel face can be automatically eliminated, effectively reducing the false detection rate of the model and making the detection results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the algorithm flowchart of the present invention.

[0022] Figure 2 It is the schematic diagram of blast hole annotation on the tunnel face of the present invention.

[0023] Figure 3 It is the original dataset of data augmentation of the present invention.

[0024] Figure 4 It is the standard convolution and deformable convolution of the present invention.

[0025] Figure 5 This is the distance constraint operation process of the present invention.

[0026] Figure 6 This is the algorithm structure of the present invention.

[0027] Figure 7 This is an example of image data of multiple blast holes of the present invention.

[0028] Figure 8 This is an example of image data of a single blast hole of the present invention.

[0029] Figure 9 This is a comparison chart of the precision rate and recall rate of model training of the present invention. Among them, a is the precision rate comparison chart, and b is the recall rate comparison chart.

[0030] Figure 10 This is the output chart of the model recognition result of the present invention. Detailed implementation manners

[0031] In order to enable those of ordinary skill in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0032] Please refer to Figures 1 - 10 , this application discloses a tunnel face blast hole recognition method based on YOLOv8, including the following steps:

[0033] S1: Collect blast hole images on the tunnel face to obtain a blast hole image data set; specifically, to ensure the applicability and scalability of the tunnel blast hole image data, relying on a specific tunnel construction project, without affecting the normal drilling operation of the tunnel, considering different surrounding rock types and grades, various lighting conditions, shooting distances and shooting angle changes, use a portable mobile phone or camera to collect tunnel blast hole images.

[0034] S2: Label the positions of the blast hole images in the blast hole image data set in S1 to obtain a labeled data set;

[0035] It should be noted that after obtaining the face blast hole images, use the labelme software to label the blast hole image data set, and label the blast hole images in the form of rectangular frames to obtain the json label file of the blast hole pictures. During the labeling process, the position information of each blast hole needs to be accurately labeled.

[0036] S3: Perform data augmentation processing on the labeled data set in S2 to obtain an augmented data set;

[0037] Due to the long construction period of the tunnel and the difficult conditions for taking photos of the tunnel face, the effective data obtained after data screening may be relatively small. However, the larger the amount of data, the higher the robustness and recognition accuracy of the model. Therefore, data augmentation is performed on the labeled dataset to improve the model performance. It should be noted that the model here refers to the deep learning model for such object recognition. The statement "the higher the robustness and recognition accuracy of the model" is to explain why horizontal flipping and brightness adjustment are performed on the pictures later to expand the dataset. That is, the "model" mentioned here is a general term for a class of object recognition models, which does not involve the specific implementation process of this solution and has not been constructed for it. Horizontal flipping and brightness adjustment are mainly used to expand the picture data to increase data diversity and improve the robustness of the algorithm.

[0038] Specifically, a large number of data pictures are required for training. The original number of pictures cannot meet the training requirements. Therefore, the number of pictures can be doubled by "horizontally flipping and adjusting the brightness of the original pictures", so as to obtain more pictures and the training results will be better.

[0039] It should be noted that the horizontal flipping in this solution is a left - right flip, that is, a 180 - degree flip. There is no fixed requirement for brightness adjustment because the original lighting conditions of each photo are different, so a fixed value cannot be given. If the original picture is relatively dark, the brightness is increased; if the picture itself is relatively bright, the brightness is decreased.

[0040] S4: Based on the improved YOLOv8 model, model training is performed on the augmented dataset in S3 to obtain a blast hole recognition model;

[0041] During the training process, the Backbone structure and Head structure of the original model are optimized.

[0042] It should be noted that the YOLO (You Only Look Once) series of algorithms, as one of the classic algorithms in the field of object detection, are characterized by fast detection speed and high accuracy. YOLOv8 consists of three main parts: the Backbone structure, the Neck, and the Head structure. The Backbone structure and the Neck part use the C2f module (CSPDarknet53 to 2-Stage FPN) module for feature fusion, obtaining richer gradient flow information while ensuring the lightweight of the model, and adjusting the number of different channels according to the model scale. The Backbone structure is composed of the CBS module, the C2f module, and the Spatial Pyramid Pooling-Fast (SPPF) module, which is used for feature extraction; the Neck part performs multi-scale fusion on the features extracted by the backbone network; the Head structure uses the extracted features for prediction, adopts a decoupled detection head, separates the classification and detection heads, and extracts the position and category information of the target respectively. In addition, YOLOv8 changes from Anchor-Based to Anchor-Free, adopts the Task-Aligned Assigner positive and negative sample assignment strategy, and uses a high-order combination of classification scores and IoU to calculate the alignment degree at the Anchor level. YOLOv8 uses BCE Loss to represent the classification loss and CIoU Loss + DFL to represent the regression loss. YOLOv8 can perform object detection and instance segmentation, and at the same time provides 5 models with different scaling sizes to meet the needs of different scenarios.

[0043] In this application, the main role of the Backbone structure of the YOLOv8 model is to extract image feature information. When training on the tunnel face blast hole dataset, the phenomenon of loss of image feature information will occur. The core of the Backbone structure is the C2f module, which is used to convert the output of the convolutional layer into the input of the fully connected layer. It is the key module that causes missed detections and misclassifications after model training. The Bottleneck structure can improve the effect in the C2f module. The deformable convolution DCNv3 module is used to optimize the model and improve its performance on the tunnel face blast hole recognition dataset. As Figure 4 shown is the comparison between deformable convolution and standard convolution. In deformable convolution (b), the sampling positions are deformed by introducing offsets, which are represented by enhanced offsets (light blue arrows). This means that in deformable convolution, it is no longer limited to regular grid positions, but can freely sample on the input feature map according to needs. Specifically, replace the C2f module of the Backbone structure with the DCNv3 module. The replaced neck structure is in Figure 6It can be observed. The specific operation process is to modify the model code: First, add the DCNv3 module code, and then replace the C2f module in the Backbone structure with the DCNv3 module in the model structure code, that is, the yolov8.yaml file, to complete the call of the new module. This module is a plug-and-play module, and it does not involve any specific modifications to the model, but only code replacement operations.

[0044] The role of the C2f module is to perform feature fusion to improve the performance of object detection. However, the C2f module divides the feature map into parts that are the same size as the convolution kernel and then performs convolution operations. The position of each part on the feature map is fixed. In this way, for objects with complex deformations, the effect of using this convolution may not be very good. The DCNv3 module is a deformable convolution, which is no longer limited to regular grid positions, but can freely sample on the input feature map as needed. This enhances the model's ability to model object deformations; that is, standard convolution samples according to regular grid positions during convolution operations, while deformable convolution achieves irregular sampling by introducing offsets, thus having stronger generalization ability in terms of shape transformation (scale, aspect ratio, rotation, etc.). Therefore, it is necessary to replace the C2f module with the DCNv3 module for the blast hole recognition task.

[0045] There are several reasons for the existence of blast hole targets on the tunnel face: (1) The size of the blast hole image is large, the number of blast holes is large, and the size of the blast holes is very small; (2) The surrounding rock background is complex, and there are many interference factors such as the shadow of the bench, support, and rock blocks. When using the original model for target recognition, there are phenomena where the recognition cannot be successful. Therefore, the Head structure of the YOLOv8 model is optimized, and an NMS model with distance constraint conditions is proposed, specifically a multi-blast hole filtering algorithm, which eliminates false blast holes through distance constraint conditions to reduce the false detection rate and re-detection rate of the model.

[0046] The optimization operation on the Head structure is to add a distance constraint condition NMS model filtering algorithm. Specifically, the NMS model is added after the detect module in the Head structure. That is, after the blast holes are recognized, there may be false blast holes. Therefore, adding the NMS model re-screens the blast holes after passing through the detect module and deletes the false blast holes. It is equivalent to that the YOLOv8 model has already labeled the blast holes, but there may be false blast holes in its labeling results. Therefore, it is necessary to further process the images after the YOLOv8 model has completed recognition to eliminate false blast holes. So, a distance constraint algorithm is added after the detect module in the Head structure. The specific principle and steps of the NMS model are pointed out below. The specific improved overall model structure can be found in Figure 6 It can be seen.

[0047] It should be noted that there are certain requirements for the distance between blast holes in the actual tunnel face. Therefore, adding the NMS model distance constraint algorithm can eliminate false blast holes according to the actual situation and is irreplaceable.

[0048] The specific operation of the model is to directly add code to the original detect module function part for calling. In the network framework diagram, it is to directly connect the NMS model behind the detect module without other connections in between. Only after connecting the detect module and the NMS model can the effect be achieved. Because the NMS model is only a distance constraint condition, the blast holes cannot be identified only by this part. Only after identifying the blast holes first can those false blast holes be eliminated based on all the blast holes.

[0049] Based on the principle of distance constraint: There is a certain distance between two adjacent blast holes in the tunnel face. There is one and only one blast hole within a certain range around each blast hole. If the deep learning method identifies false blast holes or surrounding rock shadows as blast hole targets, it will inevitably result in multiple blast holes appearing in a small area, which does not conform to the actual blast hole layout principle. Therefore, if the minimum distance between adjacent blast holes in the tunnel face is D, by setting: r = αD (0 < α < 1), the NMS model based on distance constraint, specifically a multiple blast hole filtering algorithm, can effectively eliminate the predicted false blast holes and pseudo-blast holes.

[0050] The specific operation steps are as follows: Sort all predicted targets in descending order of probability score, and select the target with the highest score and denote it as k1; then based on the upper left coordinate (x 1，1 , y 1，1 ) and the lower right coordinate (x 1，2 , y 1，2 ) of this prediction box, calculate its center point coordinate (x 1，0 , y 1，0 ); then introduce the distance constraint r, draw a circle with the center point (x 1，0 , y 1，0 ) as the center and r as the radius; finally, calculate the distance d between the center points (x i，0 , y i，0 ) of other prediction boxes k i (i = 2, 3,..., n) and the center point (x 1，0 , y 1，0 ) of k1. If d < r, then remove this target box k i , otherwise retain this target box k i .

[0051] Based on the YOLOv8 model structure, integrating deformable kernel convolution and the blast hole target filtering algorithm based on distance constraint, this application designs an algorithm suitable for blast hole recognition in the tunnel face, such asFigure 6 as shown

[0052] S5: Input the tunnel face image to be recognized into the blast hole recognition model in S4 to obtain the recognition result; the model will automatically recognize the position and size information of the blast holes in the image.

[0053] Finally, output the recognition result in S5 in a visual way. Specifically, the visual output means that annotations will be made on the image output by the model, and a rectangular box, that is, the target box k, will be generated around each blast hole. i , and at the same time, the blast hole label and label score will be marked in the upper left corner of the rectangular box. If there is a false blast hole near a real blast hole, the false blast hole will also be marked in the form of a rectangular box, but the label score is lower than that of the real blast hole, and the false blast hole can be eliminated according to the score.

[0054] Embodiment

[0055] According to the above implementation steps, first establish a blast hole image database. After obtaining available tunnel blast hole image samples, construct a blast hole target detection dataset through cropping, classification, and manual annotation. Figure 7 and Figure 8 respectively show the picture data of multiple blast holes and a single blast hole.

[0056] The shooting distance of a single blast hole image is relatively close. As can be seen from the examples in Figure 8 , the shapes of blast holes are diverse, including regular circular, elliptical blast holes, and irregularly shaped blast holes. In order to improve the accuracy of blast hole target detection, a large number of samples are required for the training of the deep learning model. Therefore, the methods of left-right flipping and random brightness adjustment in S2 above are used to perform blast hole image data augmentation, expanding the collected blast hole image samples to 3 times the original, and finally establishing a blast hole image dataset. Obtain 1500 pieces of multiple blast hole image data and 1000 pieces of single blast hole image data. In order to use more blast hole images for model training, a 9:1 ratio is used to divide the training set and test set in both the single blast hole and multiple blast hole model trainings.

[0057] Use the labelme software to perform data annotation on the established image database, and use a rectangular box (as shown in Figure 2 ) to frame the position of the blast hole target in the picture. Each blast hole image can obtain a json format file, and the file name is the same as the blast hole image name. The YOLOv8 model training requires a txt format file, so after annotating the photos, use python code to generate the txt file corresponding to the json file as the data label to form a tunnel face blast hole recognition dataset.

[0058] To improve the training speed of the model, the image size is unified to 448*448, and the number of iterations (Epochs) is set to 50 times. Currently, the common quantitative evaluation indicators for deep learning models mainly include Precision (P) and Recall. Precision is also called the precision rate, which measures the accuracy of the recognition results; Recall is also called the recall rate, which is used to measure the completeness of the recognition results. Multiple blast hole pictures are input, and the training results of the model are as Figure 9 shown below:

[0059] As can be seen from the figure, in terms of accuracy: the accuracy of the improved YOLOv8 model gradually increases during the iteration process. During the whole training process, the fluctuation range of the model accuracy curve is small. It gradually stabilizes after 35 iterations. Compared with the original model, the accuracy is increased by about 0.3; in terms of recall: the recall of the improved model is significantly higher than that of the original model during the iteration process, and the fluctuation range of the original model is larger, indicating that the training results of the model are not stable enough and the robustness is worse. In summary, the optimized model has higher accuracy and more accurate recognition results than the original model in blast hole recognition, indicating the effectiveness of the improvement scheme.

[0060] Figure 10 The image detection results of multiple blast holes are shown. The black dotted circles in the figure are the misdetected blast holes in the surrounding rock background. The detection results show that after adding the distance constraint algorithm, the model will automatically eliminate the false blast holes in the tunnel face, effectively reducing the false detection rate of the model and making the detection results more accurate.

[0061] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A tunnel face blasthole recognition method based on YOLOv8, characterized in that: The following steps are involved: S1: Collect multiple blasthole images on the tunnel face to obtain a blasthole image dataset; S2: annotate the blasthole positions in the blasthole image dataset in S1 to obtain an annotated dataset; S3: Perform data augmentation processing on the labeled data set in S2 to obtain an enhanced data set; S4: Based on the enhanced data set in S3, the improved YOLOv8 model is trained to obtain a blasthole recognition model; S5: Input the tunnel face image to be identified into the blasthole identification model in S4 to obtain the identification result.

2. The tunnel face blasthole identification method based on YOLOv8 according to claim 1 is characterized in that: In S3, horizontal flipping and brightness adjustment are used to perform data enhancement processing on the labeled data set in S2.

3. The tunnel face blasthole identification method based on YOLOv8 according to claim 1 is characterized in that: The improved YOLOv8 model in S4 is obtained by improving the Backbone structure and Head structure in the YOLOv8 model.

4. The tunnel face blasthole identification method based on YOLOv8 according to claim 3 is characterized in that: The Backbone structure improvement in the YOLOv8 model is achieved by replacing the C2f module in the Backbone structure with the DCNv3 module.

5. The tunnel face blasthole identification method based on YOLOv8 according to claim 3 is characterized in that: The improvement of the Head structure in the YOLOv8 model is achieved by adding an NMS model after the detect module of the Head structure to re-screen the blast holes after the detect module identifies them.

Images