A coal mine shaft wall crack identification method based on a deep neural network

Abstract

The application discloses a coal mine wall crack identification method based on a deep neural network, a data collection module collects image data from a mine robot; the image information collected by the mine robot is enhanced by using an image enhancement algorithm; a coal mine wall crack target image is acquired, low-quality crack images are removed, crack targets in the image are labeled, and a coal mine wall crack data set is constructed; the wall crack data set constructed by the data set construction module is sent to an improved Efficientdet model training module for training, a crack image to be detected is input into the improved Efficientdet model for detection, and a final detection result is generated; the application improves the Efficientdet model, adds a multi-space perspective fusion module and a ternary coordinate attention module on the basis of the existing model, so that the model can overcome the interference of a complex coal mine underground environment and solve the problem of missing detection of small cracks, and the detection precision is improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition and detection technology. Specifically, it is a method for identifying cracks in coal mine walls based on deep neural networks, applicable to the investigation of safety hazards in coal mine walls. Background Technology

[0002] Coal mine wall cracks have a serious impact on coal mine production, causing accidents such as changes in underground airflow, groundwater seepage, gas accumulation, spontaneous combustion of coal seams due to oxygen supply from the cracks, and collapses. From a safety and maintenance perspective, timely and rapid identification and repair of cracks helps increase the safety factor of coal mines, reduce maintenance costs, and minimize losses. However, due to poor lighting conditions and small crack sizes in underground coal mines, cracks are difficult to detect. Therefore, how to efficiently detect coal mine wall cracks is receiving increasing attention from the coal mining industry. Manual detection of coal mine wall cracks usually requires professional personnel to go down into the mine for exploration. This method is not only inefficient and costly, but also prone to missing detections and carries safety hazards. Therefore, using underground robots to replace manual labor is an effective and safe approach.

[0003] Traditional crack detection algorithms are typically based on classic digital image processing methods, such as using various edge detection operators like Sobel and Robberts. These classic methods have limited applicability and struggle to achieve accurate results when used to detect cracks in the complex backgrounds of coal mine walls. With the rise of deep learning, many deep neural networks, such as SegNet and Deepcrack, have been used to detect road cracks. However, compared to road crack detection, coal mine wall crack detection needs to overcome complex environmental interference and poor lighting conditions, which often lead to missed or false detections of small cracks in the mine wall. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to provide a method for identifying coal mine wall cracks based on deep neural networks. This method is based on the Efficientdet detection network and carefully designs a multi-spatial perspective fusion module and a ternary coordinate attention module to overcome complex environmental interference and poor lighting conditions, thereby improving the accuracy of identification and detection.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A method for identifying cracks in coal mine shafts based on deep neural networks includes the following steps:

[0007] Step 100: The data collection module collects the original coal mine wall crack video data and the original coal mine wall crack image data in chronological order from the data collected by the underground robot in the mine. The original coal mine wall crack video data is divided into frame-by-frame segmented coal mine wall crack image data, and the segmented coal mine wall crack image data and the original coal mine wall crack image data are saved to the data collection module.

[0008] Step 200: The data processing enhancement module acquires the wellbore fracture image data collected by the data collection module, and manually removes poor-quality wellbore fracture image data and image data without fractures; the filtered wellbore fracture image data is then enhanced.

[0009] Step 300: Input the high-quality wellbore fracture image data obtained after processing using the data processing enhancement module into the dataset construction module; manually label the fracture regions on the high-quality wellbore image data; divide the labeled image data into training and testing sets according to the proportion to complete the construction of the wellbore fracture dataset;

[0010] Step 400: Input the wellbore fracture dataset built by the dataset construction module into the improved Efficientdet model training module for training and learning. Save the weight model with the highest fracture detection accuracy in the test set during the training process, and use it to detect the wellbore fracture image to be inspected.

[0011] Step 500: Send the image of the well wall crack to be detected to the result detection module. The weighted model with the highest crack detection accuracy obtained in step 400 is used for detection. The detection results are displayed and saved.

[0012] The aforementioned method for identifying coal mine wall cracks based on deep neural networks includes the following sub-steps in step 200:

[0013] Step 210: First, the collected wellbore crack image data is manually screened to remove blurry wellbore crack images caused by the movement of the downhole robot and normal images without cracks; the number of wellbore crack images after screening is greater than or equal to 800.

[0014] Step 220: Use the CLAHE algorithm from the alumensations library to enhance the contrast of the image, so that the wellbore fracture image data after contrast enhancement can be detected by the deep neural network.

[0015] Step 230: In order to expand the limited database and train a robust detection model, data augmentation is performed on the high-quality wellbore fracture image data that has been manually selected. The high-quality wellbore fracture image data is then subjected to horizontal / vertical flipping, rotation, scaling, cropping, shearing and translation processing.

[0016] The aforementioned method for identifying coal mine wall cracks based on deep neural networks includes the following steps in step 300, the dataset construction module:

[0017] Step 310: Divide the amplified wellbore fracture image data into a training set and a test set in an 8:2 ratio; the training set is used to train the model, and the test set is used to test the detection accuracy of the model.

[0018] Step 320: Use Label Img software to label the crack areas in the divided wellbore crack image. When labeling, be careful to avoid interfering objects within the label box. If necessary, divide longer wellbore cracks into several segments for labeling.

[0019] The aforementioned method for identifying coal mine wall cracks based on deep neural networks, step 400, improving the Efficienedet model training module, includes the following steps:

[0020] Step 410: Construct an improved backbone feature extraction network: The improved backbone feature extraction network incorporates a multi-spatial view fusion module and a ternary coordinate attention module. The first three layers of the multi-level features are still composed of Efficient feature layers of different depths, named P3, P4, and P5 respectively; the features of the last two layers are: P6 is formed by P5 through the multi-spatial view fusion module and upsampling; P7 is formed by P6 through the multi-spatial fusion block and upsampling.

[0021] Step 420: Input the multi-level backbone features P3, P4, P5, P6, P7 constructed in step 410 into the ternary coordinate attention module to form improved multi-level backbone features P3′, P4′, P5′, P6′, P7′.

[0022] Step 430: Feed the improved multi-level backbone features P3′, P4′, P5′, P6′, P7′ into the enhanced feature extraction network BiFPN. By fusing the improved multi-level features, more comprehensive multi-level enhanced features P3″, P4″, P5″, P6″, P7″ are obtained.

[0023] Step 440: Input the multi-level enhanced features P3″, P4″, P5″, P6″, P7″ into the decoder Class PredictionNet and Box PredictionNet to obtain the predicted target category and the location of the predicted box;

[0024] Step 450: Input the position of the predicted bounding box obtained in step 440 and the actual predicted bounding box of the labeled wellbore fracture dataset into formula (1) to calculate the loss value;

[0025] L=αLce +βL focal (1)

[0026] Where α and β are hyperparameters, both set to 0.5; L ce Let L be the binary cross-entropy loss function. focal The focal loss function is used, and the two are calculated as follows:

[0027] L ce = -ylog(p) - (1-y)log(1-p) (2)

[0028] L focal = -y(1-p) γ log(p)-(1-y)p γ log(1-p) (3)

[0029] In equations (2) and (3), y represents the true label value of the data; p represents the label value predicted by the model; and γ represents the weight parameter, which is usually set to 2.

[0030] Step 460: Backpropagate the loss value calculated in step 450 to update the network parameters. Repeat steps 410-460 until the model converges. Based on practical experience, the number of training rounds is set to 30.

[0031] The aforementioned method for identifying coal mine wall cracks based on deep neural networks includes the following sub-steps in step 410: Establishing a multi-spatial perspective fusion module.

[0032] Step 411: Input the features P5 extracted by the Efficient backbone network into the multi-spatial view fusion module;

[0033] Step 412: P5∈R N×C×H×W Divide into two parts along the channel dimension: X1∈R N×αC×H×W ,X2∈R N×(1-α)C×H×W X1 is the original feature, which does not participate in the multi-spatial view fusion operation, ensuring that the original feature map is accessible; X2 participates in the multi-view modeling operation; where N, C, H, W and α represent the training data batch size, number of channels, image height, image width and channel segmentation rate, respectively.

[0034] Step 413: First, feed X2 into multi-view branch 1; X2 is then processed by 1*1 convolution, 3*1 convolution, and 1*3 convolution to obtain...

[0035] Step 414: Then feed X2 into multi-view branch 2; X2 is processed by 1*1 convolution, 5*1 convolution and 1*5 convolution respectively to obtain

[0036] Step 415: Finally, feed X2 into the multi-view branch 3; X2 is then processed by 1*1 convolution, 7*1 convolution, and 1*7 convolution to obtain...

[0037] Step 416: Calculate the multi-view spatial feature X2′ using formula (4);

[0038]

[0039] Where δ is the activation function, and β1, β2, β3 are parameters that can be learned and updated through gradient backpropagation;

[0040] Step 417: Calculate the overall fusion feature X using formula (5);

[0041] X = Concat(X1, X2′) (5)

[0042] Concat represents concatenating features along the channel dimension.

[0043] The aforementioned method for identifying coal mine wall cracks based on deep neural networks includes the following sub-steps in step 420: The ternary coordinate attention module includes the following sub-steps:

[0044] Step 421: Convert the multi-level feature P constructed in step 410 3, P4, P5, P6, and P7 are fed into the ternary coordinate attention module to obtain improved multi-level features P3′, P4′, P5′, P6′, and P7′.

[0045] Step 422: Taking feature layer P3 as an example, first set P3∈R C×H×W The input is fed into branch 1 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the H direction, aggregating information from the vertical direction to obtain X1∈R. C×H×1 Next, the pooled features are fed into a 3*1 convolution to explore the horizontal wellbore fracture region features; then, batch normalization and sigmoid activation are used to generate a spatial weight feature map M1; finally, the generated M1 is multiplied element-wise with the corresponding elements of P3 to obtain the enhanced feature map P3 of branch 1. 1 The calculation formula for the spatial weight feature map M1 of branch 1 of the ternary coordinate attention module is shown in formula (6).

[0046] Step 423: Next, P3∈R C×H×W The input is fed into branch 2 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the entire spatial direction, aggregating information from the channel dimensions to obtain X2∈R. 1×H×WNext, the pooled features are fed into a 3x3 convolution to explore the crack region features in the spatial direction; then, batch normalization and sigmoid activation are used to generate the spatial weight feature map M2; finally, the generated M2 is multiplied element-wise with the corresponding elements of P3 to obtain the enhanced feature map P3 of branch 2. 2 The calculation formula for the spatial weight feature map M2 of branch 2 of the ternary coordinate attention module is shown in formula (6).

[0047] Step 424: Finally, P3∈R C×H×W The input is fed into branch 3 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the W direction, aggregating information from the horizontal direction to obtain X3∈R. C×1×W Next, the pooled features are fed into a 1*3 convolution to explore the vertical wellbore fracture region features; then, batch normalization and sigmoid activation are used to generate spatial weight feature map M3; finally, the generated M3 is multiplied element-wise with P3 to obtain the enhanced feature map P3 of branch 3. 3 The calculation formula for the spatial weight feature map M3 of branch 3 of the ternary coordinate attention module is shown in formula (6).

[0048] Step 425: Calculate the enhanced feature map P3 obtained from each branch of the ternary coordinate attention module. 1 P3 2 P3 3 The channels are concatenated according to the channel dimension and the number of channels is restored by 1*1 convolution to obtain the final improved multi-level feature P3′, and its calculation formula is shown in formula (7).

[0049] M i =δ(Conv i (Avgpool i (P3))) (6)

[0050] P3′=Conv 1×1 (Concat(P3 1 P3 2 P3 3 (7)

[0051] Step 426: Other hierarchical features such as P4′, P5′, P6′, and P7′ are constructed in the same way as P3′, and finally the improved multi-level backbone features P3′, P4′, P5′, P6′, and P7′ are obtained.

[0052] The above-mentioned method for identifying coal mine wall cracks based on deep neural networks includes the following sub-steps in step 500:

[0053] Step 510: Deploy the optimal network weights obtained in step 400 to the robot server of the result detection module;

[0054] Step 520: The image of the well wall crack to be detected is sent from the client to the robot server to obtain the identification and detection results and save them.

[0055] The technical solution of the present invention achieves the following beneficial technical effects:

[0056] This invention utilizes an improved Efficientdet network to efficiently identify and detect cracks in coal mine shafts. By adding a multi-spatial-view fusion module, the network can extract features and detailed information about cracks of different sizes and shapes, thereby reducing the false negative rate for small-scale cracks from 25% to 17%. (The difficulty in reducing the false negative rate and improving detection accuracy lies in the fact that crack images collected from coal mine shafts differ from other obvious crack detection tasks (such as highway crack detection). Cracks in coal mine shafts are more subtle and less obvious, making it a more fine-grained detection task. When there are many small cracks in the image, some inconspicuous cracks will inevitably be missed, which is a challenge currently faced by many detection tasks.) Simultaneously, by adding a ternary coordinate attention module to redistribute and fuse spatial feature weights in different spatial directions, the network can effectively suppress noise features and emphasize crack features, thus minimizing false detections of shaft cracks.

[0057] This invention improves the Efficientdet model by adding a multi-spatial perspective fusion module and a ternary coordinate attention module to the existing Efficientdet model. This enables the model to overcome the interference of complex underground coal mine environments and solve the problem of missing small cracks, increasing the detection accuracy from 75% to 83%. Attached Figure Description

[0058] Figure 1 A flowchart of a method for identifying coal mine wall cracks based on deep neural networks according to the present invention;

[0059] Figure 2 This invention provides a visual explanation of the overall process for detecting cracks in coal mine shafts;

[0060] Figure 3 Overall structural diagram of the improved Efficientdet model of this invention;

[0061] Figure 4 A structural diagram of the multi-spatial perspective fusion module of this invention;

[0062] Figure 5 The implementation structure diagram of the ternary coordinate attention module of the present invention.

[0063] The reference numerals in the figure are as follows: 101-Data acquisition by downhole robot, 102-Data collection module, 103-Data processing enhancement module, 104-Dataset construction module, 105-Improved Efficientdet model training module, 106-Result detection module. Detailed Implementation

[0064] Example 1: Coal Mine Wall Crack Identification System Based on Deep Neural Network.

[0065] A coal mine wall crack identification system includes an underground robot for data collection 101, a data collection module 102, a data processing and enhancement module 103, a dataset construction module 104, an improved Efficientdet model training module 105, and a result detection module 106.

[0066] The data collection module 102, data processing enhancement module 103, dataset construction module 104, and improved Efficientdet model training module 105 are all stored in the computer system's memory; the result detection module 106 is deployed in the downhole robot server. The downhole robot collects data 101, including: original coal mine wall crack video data and original coal mine wall crack image data.

[0067] Example 2: A method for detecting cracks in the mine shaft using an underground robot, such as... Figure 1 As shown.

[0068] Step 100: The data collection module 102 collects visual data from the underground robot data collection 101 in chronological order. The underground robot data collection 101 includes original coal mine wall crack video data and original coal mine wall crack image data. The original coal mine wall crack video data is divided into frame-by-frame segmented coal mine wall crack image data, and the original coal mine wall crack image data and the segmented coal mine wall crack image data are saved to the data collection module 102.

[0069] Step 200: The data processing enhancement module 103 acquires the wellbore fracture image data collected by the data collection module 102, and manually removes poor-quality wellbore fracture image data and image data without fractures; the filtered wellbore fracture image data is then enhanced; 800 wellbore fracture image data are obtained after filtering.

[0070] Step 210: First, the collected wellbore crack image data are manually screened to remove blurry wellbore crack images caused by the movement of the downhole robot and normal crack-free images.

[0071] Step 220: Due to the low light intensity in the mine, the acquired image data has low clarity and contrast. The CLAHE algorithm from the alumensations library is used to enhance the contrast of the images, so that the image data of the well wall cracks after contrast enhancement can be detected by a deep neural network;

[0072] Step 230: In order to expand the limited database and train a robust detection model, data augmentation is performed on the high-quality wellbore fracture image data that has been manually selected. The high-quality wellbore fracture image data is then subjected to horizontal / vertical flipping, rotation, scaling, cropping, shearing and translation processing.

[0073] Step 300: Input the high-quality wellbore fracture image data obtained after processing by the data processing enhancement module 103 into the dataset construction module 104; manually label the fracture areas on the high-quality wellbore image data; divide the labeled image data into training set and test set according to the ratio to complete the construction of the wellbore fracture dataset;

[0074] Step 310: Divide the amplified wellbore fracture image data into a training set and a test set in an 8:2 ratio; the training set is used to train the model, and the test set is used to test the detection accuracy of the model.

[0075] Step 320: Use Label Img software to label the crack areas in the divided wellbore crack image. When labeling, be careful to avoid interfering objects within the label box. If necessary, divide longer wellbore cracks into several segments for labeling.

[0076] Step 400: The well wall fracture dataset established by the dataset construction module 104 is sent to the improved Efficientdet model training module 105 for training. The weight model with the highest fracture detection accuracy in the test set during the training process is saved and used to detect the well wall fracture image to be inspected.

[0077] Step 500: The image of the well wall crack to be detected is sent to the result detection module 106. The crack detection is performed by the weighted model with the highest crack detection accuracy obtained in step 400, and the detection results are displayed and saved.

[0078] Step 510: Deploy the optimal network weights obtained in step 400 to the robot server of the result detection module 106;

[0079] Step 520: The image of the well wall crack to be detected is sent from the client to the robot server to obtain the identification and detection results and save them.

[0080] Reference Figure 2This document provides a visual explanation of the overall process for detecting cracks in coal mine shafts. The enhanced crack image data is processed to form a dataset, which is then divided and fed into an improved Efficientdet model for training. Crack detection is then performed, and the results are reported.

[0081] The following sections will explain the improved Efficienedet model training module and the multi-spatial view fusion module, respectively.

[0082] Figure 3 The specific process for improving the Efficientdet network structure is as follows:

[0083] The improved Efficient model consists of three main parts: an improved backbone feature extraction network, an enhanced feature extraction network (BiFPN), and a prediction result decoder network (Class prediction net and Box prediction net). Our improvement to the classic Efficientdet model mainly lies in the backbone feature extraction network.

[0084] First, an improved backbone feature extraction network is constructed. The basic architecture of a typical Efficientdet network is Efficient. The first three layers of the multi-level backbone feature network are composed of feature layers of different depths and sizes from Efficient, named P3, P4, and P5 respectively; the last two feature layers are formed by upsampling P5 twice, named P6 and P7 respectively.

[0085] The improved backbone feature extraction network incorporates a multi-spatial view fusion module and a ternary coordinate attention module. The first three layers of the multi-level features are still composed of feature layers of different depths from Efficient, named P3, P4, and P5 respectively. The features of the last two layers are formed by P5 through the multi-spatial view fusion module and upsampling to form P6; and P6 is formed by P6 through the multi-spatial fusion module and upsampling to form P7.

[0086] Secondly, the constructed multi-level backbone feature P 3, P4, P5, P6, and P7 are fed into the ternary coordinate attention module to form improved multi-level backbone features P3′, P4′, P5′, P6′, and P7′.

[0087] Then, the improved multi-level backbone features P3′, P4′, P5′, P6′, P7′ are fed into the enhanced feature extraction network (BiFPN). By fusing the improved multi-level features, more comprehensive multi-level enhanced features P3″, P4″, P5″, P6″, P7″ are obtained.

[0088] Finally, the multi-level enhanced features P3″, P4″, P5″, P6″, and P7″ are fed into the decoder Class Prediction Net and Box Prediction Net to obtain the predicted target category and the location of the predicted bounding box. The obtained predicted bounding box locations and the ground truth predicted bounding boxes in the labeled dataset are input into formula (1) to calculate the loss value.

[0089] L=αL ce +βL focal (1)

[0090] Where α and β are hyperparameters, both are set to 0.5 in this application. L ce Let L be the binary cross-entropy loss function. focal The focal loss function is used, and the two are calculated as follows:

[0091] L ce = -ylog(p) - (1-y)log(1-p) (2)

[0092] L focal = -y(1-p) γ log(p)-(1-y)p γ log(1-p) (3)

[0093] In equations (2) and (3), y represents the true label value of the data; p represents the predicted label value of the model; γ represents the weight parameter, which is usually set to 2; the calculated loss value is used for backpropagation to update the network parameters, and the above steps are repeated until the model converges. Based on practical experience, the training iteration is set to 30 rounds.

[0094] The Multi-Spatial View Fusion (MSF) module achieves this through convolutions of different sizes, as shown in the reference... Figure 4 :

[0095] Step 1: Feed the P5 features extracted by the Efficient backbone network into the multi-spatial view fusion module;

[0096] Step 2: P5∈R N×C×H×W Divide into two parts along the channel dimension: X1∈R N×αC×H×W ,X2∈R N×(1-α)C×H×W , where N, C, H, W and α represent the training data batch size, number of channels, image height, image width and channel segmentation rate, respectively.

[0097] X1 represents the original feature, which does not participate in the multi-spatial view fusion operation, ensuring that the original feature map is accessible; X2 participates in the multi-view modeling operation.

[0098] First, X2 is fed into multi-view branch 1; X2 is then processed by 1*1 convolution, 3*1 convolution, and 1*3 convolution to obtain...

[0099] Then X2 is fed into multi-view branch 2; X2 is then processed by 1*1 convolution, 5*1 convolution and 1*5 convolution respectively to obtain

[0100] Finally, X2 is fed into the multi-view branch 3; X2 is obtained by performing 1*1 convolution, 7*1 convolution and 1*7 convolution respectively.

[0101] Step 3: Calculate the multi-view spatial feature X2′ using formula (4)

[0102]

[0103] Where δ is the activation function, and β1, β2, and β3 are parameters that can be learned and updated. These three parameters are defined in the network, and therefore can be set to participate in gradient backpropagation for autonomous learning.

[0104] Step 4: Calculate the overall fusion feature X using formula (5).

[0105] X = Concat(X1, X2′) (5)

[0106] Concat represents concatenating features along the channel dimension.

[0107] The final calculated X is the output of the multi-space view fusion module. X is upsampled once to obtain P6, and P6 is then passed through the multi-space view fusion module and upsampled again to obtain P7.

[0108] Reference Figure 5 This is a structural diagram of the ternary coordinate attention module (TCA) designed in this invention.

[0109] First, the multi-level features P3, P4, P5, P6, and P7 are fed into the ternary coordinate attention module to obtain improved multi-level features P3′, P4′, P5′, P6′, and P7′.

[0110] Taking P3 as an example, first let P3∈R C×H×W The input is fed into branch 1 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the H direction, aggregating information from the vertical direction to obtain X1∈R. C×H×1 .

[0111] Next, the pooled features are fed into a 3*1 convolution to explore the horizontal wellbore fracture region features; then, batch normalization and sigmoid activation are applied to generate the spatial weight feature map M1. Finally, the generated M1 is multiplied element-wise with the corresponding elements of P3 to obtain the enhanced feature map P3 of branch 1. 1 The calculation formula for the spatial weight feature map M1 of branch 1 of the ternary coordinate attention module is shown in formula (6).

[0112] Next, let P3∈R C×H×W The input is fed into branch 2 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the entire spatial direction, aggregating information from the channel dimensions to obtain X2∈R. 1×H×W Next, the pooled features are fed into a 3x3 convolution to explore the crack region features in the spatial direction; then, batch normalization and sigmoid activation are used to generate the spatial weight feature map M2. Finally, the generated M2 is multiplied element-wise with the corresponding elements of P3 to obtain the enhanced feature map P3 of branch 2. 2 The calculation formula for the spatial weight feature map M2 of branch 2 of the ternary coordinate attention module is shown in formula (6).

[0113] Finally, P3∈R C×H×W The input is fed into branch 3 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the W direction, aggregating information from the horizontal direction to obtain X3∈R. C×1×W Next, the pooled features are fed into a 1*3 convolution to explore the vertical wellbore fracture region features; then, batch normalization and sigmoid activation are used to generate a spatial weight map. Finally, the generated M3 and P3 are multiplied element-wise to obtain the enhanced feature map P3 of branch 3. 3 The calculation formula for the spatial weight feature map M3 of branch 3 of the ternary coordinate attention module is shown in formula (6).

[0114] The enhanced feature map P3 obtained from each branch of the ternary coordinate attention module 1 P3 2 P3 3 The channels are concatenated along the channel dimension and the number of channels is restored by 1*1 convolution to obtain the final improved multi-level feature P3′, and its calculation formula is shown in formula (7).

[0115] M i =δ(Conv i (Avgpool i (P3))) (6)

[0116] P3′=Conv 1×1 (Concat(P3 1 P32 P3 3 (7)

[0117] It is worth noting that P4′, P5′, P6′, and P7′ are constructed in the same way as P3′, and finally, the improved multi-level backbone features P3′, P4′, P5′, P6′, and P7′ are obtained.

[0118] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of the claims of this patent application.

Claims

1. A method for identifying coal mine wall cracks based on deep neural networks, characterized in that, Includes the following steps: Step 100: The data collection module (102) collects the original coal mine wall crack video data and the original coal mine wall crack image data in chronological order from the data collected by the underground robot (101). The original coal mine wall crack video data is divided into frame-by-frame segmented coal mine wall crack image data, and the segmented coal mine wall crack image data and the original coal mine wall crack image data are saved to the data collection module (102). Step 200: The data processing enhancement module (103) acquires the well wall crack image data collected by the data collection module (102), and manually removes the well wall crack image data with poor quality and removes the image data without cracks; and performs data enhancement on the filtered well wall crack image data. Step 300: Input the high-quality wellbore fracture image data obtained after processing by the data processing enhancement module (103) into the dataset construction module (104); manually annotate the fracture areas in the high-quality wellbore image data; The labeled image data is divided into training and testing sets according to the proportions to complete the construction of the well wall fracture dataset; Step 400: Send the well wall crack dataset established by the dataset construction module (104) into the improved Efficientdet model training module (105) for training and learning. Save the weight model with the highest crack detection accuracy in the test set during the training process to detect the well wall crack image to be detected. Step 500: Send the image of the well wall crack to be detected to the result detection module (106), and perform detection using the weighted model with the highest crack detection accuracy obtained in step 400. Display the detection results and save them. Step 400 improves the Efficienedet model training module, including the following steps: Step 410: Constructing an improved backbone feature extraction network: The improved backbone feature extraction network incorporates a multi-spatial view fusion module and a ternary coordinate attention module. The first three layers of the multi-level features still consist of Efficient feature layers of different depths, named as follows: The characteristics of the last two layers are as follows: Formed through multi-spatial perspective fusion module and upsampling ;Depend on Formed through multi-spatial perspective fusion module and upsampling ; Step 420: Develop the multi-level backbone features constructed in step 410. Feed into the ternary coordinate attention module to form improved multi-level backbone features ; Step 430: Improve the multi-level backbone features The data is fed into the BiFPN enhanced feature extraction network, which fuses the improved multi-level features to obtain more comprehensive multi-level enhanced features. ; Step 440: Enhance multi-level features The data is fed into the Class Prediction Net and Box Prediction Net decoders to obtain the predicted target class and the location of the predicted bounding box. Step 450: Input the position of the predicted box obtained in step 440 and the actual predicted box in the labeled wellbore fracture dataset into formula (1) to calculate the loss value; in For hyperparameters, All are set to 0.5; The binary cross-entropy loss function is... The focal loss function is used, and the two are calculated as follows: （2） （3） In equations (2) and (3), y This represents the actual label value of the data; p This represents the model's predicted label value; γ This represents the weight parameter, set to 2; Step 460: Backpropagate the loss value calculated in step 450 to update the network parameters. Repeat steps 410-460 until the model converges. Based on practical experience, the number of training rounds is set to 30. Step 410 includes the following sub-steps: Step 411: Extract features from the Efficient backbone network Send it to the multi-spatial perspective fusion module; Step 412: Divide into two parts along the channel dimension: , ;in These are the original features and do not participate in multi-spatial view fusion operations, ensuring that the original feature map is accessible. Participate in multi-view modeling operations; among them N , C , H , W and α These represent the training data batch size, number of channels, image height, image width, and channel segmentation ratio, respectively. Step 413: First, Send to multi-view branch 1; Obtained by 1*1 convolution, 3*1 convolution and 1*3 convolution respectively ; Step 414: Then Send to multi-view branch 2; Obtained by 1*1 convolution, 5*1 convolution and 1*5 convolution respectively ; Step 415: Finally, Send to multi-view branch 3; Obtained by 1*1 convolution, 7*1 convolution and 1*7 convolution respectively ; Step 416: Calculate multi-view spatial features using formula (4) ; （4） in For activation function, These are the parameters that can be learned and updated through gradient backpropagation; Step 417: Calculate the overall fusion features using formula (5) X ; （5） in This means that features are stitched together along the channel dimension.

2. The method for identifying coal mine wall cracks based on deep neural networks according to claim 1, characterized in that, Step 200 includes the following sub-steps: Step 210: First, the collected wellbore crack image data is manually screened to remove blurry wellbore crack images caused by the movement of the downhole robot and normal images without cracks; the number of wellbore crack images after screening is greater than or equal to 800. Step 220: Use the CLAHE algorithm from the alumensations library to enhance the contrast of the image, so that the wellbore fracture image data after contrast enhancement can be detected by the deep neural network. Step 230: In order to expand the limited database and train a robust detection model, data augmentation is performed on the high-quality wellbore fracture image data that has been manually selected. The high-quality wellbore fracture image data is then subjected to horizontal / vertical flipping, rotation, scaling, cropping, shearing and translation processing.

3. The method for identifying coal mine wall cracks based on deep neural networks according to claim 1, characterized in that, Step 300 includes the following steps: Step 310: Divide the amplified wellbore fracture image data into a training set and a test set in an 8:2 ratio; the training set is used to train the model, and the test set is used to test the detection accuracy of the model. Step 320: Use Label Img software to label the crack areas in the divided wellbore crack image. When labeling, be careful to avoid interfering objects within the label box. Divide the wellbore crack into several segments for labeling.

4. The method for identifying coal mine wall cracks based on deep neural networks according to claim 1, characterized in that, Step 420 includes the following sub-steps: Step 421: Develop the multi-level features constructed in step 410 The data are fed into the ternary coordinate attention module to obtain improved multi-level features. ; Step 422: Using the feature layer For example, firstly The input is given to branch 1 of the ternary coordinate attention module, and it is then directed along... H Adaptive global average pooling is performed in the direction to aggregate information in the vertical direction. Next, the pooled features are fed into a 3*1 convolution to explore the features of the wellbore fracture region in the horizontal direction. Then, after batch normalization and sigmoid activation, a spatial weight feature map is generated. Finally, the generated and Multiply corresponding elements to obtain the enhanced feature map of branch 1. Spatial weight feature map of branch 1 of the ternary coordinate attention module The calculation formula is shown in formula (6); Step 423: Next, The input is fed into branch 2 of the ternary coordinate attention module, where it undergoes adaptive global average pooling along the entire spatial direction, aggregating information from the channel dimensions to obtain... Next, the pooled features are fed into a 3x3 convolution to explore the crack region features in the spatial direction. Then, after batch normalization and sigmoid activation, a spatial weight feature map is generated. M 2 Finally, the generated and Multiply corresponding elements to obtain the enhanced feature map of branch 2. Spatial weight feature map of branch 2 of the ternary coordinate attention module The calculation formula is shown in formula (6); Step 424: Finally, The input is given to branch 3 of the ternary coordinate attention module, and it is then directed along... W Adaptive global average pooling is performed in the direction to aggregate information in the horizontal direction. Next, the pooled features are fed into a 1*3 convolution to explore the features of the vertical wellbore fracture region. Then, after batch normalization and sigmoid activation, a spatial weight feature map is generated. Finally, the generated and Multiply corresponding elements to obtain the enhanced feature map of branch 3. Spatial weight feature map of branch 3 of the ternary coordinate attention module The calculation formula is shown in formula (6); Step 425: Combine the enhanced feature maps obtained from each branch of the ternary coordinate attention module. The channels are concatenated along the channel dimension and then recovered using a 1x1 convolution to obtain the final improved multi-level features. The calculation formula is shown in formula (7); （6） （7） Step 426: Other hierarchical features such as Construction method and Similarly, the improved multi-level backbone features were finally obtained. .

5. The method for identifying coal mine wall cracks based on deep neural networks according to claim 1, characterized in that, Step 500 includes the following sub-steps: Step 510: Deploy the optimal network weights obtained in step 400 to the robot server of the result detection module (106); Step 520: The image of the well wall crack to be detected is sent from the client to the robot server to obtain the identification and detection results and save them.

Images