Coal mine shaft micro-crack detection method and device and storage medium

Abstract

The invention provides a coal mine shaft micro-crack detection method and device and a storage medium, and the method comprises the steps: S1, constructing a data set of shaft micro-cracks, and dividing the data set into a training set, a verification set and a test set; s2, constructing a wellbore micro-crack detection model; s21, constructing a backbone network; s22, constructing a neck feature fusion network; s23, constructing a head network; and S3, training the detection model by using the training set, and updating new parameters of the model. The problems that in an existing detection method, feature information is difficult to effectively extract, noise interference is likely to happen, and false detection and missing detection are likely to happen are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method, a device and a storage medium for detecting micro-cracks in a coal mine shaft. Background Art

[0003] The appearance of cracks in the shaft wall is an early warning of shaft wall structural failure. Since the cracks are relatively small in the initial stage, they will not bring serious consequences. However, if the cracks in the shaft wall are not monitored in time and stop-loss measures are not taken, the shaft wall cracks may further evolve into serious cracks, causing shaft wall structural failure, threatening the stability and safe operation of the shaft, and even causing huge losses of life and property. In order to avoid this situation, effective monitoring methods are needed to monitor shaft wall damage and control it to ensure the safety of underground mining.

[0004] With the continuous development of computer technology, collecting wellbore images and using algorithms to detect cracks has become a popular research direction. The YOLO algorithm has become a popular algorithm in the field of deep learning due to its advantages such as high real-time detection and simple structure. Scholars have studied defect detection from different strategies such as increasing the receptive field, multi-scale feature fusion, and model lightweighting. However, due to the particularity of the detection environment of wellbore cracks, there are still some difficulties, such as: the shapes and sizes of crack diseases vary greatly, and it is difficult for the model to effectively extract feature information; the wellbore is not illuminated enough, and the background distinction of micro-cracks is small and easily affected by noise; the details of the crack image are blurred, which is prone to false detection and missed detection.

[0005] Therefore, there is an urgent need for a coal mine shaft microcrack detection method that can maintain the lightweight of the model while strengthening the model feature characterization capability, thus achieving a balance between model accuracy and lightweight. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a method, device and storage medium for detecting micro-cracks in a coal mine shaft, so as to solve the problems that the existing detection methods are difficult to effectively extract feature information, are easily interfered by noise, and are prone to false detection and missed detection.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A method for detecting micro-cracks in a coal mine shaft comprises the following steps:

[0009] S1: Construct a dataset of wellbore microfractures and divide it into training set, validation set and test set;

[0010] S2: Construct a wellbore micro-fracture detection model;

[0011] S21: Build backbone network;

[0012] S22: Construct neck feature fusion network;

[0013] S23: build the head network;

[0014] S3: Use the training set to train the detection model and update the new model parameters.

[0015] Further, the S21 includes:

[0016] S211: Construct a basic convolution module CBS; the basic convolution module CBS consists of a convolution layer Conv, a batch normalization layer BN, and an activation function SiLU in sequence;

[0017] S212: Construct a deformable convolution module CN-CBS; the deformable convolution module CN-CBS is composed of a deformable convolution layer DCNv2, a batch normalization layer BN, and an activation function SiLU in sequence; three deformable convolution modules CN-CBS form a parallel sub-network;

[0018] In the deformable convolution layer DCNv2, the calculation formula of the deformable convolution layer is:

[0019]

[0020] Among them, y(p) represents the feature value of the output feature map at position p, x represents the input feature map, p represents the position of the output feature map, and p k represents the initial position of the kth sampling point relative to the center of the convolution kernel, Δp k represents the position offset of the kth sampling point, Δmk represents the characteristic adjustment coefficient of the kth sampling point, and w k represents the weight of the kth sampling point, k represents the sequence number of the sampling point, and K represents the total number of sampling points.

[0021] Furthermore, the S21 further includes:

[0022] S213: constructing a deformable feature fusion module CN-C2f;

[0023] In the deformable feature fusion module CN-C2f, the input feature map passes through the first basic convolution module CBS for feature extraction and activation, and then passes through the Split layer for channel segmentation operation, dividing the feature map into two branches. The first branch passes through N D-Bottleneck modules in sequence, and the second branch does not pass through the D-Bottleneck module. After that, all D-Bottleneck module outputs are concat-joined with the second branch feature map, and then pass through the second basic convolution module CBS to obtain the output feature map of the deformable feature fusion module CN-C2f;

[0024] S214: Construct a multi-scale attention feature fusion module EMA-C2f; the multi-scale attention feature fusion module EMA-C2f is composed of a basic convolution module CBS, a BottleNeck module, and an efficient multi-scale attention mechanism module EMA;

[0025] In the multi-scale attention feature fusion module EMA-C2f, the input feature map passes through the first basic convolution module CBS, and then the Split layer is used to divide the output feature map of the first basic convolution module CBS into two branches; the first branch is processed by N Bottleneck modules in sequence; the Bottleneck module is sequentially constructed by convolution, residual structure, and SiLU activation function; the second branch is not processed by the Bottleneck module, and all Bottleneck module outputs are concat-concatenated with the second branch feature map, and pass through the second basic convolution module CBS and the efficient multi-scale attention mechanism module EMA in sequence to obtain the output of the multi-scale attention feature fusion module EMA-C2f.

[0026] Furthermore, the S21 further includes:

[0027] S215: constructing a spatial depth conversion convolution module SPDI;

[0028] In the spatial depth conversion convolution module SPDI, for the input feature map S1×S2×C1, S1, S2, and C1 represent the height, width, and number of channels of the feature map respectively; first convert the input feature map size to S1 / 2×S2 / 2×4C1, and then obtain four sub-feature maps through slicing operation Then the four sub-feature maps are concat-joined and passed through a convolution layer. The Split operation is used to divide the feature map into four branches. The four branches pass through different depth-wise separable convolution layers DWConv respectively. The convolution kernel sizes of the depth-wise separable convolution layers DWConv in the first three branches are 3x3, 1×11, and 11×1 respectively. The depth-wise separable convolution layer DWConv operation is not performed in the last branch. The outputs of each branch are concat-joined and the outputs of all branches are fused together through a convolution layer to generate the output feature map of the spatial depth conversion convolution module SPDI.

[0029] Furthermore, the S21 further includes:

[0030] S216: constructing a spatial pyramid pooling module E-SPPFCSPC;

[0031] In the spatial pyramid pooling module E-SPPFCSPC, the input feature map is divided into two branches after passing through the first basic convolution module CBS. The first branch inputs the obtained feature map into the SPP structure, which includes pooling layers of different sizes, and then concats the outputs of different pooling layers, and then passes through the second basic convolution module CBS; the second branch inputs the obtained feature map into the third basic convolution module CBS, concats the output feature map of the second basic convolution module CBS with the output feature map of the third basic convolution module CBS, and then passes through the efficient multi-scale attention mechanism module EMA and the fourth basic convolution module CBS in turn to obtain the output of E-SPPFCSPC.

[0032] Furthermore, in the backbone network, the input image passes through the first basic convolution module CBS, the second basic convolution module CBS, the first spatial depth conversion convolution module SPDI, the third basic convolution module CBS, the second spatial depth conversion convolution module SPDI module, the first deformable feature fusion module CN-C2F, the fourth basic convolution module CBS, the third spatial depth conversion convolution module SPDI, the second deformable feature fusion module CN-C2F, the fifth basic convolution module CBS, the multi-scale attention feature fusion module EMA-C2f, and the E-SPPFCSPC module in sequence. At the same time, the feature map passing through the first spatial depth conversion convolution module SPDI is input into the first deformable convolution module CN-CBS through another branch; the feature map obtained through the first deformable convolution module CN-CBS module is divided into two The first branch enters the first deformable feature fusion module CN-C2F, and the second branch enters the second deformable convolution module CN-CBS; the feature map obtained by the second deformable convolution module CN-CBS is divided into two branches, the first branch enters the second deformable feature fusion module CN-C2F, and the second branch enters the third deformable convolution module CN-CBS; the feature map passing through the third deformable convolution module CN-CBS is input into the multi-scale attention feature fusion module EMA-C2f to extract the surface features of the wellbore fractures; the output of the backbone network is three-way, namely the output of the first deformable feature fusion module CN-C2F, the output of the second deformable feature fusion module CN-C2F, and the output of the E-SPPFCSPC module; the three-way features are simultaneously sent to the neck feature fusion network Neck.

[0033] Furthermore, the neck feature fusion network Neck is composed of a basic convolution module CBS, a multi-scale attention feature fusion module EMA-C2f, a deformable feature fusion module CN-C2F module, and an upsampling layer;

[0034] In the neck feature fusion network, the output of the E-SPPFCSPC module in the head network is divided into two branches, and the first upsampling operation is performed in the first branch, and then the feature map after the first upsampling operation is subjected to the first concat splicing operation with the output feature map of the second deformable feature fusion module CN-C2F of the backbone network; the feature map is then divided into two branches through the deformable feature fusion module CN-C2F, and the first branch is subjected to the second upsampling operation, and then the feature map obtained by the second upsampling operation is subjected to the second concat splicing operation with the output feature map of the first deformable feature fusion module CN-C2F of the backbone network; the feature map obtained by the second concat splicing operation is divided into two branches after passing through the first multi-scale attention feature fusion module EMA-C2f, and the first branch is used as the first output of the neck network; the second branch is input to the first basic convolution of the neck network. Module CBS; perform a third concat splicing operation on the feature map passing through the first basic convolution module CBS and the second branch feature map passing through the deformable feature fusion module CN-C2F, and input the feature map obtained by the third concat splicing operation into the second multi-scale attention feature fusion module EMA-C2f, and divide the output of the second multi-scale attention feature fusion module EMA-C2f into two branches, the first branch is used as the second output of the neck network, and the second branch feature map is input into the second basic convolution module CBS; then perform a fourth concat splicing operation on the output feature map of the second basic convolution module CBS and the second branch feature map output by the backbone network E-SPPFCSPC module; input the output of the fourth concat splicing operation into the third multi-scale attention feature fusion module EMA-C2f as the third output of the neck network.

[0035] Further, the head network is composed of a detection output module; the detection output module receives the output of the neck network;

[0036] The detection output module consists of two branches. The first branch consists of the basic convolution module CBS and Conv2d in sequence, and then optimized using the box regression loss. The second branch consists of the basic convolution module CBS module and Conv2d in sequence, and then optimized using the category classification loss.

[0037] In the basic convolution module CBS of the detection output module, FReLU is used as the activation function. The formula of FReLU is as follows:

[0038] f(x c,i,j )=max(x c,i,j ,T(x c,i,j ))

[0039]

[0040] in,

[0041] f(x c,i,j ) represents the output of the feature value at position (i, j) on the cth channel after the FReLU activation function, T(x c,i,j ) represents the spatial condition function; x c,i,j represents the original eigenvalue at position (i, j) on the cth channel, represents the parameters shared in the cth channel, Indicates that in x c,i,j The result of pooling the eigenvalues ​​in a local pooling window of size kh×kw around it, where kh and kw represent the height and width of the local pooling window.

[0042] Another technical solution adopted by the present invention is a coal mine shaft micro-crack detection device, which adopts the above method to realize the detection of coal mine shaft micro-cracks.

[0043] Another technical solution adopted by the present invention is a storage medium for detecting micro-cracks in a coal mine shaft, which adopts the above method to realize the detection of micro-cracks in the coal mine shaft.

[0044] The beneficial effects of the present invention are

[0045] 1. The method of the present invention integrates the deformable convolution module CN-CBS to build a parallel sub-network to improve the network's ability to extract feature information.

[0046] 2. The method of the present invention constructs a deformable feature fusion module CN-C2f, which performs feature transformation on the input data through two convolutional layers, which helps to extract features of different levels and abstract degrees in the input data.

[0047] 3. The method of the present invention combines the advantages of the space-to-depth module SPDConv, the large kernel decomposition operation Inception, and the depth-separable convolution DWConv to design SPDI, which avoids the information loss caused by the strided convolution in the backbone network of the existing detection method. By combining different small-size convolutions in parallel and splicing the outputs in the non-strided convolution, i.e., the convolution layer with a step size of 1 in SPDI and the decomposition of the large convolution kernel, i.e., the SPDI module, the feature learning efficiency and global modeling ability of the model are improved, thereby improving the detection efficiency.

[0048] 4. The method of the present invention introduces an efficient multi-scale attention mechanism EMA, constructs a multi-scale attention feature fusion module EMA-C2F and a spatial pyramid pooling module E-SPPFCSCP. The E-SPPFCSPC module can effectively avoid the distortion after image cropping and scaling, and solve the problem of repeated feature extraction of convolutional neural networks. The EMA-C2F module and the E-SPPFCSPC module can improve the model's ability to characterize microcracks and avoid noise interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a process architecture diagram of an embodiment of the present invention.

[0051] Figure 2 It is a flow chart of the deformable feature fusion module CN-C2f according to an embodiment of the present invention.

[0052] Figure 3 It is a flow chart of the deformable convolution module CN-CBS according to an embodiment of the present invention.

[0053] Figure 4 It is a process architecture diagram of the spatial pyramid pooling module E-SPPFCSPC according to an embodiment of the present invention.

[0054] Figure 5 It is a flow chart of the spatial depth conversion convolution module SPDI according to an embodiment of the present invention.

[0055] Figure 6 It is a process architecture diagram of the multi-scale attention feature fusion module EMA-C2f of an embodiment of the present invention.

[0056] Figure 7 It is a comparison chart of the detection results of the detection method of the embodiment of the present invention and the existing detection method in the wellbore fracture data set.

[0057] Figure 8 It is a comparison chart of the detection results of the detection method of the embodiment of the present invention and the existing detection method on the public data set. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The embodiment of the present invention provides a method, device and storage medium for detecting micro-cracks in a coal mine shaft. The complete steps are as follows:

[0060] S1: Construct a dataset of shaft microcracks; and preprocess the dataset, preprocess the coal mine shaft defect images to convert the image size into the standard size required by the model, which is 640×640.

[0061] Since the number of various crack images in the data set is unevenly distributed, in order to avoid overfitting affecting the training effect, the number of well wall crack images is increased by data enhancement. In order to increase the number of samples, some road crack images are incorporated to simulate well wall crack images; in order to simulate the dim shooting environment underground and the influence of factors such as dust and fog, the brightness of the image is adjusted, and salt and pepper noise and Gaussian noise are added; geometric transformation operations such as translation, flipping, and rotation are used to simulate images taken by the camera from different angles. A total of 2223 usable well wall crack images are generated through the above methods, and the training set, validation set, and test set are randomly divided into 7:2:1.

[0062] S2: Construct a wellbore microfracture detection model, see Figure 1 ,The model consists of three modules: backbone network, neck feature fusion network, and prediction output; the specific steps are described as follows:

[0063] S21: Construct the backbone network. The backbone network Backbone consists of 5 basic convolution modules CBS, 3 deformable convolution modules CN-CBS, 3 spatial depth conversion convolution modules SPDI, 1 attention feature fusion module EMA-C2F, 2 deformable feature fusion modules CN-C2f and a spatial pyramid pooling module E-SPPFCSPC, which is responsible for enhancing the network's extraction effect on wellbore microfracture feature information.

[0064] S211: Construct a CBS module. The CBS module consists of a convolution layer Conv, a batch normalization layer BN, and an activation function SiLU in sequence. The CBS module integrates and compresses the channel information of the input feature map to extract more feature map features; in this embodiment, the size of the convolution layer Conv is 3×3.

[0065] S212: Construct CN-CBS modules. The backbone network Backbone integrates deformable convolution (ConvNets v2) to construct a parallel sub-network, in order not to increase too much computation and to retain feature map information as much as possible. The parallel sub-network consists of three CN-CBS modules. Figure 3 ,The CN-CBS module consists of a deformable convolution layer DCNv2, a batch normalization layer BN, and an activation function SiLU in sequence. A 3×3 convolution kernel is used in the deformable convolution layer DCNv2.

[0066] In Deformable Convolution DCNv2, random sampling around the current position is achieved by adding an offset variable to each sampling point in the convolution kernel. By adjusting the amplitude of the input features at different spatial positions and the weight of each sampling point, the model's ability to focus on the target image area is enhanced. The calculation formula for deformable convolution is:

[0067]

[0068] Among them, y(p) represents the value of the output feature map at position p, x represents the input feature map, p represents the position of the output feature map, and p k represents the initial position of the kth sampling point relative to the center of the convolution kernel, Δp k represents the position offset of the kth sampling point, Δmk represents the characteristic adjustment coefficient of the kth sampling point, and w k represents the weight of the kth sampling point, k represents the sequence number of the sampling point, and K represents the total number of sampling points.

[0069] S213: Construct a deformable feature fusion module CN-C2f. In order to enhance the adaptability of the model, a CN-C2f module is also designed. The CN-C2F module performs feature transformation on the input data through two convolutional layers, which helps to extract features of different levels and abstraction levels in the input data. Figure 2 The CN-C2f module consists of a CBS module, a channel segmentation layer, and a deformable bottleneck module D-Bottleneck module, where D-Bottleneck is composed of two DCNv2 modules connected by residual connections; D-Bottleneck is used to extract multi-scale crack feature information and improve the network's modeling ability for complex data.

[0070] In the CN-C2f module, the input feature map passes through the first CBS module in sequence for feature extraction and activation. This process helps the model capture more contextual information and thus identify the target more accurately. The step size of the convolution layer Conv in this CBS module is 1, and the number of output channels is 1. Then the channel segmentation operation is performed through the Split layer, and the feature map is divided into two branches. The first branch passes through N D-Bottleneck modules in sequence, and the second branch does not pass through the D-Bottleneck module. After that, all D-Bottleneck module outputs are concat-joined with the second branch feature map, and then passed through the second CBS module. The function of this CBS module is to improve the expressiveness of the features and prepare for subsequent feature fusion and dense connection. The step size of the convolution layer Conv in this CBS module is 1, and the number of output channels is 1. The output feature map of the CN-C2f module is obtained.

[0071] S214: Construct a multi-scale attention feature fusion module EMA-C2f. Figure 6 ,The EMA-C2f module is composed of two CBS modules, a bottleneck module BottleNeck, and an EMA attention module;

[0072] In the EMA-C2f module, the input feature map passes through the first CBS module, and then the Split layer is used to divide the output feature map of the first CBS module into two branches; the Split operation splits the convolutional feature map tensor into multiple small tensors to achieve parallel calculation of feature maps and speed up the training speed and efficiency of the model; the first branch is processed by N Bottleneck modules in turn. The Bottleneck module is composed of residual structure, 3×3 convolution and SiLU activation function. It uses partial convolution to implement convolution operation, realizes efficient capture of spatial features and optimization of parameter calculation, and improves the efficiency and performance of the model. The second branch is not processed by the Bottleneck module, and then all Bottleneck module outputs are concat spliced ​​with the second branch feature map in the channel dimension, and then pass through the second CBS module and EMA attention module in turn to obtain the final EMA-C2f module output.

[0073] Among them, the Efficient Multi-Scale Attention (EMA) is an efficient multi-scale attention mechanism without dimensionality reduction. It uses a parallel sub-network structure to improve the efficiency of feature extraction while maintaining a small depth and low latency, reducing the amount of calculation and parameters. The global average pooling operation is used to generate channel statistics, where the global spatial position information is compressed into the channel descriptor, and the 1×1 and 3×3 convolution kernels are placed in the parallel sub-networks respectively. Without reducing the channel dimension, local cross-channel interactions are established in each parallel sub-network. The output of the 1x1 branch encodes the global spatial information through two-dimensional global average pooling, while the output of the 3x3 branch is directly converted to the corresponding dimensional shape. Finally, the feature outputs of the two parallel sub-networks are fused through cross-space learning. The formula for two-dimensional global average pooling is as follows:

[0074]

[0075] Among them, Z C represents the feature value of the feature map at the (i, j)th position in channel c after global average pooling, i and j represent the horizontal and vertical coordinates of the feature map in channel c respectively. H represents the height of the feature map, W represents the width of the feature map, and X c (i,j) represents the eigenvalue of the feature map at the (i,j)th position of channel c.

[0076] In order to facilitate hardware deployment and avoid complex floating-point operations of the Sigmoid function, the Hard Sigmoid activation function is used for fitting to improve computing efficiency.

[0077]

[0078] x represents the input of the Hard Sigmoid function, that is, the output value of the previous layer of the neural network.

[0079] S215: Build SPDI module. Figure 5 Combining the advantages of the space-to-depth module SPDConv and the large kernel decomposition operation Inception, the SPDI module is proposed to improve the feature learning efficiency and global modeling ability of the model through non-strided convolution and large kernel decomposition.

[0080] Non-strided convolution is the convolution layer with a stride of 1 in SPDConv. There are a large number of convolution operations with a stride of 2 in the backbone network of the existing detection method, which is easy to lose feature information. SPDI is used to replace the strided convolution with a stride of 2.

[0081] Using large-kernel deep convolution can enhance the feature extraction effect of the model, but it will increase the number of parameters and computation and reduce the detection speed. The idea of ​​the Inception structure is to combine different small-size convolutions in parallel, and splice the processed results to form a deeper structure. The advantage of the Inception structure is that it can efficiently expand the depth and width of the network, while improving the accuracy of the deep learning network and preventing overfitting. Aggregating information at different sizes makes it easier to extract features from different scales.

[0082] In the SPDI module, for the input feature map (S1, S2, C1), S1, S2, and C1 represent the height, width, and number of channels of the feature map respectively; the input feature map is first converted to S1 / 2×S2 / 2×4C1, that is, the spatial dimension is halved and the number of channels is doubled, and then four sub-feature maps are obtained through slicing operations Then the four sub-feature maps are concat-joined in the channel dimension, and then passed through a common convolution layer (Conv2d K=1). The Split operation is used to divide the feature map into four branches according to the channel dimension. The four branches pass through different DWConv layers. The DWConv layers in the first three branches have different convolution kernel sizes: 3x3, 1×11, and 11×1. The last branch does not perform a DWConv layer operation. The outputs of each branch are concat-joined, and the outputs of all branches are fused together through a common convolution layer (Conv2d K=1) to generate the final output feature map and output it. The SPDI module enhances detection accuracy and learning efficiency by decomposing the parallel operation of the large kernel and the non-strided convolution layer that does not lose feature information.

[0083] S216: Build E-SPPFCSPC module. Figure 4 The E-SPPFCSPC module is composed of a spatial pyramid pooling SPP module, a cross-stage partial network CSP module, and an EMA module. The E-SPPFCSPC module is a serial structure. The E-SPPFCSPC module can effectively avoid the distortion after image cropping and scaling, and solve the problem of repeated feature extraction in convolutional neural networks. The parallel Maxpool structure can increase the network speed and reduce the amount of model calculation. Multiple pooling kernels of different sizes are connected in parallel, which is conducive to retaining more detailed information in the feature map. Through the output of the EMA attention module, the deep network is conducive to focusing on the feature representation of tiny cracks.

[0084] In the CSP module, the input feature map is used to extract features through the first CBS module, and then divided into two branches. The first branch inputs the obtained feature map into the SPP structure. The SPP (Spatial Pyramid Pooling) structure is a pooling operation used to extract receptive field features of different sizes. It includes multiple pooling layers, each of which has a different size and is used to summarize features of different ranges. In this embodiment, in the main branch, the feature map obtained by the first CBS module is divided into three branches, and the maximum pooling operations of sizes: 5×5, 9×9, and 13×13 are performed respectively. The three branch outputs are then concat operated and then passed through the second CBS module; the second branch inputs the obtained feature map into the third CBS module to reduce the amount of calculation and memory consumption. The output feature map of the second CBS module in the first branch is partially concat-connected with the output feature map of the third CBS module in the second branch to form a new feature map, and then further processed by the EMA attention module and the fourth CBS module in turn. By inserting the SPP and EMA modules into the CSP block, the receptive field of the model can be increased, and the feature extraction capability and computational efficiency of the model can be improved.

[0085] In the backbone network, the main branch is: the input image passes through the first CBS module, the second CBS module, the first SPDI module, the third CBS module, the second SPDI module, the first CN-C2F module, the fourth CBS module, the third SPDI module, the second CN-C2F module, the fifth CBS module, the EMA-C2f module, and the E-SPPFCSPC module in sequence. At the same time, the feature map passing through the first SPDI module is input into the first CN-CBS module through another branch; the feature map obtained through the first CN-CBS module is divided into two branches, the first branch enters the first CN-C2F In the module, the second branch enters the second CN-CBS module; the feature map obtained by the second CN-CBS module is divided into two branches, the first branch enters the second CN-C2F module, and the second branch enters the third CN-CBS module; the feature map after the third CN-CBS module is input into the EMA-C2f module; the surface feature extraction of the wellbore fracture is completed, and the output of the backbone network is three-way, namely the output of the first CN-C2F module, the output of the second CN-C2F module, and the output of the E-SPPFCSPC module; the three shallow and deep features are simultaneously sent to the neck feature fusion Neck;

[0086] S22: Construct the neck feature fusion network. The neck feature fusion network Neck is composed of 2 CBS units, 3 EMA-C2f modules, 1 CN-C2F module, and multiple upsampling and splicing operations. It is responsible for improving the model's ability to extract multi-scale cracks and complex features, reducing the amount of model calculations while speeding up the network detection speed.

[0087] In the neck network, the output of the E-SPPFCSPC module in the head network is divided into two branches, and the first upsampling operation is performed in the first branch, and then the feature map after the first upsampling operation is concatenated with the output feature map of the second CN-C2F module of the backbone network for the first concat splicing operation; the feature map is then divided into two branches through the CN-C2F module, and the first branch is subjected to the second upsampling operation, and then the feature map obtained by the second upsampling operation is concatenated with the output feature map of the first CN-C2F module of the backbone network for the second concat splicing operation; the feature map obtained by the second concat splicing operation is divided into two branches after passing through the first EMA-C2f module, and the first branch is used as the first output of the neck network; the second branch is input to the neck In the first CBS module of the network; the feature map passing through the first CBS module and the second branch feature map passing through the CN-C2F module are subjected to a third concat splicing operation, and the feature map obtained by the third concat splicing operation is input into the second EMA-C2f module, and the output of the second EMA-C2f module is divided into two branches, the first branch is used as the second output of the neck network, and the second branch feature map is input into the second CBS module; then, the output feature map of the second CBS module and the second branch feature map output by the backbone network E-SPPFCSPC module are subjected to a fourth concat splicing operation; the output of the fourth concat splicing operation is input into the third EMA-C2f module as the third output of the neck network.

[0088] S23: Build a head network. The head network consists of a detection output module.

[0089] In the head network, the microcracks in the wellbore are predicted through the detection output module, and the classification detection results of the wellbore surface defects are output.

[0090] See also Figure 1 The detection output module receives the three-way output of the neck network; the detection output module consists of two branches, the first branch is sequentially composed of the CBS module, Conv2d, and finally optimized using the box regression loss box-loss. The other branch is sequentially composed of the CBS module, Conv2d, and finally optimized using the category classification loss cls-loss.

[0091] In existing detection methods, SiLU is generally used as the activation function. In the detection output module of the present invention, FReLU is used as the activation function. FReLU is an activation function specifically for visual tasks. It introduces spatial conditions to expand ReLU and PReLU into 2D activation functions, which solves the spatial insensitivity problem in general activation functions and is more suitable for microcrack detection. The formula of FReLU is as follows:

[0092] f(xc,i,j )=max(x c,i,j ,T(x c,i,j ))

[0093]

[0094] in,

[0095] f(x c,i,j ) represents the output of the feature value at position (i, j) on the cth channel after the FReLU activation function.

[0096] T(x c,i,j ) represents the spatial condition function. By introducing the spatial condition function, the activation function can take into account the spatial relationship in the local area, which is particularly important for slender and possibly variable directional features such as microcracks; x c,i,j represents the original eigenvalue at position (i, j) on the cth channel, represents the parameters shared in the cth channel, which means that the activation values ​​at different positions are adjusted according to the same conditions, which helps to maintain the consistency of features and reduce the number of model parameters. Indicates that in x c,i,j The result of pooling the eigenvalues ​​in a local pooling window of size kh×kw around it, where kh and kw represent the height and width of the local pooling window.

[0097] S3: Use the training set to train the detection model and update the new model parameters.

[0098] Test model:

[0099] In order to verify the detection effect of YOLO-DECT of the present invention, a comparative experiment was carried out. Relying on the Key Laboratory of Deep Coal Mine Mining Response and Disaster Prevention and Control of Anhui University of Science and Technology, a shaft detection simulation experimental platform was built for actual shooting. The image types of the test set are divided into three categories: slight cracks, mesh cracks and severe cracks. Among them, the appearance of tiny cracks has no peeling deformation and no branch cracks. The mesh cracks have more branch cracks, a large coverage span and a long length. Severe cracks have obvious cracks, large gaps, and obvious peeling.

[0100] Six target detection networks are compared, namely Faster-RCNN, YOLOv5, YOLOv7, YOLOv8, YOLOv8-RD, and YOLOv8-based. The comparative test results are shown in Table 1, where bold represents the best indicator.

[0101] Table 1 Comparison of experimental results of the method of the present invention and different detection methods As shown in Table 1, compared with the two-stage algorithm Faster R-CNN, the detection speed FPS of the proposed method has a certain gap with it, but the number of parameters and computing power required by YOLO-DECT are much smaller than Faster R-CNN, and the detection accuracy is improved by 9.5%. Comparing YOLO-DECT with YOLOv5s, YOLOv7tiny, and YOLOv8n, it can be seen from Table 1 that YOLO-DECT reduces the floating point operation value by 53.1%, 43.2%, and 8.5% respectively, and the mAP value increases by 15.2%, 10%, and 3% respectively, while ensuring the detection speed, and the detection effect is the best. YOLOv8-RD and YOLOv8-based are improved networks for road and steel defect detection, respectively. Compared with YOLOv8n, the number of parameters of YOLOv8-RD has decreased, and the floating point operation value has decreased. YOLOv8-based only introduces the SE attention mechanism, and the improvement is relatively simple. For the detection of microcracks in wellbore, their improvement effect is not obvious. Although the detection speed of YOLOv5-DECT is slightly higher than that of the method of the present invention, the other three indicators are not as good as those of the method of the present invention.

[0102] In order to more comprehensively demonstrate the performance of the YOLO-DECT network model in coal mine shaft microcrack detection, this paper Figure 7 A detailed comparative analysis is given in Table 2. Figure 7 Contains four rows of pictures, in each row of pictures the left side is the detection result of the YOLOv8 network, and the right side is the detection result of the method of the present invention. The first row of pictures shows the detection results of the two detection methods for serious cracks; the second row of pictures shows the detection results of the two detection methods for reticulate cracks; the third row of pictures shows the detection results of the two detection methods for slight cracks. In the figure, each box represents a detected defect area, and the defect category and confidence level of the defect are marked above each defect area. This detail increases the interpretability and practical value of the results. The detection accuracy of the YOLO-DECT algorithm is higher for all three types of defects. There are some false detections and repeated detections in the YOLOv8 algorithm, while the YOLO-DECT algorithm accurately identifies the crack features.

[0103] The fourth row of pictures is a heat map during the detection of slight cracks. The defect category and confidence level are marked above the defect area. It can be seen that the method of the present invention pays more attention to the key crack features and has a better detection effect.

[0104] Table 2 Comparison of detection results of different types of cracks by different detection methods

[0105]

[0106] As shown in Table 2, the average detection accuracy of the method of the present invention in the three types of cracks, slight, reticulate and serious, is improved by 6.2%, 1.3% and 2.1% respectively compared with the YOLOv8n network. Among them, the detection effect of tiny cracks is improved most significantly, and it has advantages in detecting small targets such as tiny cracks.

[0107] In order to verify the generalization ability and robustness of the YOLO-DECT network, the GRDDC2020 dataset provided by the Global Road Damage Detection Challenge was used for verification. Since the dataset is large in number and the images are collected from different countries, a large number of images do not have the required annotations. Therefore, 10,000 samples were selected from the Japanese category dataset for testing. There are 5 types of annotations: d00 (longitudinal cracks); d10 (transverse cracks); d20 (cracking); d40 (potholes); d50 (manhole covers). The verification results are shown in the figure below. Figure 8 As shown, Figure 8 There are four rows of pictures in total. The left side of each row shows the detection results of the YOLOv8 network, and the right side shows the detection results of the method of the present invention. From the results, it can be seen that the verification accuracy of YOLO-DECT for various types of annotated cracks is better than that of the YOLO v8 network, and has higher verification accuracy. This shows that YOLO-DECT is not only effective in detecting microcracks in wellbore on self-built data sets, but also has a certain degree of versatility.

[0108] In summary, the YOLO-DECT network shows good detection results for various shaft surface defects, indicating that the network has strong generalization ability and robustness. This is of great significance for maintaining the safety of coal mine shaft environment in practical applications.

[0109] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting micro-cracks in a coal mine shaft, characterized in that: The following steps are involved: S1: Construct a dataset of wellbore microfractures and divide it into training set, validation set and test set; S2: Construct a wellbore micro-fracture detection model; S21: Build backbone network; S22: Construct neck feature fusion network; S23: build the head network; S3: Use the training set to train the detection model and update the new model parameters.

2. A method for detecting micro-cracks in a coal mine shaft according to claim 1, characterized in that: The S21 includes: S211: Construct a basic convolution module CBS; the basic convolution module CBS consists of a convolution layer Conv, a batch normalization layer BN, and an activation function SiLU in sequence; S212: Construct a deformable convolution module CN-CBS; the deformable convolution module CN-CBS is composed of a deformable convolution layer DCNv2, a batch normalization layer BN, and an activation function SiLU in sequence; three deformable convolution modules CN-CBS form a parallel sub-network; In the deformable convolution layer DCNv2, the calculation formula of the deformable convolution layer is: Among them, y(p) represents the feature value of the output feature map at position p, x represents the input feature map, p represents the position of the output feature map, and p k represents the initial position of the kth sampling point relative to the center of the convolution kernel, Δp k represents the position offset of the kth sampling point, Δmk represents the characteristic adjustment coefficient of the kth sampling point, and w k represents the weight of the kth sampling point, k represents the sequence number of the sampling point, and K represents the total number of sampling points.

3. A method for detecting micro-cracks in a coal mine shaft according to claim 2, characterized in that: The S21 further includes: S213: constructing a deformable feature fusion module CN-C2f; In the deformable feature fusion module CN-C2f, the input feature map passes through the first basic convolution module CBS for feature extraction and activation, and then passes through the Split layer for channel segmentation operation, dividing the feature map into two branches. The first branch passes through N D-Bottleneck modules in sequence, and the second branch does not pass through the D-Bottleneck module. After that, all D-Bottleneck module outputs are concat-joined with the second branch feature map, and then pass through the second basic convolution module CBS to obtain the output feature map of the deformable feature fusion module CN-C2f; S214: Construct a multi-scale attention feature fusion module EMA-C2f; the multi-scale attention feature fusion module EMA-C2f is composed of a basic convolution module CBS, a BottleNeck module, and an efficient multi-scale attention mechanism module EMA; In the multi-scale attention feature fusion module EMA-C2f, the input feature map passes through the first basic convolution module CBS, and then the Split layer is used to divide the output feature map of the first basic convolution module CBS into two branches; the first branch is processed by N Bottleneck modules in sequence; the Bottleneck module is sequentially constructed by convolution, residual structure, and SiLU activation function; the second branch is not processed by the Bottleneck module, and all Bottleneck module outputs are concat-concatenated with the second branch feature map, and pass through the second basic convolution module CBS and the efficient multi-scale attention mechanism module EMA in sequence to obtain the output of the multi-scale attention feature fusion module EMA-C2f.

4. A method for detecting micro-cracks in a coal mine shaft according to claim 3, characterized in that: The S21 further includes: S215: constructing a spatial depth conversion convolution module SPDI; In the spatial depth conversion convolution module SPDI, for the input feature map S1×S2×C1, S1, S2, and C1 represent the height, width, and number of channels of the feature map respectively; first convert the input feature map size to S1 / 2×S2 / 2×4C1, and then obtain four sub-feature maps through slicing operation Then the four sub-feature maps are concat-joined and passed through a convolution layer. The Split operation is used to divide the feature map into four branches. The four branches pass through different depth-wise separable convolution layers DWConv respectively. The convolution kernel sizes of the depth-wise separable convolution layers DWConv in the first three branches are 3x3, 1×11, and 11×1 respectively. The depth-wise separable convolution layer DWConv operation is not performed in the last branch. The outputs of each branch are concat-joined and the outputs of all branches are fused together through a convolution layer to generate the output feature map of the spatial depth conversion convolution module SPDI.

5. A method for detecting micro-cracks in a coal mine shaft according to claim 4, characterized in that: The S21 further includes: S216: constructing a spatial pyramid pooling module E-SPPFCSPC; In the spatial pyramid pooling module E-SPPFCSPC, the input feature map is divided into two branches after passing through the first basic convolution module CBS. The first branch inputs the obtained feature map into the SPP structure, which includes pooling layers of different sizes, and then concats the outputs of different pooling layers, and then passes through the second basic convolution module CBS; the second branch inputs the obtained feature map into the third basic convolution module CBS, concats the output feature map of the second basic convolution module CBS with the output feature map of the third basic convolution module CBS, and then passes through the efficient multi-scale attention mechanism module EMA and the fourth basic convolution module CBS in turn to obtain the output of E-SPPFCSPC.

6. A method for detecting micro-cracks in a coal mine shaft according to claim 5, characterized in that: In the backbone network, the input image passes through the first basic convolution module CBS, the second basic convolution module CBS, the first spatial depth conversion convolution module SPDI, the third basic convolution module CBS, the second spatial depth conversion convolution module SPDI module, the first deformable feature fusion module CN-C2F, the fourth basic convolution module CBS, the third spatial depth conversion convolution module SPDI, the second deformable feature fusion module CN-C2F, the fifth basic convolution module CBS, the multi-scale attention feature fusion module EMA-C2f, and the E-SPPFCSPC module in sequence. At the same time, the feature map passing through the first spatial depth conversion convolution module SPDI is input into the first deformable convolution module CN-CBS through another branch; the feature map obtained through the first deformable convolution module CN-CBS module is divided into two branches. The first branch enters the first deformable feature fusion module CN-C2F, and the second branch enters the second deformable convolution module CN-CBS; the feature map obtained by the second deformable convolution module CN-CBS is divided into two branches, the first branch enters the second deformable feature fusion module CN-C2F, and the second branch enters the third deformable convolution module CN-CBS; the feature map passing through the third deformable convolution module CN-CBS is input into the multi-scale attention feature fusion module EMA-C2f to extract the surface features of the wellbore fractures; the output of the backbone network is three-way, namely the output of the first deformable feature fusion module CN-C2F, the output of the second deformable feature fusion module CN-C2F, and the output of the E-SPPFCSPC module; the three-way features are simultaneously sent to the neck feature fusion network Neck.

7. A method for detecting micro-cracks in a coal mine shaft according to claim 6, characterized in that: The neck feature fusion network Neck is composed of a basic convolution module CBS, a multi-scale attention feature fusion module EMA-C2f, a deformable feature fusion module CN-C2F module, and an upsampling layer; In the neck feature fusion network, the output of the E-SPPFCSPC module in the head network is divided into two branches, and the first upsampling operation is performed in the first branch, and then the feature map after the first upsampling operation is subjected to the first concat splicing operation with the output feature map of the second deformable feature fusion module CN-C2F of the backbone network; the feature map is then divided into two branches through the deformable feature fusion module CN-C2F, and the first branch is subjected to the second upsampling operation, and then the feature map obtained by the second upsampling operation is subjected to the second concat splicing operation with the output feature map of the first deformable feature fusion module CN-C2F of the backbone network; the feature map obtained by the second concat splicing operation is divided into two branches after passing through the first multi-scale attention feature fusion module EMA-C2f, and the first branch is used as the neck The first output of the network; the second branch is input into the first basic convolution module CBS of the neck network; the feature map passing through the first basic convolution module CBS is subjected to a third concat splicing operation with the second branch feature map passing through the deformable feature fusion module CN-C2F, and the feature map obtained by the third concat splicing operation is input into the second multi-scale attention feature fusion module EMA-C2f, and the output of the second multi-scale attention feature fusion module EMA-C2f is divided into two branches, the first branch is used as the second output of the neck network, and the second branch feature map is input into the second basic convolution module CBS; and then the output feature map of the second basic convolution module CBS is subjected to a fourth concat splicing operation with the second branch feature map output by the backbone network E-SPPFCSPC module; The output of the fourth concat operation is input into the third multi-scale attention feature fusion module EMA-C2f as the third output of the neck network.

8. A method for detecting micro-cracks in a coal mine shaft according to claim 7, characterized in that: The head network is composed of a detection output module; the detection output module receives the output of the neck network; The detection output module consists of two branches. The first branch consists of the basic convolution module CBS and Conv2d in sequence, and then optimized using the box regression loss. The second branch consists of the basic convolution module CBS module and Conv2d in sequence, and then optimized using the category classification loss. In the basic convolution module CBS of the detection output module, FReLU is used as the activation function. The formula of FReLU is as follows: f(x c,i,j )=max(x c,i,j ,T(x c,i,j )) in, f(x c,i,j ) represents the output of the feature value at position (i, j) on the cth channel after the FReLU activation function, T(x c,i,j ) represents the spatial condition function; x c,i,j represents the original eigenvalue at position (i, j) on the cth channel, represents the parameters shared in the cth channel, Indicates that in x c,i,j The result of pooling the eigenvalues ​​in a local pooling window of size kh×kw around it, where kh and kw represent the height and width of the local pooling window.

9. A coal mine shaft micro-crack detection device, characterized in that: The method as described in any one of claims 1 to 8 is used to detect micro cracks in coal mine shafts.

10. A storage medium for detecting micro-cracks in a coal mine shaft, characterized in that: The method as described in any one of claims 1 to 8 is used to detect micro cracks in coal mine shafts.

Images