A coal mine shaft wall water seepage identification method based on a deep neural network

Abstract

The application discloses a coal mine wall water seepage identification method based on a deep neural network, obtains a coal mine wall image by using an image acquisition device, selects an image containing water seepage as an original coal mine wall water seepage image, pre-processes the original coal mine wall water seepage image, constructs a coal mine wall water seepage detection data set, constructs an improved YOLOv5 model, inputs the coal mine wall water seepage detection data set into the improved YOLOv5 model for training, and inputs an image to be detected into the trained model to complete water seepage target detection. The improved YOLOv5 model adds a Cross-Transformer module in a neck network of an original YOLOv5 model, adopts a multi-head cross attention mechanism in the Cross-Transformer module, realizes the interaction and fusion of key information between different level feature maps, solves a long distance dependence problem, and improves detection precision from 78% to 85%.

Description

Technical Field

[0001] This invention relates to the field of coal mine shaft image detection technology. Specifically, it is a method for identifying coal mine shaft seepage based on deep neural networks. Background Technology

[0002] Water hazards during coal mining often cause enormous losses. Regularly conducting seepage detection on coal mine walls can promptly eliminate greater property damage caused by seepage and protect the lives of coal mine workers.

[0003] Currently, methods for detecting seepage in coal mine walls are relatively limited, mostly relying on manual inspections. However, manual inspections have significant drawbacks in terms of accuracy, efficiency, and cost. In recent years, the field of deep learning, represented by convolutional neural networks, has developed rapidly, and deep learning-based methods have become an important tool in image detection.

[0004] Currently, image detection can employ various neural network methods, such as the popular YOLO series of network models. However, the YOLO series of network models are based on convolutional neural networks, lacking effective utilization of spatial location information and unable to solve long-distance dependency problems. Neural network models contain multiple convolutional layers during feature extraction, generating feature maps at different levels. Low-level feature maps contain more detailed and locational information, such as color and texture. High-level feature maps contain more semantic information. Existing methods largely lack information interaction between features at different levels, easily leading to information loss and reduced detection accuracy. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to provide a coal mine well wall seepage identification method based on deep neural networks to make up for the shortcomings of convolutional neural networks and improve the accuracy of seepage target detection in complex coal mine well wall environments.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A method for identifying coal mine wellbore seepage based on deep neural networks.

[0008] Step 100: Use image acquisition equipment to acquire images of the coal mine wall, and select images containing clear seepage areas as the original coal mine wall seepage images;

[0009] Step 200: Preprocess the original coal mine wall seepage images to construct a coal mine wall seepage detection dataset;

[0010] Step 300: Construct an improved YOLOv5 model by adding a Cross-Transformer module to the neck network of the original YOLOv5 model. The Cross-Transformer module employs a multi-head cross-attention mechanism.

[0011] Step 400: Input the coal mine wall seepage detection dataset into the improved YOLOv5 model for training;

[0012] Step 500: Input the image to be detected into the trained model to complete the water seepage target detection.

[0013] In the above-mentioned method for identifying seepage in coal mine walls based on deep neural networks, in step 100, the number of images containing seepage is greater than or equal to 300.

[0014] The above-mentioned method for identifying seepage in coal mine walls based on deep neural networks includes the following sub-steps in step 200: Establishing a seepage detection dataset.

[0015] Step 210: Use LabelImg image annotation software to manually annotate the seepage areas in the original coal mine wall seepage images to form a coal mine wall seepage image dataset. The samples in the coal mine wall seepage image dataset include seepage images and seepage target annotation result images corresponding to the seepage images.

[0016] Step 220: The images in the coal mine wall seepage image dataset are expanded by random rotation, translation, horizontal flipping and vertical flipping operations to obtain a coal mine wall seepage image dataset that is four times larger and contains more than 1200 seepage images.

[0017] Step 230: The samples in the expanded coal mine wall seepage image dataset are randomly divided into training set and validation set according to a preset ratio of 8:2.

[0018] In the aforementioned deep neural network-based method for identifying seepage in coal mine walls, step 300 involves a multi-head cross-attention mechanism employing a dual-input structure. Its goal is to obtain the correlation between an input feature map query and a key, and then multiply it by the value of another input to obtain an attention map. This process can be described as follows:

[0019]

[0020] In equation (1), Q1 is a feature map query for one input, K1 is a key for one input, and V1 is a value for one input; Q2 is a feature map query for another input, K2 is a key for another input, and V2 is a value for another input; Attention represents the attention mechanism algorithm.

[0021] C k Let T be the dimension of the input data, and T be the matrix transpose operation.

[0022] MCA1 and MCA2 represent the multi-head cross-attention corresponding to the two inputs.

[0023] The aforementioned method for identifying coal mine wall seepage based on deep neural networks uses a Cross-Transformer module with a dual-branch structure, including linear projection, feedforward network, and layer normalization operation, and employs a multi-head cross-attention mechanism to achieve information interaction between the two branches.

[0024] The aforementioned method for identifying coal mine wall seepage based on deep neural networks performs dimensionality reduction on two input feature maps x1 and x2. To encode the positional information of the feature maps, a learnable position embedding is introduced and fused with the input feature maps respectively by direct addition. Then, layer normalization is performed on y1 and y2 to obtain the embedded features y1 and y2. This process can be described as follows:

[0025]

[0026] In equation (2), LP is the linear projection operation, PE is the position encoding, and LN represents layer normalization.

[0027] The aforementioned method for identifying coal mine wall seepage based on deep neural networks utilizes a multi-head cross-attention mechanism to achieve information interaction between feature maps y1 and y2, and adds them to the original feature maps to obtain feature maps z1 and z2. This process can be described as follows:

[0028]

[0029] In the formula, MCA1 and MCA2 represent the multi-head cross-attention corresponding to the two inputs y1 and y2, respectively.

[0030] The above-mentioned method for identifying seepage in coal mine walls based on deep neural networks, in order to adapt to the next dimension of the network, adopts layer normalization, feedforward network, and feature mapping operation, and adds two feature maps z1 and z2 to obtain feature map f, thereby achieving further information fusion. This process can be described as follows:

[0031] f=FM[LN(z1)+LN(FFN(z1))]+FM[LN(z2)+LN(FFN(z2))] (3);

[0032] In equation (3), LN represents layer normalization, FM represents feature mapping, and FFN is a feedforward network.

[0033] The technical solution of the present invention achieves the following beneficial technical effects:

[0034] This application proposes a method for identifying seepage in coal mine walls based on deep neural networks. A Cross-Transformer module is introduced into the neck network of the original YOLOv5 model to make up for the shortcomings of convolutional neural networks and effectively solve the problem of modeling long-range dependencies.

[0035] In addition, this invention also uses a multi-head cross-attention mechanism in the Cross-Transformer module to achieve information interaction between feature maps of different levels. This enables information interaction and fusion between feature maps of the same scale but different levels in the neck network of the YOLOv5 model, improving the network's representational ability.

[0036] The improved YOLOv5 model adds a Cross-Transformer module to the neck network of the original YOLOv5 model. The Cross-Transformer module adopts a multi-head cross-attention mechanism, which can realize the interaction and fusion of key information between feature maps at different levels, solve the problem of long-distance spatial dependence, and improve the detection accuracy of seepage targets in complex coal mine wall environments from 78% to 85%. Attached Figure Description

[0037] Figure 1 A flowchart of a method for identifying coal mine well wall seepage based on deep neural networks in this invention;

[0038] Figure 2 Existing YOLOv5 model framework diagram;

[0039] Figure 3 The improved YOLOv5 model framework diagram in this invention;

[0040] Figure 4 A schematic diagram of the Cross-Transformer module structure in this invention. Detailed Implementation

[0041] Reference Figure 1 This invention provides a method for identifying coal mine wall seepage based on deep neural networks, which can be used for coal mine underground robots to detect coal mine wall seepage.

[0042] Step 100: Use image acquisition equipment to acquire images of the coal mine wall, and select images containing water seepage as the original coal mine wall water seepage images; a total of 300 coal mine wall water seepage images are obtained;

[0043] Step 200: Preprocess the original coal mine wall seepage images to construct a coal mine wall seepage detection dataset;

[0044] Step 210: Use LabelImg image annotation software to manually annotate the seepage areas in the original coal mine wall seepage images to form a coal mine wall seepage image dataset. The samples in the coal mine wall seepage image dataset include seepage images and seepage target annotation result images corresponding to the seepage images.

[0045] Step 220: The images in the coal mine wall seepage image dataset are augmented by random rotation, translation, horizontal flipping, and vertical flipping operations to obtain an augmented coal mine wall seepage image dataset containing 1200 seepage images;

[0046] Step 230: Randomly divide the samples in the expanded coal mine wall seepage image dataset into a training set and a validation set at an 8:2 ratio. The training set contains 1080 samples, and the validation set contains 120 samples.

[0047] Step 300: Construct an improved YOLOv5 model by adding a Cross-Transformer module to the neck network of the original YOLOv5 model. The Cross-Transformer module employs a multi-head cross-attention mechanism.

[0048] The multi-head cross-attention mechanism employs a dual-input structure. Its goal is to obtain the correlation between an input feature map query and a key, and then multiply it by the value of the other input to obtain an attention map. This process can be described as follows:

[0049]

[0050] In equation (1), Q1 is a feature map query for one input, K1 is a key for one input, and V1 is a value for one input; Q2 is a feature map query for another input, K2 is a key for another input, and V2 is a value for another input; Attention represents the attention mechanism algorithm.

[0051] C k The dimension of the input data is T; T represents the matrix transpose operation.

[0052] MCA1 and MCA2 represent the multi-head cross-attention corresponding to the two inputs.

[0053] like Figure 4 As shown, the Cross-Transformer module adopts a dual-branch structure, which includes linear projection, feedforward network, layer normalization operation, and uses a multi-head cross attention mechanism to realize information interaction between the two branches.

[0054] Step 310: Perform dimensionality reduction on the two input feature maps x1 and x2. To encode the positional information of the feature maps, a learnable positional embedding is introduced and fused with the input feature maps respectively by direct addition. Then, layer normalization is performed on y1 and y2 to obtain the embedded features y1 and y2. This process can be described as follows:

[0055]

[0056] In equation (2), LP is the linear projection operation, PE is the position encoding, and LN represents layer normalization.

[0057] Step 320: Utilize a multi-head cross-attention mechanism to achieve information exchange between feature maps y1 and y2, and add them to the original feature maps to obtain feature maps z1 and z2. This process can be described as follows:

[0058]

[0059] In the formula, MCA1 and MCA2 represent the multi-head cross-attention corresponding to the two inputs y1 and y2, respectively.

[0060] Step 330: In order to adapt to the next dimension of the network, layer normalization, feedforward network, and feature mapping operations are adopted, and the two feature maps z1 and z2 are added together to obtain feature map f, so as to achieve further information fusion. This process can be described as:

[0061] f=FM[LN(z1)+LN(FFN(z1))]+FM[LN(z2)+LN(FFN(z2))] (3);

[0062] In equation (3), LN represents layer normalization, FM represents feature mapping, and FFN is a feedforward network.

[0063] Step 400: Input the coal mine wall seepage detection dataset into the improved YOLOv5 model for training;

[0064] Step 500: Input the image to be detected into the trained model to complete the water seepage target detection.

[0065] The improved YOLOv5 model adds a Cross-Transformer module to the neck network of the original YOLOv5 model. The Cross-Transformer module adopts a multi-head cross-attention mechanism, which can realize the interaction and fusion of key information between feature maps at different levels, solve the problem of long-distance spatial dependence, and improve the detection accuracy of seepage targets in complex coal mine wall environments from 78% to 85%.

[0066] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of the claims of this patent application.

Claims

1. A method for identifying seepage water in coal mine walls based on deep neural networks, characterized in that, Step 100: Use image acquisition equipment to acquire images of the coal mine wall, and select images containing clear seepage areas as the original coal mine wall seepage images; Step 200: Preprocess the original coal mine wall seepage images to construct a coal mine wall seepage detection dataset; Step 300: Construct an improved YOLOv5 model by adding a Cross-Transformer module to the neck network of the original YOLOv5 model. The Cross-Transformer module employs a multi-head cross-attention mechanism. The Cross-Transformer module is placed in the four key feature fusion nodes of the YOLOv5 model neck network, i.e., the positions of each branch feature before entering Concat. The Cross-Transformer module is inserted in four positions respectively: (1) After the first upsampling and BottleNeckCSP, and before entering the feature concatenation (Concat) step; (2) After the second 1×1 convolution and 3×3 convolution, and before entering the feature concatenation (Concat) step; (3) After Upsample and BottleNeckCSP on another path, and before entering feature concatenation (Concat); (4) After the 1×1 convolution and 3×3 convolution in the bottom branch, and before entering the feature concatenation (Concat) stage; Step 400: Input the coal mine wall seepage detection dataset into the improved YOLOv5 model for training; Step 500: Input the image to be detected into the trained model to complete the detection of water seepage targets; The Cross-Transformer module adopts a dual-branch structure, which includes linear projection, feedforward network, and layer normalization operation, and uses a multi-head cross attention mechanism to realize information interaction between the two branches. Mapping two features of the input x 1 and x 2. Dimensionality reduction is performed. To encode the positional information of the feature maps, learnable positional embeddings are introduced and fused with the input feature maps separately through direct addition. Then... y 1 and y 2. Perform layer normalization to obtain the features after feature embedding. y 1 and y 2. This process is described as follows: （2） ； In equation (2), LP For linear projection operations, PE For position encoding, LN Representation layer normalization; Achieve this using a multi-head cross-attention mechanism y 1 and y 2. Information exchange between feature maps, and addition to the original feature map to obtain the new feature map. z 1 and z 2. This process is described as follows: （3）； In equation (3), MCA 1 and MCA 2 Representing two inputs respectively y 1 and y 2 Corresponding multi-head cross attention; To accommodate the next dimension of the network, layer normalization, feedforward networks, and feature mapping operations are employed, and the two feature maps are combined. z 1 and z 2 Add them together to obtain the feature map. f To achieve further information fusion, this process is described as follows: （4）； In equation (4), LN represents layer normalization, FM represents feature mapping, and FFN is a feedforward network.

2. The method for identifying coal mine wall seepage based on deep neural networks according to claim 1, characterized in that, In step 100, the number of images containing water seepage is greater than or equal to 300.

3. The method for identifying coal mine wall seepage based on deep neural networks according to claim 1, characterized in that, In step 200, a seepage detection dataset is established, including the following sub-steps: Step 210: Use LabelImg image annotation software to manually annotate the seepage areas in the original coal mine wall seepage images to form a coal mine wall seepage image dataset. The samples in the coal mine wall seepage image dataset include seepage images and seepage target annotation result images corresponding to the seepage images. Step 220: Expand the coal mine wall seepage image dataset by randomly rotating, translating, horizontally flipping and vertically flipping the images to obtain a coal mine wall seepage image dataset that is four times larger, containing more than or equal to 1200 seepage images. Step 230: The samples in the expanded coal mine wall seepage image dataset are randomly divided into training set and validation set according to a preset ratio of 8:

2.

4. The method for identifying coal mine wall seepage based on deep neural networks according to claim 1, characterized in that, In step 300, the multi-head cross-attention mechanism employs a dual-input structure. Its goal is to obtain the correlation between an input feature map query and a key, and then multiply it by the value of another input to obtain an attention map. This process is described as follows: （1）； In equation (1), Q 1 Query a feature map as input. K 1 For an input key, V 1 Value is an input value. Q 2 Query the feature map of another input. K 2 For another input key, V 2 For another input value, Value; Attention This represents the attention mechanism algorithm; C k Let T be the dimension of the input data, and T be the matrix transpose operation. MCA 1 and MCA 2 This represents the multi-head cross-attention corresponding to two inputs.

Images