Coal rock image classification method and device based on deep learning, and storage medium

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a coal rock image classification method and device based on deep learning and a storage medium, and the method comprises the following steps: constructing a coal rock image data set, and carrying out the preprocessing of the data set; constructing an attention residual network Att-Resnet, wherein the Att-Resnet comprises a feature extraction module A1 and a plurality of residual attention modules A2; inputting images in the coal rock image data set into an attention residual network Att-Resnet for feature extraction to obtain a feature map # imgabs0 #; and inputting the feature map # imgabs1 # into a classifier for classification to obtain probability values of the feature map # imgabs2 # corresponding to each category, and then selecting the category with the highest probability value as a predicted category. According to the method, a large-kernel parallel convolution mechanism is adopted to provide richer feature maps for downstream tasks, residual connection and an attention introduction mechanism are adopted, multi-scale space information with finer granularity can be effectively extracted, and meanwhile, a channel dependency relationship with a longer distance can be established to provide a premise for the improvement of coal rock image classification precision.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a coal rock image classification method based on deep learning. Background Art

[0002] Coal is the most economical fossil energy in the world and the main energy source in my country. It plays a vital role in my country's energy security and economic and social development. Intelligent and unmanned coal mining is not only increasing coal production, but also the main way to reduce coal mine accidents, especially working face accidents. Coal rock image recognition is the process of automatically identifying and classifying coal rock images using computer vision technology. The background technology of coal rock image recognition mainly includes image preprocessing, feature extraction, classification algorithm and model evaluation, through which the automatic recognition and classification of coal rock images can be realized. However, since coal rock images usually have complex texture and structural features, such as the longitudinal and transverse textures of coal seams, the contact surface between rock layers, etc., effective feature extraction methods are the key to solving the problem of coal rock image classification. Commonly used feature extraction methods include texture features, shape features, color features, etc. In short, the background technology and difficulties in the field of coal rock image classification include feature extraction, classification algorithms, data set problems, category imbalance problems and real-time requirements. Solving these problems requires the comprehensive use of technologies in the fields of computer vision, machine learning, data processing, etc., and combined with the characteristics of coal rock images for targeted research and optimization. Summary of the invention

[0003] In response to the above problems, the present invention provides a coal rock image classification method, device and storage medium based on deep learning. It adopts a large-core parallel convolution mechanism to provide richer feature maps for downstream tasks. At the same time, it adopts residual connection and introduces attention mechanism, which can effectively extract finer-grained multi-scale spatial information and establish longer-distance channel dependencies, providing richer feature maps for classification tasks and providing a prerequisite for improving the classification accuracy of coal rock images.

[0004] The technical solution adopted by the present invention to overcome the technical problems is: In a first aspect, the present invention provides a coal rock image classification method based on deep learning, comprising the following steps: S1. Collect coal and rock images and construct a coal and rock image dataset, and then perform preprocessing operations on the dataset; S2. Construct an attention residual network Att-Resnet, which includes a feature extraction module A1 and multiple residual attention modules A2; S3. Construct feature extraction module A1, which is composed of large-core parallel convolution modules. Input the processed images in the data set into feature extraction module A1, and extract the feature map of coal and rock images through the large-core parallel convolution module. ; S4. Construct multiple residual attention modules A2. The residual attention module A2 is a variant residual module that replaces the 3×3 convolution in the classic residual module with an attention mechanism. Input to multiple residual attention modules A2 for further feature extraction to obtain the final feature map ; S5. The final feature map Input to the classifier for classification and obtain the feature map Corresponding to the probability values ​​of each category, the category with the highest probability value is selected as the predicted category.

[0005] S1 is as follows: The coal and rock images in the dataset are scaled to a uniform size, and then the images are further enhanced by random cropping and rotation operations to obtain the preprocessed dataset. , , Representation and processed dataset Middle Zhang Meiyan Image, Represents the preprocessed dataset The number of coal and rock images, Represents the preprocessed dataset Any coal rock image in .

[0006] S3 is as follows: The large-core parallel convolution module consists of four branches, feature maps In the first branch, a 1×1 convolution is performed, and in the second branch, a Convolution, pass a 3×3 convolution in the third branch, directly pass through the fourth branch, add the results of the four branches, and then pass through a normalization layer and an activation function in turn, and finally output the feature map , the calculation process is as follows: , in, Represents the activation function Operation, is the normalization layer, , and They represent the 3×3 convolution in the large-core parallel convolution module, Convolution and 1×1 convolution operations, express The size of the convolution kernel in the convolution, and the 2D convolution 3×3 convolution in the large-core parallel convolution module, Convolution and 1×1 convolution are replaced by deep over-parameterized convolution , The convolution is calculated as follows: , in, is a two-dimensional tensor, The number of input channels is Gao Wei , width , represents the deep convolutional trainable kernel, , represents the depth multiplier in the depth convolution, represents the traditional convolutional trainable kernel, , Indicates the number of output channels, represents the regular convolution operator, represents the depthwise convolution operator, Represents the output convolution result.

[0007] S4 is as follows: S4.1Att-Resnet includes A residual attention module A2, which includes the first 1×1 convolution, the attention module and the second 1×1 convolution in sequence; S4.2 Feature map Input to the first residual attention module A2, the feature map is transformed into The number of channels is reduced to the level that can pass through the attention module, and then the feature map after the number of channels is reduced Input to the attention module, the attention module first divides the feature map into image channels, the number of channels after division is n, and then extracts features in each channel respectively, corresponding to n feature maps, expressed as , Indicates The feature map output by each channel, , the calculation process is as follows: , in, Indicates The channel is the convolution kernel Convolution; Then, the feature map of each channel output is extracted through the attention mechanism in the attention module The attention weight , the calculation process is as follows: , in, represents the attention mechanism operation, and Represents the height and width of the input feature map, The feature map Any point in , Representatives passed the The feature map output by each channel, Represents the Sigmoid function operation, Represents the relu activation function operation, and are two different weight parameters in the attention mechanism, , , is the reduction ratio, is the number of input channels; Then use The activation function is used to recalibrate the weights of the channel attention information. The calculation process is as follows: , in, Indicates the image The attention weights after recalibration of the channels, express Operation of activation function; S4.3 Feature map Attention weights recalibrated with corresponding weights Perform channel-wise multiplication to obtain feature maps , the calculation process is as follows: , in, Represents channel-wise level multiplication; Then you will get Perform dimension splicing to obtain a feature map after multi-scale feature information attention weighting , the calculation process is as follows: , in, Represents a splicing operation; Final feature map After the second 1×1 convolution operation, the feature map The number of channels is restored and the feature map is obtained ; S4.4. Output feature map of the first residual attention module A2 As the second residual attention module A2, repeat the operations in steps S4.2 and S4.3 to obtain the output feature map of the first residual attention module A2 , take the output of the previous residual attention module A2 as the input of the next residual attention module A2, iterate in sequence, and get the first Output feature map of residual attention module A2 ; S4.5. Feature Map It passes through the BN layer, the ReLU layer, the BN layer in turn, and then connects to the feature map through the residual connection After adding, it passes through a relu layer to get the final feature map , the calculation process is as follows: .

[0008] S5 is as follows: The feature map Input to the classifier for classification. The classifier includes an average pooling layer and a fully connected layer. The feature map First, the feature space vector is obtained through the average pooling layer, and then the category with the highest probability value is obtained through the mapping of the fully connected layer. The calculation process is as follows: , in, Represents the prediction result, represents the operation of the average pooling layer, Represents the operation of a fully connected layer.

[0009] In a second aspect, the present invention provides a coal rock image classification device based on deep learning, comprising the following units: (1) Preprocessing unit: used to construct coal rock image dataset and preprocess the dataset; (2) Feature extraction module A1: extracts the features of coal and rock images using a large-core parallel convolution module; (3) Residual attention module A2: It consists of a variant residual module and an attention module, and is used to extract the channel attention of feature maps of different scales, obtain the channel attention vector at each different scale, and output a feature map after multi-scale feature information attention weighting; (4) Classification unit: used to classify feature maps.

[0010] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes a coal-rock image classification method based on deep learning.

[0011] The advantages of the present invention are: The features of the original coal and rock images are extracted through a large-core parallel convolution module, avoiding the size limitation of the fixed convolution kernel and the limitation of the receptive field, and the extracted features are richer. The large-core parallel convolution module is a plug-and-play module and can be embedded in any convolutional neural network architecture to improve performance. The network structure of the present invention uses the residual module as the backbone network, avoiding problems such as gradient disappearance and gradient explosion caused by the network being too deep. In addition, the present invention also embeds an attention mechanism in the backbone network, and dimensionalizes the feature map obtained after the attention weight, and finally outputs a feature map with richer multi-scale information, which more effectively provides a strong guarantee for the implementation of downstream classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0013] Figure 1 It is a schematic diagram of the Att-Resnet structure of the present invention.

[0014] Figure 2 This is the heat map of the focus area in coal and rock image feature extraction.

[0015] Figure 3 This is the calculation principle diagram of Do-conv convolution. DETAILED DESCRIPTION

[0016] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0017] Example 1 Aiming at the complex coal and rock underground environment, this method uses a coal and rock image classification network based on deep learning to solve the problem of low coal and rock image classification accuracy. At the same time, based on the characteristics of coal and rock images, the channel attention of feature maps of different scales is extracted on each channel, and a feature map after multi-scale feature information attention weighting is output. At the same time, the large-core parallel convolution block can handle the problem of long-distance spatial position information dependency.

[0018] Combine the following Figure 1A coal rock image classification method based on deep learning implemented by the present invention is specifically described, and the specific steps are as follows: S1. Collect coal and rock images and construct a coal and rock image dataset, and then perform preprocessing operations on the dataset; S2. Construct an attention residual network Att-Resnet, which includes a feature extraction module A1 and multiple residual attention modules A2; S3. Construct feature extraction module A1, which is composed of large-core parallel convolution modules. Input the processed images in the data set into feature extraction module A1, and extract the feature map of coal and rock images through the large-core parallel convolution module. ; S4. Construct multiple residual attention modules A2. The residual attention module A2 is a variant residual module that replaces the 3×3 convolution in the classic residual module with an attention mechanism. Input to multiple residual attention modules A2 for further feature extraction to obtain the final feature map ; S5. The final feature map Input to the classifier for classification and obtain the feature map Corresponding to the probability values ​​of each category, the category with the highest probability value is selected as the predicted category.

[0019] S1 is as follows: The coal and rock images in the dataset are scaled to a uniform size, and then the images are further enhanced by random cropping and rotation operations to obtain the preprocessed dataset. , , Representation and processed dataset Middle Zhang Meiyan Image, Represents the preprocessed dataset The number of coal and rock images, Represents the preprocessed dataset Any coal rock image in .

[0020] S3 is as follows: The large-core parallel convolution module consists of four branches, feature maps In the first branch, a 1×1 convolution is performed, and in the second branch, a Convolution, pass a 3×3 convolution in the third branch, directly pass through the fourth branch, add the results of the four branches, and then pass through a normalization layer and an activation function in turn, and finally output the feature map , the calculation process is as follows: , in, Represents the activation function Operation, is the normalization layer, , and They represent the 3×3 convolution in the large-core parallel convolution module, Convolution and 1×1 convolution operations, express The size of the convolution kernel in the convolution, and the 2D convolution 3×3 convolution in the large-core parallel convolution module, Convolution and 1×1 convolution are replaced by deep over-parameterized convolution , The convolution is calculated as follows: , in, is a two-dimensional tensor, The number of input channels is Gao Wei , width , represents the deep convolutional trainable kernel, , represents the depth multiplier in the depth convolution, represents the traditional convolutional trainable kernel, , Indicates the number of output channels, represents the regular convolution operator, represents the depthwise convolution operator, Represents the output convolution result.

[0021] S4 is as follows: S4.1Att-Resnet includes A residual attention module A2, which includes the first 1×1 convolution, the attention module and the second 1×1 convolution in sequence; S4.2 Feature map Input to the first residual attention module A2, the feature map is transformed into The number of channels is reduced to the level that can pass through the attention module, and then the feature map after the number of channels is reduced Input to the attention module, the attention module first divides the feature map into image channels, the number of channels after division is n, and then extracts features in each channel respectively, corresponding to n feature maps, expressed as , Indicates The feature map output by each channel, , the calculation process is as follows: , in, Indicates The channel is the convolution kernel Convolution; Then, the feature map of each channel output is extracted through the attention mechanism in the attention module The attention weight , the calculation process is as follows: , in, represents the attention mechanism operation, and Represents the height and width of the input feature map, The feature map Any point in , Representatives passed the The feature map output by each channel, Represents the Sigmoid function operation, Represents the relu activation function operation, and are two different weight parameters in the attention mechanism, , , is the reduction ratio, is the number of input channels; Then use The activation function is used to recalibrate the weights of the channel attention information. The calculation process is as follows: , in, Indicates the image The attention weights after recalibration of the channels, express Operation of activation function; S4.3 Feature map Attention weights recalibrated with corresponding weights Perform channel-wise multiplication to obtain feature maps , the calculation process is as follows: , in, Represents channel-wise level multiplication; Then you will get Perform dimension splicing to obtain a feature map after multi-scale feature information attention weighting , the calculation process is as follows: , in, Represents a splicing operation; Final feature map After the second 1×1 convolution operation, the feature map The number of channels is restored and the feature map is obtained ; S4.4. Output feature map of the first residual attention module A2 As the second residual attention module A2, repeat the operations in steps S4.2 and S4.3 to obtain the output feature map of the first residual attention module A2 , take the output of the previous residual attention module A2 as the input of the next residual attention module A2, iterate in sequence, and get the first Output feature map of residual attention module A2 ; S4.5. Feature Map It passes through the BN layer, the ReLU layer, the BN layer in turn, and then connects to the feature map through the residual connection After adding, it passes through a relu layer to get the final feature map , the calculation process is as follows: .

[0022] S5 is as follows: The feature map Input to the classifier for classification. The classifier includes an average pooling layer and a fully connected layer. The feature map First, the feature space vector is obtained through the average pooling layer, and then the category with the highest probability value is obtained through the mapping of the fully connected layer. The calculation process is as follows: , in, Represents the prediction result, represents the operation of the average pooling layer, Represents the operation of a fully connected layer.

[0023] In order to further prove that the present invention can more effectively provide a strong guarantee for the downstream classification tasks, the classification results of coal and rock images were analyzed based on the present method. The experimental data set is divided into seven categories (coal, basalt, granite, limestone, marble, quartzite and sandstone), with a total of 1503 training images and 377 test images. The above data are uniformly preprocessed by the method of the present invention. In this small data set, the classification accuracy of the model reached 90.343% after only 60 rounds of training. The model was experimentally compared with the existing image classification method. The comparative experimental results are shown in Table 1. Att-Resnet50 is the method of the present invention, EPSANet50 and PreActResNet34 are existing methods. Under the same training conditions, the accuracy of the method of the present invention is higher.

[0024] Table 1 Comparison of recognition accuracy of this method and existing methods on this small dataset Example 2 The present invention provides a coal-rock image classification device based on deep learning, comprising the following units: (1) Preprocessing unit: used to construct coal rock image dataset and preprocess the dataset; (2) Feature extraction module A1: extracts the features of coal and rock images using a large-core parallel convolution module; (3) Residual attention module A2: It consists of a variant residual module and an attention module, and is used to extract the channel attention of feature maps of different scales, obtain the channel attention vector at each different scale, and output a feature map after multi-scale feature information attention weighting; (4) Classification unit: used to classify feature maps.

[0025] Example 3 The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, a coal-rock image classification method based on deep learning is executed.

[0026] Storage media include: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store programs.

[0027] Example 4 like Figure 2As shown in the figure, the present invention uses training parameters to realize a visualized heat map of the network's attention area on the feature map. The heat map can be used to display key points or attention areas in tasks such as target detection, image classification or face recognition in computer vision. Figure 2 The middle blue area belongs to the network's key focus area. This method can focus on the coal and rock areas of the image over a large area in the coal and rock image feature extraction task, providing effective features for downstream classification tasks.

[0028] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A coal-rock image classification method based on deep learning, characterized in that: The following steps are involved: S1. Collect coal and rock images and construct a coal and rock image dataset, and then perform preprocessing operations on the dataset; S2. Construct an attention residual network Att-Resnet, which includes a feature extraction module A1 and multiple residual attention modules A2; S3. Construct feature extraction module A1, which is composed of large-core parallel convolution modules. Input the processed images in the data set into feature extraction module A1, and extract the feature map of coal and rock images through the large-core parallel convolution module. ; S4. Construct multiple residual attention modules A2. The residual attention module A2 is a variant residual module that replaces the 3×3 convolution in the classic residual module with an attention mechanism. Input to multiple residual attention modules A2 for further feature extraction to obtain the final feature map ; S5. The final feature map Input to the classifier for classification and obtain the feature map Corresponding to the probability values ​​of each category, the category with the highest probability value is selected as the predicted category.

2. The coal-rock image classification method based on deep learning according to claim 1 is characterized in that: S1 is as follows: The coal and rock images in the dataset are scaled to a uniform size, and then the images are further enhanced by random cropping and rotation operations to obtain the preprocessed dataset. , , Representation and processed dataset Middle Zhang Meiyan Image, Represents the preprocessed dataset The number of coal and rock images, Represents the preprocessed dataset Any coal rock image in .

3. The coal-rock image classification method based on deep learning according to claim 2 is characterized in that: S3 is as follows: The large-core parallel convolution module consists of four branches, feature maps In the first branch, a 1×1 convolution is performed, and in the second branch, a Convolution, pass a 3×3 convolution in the third branch, directly pass through the fourth branch, add the results of the four branches, and then pass through a normalization layer and an activation function in turn, and finally output the feature map , the calculation process is as follows: ， in, Represents the activation function Operation, is the normalization layer, , and They represent the 3×3 convolution in the large-core parallel convolution module, Convolution and 1×1 convolution operations, express The size of the convolution kernel in the convolution, and the 2D convolution 3×3 convolution in the large-core parallel convolution module, Convolution and 1×1 convolution are replaced by deep over-parameterized convolution , The convolution is calculated as follows: ， in, is a two-dimensional tensor, The number of input channels is Gao Wei , width , represents the deep convolutional trainable kernel, , represents the depth multiplier in the depth convolution, represents the traditional convolutional trainable kernel, , Indicates the number of output channels, represents the regular convolution operator, represents the depthwise convolution operator, Represents the output convolution result.

4. The coal-rock image classification method based on deep learning according to claim 1 is characterized in that: S4 is as follows: S4.1Att-Resnet includes A residual attention module A2, which includes the first 1×1 convolution, the attention module and the second 1×1 convolution in sequence; S4.2 Feature map Input to the first residual attention module A2, the feature map is transformed into The number of channels is reduced to the level that can pass through the attention module, and then the feature map after the number of channels is reduced Input to the attention module, the attention module first divides the feature map into image channels, the number of channels after division is n, and then extracts features in each channel respectively, corresponding to n feature maps, expressed as , Indicates The feature map output by each channel is , the calculation process is as follows: ， in, Indicates The channel is the convolution kernel Convolution; Then, the feature map of each channel output is extracted through the attention mechanism in the attention module The attention weight , the calculation process is as follows: ， in, represents the attention mechanism operation, and Represents the height and width of the input feature map, The feature map Any point in , Representatives passed the The feature map output by each channel is Represents the Sigmoid function operation, Represents the relu activation function operation, and are two different weight parameters in the attention mechanism, , , is the reduction ratio, is the number of input channels; Then use The activation function is used to recalibrate the weights of the channel attention information. The calculation process is as follows: ， in, Indicates the image The attention weights after recalibration of the channels, express Operation of activation function; S4.3 Feature map Attention weights recalibrated with corresponding weights Perform channel-wise multiplication to obtain the feature map , the calculation process is as follows: ， in, Represents channel-wise level multiplication; Then you will get Perform dimension splicing to obtain a feature map after multi-scale feature information attention weighting , the calculation process is as follows: ， in, Represents a splicing operation; Final feature map After the second 1×1 convolution operation, the feature map The number of channels is restored and the feature map is obtained ; S4.

4. Output feature map of the first residual attention module A2 As the second residual attention module A2, repeat the operations in steps S4.2 and S4.3 to obtain the output feature map of the first residual attention module A2 , take the output of the previous residual attention module A2 as the input of the next residual attention module A2, iterate in sequence, and get the first Output feature map of residual attention module A2 ; S4.

5. Feature Map It passes through the BN layer, the ReLU layer, the BN layer in turn, and then connects to the feature map through the residual connection After adding, it passes through a relu layer to get the final feature map , the calculation process is as follows: 。 5. The coal-rock image classification method based on deep learning according to claim 1 is characterized in that: S5 is as follows: The feature map Input to the classifier for classification. The classifier includes an average pooling layer and a fully connected layer. The feature map First, the feature space vector is obtained through the average pooling layer, and then the category with the highest probability value is obtained through the mapping of the fully connected layer. The calculation process is as follows: ， in, Represents the prediction result, represents the operation of the average pooling layer, Represents the operation of a fully connected layer.

6. A coal-rock image classification device based on deep learning, characterized in that: The following units are included: (1) Preprocessing unit: used to construct coal rock image dataset and preprocess the dataset; (2) Feature extraction module A1: extracts the features of coal and rock images using a large-core parallel convolution module; (3) Residual attention module A2: It consists of a variant residual module and an attention module, and is used to extract the channel attention of feature maps of different scales, obtain the channel attention vector at each different scale, and output a feature map after multi-scale feature information attention weighting; (4) Classification unit: used to classify feature maps.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the coal-rock image classification method based on deep learning as described in any one of claims 1 to 5.

Images