A coal-rock structure damage classification method and device based on convolutional neural network

Abstract

The present invention provides a coal-rock structure damage classification method and device based on convolutional neural network, and relates to the technical field of coal-rock structure damage classification detection. The coal-rock structure damage classification method based on convolutional neural network includes: obtaining acoustic emission parameter data about coal-rock mass fracture; processing the data using Morse wavelet to obtain RGB image data; building a coal-rock structure damage recognition model; inputting the image data into the model, extracting features through a pre-trained ResNet and multi-head attention mechanism network to obtain a first feature representation; extracting features through a pre-trained LSTM and multi-head attention mechanism residual network to obtain a second feature representation; extracting features through a pre-trained convolutional neural and LSSVM network to obtain a third feature representation; fusing all the obtained feature representations to obtain a fusion result; and obtaining a classification prediction result through the fusion result. The present invention can improve the accuracy of coal-rock structure damage classification.

Description

Technical Field

[0001] The present invention relates to the technical field of coal-rock structure damage classification detection, and in particular to a coal-rock structure damage classification method and device based on convolutional neural network. Background Art

[0002] As the depth of coal mining in my country continues to increase, the risk of coal-rock structural damage is increasing, becoming one of the main disasters that restrict safe and efficient production in mines. Existing sensors are easily affected by dust, moisture and high temperature in the coal mine environment, resulting in reduced accuracy of monitoring data, and the accuracy and reliability of classification monitoring equipment need to be improved, which is prone to false alarms or missed alarms. Classification monitoring equipment can capture and record acoustic emission signals. Acoustic emission signals with different parameters correspond to different degrees of coal-rock fractures. Acoustic emission signal data obtained through classification monitoring technology can provide classification indicators for coal-rock fracture structural damage. Therefore, how to establish an effective classification detection method and accurately detect and classify coal-rock fracture structural damage is of great significance for preventing and mitigating safety hazards. Summary of the invention

[0003] In order to solve the technical problems of low accuracy of existing sensor monitoring data, classification errors in classification monitoring equipment, and poor classification algorithm capabilities of deep learning models, the embodiment of the present invention provides a coal rock structure damage classification method and device based on convolutional neural network. The technical solution is as follows:

[0004] On the one hand, a coal-rock structure damage classification method based on a convolutional neural network is provided, the method is implemented by a coal-rock structure damage classification device based on a convolutional neural network, and the method includes:

[0005] S1. Acquire acoustic emission parameter data about coal rock mass fracture and stress state; process the acoustic emission parameter data using Morse wavelet according to signal characteristics to obtain RGB image data;

[0006] S2. Construct a coal-rock structure damage recognition model; wherein the coal-rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a fully connected layer, and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM;

[0007] S3, inputting the RGB image data into the input layer of the coal-rock structure damage recognition model, performing feature extraction through the pre-trained ResNet-based and multi-head attention mechanism network, and obtaining a first feature representation;

[0008] S4, inputting the RGB image data into the coal-rock structure damage recognition model, performing feature extraction through the pre-trained residual network based on LSTM and multi-head attention mechanism, and obtaining a second feature representation;

[0009] S5, inputting the RGB image data into the coal-rock structure damage recognition model, performing feature extraction through the pre-trained convolutional neural network and LSSVM network, and obtaining a third feature representation;

[0010] S6. Input the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; input the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result.

[0011] On the other hand, a coal-rock structure damage classification device based on a convolutional neural network is provided, and the device is applied to a coal-rock structure damage classification method based on a convolutional neural network, and the device comprises:

[0012] The first acquisition unit is used to acquire acoustic emission parameter data about coal rock mass fracture and stress state; according to signal characteristics, the acoustic emission parameter data is processed by using Morse wavelet to obtain RGB image data;

[0013] A construction unit is used to construct a coal-rock structure damage recognition model; wherein the coal-rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a fully connected layer, and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM;

[0014] A second acquisition unit is used to input the RGB image data into the input layer of the coal-rock structure damage recognition model, perform feature extraction through the pre-trained ResNet-based and multi-head attention mechanism network, and obtain a first feature representation;

[0015] A third acquisition unit is used to input the RGB image data into the coal-rock structure damage recognition model, perform feature extraction through the pre-trained residual network based on LSTM and multi-head attention mechanism, and obtain a second feature representation;

[0016] A fourth acquisition unit is used to input the RGB image data into the coal-rock structure damage recognition model, and extract features through the pre-trained convolutional neural network and LSSVM network to obtain a third feature representation;

[0017] A fusion unit is used to input the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; and input the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result.

[0018] On the other hand, a coal rock structure damage classification device based on a convolutional neural network is provided, and the coal rock structure damage classification device based on a convolutional neural network includes: a processor; a memory, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, any one of the above-mentioned coal rock structure damage classification methods based on a convolutional neural network is implemented.

[0019] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned coal and rock structure damage classification methods based on convolutional neural networks.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] In the embodiment of the present invention, acoustic emission parameter data on coal rock fracture and stress state are first obtained; according to the signal characteristics, the acoustic emission parameter data is processed by using Morse wavelet to obtain RGB image data; secondly, a coal rock structure damage recognition model is constructed; wherein the coal rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a full connection layer and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM; the RGB image data is input into the input layer of the coal rock structure damage recognition model, and the RGB image data is input into the input layer of the coal rock structure damage recognition model through the pre-trained The trained ResNet-based and multi-head attention mechanism network is used to extract features to obtain the first feature representation; the RGB image data is input into the coal-rock structure damage recognition model, and the second feature representation is obtained by the pre-trained LSTM-based and multi-head attention mechanism residual network; the RGB image data is input into the coal-rock structure damage recognition model, and the third feature representation is obtained by the pre-trained convolutional neural and LSSVM network; finally, the first feature representation, the second feature representation and the third feature representation are input into the fusion layer for fusion to obtain the fused feature vector; the fused feature vector is input into the fully connected layer and the classification layer for classification to obtain the classification prediction result.

[0022] The embodiment of the present invention processes images and extracts features through three neural networks, thereby solving the difficult and complex problems of classifying coal rock structure damage, improving the accuracy of coal rock structure damage classification, and accurately identifying shear and tension damage to coal rock structures, thereby improving classification accuracy; the embodiment of the present invention can be used to identify precursor features of coal rock damage to prevent the occurrence of coal rock structure damage accidents. The embodiment of the present invention can monitor and classify various types of damage in coal rock structures in real time, and provide accurate identification information. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a schematic diagram of the overall structure of a coal-rock structure damage classification method based on a convolutional neural network provided by an embodiment of the present invention;

[0025] Figure 2 It is a flow chart of a coal-rock structure damage classification method based on a convolutional neural network provided by an embodiment of the present invention;

[0026] Figure 3 It is a structural diagram of a ResNet network and a multi-head attention mechanism network provided by an embodiment of the present invention;

[0027] Figure 4 It is a block diagram of a coal-rock structure damage classification device based on a convolutional neural network provided by an embodiment of the present invention;

[0028] Figure 5 It is a structural schematic diagram of a coal-rock structure damage classification device based on a convolutional neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0031] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0032] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0033] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0034] The embodiment of the present invention provides a coal rock structure damage classification method based on a convolutional neural network, which can be implemented by a coal rock structure damage classification device based on a convolutional neural network, and the coal rock structure damage classification device based on a convolutional neural network can be a terminal or a server. Figure 1The overall structural schematic diagram of the coal-rock structural damage classification method based on convolutional neural network is shown; in a feasible implementation mode, the original coal-rock waveform signal is obtained, and RGB image data is obtained through CWT transformation processing; the RGB image data is input into a pre-trained network based on ResNet and multi-head attention mechanism, and processed through convolution layer and maximum pooling layer to obtain image features; the image features are input into the residual convolution module and the multi-head attention module for processing to obtain the processing results; the processing results are input into the global average pooling layer and the fully connected layer for processing to obtain the first feature representation; the RGB image data is input into a pre-trained residual network based on LSTM and multi-head attention mechanism, and the processing results are obtained through flattening and adjustment processing; the processing results are input into two LSTM layers, and processed through the multi-head attention mechanism to obtain image features; the image features are processed through flattening layer and fully connected layer to obtain the second feature representation. The RGB image data is input into the pre-trained convolutional neural network and LSSVM network, and convolution processing is performed through the three-layer KAN convolution layer to obtain image features; the image features are input into the adaptive average pooling and multi-head attention mechanism for processing to obtain the processing results; the processing results are input into the fully connected layer and the BKA-LSSVM module for processing, and the third feature representation is obtained through the fully connected layer; the first feature representation, the second feature representation and the third feature representation are input into the fusion layer for feature fusion to obtain the fused feature vector; the fused feature vector is input into the fully connected layer and the classification layer, and classified through the Softmax layer to obtain the classification prediction result; the classification evaluation indicators: recall rate, accuracy rate and precision rate are used to evaluate the classification prediction results, and the final classification prediction results are obtained and the category probability is output.

[0035] like Figure 2 The flowchart of the coal-rock structure damage classification method based on convolutional neural network is shown. The processing flow of the method may include the following steps:

[0036] S1. Acquire acoustic emission parameter data about coal rock fracture and stress state; process the acoustic emission parameter data using Morse wavelet according to signal characteristics to obtain RGB image data.

[0037] Among them, the acoustic emission parameter data about the fracture and stress state of coal rock mass can be obtained through Brazilian splitting test and direct shear test.

[0038] Among them, the acoustic emission parameter data about coal rock fracture and stress state include: parameter amplitude, rise time, duration, energy, average frequency, count and peak frequency.

[0039] Optionally, the specific implementation process of S1 may include S11-S13:

[0040] S11, according to the signal characteristics, using a preset threshold value for detecting signal events, detecting events of the acoustic emission parameter data to obtain event segments;

[0041] In a feasible implementation, before event detection of the acoustic emission parameter data, the time series corresponding to each parameter in the acoustic emission parameter data is normalized to obtain a normalized result; wherein the normalization process can be expressed by the following formula (1):

[0042] (1)

[0043] Among them, among them, Represents the original coal rock data The parameter value; The first The minimum value among the parameters; The first The maximum value among the parameters; Represents the normalized coal rock data The parameter value.

[0044] In a feasible implementation, according to the signal characteristics, 0.004V is selected as the preset threshold for detecting signal events; wherein, the preset threshold for detecting signal events is used to detect events in the signal, traverse each time point of the signal, and detect whether the amplitude exceeds 0.004V; wherein, when the signal exceeds 0.004V, an event is recorded, and when the event recording starts, the recording is continued until 20 consecutive signals are below 0.004V, which is used as the end event point of the event, and the event segment is obtained to complete the process of detecting signal events.

[0045] S12, processing the damage signal of the event segment according to a preset threshold for determining the damage signal, and obtaining processed acoustic emission parameter data;

[0046] In a feasible implementation, the rise time / amplitude value RA of each event segment is calculated, and the category of the damage signal is determined by comparing the distribution changes of the RA value and the AF value during the damage process, and by a set threshold; wherein the AF value represents the average frequency of each event segment; the threshold is set to 1, and when the calculated RA value is greater than 1, the damage signal is determined to be shear rupture; when the AF value is less than 1, the damage signal is determined to be tensile rupture.

[0047] In a feasible implementation, the damage signal of each event segment is marked as a corresponding classification to generate a classification label; based on the comparison between the RA value and the threshold value 1, the event segment damage signal is divided into two categories, including: when the RA value is greater than 1, the event segment is marked as a shear rupture signal, and the label "0" represents the shear state; when the RA value is less than 1, the event segment is marked as a tension rupture signal, and the label "1" represents the tension state; the acoustic emission parameter data is processed through the above process to obtain the processed acoustic emission parameter data.

[0048] S13, using Morse wavelet to perform wavelet transform on the processed acoustic emission parameter data to obtain RGB image data.

[0049] In a feasible implementation manner, the Morse wavelet is used to perform wavelet transform processing on the processed acoustic emission parameter data, and the specific process of obtaining RGB image data includes:

[0050] S131, using Morse wavelet to perform continuous wavelet transform on the processed acoustic emission parameter data, wherein, given a signal and a mother wavelet, the process of performing continuous wavelet transform on the acoustic emission waveform data can be expressed by the following formula (2):

[0051] (2)

[0052] in, Representation parameters and parameters The continuous wavelet transform of Indicates time, represents the original time domain signal obtained from the sensor network; Representing the continuous complex function in the time and frequency domains is called the mother wavelet; represents the complex conjugate of the analytical wavelet function; Represents the normalization factor of the wavelet function, ensuring that the transformed results have the same energy; Represents the mathematical operation of the entire time domain integration; represents the complex conjugate of the analytical wavelet function, with the parameter and parameters To scale and move, represents the scaling parameter, which refers to the scale scaling parameter and determines the width of the wavelet; represents the translation parameter, which determines the position of the wavelet; where the Morse wavelet is selected as the basic wavelet function for analysis and can be expressed by the following formula (3):

[0053] (3)

[0054] in, Indicates time Function of Represents parameters related to signal bandwidth; Represents frequency-related parameters; Represents an imaginary unit.

[0055] When Morse wavelet is selected as the basic wavelet function, the expression of the change of continuous wavelet can be expressed by the following formula (4):

[0056] (4)

[0057] in, represents the Morse wavelet function, represents the complex conjugate of the Morse wavelet function, represents the frequency parameter, represents the shape parameter, Indicates translating the wavelet function along the time axis units.

[0058] S132. The CWT continuous wavelet transform convolution operation can form a two-dimensional matrix, called a coefficient matrix; wherein the rows of the coefficient matrix correspond to the scale, the columns of the coefficient matrix correspond to the position, and each element is the coefficient value at the corresponding scale and position; the characteristics of the signal at different frequency scales and time positions can be obtained through the continuous wavelet transform; the 'cwt_to_rgb' function is used to map the continuous wavelet transform image to the three channels of the RGB image, and merge them into an RGB image to obtain the RGB image data; wherein the RGB image data is RGB image data; wherein the 'np.clip' function is used to ensure that the intermediate frequency component is between 0 and 1 to prevent the channel value from exceeding the RGB range.

[0059] S2. Construct a coal-rock structure damage identification model; wherein, the coal-rock structure damage identification model includes: an input layer, a feature extraction module, a fusion layer, a fully connected layer and a classification layer; wherein, the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM.

[0060] Optionally, S2's pre-trained ResNet-based and multi-head attention network includes:

[0061] Convolutional layer, max pooling layer, four residual convolution modules, multi-head attention mechanism module, global average pooling layer and fully connected layer;

[0062] S2’s pre-trained LSTM and multi-head attention residual network, including two LSTM layers, a multi-head attention module, a flat layer, and a fully connected layer;

[0063] S2's pre-trained convolutional neural and LSSVM network includes: three KAN convolutional layers, an adaptive average pooling layer, a multi-head attention mechanism module, two fully connected layers, and a BKA-LSSVM module.

[0064] S3. Input the RGB image data into the input layer of the coal-rock structure damage recognition model, extract features through the pre-trained ResNet-based and multi-head attention mechanism network, and obtain the first feature representation.

[0065] Figure 3 is a schematic diagram of a structure of a ResNet network and a multi-head attention mechanism network provided by an embodiment of the present invention; in a feasible implementation manner, the collected RGB image is used as the input of the convolutional neural network, and the The convolution kernel of the input image is convolved, the feature map is downsampled through the maximum pooling layer to reduce the size of the feature map, and the outputs of the convolution layer and the maximum pooling layer are added through the ReLU activation function layer as the input X of the residual block. The input feature map is wavelet transformed using the CWT layer to generate a multi-scale feature map. After 3 KAN convolution layers, 2 batch normalization layers, 2 ReLU activation function layers and 1 downsample layer, the input X and the output of the residual convolution module are added and output in a residual connection; the output result is processed by the multi-head attention module, and feature extraction and enhancement are performed alternately through 4 residual convolution modules and 4 multi-head attention modules; a global average pooling operation is performed on the entire feature map to generate a fixed-length feature vector; the pooled feature vector is processed using a fully connected layer to output the final feature representation.

[0066] Optionally, the specific implementation process of S3 may include S31-S33:

[0067] S31, inputting the RGB image data into the convolution layer and the maximum pooling layer for processing to obtain the features of the RGB image data;

[0068] Among them, the ResNet network is a convolutional neural network based on the ResNet-18 architecture, and the size of the convolutional layer kernel is pixels, and the step size is pixels, the padding size is pixels.

[0069] Among them, the size of the maximum pooling layer kernel is pixels, and the step size is pixels, the padding size is pixels.

[0070] S32, inputting the features of the RGB image data into four residual convolution modules and a multi-head attention module to obtain deep features of the RGB image data;

[0071] Among them, each of the four residual convolution modules is composed of residual blocks; each residual block includes: 3 KAN convolution layers, 1 CWT layer, 2 batch normalization layers, 2 ReLU activation function layers and 1 downsample layer.

[0072] In a feasible implementation, a CWT layer is inserted before the convolution operation of each residual block, and the CWT layer is used to perform wavelet transformation on the input feature image. The feature map is at different scales [1, 2, 3] to generate a multi-scale feature map.

[0073] In a feasible implementation, in the residual block processing, a batch normalization layer and a ReLU activation function are set after each convolution layer, which helps to stabilize the model and normalize the output of the convolution layer and can provide nonlinear transformation.

[0074] Among them, in the residual block processing, the KAN convolution layer processes multi-channel RGB image data, and the convolution operation is performed on each channel to obtain the processing result; according to the processing result, the batch normalization layer is used to merge to obtain the merged result; among them, the specific implementation process of the batch normalization layer includes:

[0075] (1) Calculating the mean and variance of small batches of data can be expressed by the following formulas (5) and (6):

[0076] (5)

[0077] in, is the number of samples in the mini-batch; represents the mean; Indicates samples.

[0078] (6)

[0079] in, represents variance; represents a very small constant, take , used to prevent the denominator from being zero.

[0080] (2) Introducing learnable parameters, performing scaling and translation normalization, and obtaining the output variable after batch normalization, which can be expressed by the following formula (7):

[0081] (7)

[0082] in, represents the output variable after batch normalization; represents the first learnable parameter; represents the second learnable parameter.

[0083] Among them, the batch norm layer can keep the input of each layer stable and improve the robustness of the training steps and the model.

[0084] In a feasible implementation, the input data and the output of the convolution in the residual connection can be added through the ReLU activation function. The specific implementation process includes:

[0085] (1) According to the input vector, the linear transformation result is calculated using the following formula (8):

[0086] (8)

[0087] in, represents the result after linear transformation, represents the weight matrix, represents the input vector, Represents the bias term.

[0088] (2) Based on the result after linear transformation, the weighted sum of all input units plus the bias term is calculated using the following formula (9):

[0089] (9)

[0090] in, Represents the weighted sum of all input units plus the bias term As a result, Indicates that from Input unit to The weights of the output neurons, Indicates that from The value of the input cell, Indicates The bias term of the output neuron, Represents the total number of input cells.

[0091] (3) Based on the weighted sum of all input units plus the bias term, the activation value of each neuron is calculated using the following formula (10):

[0092] (10)

[0093] in, It is The activation value of a neuron.

[0094] Among them, the downsample layer is used to adjust the size mismatch in the residual connection.

[0095] In a feasible implementation, after each residual convolution module is processed, a multi-head attention module is set up, including: a multi-head attention mechanism and LayerNorm; wherein the multi-head attention mechanism and LayerNorm; structure can introduce global context information while retaining the deep features of ResNet, thereby improving the overall performance of the model.

[0096] Among them, the specific implementation process of the multi-head attention mechanism may include:

[0097] (1) Convert the input sequence into a vector representation through the embedding layer to obtain the input vector;

[0098] (2) Perform a linear transformation on the input vector and obtain the query according to the following formula (11):

[0099] (11)

[0100] in, Represents the input sequence, the shape is , represents the query vector; represents the first learnable parameter mean.

[0101] The key is obtained according to the following formula (12):

[0102] (12)

[0103] in, is the key vector; represents the second learnable parameter mean.

[0104] The value vector is obtained according to the following formula (13):

[0105] (13)

[0106] in, represents the third learnable parameter mean.

[0107] (3) Calculate the attention score based on the obtained query, key, and value vectors, and obtain the attention weight through the Softmax function. The specific process of obtaining the attention score can be expressed by the following formula (14):

[0108] (14)

[0109] in, Represents the self-attention mechanism algorithm; Represents the multi-head attention corresponding to the input; Indicates the dimension of the key vector.

[0110] (4) Repeat steps (1) to (3) eight times. When the preset number of heads is calculated, concatenate the outputs of each head and perform linear transformation using the following formula (15) to obtain the final output result:

[0111] (15)

[0112] Among them, it means; Represents the final linear transformation matrix. The final output result is obtained by concatenation and linear transformation, and the shape is restored to ,in, represents the length of the sequence, Represents the dimension of the embedding vector.

[0113] Among them, LayerNorm is used on the output of the multi-head attention mechanism to ensure the feature normalization of each channel.

[0114] S33. Input the deep features of the RGB image data into the global average pooling layer for processing to obtain a feature vector; input the feature vector into the fully connected layer to obtain a first feature representation.

[0115] In one feasible implementation, the depth features of the RGB image data are input into the global average pooling layer for spatial dimension reduction processing, and the spatial dimension is reduced to , generate a feature vector, input the feature vector into the fully connected layer, and obtain the first feature representation.

[0116] S4. Input the RGB image data into the coal-rock structure damage recognition model, extract features through the pre-trained residual network based on LSTM and multi-head attention mechanism, and obtain the second feature representation.

[0117] Optionally, the specific implementation process of S4 may include S41-S43:

[0118] S41, input the RGB image data into two LSTM layers for processing, and output the processing results;

[0119] In a feasible implementation, the specific implementation process of using the anti-aliasing method to process the RGB image data and obtain the sequence data includes:

[0120] (1) Using the anti-aliasing method, the obtained RGB image data is uniformly adjusted to Pixels are used as input to the LSTM layer, and the shape of the selected input data is (32, 20, 24, 224, 3);

[0121] (2) Normalize the pixel values ​​of the RGB image to between 0 and 1;

[0122] (3) Flatten each RGB image into a one-dimensional vector. The size of the RGB image is , after flattening into a one-dimensional vector, it has 150528 elements;

[0123] (4) The flattened images are organized into time series, each of which contains 20 consecutive CWT images, and an input sequence data of size (20, 150528) is created;

[0124] In a feasible implementation, the sequence data is input into a 2-layer LSTM model to process the sequence data.

[0125] Among them, the specific implementation process of the LSTM model includes:

[0126] (1) Input the data at the current moment and the output data at the previous moment;

[0127] (2) According to the input data, the forget gate is calculated by the following formula (16):

[0128] (16)

[0129] in, represents the first activation function; represents the learnable weight of the forget gate; Represents the bias of the forget gate; Represents data at the current moment; Indicates the output data at the previous moment.

[0130] The input gate is calculated according to the following formula (17):

[0131] (17)

[0132] in, Represents the input gate The learnable weights of Represents the input gate The bias.

[0133] Among them, the candidate cell state can be expressed by the following formula (18):

[0134] (18)

[0135] in, Represents candidate cell states The learnable weights of Represents candidate cell states The bias.

[0136] Among them, the updated cell state can be expressed by the following formula (19):

[0137] (19)

[0138] Among them, the output gate can be expressed by the following formula (20):

[0139] (20)

[0140] in, Output gate The learnable weights of Output gate The bias.

[0141] Among them, the hidden state can be expressed by the following formula (21):

[0142] (twenty one)

[0143] in, represents the second activation function.

[0144] S42, inputting the processing results into the multi-head attention mechanism module for fusion processing to obtain the key features of the RGB image data;

[0145] In a feasible implementation, a multi-head attention mechanism is used to fuse the processing results of LSTM, and the processing results of LSTM and the output of the attention layer are added through the Add layer to form a residual connection; wherein the learning residual is used to capture the key features in the data.

[0146] S43. Input the key features of the RGB image data into the flat layer for processing to obtain a one-dimensional feature vector; input the one-dimensional feature vector into the fully connected layer to obtain a second feature representation.

[0147] Among them, a flattening layer is used to flatten the multi-dimensional residual output into a one-dimensional vector.

[0148] S5. Input the RGB image data into the coal-rock structure damage recognition model, extract features through the pre-trained convolutional neural and LSSVM network, and obtain the third feature representation.

[0149] Optionally, the specific implementation process of S5 may include S51-S55:

[0150] S51, inputting the RGB image data into three KAN convolutional layers and an adaptive average pooling layer for processing to obtain local features;

[0151] In a feasible implementation, the RGB image data is input into a pre-trained convolutional neural and LSSVM network, and the local features of the image are extracted according to the multi-scale convolution kernel, including three KAN convolution layers and one adaptive average pooling layer, the output of the first KAN convolution layer, the output of the second KAN convolution layer, the output of the third KAN convolution layer and the adaptive average pooling layer are added, and the ReLU activation function is used to integrate the features to obtain local features.

[0152] S52, inputting the local features into the multi-head attention mechanism module for processing to obtain comprehensive extracted features;

[0153] S53, inputting the comprehensively extracted features into the first fully connected layer for flattening to obtain flattened features;

[0154] S54, inputting the flattened features into the BKA-LSSVM module for processing, and outputting the processed features;

[0155] In one feasible implementation, the flattened features are used to train the LSSVM model. Given a training dataset ,in, represents the flattened eigenvector, is the corresponding category label. The optimization process of LSSVM can be expressed by the following formula (22) and formula (23):

[0156] (twenty two)

[0157] (twenty three)

[0158] in, represents the weight vector, represents the bias, represents the error vector, represents the feature mapping function, represents the regularization parameter.

[0159] In one feasible implementation, the Black-winged Kite Algorithm (BKA) is used to regularize the LSSVM parameters. and kernel parameters The specific implementation steps of BKA-LSSVM optimization include:

[0160] (1) Initialize the population: Randomly generate a population within the parameter range, each containing The value and The value of

[0161] (2) Evaluate fitness: Train the LSSVM model based on the individual parameters and use the remaining 20% ​​test set to evaluate the classification performance;

[0162] (3) Updating individual positions specifically includes:

[0163] Select the best individual: Select the best individual according to the fitness value. ;

[0164] Update strategy: Update the positions of other individuals according to the foraging behavior of the black kite. The update formula can be expressed by the following formula (24):

[0165] (twenty four)

[0166] in, Indicates The individual in The position of the generation, Indicates The individual in The position of the generation, represents the parameter that controls the best individual influence, represents the parameter controlling the random disturbance, Represents a random vector, each category is uniformly distributed in .

[0167] (4) Iterative update: Repeatedly evaluate the fitness and update the individual position until the maximum number of iterations is reached or the fitness value converges;

[0168] (5) Output optimal parameters: After the optimization process is completed, output the individual with the highest fitness value. Value and The value is taken as the optimal parameter of LSSVM.

[0169] S55. Input the processed features into the second fully connected layer for processing to obtain a third feature representation.

[0170] S6. Input the first feature representation, the second feature representation, and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; input the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result.

[0171] In a feasible implementation, a fusion layer is used to fuse the first feature representation, the second feature representation, and the third feature into a unified feature vector; the fused feature vector is input into a fully connected layer, classified by a Softmax classifier, and the features are mapped to different categories to obtain classification prediction results.

[0172] Optionally, after the step of inputting the first feature representation, the second feature representation and the third feature representation into a fusion layer for fusion to obtain a fused feature vector; and inputting the fused feature vector into a fully connected layer and a classification layer for classification to obtain a classification prediction result in S6, the method further includes:

[0173] The classification evaluation indicators are used to evaluate the classification prediction results to obtain the final classification prediction results.

[0174] In a feasible implementation, classification evaluation indicators are used to evaluate and analyze the classification prediction results, where the classification evaluation indicators include: accuracy, precision and recall rate; the accuracy can be expressed by the following formula (25):

[0175] (25)

[0176] The accuracy can be expressed by the following formula (26):

[0177] (26)

[0178] The recall rate can be expressed by the following formula (27):

[0179] (27)

[0180] Among them, when the detected event segment and the true label are both cut, it is defined as ; When the event segment of the sample is shear but the label is tension, it is defined as ; When the detected event segment and the true label are both tensile, it is defined as , when the detected event segment is tension but the true label is shear, it is defined as .

[0181] In a feasible implementation, a confusion matrix is ​​used for visual display and analysis, and the matrix is ​​in the form of:

[0182]

[0183] Among them, the confusion matrix can show specific error types, including and By analyzing the errors, we can find that the model as well as Weaknesses in both categories.

[0184] In the embodiment of the present invention, acoustic emission parameter data on coal rock fracture and stress state are first obtained; according to the signal characteristics, the acoustic emission parameter data is processed by using Morse wavelet to obtain RGB image data; secondly, a coal rock structure damage recognition model is constructed; wherein the coal rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a full connection layer and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM; the RGB image data is input into the input layer of the coal rock structure damage recognition model, and the RGB image data is input into the input layer of the coal rock structure damage recognition model through the pre-trained The trained ResNet-based and multi-head attention mechanism network is used to extract features to obtain the first feature representation; the RGB image data is input into the coal-rock structure damage recognition model, and the second feature representation is obtained by the pre-trained LSTM-based and multi-head attention mechanism residual network; the RGB image data is input into the coal-rock structure damage recognition model, and the third feature representation is obtained by the pre-trained convolutional neural and LSSVM network; finally, the first feature representation, the second feature representation and the third feature representation are input into the fusion layer for fusion to obtain the fused feature vector; the fused feature vector is input into the fully connected layer and the classification layer for classification to obtain the classification prediction result.

[0185] The embodiment of the present invention processes images and extracts features through three neural networks, thereby solving the difficult and complex problems of classifying coal rock structure damage, improving the accuracy of coal rock structure damage classification, and accurately identifying shear and tension damage to coal rock structures, thereby improving classification accuracy; the embodiment of the present invention can be used to identify precursor features of coal rock damage to prevent the occurrence of coal rock structure damage accidents. The embodiment of the present invention can monitor and classify various types of damage in coal rock structures in real time, and provide accurate identification information.

[0186] Figure 4 The present invention is a block diagram of a coal-rock structure damage classification device based on a convolutional neural network according to an exemplary embodiment. The device is used in a coal-rock structure damage classification method based on a convolutional neural network. Figure 4 The device includes a first acquisition unit 310, a construction unit 320, a second acquisition unit 330, a third acquisition unit 340, a fourth acquisition unit 350 and a fusion unit 360. Wherein:

[0187] The first acquisition unit 310 is used to acquire acoustic emission parameter data about coal rock mass fracture and stress state; according to signal characteristics, the acoustic emission parameter data is processed using Morse wavelet to obtain RGB image data;

[0188] A construction unit 320 is used to construct a coal-rock structure damage recognition model; wherein the coal-rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a fully connected layer, and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM;

[0189] A second acquisition unit 330 is used to input the RGB image data into the input layer of the coal-rock structure damage recognition model, perform feature extraction through the pre-trained ResNet-based and multi-head attention mechanism network, and obtain a first feature representation;

[0190] A third acquisition unit 340 is used to input the RGB image data into the coal-rock structure damage recognition model, perform feature extraction through the pre-trained LSTM-based and multi-head attention mechanism residual network, and obtain a second feature representation;

[0191] A fourth acquisition unit 350 is used to input the RGB image data into the coal-rock structure damage recognition model, perform feature extraction through the pre-trained convolutional neural network and LSSVM network, and obtain a third feature representation;

[0192] The fusion unit 360 is used to input the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; input the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result.

[0193] Optionally, the first acquiring unit 310 is configured to:

[0194] According to the signal characteristics, using a preset threshold value for detecting signal events, detecting events of the acoustic emission parameter data to obtain event segments;

[0195] Processing the damage signal of the event segment according to a preset threshold for determining the damage signal to obtain processed acoustic emission parameter data;

[0196] The processed acoustic emission parameter data is subjected to wavelet transform processing using Morse wavelet to obtain RGB image data.

[0197] Optionally, the pre-trained network based on ResNet and multi-head attention mechanism includes:

[0198] Convolutional layer, max pooling layer, four residual convolution modules, multi-head attention mechanism module, global average pooling layer and fully connected layer;

[0199] The pre-trained LSTM and multi-head attention mechanism based residual network includes: two LSTM layers, a multi-head attention mechanism module, a flat layer and a fully connected layer;

[0200] The pre-trained convolutional neural and LSSVM network includes: three KAN convolutional layers, an adaptive average pooling layer, a multi-head attention mechanism module, two fully connected layers and a BKA-LSSVM module.

[0201] Optionally, extracting features through the pre-trained ResNet-based and multi-head attention mechanism network to obtain a first feature representation includes:

[0202] Inputting the RGB image data into the convolution layer and the maximum pooling layer for processing to obtain features of the RGB image data;

[0203] Input the features of the RGB image data into the four residual convolution modules and the multi-head attention module to obtain the deep features of the RGB image data;

[0204] The depth features of the RGB image data are input into the global average pooling layer for processing to obtain a feature vector; the feature vector is input into the fully connected layer to obtain a first feature representation.

[0205] Optionally, the step of extracting features after the preprocessing by using the pretrained residual network based on LSTM and multi-head attention mechanism to obtain a second feature representation includes:

[0206] Input the RGB image data into the two LSTM layers for processing, and output the processing results;

[0207] Inputting the processing results into the multi-head attention mechanism module for fusion processing to obtain key features of RGB image data;

[0208] The key features of the RGB image data are input into the flat layer for processing to obtain a one-dimensional feature vector; the one-dimensional feature vector is input into the fully connected layer to obtain a second feature representation.

[0209] Optionally, the extracting features based on the pre-trained convolutional neural network and LSSVM network to obtain a third feature representation includes:

[0210] Input the RGB image data into the three KAN convolutional layers and the adaptive average pooling layer for processing to obtain local features;

[0211] Inputting the local features into the multi-head attention mechanism module for processing to obtain comprehensive extracted features;

[0212] Inputting the comprehensively extracted features into the first fully connected layer for flattening to obtain flattened features;

[0213] Inputting the flattened features into the BKA-LSSVM module for processing, and outputting the processed features;

[0214] The processed features are input into the second fully connected layer for processing to obtain a third feature representation.

[0215] Optionally, after the step of inputting the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; and inputting the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result, the step further includes:

[0216] The classification evaluation indicators are used to evaluate the classification prediction results to obtain the final classification prediction results.

[0217] In the embodiment of the present invention, acoustic emission parameter data on coal rock fracture and stress state are first obtained; according to the signal characteristics, the acoustic emission parameter data is processed by using Morse wavelet to obtain RGB image data; secondly, a coal rock structure damage recognition model is constructed; wherein the coal rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a full connection layer and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM; the RGB image data is input into the input layer of the coal rock structure damage recognition model, and the RGB image data is input into the input layer of the coal rock structure damage recognition model through the pre-trained The trained ResNet-based and multi-head attention mechanism network is used to extract features to obtain the first feature representation; the RGB image data is input into the coal-rock structure damage recognition model, and the second feature representation is obtained by the pre-trained LSTM-based and multi-head attention mechanism residual network; the RGB image data is input into the coal-rock structure damage recognition model, and the third feature representation is obtained by the pre-trained convolutional neural and LSSVM network; finally, the first feature representation, the second feature representation and the third feature representation are input into the fusion layer for fusion to obtain the fused feature vector; the fused feature vector is input into the fully connected layer and the classification layer for classification to obtain the classification prediction result.

[0218] The embodiment of the present invention processes images and extracts features through three neural networks, thereby solving the difficult and complex problems of classifying coal rock structure damage, improving the accuracy of coal rock structure damage classification, and accurately identifying shear and tension damage to coal rock structures, thereby improving classification accuracy; the embodiment of the present invention can be used to identify precursor features of coal rock damage to prevent the occurrence of coal rock structure damage accidents. The embodiment of the present invention can monitor and classify various types of damage in coal rock structures in real time, and provide accurate identification information.

[0219] Figure 5 is a structural schematic diagram of a coal-rock structure damage classification device based on a convolutional neural network provided by an embodiment of the present invention, such as Figure 5 As shown, the coal rock structure damage classification device based on convolutional neural network may include the above Figure 4 The coal-rock structure damage classification device based on convolutional neural network is shown. Optionally, the coal-rock structure damage classification device based on convolutional neural network 410 may include a first processor 2001.

[0220] Optionally, the coal-rock structure damage classification device 410 based on convolutional neural network may also include a memory 2002 and a transceiver 2003 .

[0221] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0222] Combine the following Figure 5 The various components of the coal-rock structure damage classification device 410 based on convolutional neural network are specifically introduced:

[0223] The first processor 2001 is the control center of the coal-rock structure damage classification device 410 based on a convolutional neural network, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0224] Optionally, the first processor 2001 can perform various functions of the coal and rock structure damage classification device 410 based on convolutional neural network by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0225] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 are shown in FIG.

[0226] In a specific implementation, as an embodiment, the coal-rock structure damage classification device 410 based on convolutional neural network may also include multiple processors, such as Figure 5 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0227] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0228] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0229] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0230] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 5 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0231] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0232] It should be noted that Figure 5 The structure of the coal-rock structure damage classification device 410 based on convolutional neural network shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0233] In addition, the technical effects of the coal-rock structure damage classification device 410 based on convolutional neural network can refer to the technical effects of the coal-rock structure damage classification method based on convolutional neural network described in the above method embodiment, and will not be repeated here.

[0234] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0235] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0236] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0237] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0238] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0239] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0240] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0241] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0242] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0243] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0244] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0245] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0246] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A coal-rock structure damage classification method based on convolutional neural network, characterized in that: The method comprises: S1. Acquire acoustic emission parameter data about coal rock mass fracture and stress state; process the acoustic emission parameter data using Morse wavelet according to signal characteristics to obtain RGB image data; S2. Construct a coal-rock structure damage recognition model; wherein the coal-rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a fully connected layer, and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM; S3, inputting the RGB image data into the input layer of the coal-rock structure damage recognition model, performing feature extraction through the pre-trained ResNet-based and multi-head attention mechanism network, and obtaining a first feature representation; S4, inputting the RGB image data into the coal-rock structure damage recognition model, performing feature extraction through the pre-trained residual network based on LSTM and multi-head attention mechanism, and obtaining a second feature representation; S5, inputting the RGB image data into the coal-rock structure damage recognition model, performing feature extraction through the pre-trained convolutional neural network and LSSVM network, and obtaining a third feature representation; S6. Input the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; input the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result.

2. The coal-rock structure damage classification method based on convolutional neural network according to claim 1 is characterized in that: The step S1 processes the acoustic emission parameter data according to the signal characteristics to obtain RGB image data, including: S11, according to the signal characteristics, using a preset threshold value for detecting signal events, detecting events of the acoustic emission parameter data to obtain event segments; S12, processing the damage signal of the event segment according to a preset threshold for determining the damage signal, and obtaining processed acoustic emission parameter data; S13, using Morse wavelet to perform wavelet transform processing on the processed acoustic emission parameter data to obtain RGB image data.

3. The coal-rock structure damage classification method based on convolutional neural network according to claim 1 is characterized in that: The pre-trained S2 network based on ResNet and multi-head attention mechanism includes: Convolutional layer, max pooling layer, four residual convolution modules, multi-head attention mechanism module, global average pooling layer and fully connected layer; The pre-trained LSTM and multi-head attention mechanism residual network of S2 includes: two LSTM layers, a multi-head attention mechanism module, a flat layer and a fully connected layer; The pre-trained S2 is based on convolutional neural and LSSVM networks, including: three KAN convolutional layers, an adaptive average pooling layer, a multi-head attention mechanism module, two fully connected layers and a BKA-LSSVM module.

4. The coal-rock structure damage classification method based on convolutional neural network according to claim 3 is characterized in that: The S3 extracts features through the pre-trained ResNet-based and multi-head attention mechanism network to obtain a first feature representation, including: S31, inputting the RGB image data into the convolution layer and the maximum pooling layer for processing to obtain the features of the RGB image data; S32, inputting the features of the RGB image data into four residual convolution modules and a multi-head attention module to obtain deep features of the RGB image data; S33. Input the deep features of the RGB image data into the global average pooling layer for processing to obtain a feature vector; input the feature vector into a fully connected layer to obtain a first feature representation.

5. The coal-rock structure damage classification method based on convolutional neural network according to claim 3 is characterized in that: The S4 extracts features through the pre-trained residual network based on LSTM and multi-head attention mechanism to obtain a second feature representation, including: S41, inputting the RGB image data into the two LSTM layers for processing, and outputting the processing result; S42, inputting the processing results into a multi-head attention mechanism module for fusion processing to obtain key features of the RGB image data; S43, inputting the key features of the RGB image data into the flat layer for processing to obtain a one-dimensional feature vector; inputting the one-dimensional feature vector into the fully connected layer to obtain a second feature representation.

6. The coal-rock structure damage classification method based on convolutional neural network according to claim 3 is characterized in that: The S5 extracts features through the pre-trained convolutional neural network and LSSVM network to obtain a third feature representation, including: S51, inputting the RGB image data into the three KAN convolutional layers and the adaptive average pooling layer for processing to obtain local features; S52, inputting the local features into a multi-head attention mechanism module for processing to obtain comprehensively extracted features; S53, inputting the comprehensively extracted features into the first fully connected layer for flattening to obtain flattened features; S54, inputting the flattened features into the BKA-LSSVM module for processing, and outputting the processed features; S55: Input the processed features into the second fully connected layer for processing to obtain a third feature representation.

7. The coal-rock structure damage classification method based on convolutional neural network according to claim 1 is characterized in that: After the step of inputting the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; and inputting the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result, the step further includes: The classification evaluation indicators are used to evaluate the classification prediction results to obtain the final classification prediction results.

8. A coal-rock structure damage classification device based on a convolutional neural network, the coal-rock structure damage classification device based on a convolutional neural network is used to implement the coal-rock structure damage classification method based on a convolutional neural network as claimed in any one of claims 1 to 7, characterized in that: The device comprises: The first acquisition unit is used to acquire acoustic emission parameter data about coal rock mass fracture and stress state; according to signal characteristics, the acoustic emission parameter data is processed by using Morse wavelet to obtain RGB image data; A construction unit is used to construct a coal-rock structure damage recognition model; wherein the coal-rock structure damage recognition model includes: an input layer, a feature extraction module, a fusion layer, a fully connected layer, and a classification layer; wherein the feature extraction module includes: a pre-trained network based on ResNet and a multi-head attention mechanism, a pre-trained residual network based on LSTM and a multi-head attention mechanism, and a pre-trained network based on convolutional neural network and LSSVM; A second acquisition unit is used to input the RGB image data into the input layer of the coal-rock structure damage recognition model, perform feature extraction through the pre-trained ResNet-based and multi-head attention mechanism network, and obtain a first feature representation; A third acquisition unit is used to input the RGB image data into the coal-rock structure damage recognition model, perform feature extraction through the pre-trained residual network based on LSTM and multi-head attention mechanism, and obtain a second feature representation; A fourth acquisition unit is used to input the RGB image data into the coal-rock structure damage recognition model, and extract features through the pre-trained convolutional neural network and LSSVM network to obtain a third feature representation; A fusion unit is used to input the first feature representation, the second feature representation and the third feature representation into the fusion layer for fusion to obtain a fused feature vector; and input the fused feature vector into the fully connected layer and the classification layer for classification to obtain a classification prediction result.

9. A coal-rock structure damage classification device based on convolutional neural network, characterized in that: The coal-rock structure damage classification device based on convolutional neural network includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images