Coal and rock identification method and system based on audio and vibration signal fusion

Abstract

The present invention relates to the technical field of engineering geological rock identification and classification, and discloses a coal rock identification method and system based on the fusion of audio and vibration signals, wherein the method comprises: obtaining vibration signals and audio signals in the coal mining process; performing denoising on the vibration signals and the audio signals respectively; cutting the denoised signals respectively according to set time intervals to obtain a plurality of vibration signal segments and a plurality of audio signal segments; performing normalization on all vibration signal segments and audio signal segments respectively; inputting the normalized signal segments into the trained coal rock identification model at the corresponding moment in time sequence, the trained coal rock identification model outputs the coal rock identification result, and the input end of the third type hidden layer of the coal rock identification model at the next moment is connected to the output end of the third type hidden layer of the coal rock identification model at the previous moment. The present invention has the advantages of high automation, fast identification speed, high lithology identification accuracy, low cost, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering geological rock identification and classification, and in particular to a coal rock identification method and system based on the fusion of audio and vibration signals. Background Art

[0002] At present, comprehensive mechanized coal mining (fully mechanized mining) as a pillar technology of coal mining continues to support the stability of energy supply. Faced with the urgent need for transformation and upgrading of the coal industry, promoting fully mechanized mining towards less-manpowered or even unmanned mining is not only the key to improving production efficiency, but also the only way to ensure the safety of miners and achieve sustainable development. However, the complex and changeable coal seam conditions, such as the heterogeneity of coal rock hardness and thickness, coupled with the frequent interference of geological structures, have caused the coal seam to have complex and changeable occurrence forms, such as frequent fractures and folds, which have brought great challenges to the precise operation of coal mining machines. Practice has shown that when cutting gangue-containing coal seams or rock formations, the vibrations that coal mining machines are subjected to are more severe, and the wear of the drum picks is aggravated, resulting in a decrease in cutting efficiency, which in turn affects the coal mining efficiency of the entire machine. In this context, the research and development and application of on-site coal and rock identification technology is particularly important. It is like the "smart eye" of the coal mining machine, which can analyze the characteristics of coal and rock in real time and guide the coal mining machine to cut precisely. It not only improves the mining efficiency, but also significantly reduces the equipment loss and energy consumption caused by mistakenly cutting rocks, laying a solid foundation for the intelligent and safe transformation of coal mining.

[0003] In the early days, coal mining machine operators would perform preliminary coal and rock identification by observing the color, texture, hardness and other characteristics of the mined coal and rock samples during operation. This method relies on the operator's experience and intuitive judgment. Although it is simple and direct, it has certain subjectivity and uncertainty. In addition, attempts have been made to use radioactive detection technology for coal and rock identification, such as artificial The backscatter detection method of radioactive sources. However, due to the potential risks of radiation safety and the impact of contact stability on the detection results, this method has been gradually replaced by more advanced technologies. The radiometric detection method measures the radiation emitted by the top (bottom) slate. The energy and intensity attenuation of the rays is used to identify the coal-rock interface, which has become a more practical method for coal-rock identification. However, the harsh environment of the coal mining face, high noise, high dust concentration and high humidity will have different degrees of influence on infrared, spectrum, vibration and radar detection, and the identification effect is not stable.

[0004] When the coal mining machine is cutting coal / rock, there is a lot of vibration and the data is noisy. Using data from a single source to identify coal and rock will result in unstable accuracy. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the present invention provides a coal-rock identification method and system based on the fusion of audio and vibration signals; it has the advantages of high degree of automation, fast identification speed, high accuracy of rock identification, low cost, etc., and has good application prospects. There are obvious differences in the strength of coal, gangue and rock. Therefore, when the coal mining machine cuts coal, gangue and rock, the vibration and audio are significantly different. Based on this phenomenon, it is planned to use machine learning to deeply explore the nonlinear relationship between audio, mechanical vibration and coal and rock, build a fusion identification model, and achieve high-accuracy identification of coal / gangue / rock.

[0006] On the one hand, a coal rock identification method based on the fusion of audio and vibration signals is provided, including: obtaining the vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the cutting process of the coal rock to be mined; denoising the vibration signal and the audio signal respectively; cutting the denoised vibration signal and the denoised audio signal respectively according to the set time interval to obtain a plurality of vibration signal segments and a plurality of audio signal segments; normalizing all the vibration signal segments and the audio signal segments respectively; inputting the normalized vibration signal segments and the normalized audio signal segments into the trained image at the corresponding time in chronological order. In the coal-rock identification model, the trained coal-rock identification model outputs the coal-rock identification result, and the input end of the third type of hidden layer of the coal-rock identification model at the next moment is connected to the output end of the third type of hidden layer of the coal-rock identification model at the previous moment; the training process of the trained coal-rock identification model is divided into three stages: the first stage is the conventional training stage, and when the first loss function value of the first stage no longer decreases, it enters the second stage; the second stage is the difficult sample training stage, and when the second loss function value of the second stage no longer decreases, it enters the third stage; the third stage is the error-prone sample enhancement training stage, and when the third loss function value of the third stage no longer decreases, it ends and the trained coal-rock identification model is obtained.

[0007] On the other hand, a coal rock identification system based on the fusion of audio and vibration signals is provided, including: an acquisition module, which is configured to: acquire the vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the cutting process of the coal rock to be mined; a preprocessing module, which is configured to: denoise the vibration signal and the audio signal respectively; cut the denoised vibration signal and the denoised audio signal respectively according to the set time interval to obtain a plurality of vibration signal segments and a plurality of audio signal segments; normalize all vibration signal segments and audio signal segments respectively; an identification module, which is configured to: divide the normalized vibration signal segments and the normalized audio signal segments into segments according to the set time interval; The data are input into the trained coal-rock identification model at the corresponding moment in chronological order. The trained coal-rock identification model outputs the coal-rock identification result. The input end of the third hidden layer of the coal-rock identification model at the latter moment is connected to the output end of the third hidden layer of the coal-rock identification model at the previous moment. The training process of the trained coal-rock identification model is divided into three stages: the first stage is the conventional training stage. When the first loss function value of the first stage no longer decreases, the second stage is entered; the second stage is the difficult sample training stage. When the second loss function value of the second stage no longer decreases, the third stage is entered; the third stage is the error-prone sample enhancement training stage. When the third loss function value of the third stage no longer decreases, the training ends and the trained coal-rock identification model is obtained.

[0008] The above technical solution has the following advantages or beneficial effects: (1) A real-time identification method for coal gangue by fusing vibration signals and audio signals is proposed, which reduces the interference of noise on the effective information of a single data source and improves the recognition accuracy. (2) A coal gangue identification model based on the fusion of audio and vibration time series is constructed. The model comprehensively considers the historical and current coal rock characteristics, and can condense the audio and vibration dynamic characteristics of coal rock into the model, more comprehensively explore the dynamic characteristics of coal rock, and effectively improve the recognition accuracy. (3) A grouped fully connected layer is designed to construct the recognition model, which effectively reduces the number of model parameters, reduces the amount of data used for model training, and improves the model recognition speed, laying the foundation for the deployment of portable equipment with small computing power at coal mining sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0010] Figure 1 This is a flow chart of the method of embodiment 1.

[0011] Figure 2 Schematic diagram of the internal structure of the model of Example 1.

[0012] Figure 3Schematic diagram of the internal structure of the grouped fully connected layer of Example 1.

[0013] Figure 4 This is a schematic diagram of the classification result of Example 1. DETAILED DESCRIPTION

[0014] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0015] Embodiment 1, as Figure 1 As shown, the coal rock identification method based on the fusion of audio and vibration signals includes: S101: obtaining the vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the cutting process of the coal rock to be mined; S102: denoising the vibration signal and the audio signal respectively; cutting the denoised vibration signal and the denoised audio signal respectively according to the set time interval to obtain a plurality of vibration signal segments and a plurality of audio signal segments; normalizing all vibration signal segments and audio signal segments respectively; S103: inputting the normalized vibration signal segments and the normalized audio signal segments into the corresponding time sequence in time. In the trained coal-rock identification model, the trained coal-rock identification model outputs the coal-rock identification result, and the input end of the third type hidden layer of the coal-rock identification model at the latter moment is connected to the output end of the third type hidden layer of the coal-rock identification model at the previous moment; the training process of the trained coal-rock identification model is divided into three stages: the first stage is the conventional training stage, when the first loss function value of the first stage no longer decreases, it enters the second stage; the second stage is the difficult sample training stage, when the second loss function value of the second stage no longer decreases, it enters the third stage; the third stage is the error-prone sample enhancement training stage, when the third loss function value of the third stage no longer decreases, it ends, and the trained coal-rock identification model is obtained.

[0016] Further, S101: obtain the vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the coal mining rock cutting process; wherein, the vibration signal is collected by a vibration sensor installed at the rocker arm of the coal mining machine; the audio signal is collected by a microphone installed in the drum of the coal mining machine.

[0017] Furthermore, the S102: denoising the vibration signal and the audio signal respectively, specifically includes: using a filtering algorithm to remove noise or abnormal signals in the original signal; wherein the filtering algorithm includes: a Kalman filtering algorithm, a wavelet transform algorithm or a median filtering algorithm.

[0018] Furthermore, the denoised vibration signal and the denoised audio signal are cut at set time intervals to obtain a plurality of vibration signal segments and a plurality of audio signal segments, including: taking the signal mean in each signal segment as representative data of the current signal segment.

[0019] Furthermore, the normalizing of all vibration signal segments and audio signal segments respectively includes: using linear normalization or Z-Score normalization to normalize all vibration signal segments and audio signal segments respectively.

[0020] It should be understood that in order to eliminate the amplitude differences caused by different sensors or measurement conditions, vibration data from different sources or under different conditions can be compared and analyzed on the same scale.

[0021] The normalization of audio data is mainly to adjust the amplitude of the audio signal to avoid unexpected volume changes during the processing and ensure the consistency of the audio.

[0022] Further, S103: the normalized vibration signal segment and the normalized audio signal segment are input into the trained coal-rock identification model at the corresponding moment in time sequence, the trained coal-rock identification model outputs the coal-rock identification result, and the input end of the third type of hidden layer of the coal-rock identification model at the next moment is connected to the output end of the third type of hidden layer of the coal-rock identification model at the previous moment; wherein the trained coal-rock identification model includes: Figure 2 As shown, an input layer, a first type of hidden layer, a second type of hidden layer, a third type of hidden layer and an output layer are connected in series in sequence; the first type of hidden layer is a fully connected layer; the input value of the first type of hidden layer is the normalized vibration signal segments in the x, y and z directions and the normalized audio signal segments; as shown Figure 3 As shown, the second type of hidden layer is a grouped fully connected layer, which includes: a feature grouping sublayer and a feature extraction sublayer; the feature grouping sublayer is used to group all features; the feature extraction sublayer is used to extract features from each group of features using a fully connected neural network.

[0023] The second type of hidden layer has the characteristics of small number of parameters, fast calculation speed, and deep feature fusion. Assume that the input features of the second type of hidden layer are A, B, C, D, E, F, G, H, and I; divide the features into three groups, the first group is A, B, and C; the second group is D, E, and F; the third group is G, H, and I; regroup the features, that is, divide the first group of each group into The features are combined into From the new groups, we get the new first group A, D and G; the new second group B, E and H; the new third group C, F, I; for each new group, we use a fully connected neural network to extract features. The working process of the third type of hidden layer is: ; ; ; ;in, is the input value of the third hidden layer, is the feature to be extracted; , and is the weight to be learned; is the normalization function; for The moment corresponds to the output feature of the third hidden layer of the coal-rock recognition model; for The moment corresponds to the output feature of the third hidden layer of the coal-rock recognition model, as The moment corresponds to the input feature of the third hidden layer of the coal-rock recognition model. represents the first intermediate variable, represents the second intermediate variable, Represents the third intermediate variable.

[0024] Further, S103: the normalized vibration signal segments and the normalized audio signal segments are input into the trained coal and rock identification model at the corresponding moments in chronological order, the trained coal and rock identification model outputs the coal and rock identification results, and the input end of the third type of hidden layer of the coal and rock identification model at the latter moment is connected to the output end of the third type of hidden layer of the coal and rock identification model at the previous moment; wherein, the training process of the trained coal and rock identification model includes: constructing a training set, the training set being the coal and rock vibration signal segments and audio signal segments with known classification results; inputting the training set into the coal and rock identification model, and stopping the training when the loss function value of the model no longer decreases or the number of iterations reaches the set number, thereby obtaining the trained coal and rock identification model.

[0025] Furthermore, the training set is input into the coal-rock recognition model. When the loss function value of the model no longer decreases or the number of iterations reaches the set number, the training is stopped to obtain the trained coal-rock recognition model. The training is divided into three stages: the first stage is the simple sample training stage. When the first loss function value of the first stage no longer decreases, the second stage is entered; the simple sample refers to the sample whose error with the true value label is less than the set threshold during prediction; the second stage is the difficult sample training stage. When the second loss function value of the second stage no longer decreases, the third stage is entered; the difficult sample refers to the sample whose error with the true value label is greater than the set threshold during prediction; the third stage is the error-prone sample enhancement training stage. When the third loss function value of the third stage no longer decreases, the training is terminated to obtain the trained coal-rock recognition model; the error-prone sample refers to the samples of all other target categories that do not correspond to the true value.

[0026] Furthermore, the first loss function , whose expression is: ; is the output probability, Represents the label of the sample; the second loss function , whose expression is: ; is the number of vectors in the feature matrix, is the total number of times the predicted label is the same as the label in the feature matrix; It is a hyperparameter that affects the smoothness of the similarity distribution between samples; is the feature vector output by the last layer of the model, For the corresponding category; is the first feature vectors, For the corresponding category; and is a similarity evaluation indicator. If the label of the feature vector is equal to the sample feature label, then ,otherwise, , if the feature vector label is not equal to the sample feature label, then ,otherwise, ; is the output probability; the higher T The value will make the similarity distribution smoother, that is, the difference between similarities becomes relatively small, and vice versa, a lower T The value will make the similarity distribution more sharp and highlight the differences between samples. Furthermore, the third loss function , whose expression is: ;in, is the first hyperparameter, is the second hyperparameter. The parameter α is used to adjust the weights of difficult samples and error-prone samples, increase the penalty factor of related error outputs, and force the model to learn from error samples.

[0027] Further, S103: the normalized vibration signal segments and the normalized audio signal segments are input into the trained coal rock recognition model at the corresponding moment in time sequence, the trained coal rock recognition model outputs the coal rock recognition result, and the input end of the third type of hidden layer of the coal rock recognition model at the latter moment is connected to the output end of the third type of hidden layer of the coal rock recognition model at the previous moment; including: inputting the normalized vibration signal segments in the x, y and z directions and the normalized audio signal segments at the t-1th moment into the trained coal rock recognition model corresponding to the t-1th moment, and outputting the prediction result at the t-1th moment; inputting the normalized vibration signal segments in the x, y and z directions at the t-1th moment into the trained coal rock recognition model corresponding to the t-1th moment, and outputting the prediction result at the t-1th moment; The signal segment and the normalized audio signal segment are input into the trained coal-rock identification model corresponding to time t, and the prediction result at time t is output; the normalized vibration signal segments in the x, y and z directions at time t+1 and the normalized audio signal segment are input into the trained coal-rock identification model corresponding to time t+1, and the prediction result at time t+1 is output; the input end of the third type of hidden layer of the trained coal-rock identification model corresponding to time t+1 is also connected to the output end of the third type of hidden layer of the coal-rock identification model at time t; the input end of the third type of hidden layer of the trained coal-rock identification model corresponding to time t is also connected to the output end of the third type of hidden layer of the coal-rock identification model at time t-1.

[0028] like Figure 4 As shown, the prediction results include three types: coal, coal gangue and rock.

[0029] This time series model comprehensively considers the vibration signal, audio signal and previous coal rock characteristics at this moment, reduces the impact of noise on the recognition result, and improves the recognition accuracy. Specifically, when using this model for recognition, the model uses the vibration signal, audio signal at a certain moment and the output signal of the model at the previous moment ( ) is the input signal, and the output results include two parts: (1) the coal rock type and the corresponding probability at this moment; (2) the output signal ( ).

[0030] Embodiment 2. This embodiment provides a coal rock identification system based on the fusion of audio and vibration signals, including: an acquisition module, which is configured to: acquire the vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the cutting process of the coal rock to be mined; a preprocessing module, which is configured to: perform denoising on the vibration signal and the audio signal respectively; cut the denoised vibration signal and the denoised audio signal respectively according to the set time interval to obtain a plurality of vibration signal segments and a plurality of audio signal segments; normalize all vibration signal segments and audio signal segments respectively; an identification module, which is configured to: perform denoising on the normalized vibration signal segments and the normalized audio signal segments , and are input into the trained coal-rock identification model at the corresponding moment in chronological order. The trained coal-rock identification model outputs the coal-rock identification result. The input end of the third type of hidden layer of the coal-rock identification model at the latter moment is connected to the output end of the third type of hidden layer of the coal-rock identification model at the previous moment. The training process of the trained coal-rock identification model is divided into three stages: the first stage is the conventional training stage. When the first loss function value of the first stage no longer decreases, the second stage is entered; the second stage is the difficult sample training stage. When the second loss function value of the second stage no longer decreases, the third stage is entered; the third stage is the error-prone sample enhancement training stage. When the third loss function value of the third stage no longer decreases, it ends and the trained coal-rock identification model is obtained.

[0031] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A coal-rock identification method based on the fusion of audio and vibration signals, characterized in that: include: Acquire the vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the coal mining rock cutting process; The vibration signal and the audio signal are subjected to denoising processing respectively; the denoised vibration signal and the denoised audio signal are cut according to set time intervals to obtain a plurality of vibration signal segments and a plurality of audio signal segments; all the vibration signal segments and the audio signal segments are subjected to normalization processing respectively; The normalized vibration signal segment and the normalized audio signal segment are input into the trained coal-rock recognition model at the corresponding moment in time sequence, the trained coal-rock recognition model outputs the coal-rock recognition result, and the input end of the third type of hidden layer of the coal-rock recognition model at the next moment is connected to the output end of the third type of hidden layer of the coal-rock recognition model at the previous moment; The training process of the trained coal-rock identification model is divided into three stages: the first stage is the simple sample training stage, and when the first loss function value of the first stage no longer decreases, it enters the second stage; The second stage is the difficult sample training stage. When the second loss function value of the second stage no longer decreases, the third stage is entered. The third stage is the error-prone sample enhancement training stage. When the third loss function value of the third stage no longer decreases, the training ends and the trained coal-rock recognition model is obtained. The trained coal-rock recognition model includes: an input layer, a first type of hidden layer, a second type of hidden layer, a third type of hidden layer and an output layer connected in series in sequence; the first type of hidden layer is a fully connected layer; the input value of the first type of hidden layer is a normalized vibration signal segment in the x, y and z directions and a normalized audio signal segment; The second type of hidden layer is a grouped fully connected layer, which includes: a feature grouping sublayer and a feature extraction sublayer; the feature grouping sublayer is used to group all features; the feature extraction sublayer is used to extract features from each group of features using a fully connected neural network; The working process of the third type of hidden layer is as follows: ； ； ； ； in, is the input value of the third hidden layer, is the feature to be extracted; , and is the weight to be learned; is the normalization function; for The moment corresponds to the output feature of the third hidden layer of the coal-rock recognition model; for The moment corresponds to the output feature of the third hidden layer of the coal-rock recognition model, as The moment corresponds to the input feature of the third hidden layer of the coal-rock recognition model. represents the intermediate variable, represents the first intermediate variable, represents the second intermediate variable, Represents the third intermediate variable.

2. The coal-rock identification method based on audio and vibration signal fusion according to claim 1 is characterized in that: The vibration signal of the set position of the coal mining machine during the coal mining process and the audio signal generated by the coal mining machine during the coal mining rock cutting process are obtained; wherein, the vibration signal is collected by a vibration sensor installed at the rocker arm of the coal mining machine; the audio signal is collected by a microphone installed in the drum of the coal mining machine.

3. The coal-rock identification method based on audio and vibration signal fusion according to claim 1 is characterized in that: The vibration signal and the audio signal are subjected to denoising processing respectively, specifically comprising: using a filtering algorithm to remove noise or abnormal signals in the original signal; wherein the filtering algorithm comprises: a Kalman filtering algorithm, a wavelet transform algorithm or a median filtering algorithm.

4. The coal-rock identification method based on audio and vibration signal fusion according to claim 1 is characterized in that: The denoised vibration signal and the denoised audio signal are cut at set time intervals to obtain a plurality of vibration signal segments and a plurality of audio signal segments, including: taking the signal mean value in each signal segment as representative data of the current signal segment.

5. The coal-rock identification method based on audio and vibration signal fusion according to claim 1 is characterized in that: The normalizing of all vibration signal segments and audio signal segments respectively includes: using linear normalization or Z-Score normalization to normalize all vibration signal segments and audio signal segments respectively.

6. The coal-rock identification method based on audio and vibration signal fusion according to claim 1 is characterized in that: The trained coal-rock recognition model. The training process includes: Constructing a training set, wherein the training set is a coal-rock vibration signal segment and an audio signal segment with known classification results; The training set is input into the coal-rock recognition model. When the loss function value of the model no longer decreases or the number of iterations reaches the set number, the training is stopped to obtain the trained coal-rock recognition model.

7. The coal-rock identification method based on audio and vibration signal fusion according to claim 6 is characterized in that: The first loss function , whose expression is: ; is the output probability, Indicates the label of the sample; The second loss function , whose expression is: ； is the number of vectors in the feature matrix, is the total number of times the predicted label is the same as the label in the feature matrix; It is a hyperparameter that affects the smoothness of the similarity distribution between samples; is the feature vector output by the last layer of the model, For the corresponding category; is the first feature vectors, For the corresponding category; and is a similarity evaluation indicator. If the label of the feature vector is equal to the sample feature label, then ,otherwise, , if the feature vector label is not equal to the sample feature label, then ,otherwise, ; is the output probability; The third loss function , whose expression is: ； in, is the first hyperparameter, is the second hyperparameter.

8. The coal-rock identification method based on audio and vibration signal fusion according to claim 1 is characterized in that: The normalized vibration signal segment and the normalized audio signal segment are input into the trained coal-rock identification model at the corresponding moment in time sequence, the trained coal-rock identification model outputs the coal-rock identification result, and the input end of the third type of hidden layer of the coal-rock identification model at the next moment is connected to the output end of the third type of hidden layer of the coal-rock identification model at the previous moment; including: The normalized vibration signal segments in the x, y and z directions and the normalized audio signal segments at the time t-1 are input into the trained coal-rock recognition model corresponding to the time t-1, and the prediction result at the time t-1 is output; Input the normalized vibration signal segments in the x, y and z directions and the normalized audio signal segments at time t into the trained coal-rock recognition model corresponding to time t, and output the prediction result at time t; The normalized vibration signal segments in the x, y and z directions and the normalized audio signal segments at the time t+1 are input into the trained coal-rock recognition model corresponding to the time t+1, and the prediction result at the time t+1 is output; The input end of the third type of hidden layer of the trained coal and rock identification model corresponding to time t+1 is also connected to the output end of the third type of hidden layer of the coal and rock identification model at time t; the input end of the third type of hidden layer of the trained coal and rock identification model corresponding to time t is also connected to the output end of the third type of hidden layer of the coal and rock identification model at time t-1.

9. The coal-rock identification system based on the fusion of audio and vibration signals is characterized by: include: An acquisition module is configured to: acquire a vibration signal of a set position of a coal mining machine during the coal mining process and an audio signal generated by the coal mining machine during the coal mining rock cutting process; A preprocessing module is configured to: perform denoising on the vibration signal and the audio signal respectively; cut the denoised vibration signal and the denoised audio signal respectively according to a set time interval to obtain a plurality of vibration signal segments and a plurality of audio signal segments; perform normalization on all vibration signal segments and audio signal segments respectively; The recognition module is configured to: input the normalized vibration signal segment and the normalized audio signal segment into the trained coal-rock recognition model at the corresponding moment in time sequence, the trained coal-rock recognition model outputs the coal-rock recognition result, and the input end of the third type of hidden layer of the coal-rock recognition model at the next moment and the output end of the third type of hidden layer of the coal-rock recognition model at the previous moment are connected to the trained coal-rock recognition model; The trained coal-rock recognition model includes: an input layer, a first type of hidden layer, a second type of hidden layer, a third type of hidden layer and an output layer connected in series in sequence; the first type of hidden layer is a fully connected layer; the input value of the first type of hidden layer is a normalized vibration signal segment in the x, y and z directions and a normalized audio signal segment; The second type of hidden layer is a grouped fully connected layer, which includes: a feature grouping sublayer and a feature extraction sublayer; the feature grouping sublayer is used to group all features; the feature extraction sublayer is used to extract features from each group of features using a fully connected neural network; The working process of the third type of hidden layer is as follows: ； ； ； ； in, is the input value of the third hidden layer, is the feature to be extracted; , and is the weight to be learned; is the normalization function; for The moment corresponds to the output feature of the third hidden layer of the coal-rock recognition model; for The moment corresponds to the output feature of the third hidden layer of the coal-rock recognition model, as The moment corresponds to the input feature of the third hidden layer of the coal-rock recognition model. represents the intermediate variable, represents the first intermediate variable, represents the second intermediate variable, Represents the third intermediate variable.

Images