Coal gangue identification method based on audio and vibration

Abstract

The invention provides a coal gangue identification method based on audio and vibration, and the method comprises the steps: respectively installing a vibration acceleration sensor and a microphone on a tail beam of a hydraulic support and a pedestal of the hydraulic support, so as to obtain a vibration signal and an audio signal generated in a mining process, and extracting audio Mel spectrum features by combining the vibration features of the vibration signals and the audio signals, and identifying the coal gangue by using a coal gangue identification model. The two types of features utilized by the method can cover full-band information from low-frequency acoustics to high-frequency vibration, the comprehensiveness of coal gangue detection or scene recognition can be improved, and the noise robustness is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fully-mechanized caving coal mining, and in particular to a coal gangue identification method based on audio and vibration. Background Art

[0002] Fully mechanized top coal caving (LCC) is one of the key technologies for safe and efficient mining of extremely thick coal seams in my country. This technology has been applied in most working faces in my country's extremely thick coal seams, achieving breakthroughs and developments in both theory and key technologies. However, the caving process in fully mechanized top coal caving still relies on manual control based on the principle of "closing the window when gangue is seen" or "closing the window when gangue is heard." This approach presents the following problems: 1. Personal safety. The mine environment is complex and harsh, and long periods of underground work pose serious risks to workers' physical and mental health. 2. Timing is influenced by subjective factors. The timing of closing the window is influenced by workers' subjective judgment, which can easily lead to misjudgment, resulting in over- or under-caulking. Over-caulking can cause gangue to mix with the raw coal, degrading coal quality; under-caulking can waste resources. 3. Inefficiency. Fully mechanized top coal caving faces have a large number of scaffolds, resulting in a poor working environment during the caving process. Manual control of the caving opening is labor-intensive and inefficient. Separation of coal and gangue is a critical step. Traditional manual identification methods are not only inefficient but also pose safety risks. With the development of science and technology, coal gangue identification methods based on vibration acceleration sensors and audio processing technology have gradually attracted attention. However, existing identification methods often have problems such as low recognition accuracy and significant environmental influences.

[0003] Therefore, there is an urgent need to realize the unmanned, automated and intelligent coal discharge process, and coal gangue identification is a key technology that must be realized. Summary of the Invention

[0004] The present invention describes a coal gangue identification method based on audio and vibration, which can solve the above technical problems.

[0005] Provided is a coal gangue identification method based on audio and vibration. The method comprises: installing a vibration acceleration sensor on the tail beam of a hydraulic support to collect vibration signals generated by falling coal and coal gangue impacting the hydraulic support; installing a microphone on the base of the hydraulic support to obtain audio signals generated during the mining process; extracting vibration features based on the vibration signals, and extracting audio Mel spectrum features based on the audio signals; forming a feature vector from the extracted vibration signal features and audio Mel spectrum features, and inputting the feature vector into a coal gangue identification model to identify coal gangue.

[0006] In some embodiments, a method for extracting audio Mel spectrum features from an audio signal includes: converting each frame of audio signal from the time domain to the frequency domain to obtain an audio spectrogram; calculating a power spectrum based on the audio spectrogram; constructing a set of triangular filters according to the Mel frequency scale, mapping the power spectrum to the Mel frequency scale, and calculating the energy of each Mel filter output; and performing logarithmic compression on the filter group output to simulate the human ear's perception of sound intensity.

[0007] In some embodiments, before Fourier transform is performed on the audio signal, pre-emphasis is performed on the sound signal to increase the energy of the high-frequency portion; and windowing is performed on the pre-emphasized sound signal to make the edge of each frame signal smooth.

[0008] In some embodiments, the vibration characteristics include: one or more of: mean, absolute mean, variance, standard deviation, root amplitude, root mean square value, peak value, maximum value, minimum value, waveform index, peak index, pulse index, margin index, skewness, and kurtosis.

[0009] In some embodiments, a multilayer perceptron is used as a coal gangue recognition model, wherein the multilayer perceptron includes an input layer, two hidden layers, a ReLU activation function, and an output layer; wherein,

[0010] The input layer configures the number of input layer nodes by forming a feature space based on vibration signal features and audio Mel spectrum features; the first hidden layer performs nonlinear transformation on the input features, the first hidden layer contains 64 nodes, and the activation function is ReLU; the second hidden layer performs feature extraction on the output of the first hidden layer, the second hidden layer contains 32 nodes, and the activation function is ReLU; the output layer is used to output the probability of coal gangue category, and the number of nodes in the output layer is configured according to the coal gangue category.

[0011] The audio- and vibration-based coal gangue identification method provided by this invention utilizes a vibration accelerometer and microphone installed on the hydraulic support tail beam and the hydraulic support base, respectively, to capture vibration and audio signals generated during mining. By combining the vibration characteristics of the vibration signal with the audio signal to extract the audio mel-spectrum features, the method utilizes a coal gangue identification model to identify the gangue. The two types of features utilized by the method cover the full frequency range, from low-frequency acoustics to high-frequency vibrations, improving the comprehensiveness of coal gangue detection or scene recognition and enhancing noise robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A schematic flow chart of a coal gangue identification method based on audio and vibration provided in an embodiment of this specification is shown;

[0014] Figure 2 A schematic diagram of installing a vibration acceleration sensor provided in an embodiment of this specification is shown;

[0015] Figure 3 A schematic diagram of microphone installation provided in an embodiment of this specification is shown;

[0016] Figure 4 A schematic diagram of a Mel filter bank provided in an embodiment of this specification is shown;

[0017] Figure 5 shows a time-frequency diagram after being filtered by a Mel filter provided in an embodiment of this specification;

[0018] Figure 6 A coal gangue identification device based on audio and vibration provided in an embodiment of this specification is shown. DETAILED DESCRIPTION

[0019] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0021] In the description of the embodiments of the present application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of the present application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0022] In the description of the embodiments of this application, the term "and / or" is merely a description of an association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent the following three situations: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more.

[0023] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly identifying the technical features being referred to. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0024] Before introducing this solution, the technical terms used in this solution are first introduced.

[0025] Figure 1 A schematic flow chart of a coal gangue identification method based on audio and vibration provided in an embodiment of this specification is shown as follows: Figure 1 As shown, the method includes the following steps:

[0026] Step S101: Install a vibration acceleration sensor on the tail beam of the hydraulic support to collect vibration signals generated by coal and coal gangue falling and hitting the hydraulic support, and install a microphone on the base of the hydraulic support to obtain audio signals generated during the mining process.

[0027] Specifically, refer to Figure 2 A vibration accelerometer can be installed on the tail beam of a hydraulic support. It monitors the vibration characteristics of the impact between ore and gangue and the tail beam of the hydraulic support during fully-mechanized coal mining. Due to the differences in physical properties such as density, hardness, and particle size between gangue and coal seams, the vibration signals differ significantly. Furthermore, to prevent damage from gangue, both sensors can be installed behind the tail beam.

[0028] The vibration acceleration sensor installed on the tail beam of the hydraulic support can efficiently capture high-frequency impact signals and reduce signal attenuation and noise interference.

[0029] refer to Figure 3 The microphone can be mounted on the base of the hydraulic support. It is easy to understand that the microphone can be a single microphone or a microphone array composed of multiple microphones. The microphone can be used to collect audio signals generated by gangue or coal impacting the hydraulic support during mining.

[0030] The base of the hydraulic support is close to the coal mining operation surface (such as scraper conveyor, coal mining machine), which can directly capture the broadband audio of the mining process (mechanical friction, coal block crushing sound), avoiding the attenuation of far-field sound waves due to air propagation.

[0031] Step S102: extracting vibration features based on the vibration signal, and extracting audio Mel spectrum features based on the audio signal.

[0032] Specifically, the vibration signal data can be analyzed to extract the vibration features of the vibration signal, and the Mel spectrum features in the sound signal can be extracted using a bionic acoustic representation method.

[0033] In one embodiment, the vibration characteristics may include one or more of the mean, absolute mean, variance, standard deviation, root square amplitude, root mean square value, peak value, maximum value, minimum value, waveform index, peak index, pulse index, margin index, skewness, and kurtosis of the vibration signal.

[0034] Specifically, the above features can be divided into time domain features and frequency domain features. In the time domain, the mean / absolute average value can reflect the overall energy level of the impact signal. Due to the higher density of coal gangue, the mean value is usually larger during impact. The variance / standard deviation can characterize the intensity of signal fluctuations. The brittle impact of coal gangue may cause more intense transient fluctuations. The kurtosis is sensitive to the impact pulse. The hard and brittle nature of coal gangue easily produces transient peak values, and the kurtosis value is significantly higher than that of coal. The skewness can distinguish signal asymmetry. After the impact of coal gangue, the vibration signal decays faster, which may lead to negative skew.

[0035] In the frequency domain characteristics, the root square amplitude / root mean square value can be used as a comprehensive energy index. If the coal gangue has more high-frequency components, the root mean square value will be in a specific frequency band; the peak index, the instantaneous impact peak of the coal gangue is more prominent, and the peak index is larger; the waveform index / margin index can reflect the complexity of the signal waveform. Because coal gangue has rich high-frequency resonance modes, the index value is lower.

[0036] In one embodiment, a method for extracting audio mel-spectrogram features from an audio signal is provided. The method comprises the following steps:

[0037] Step S1021 : Convert each frame of audio signal from the time domain to the frequency domain to obtain an audio spectrum.

[0038] Specifically, the audio signal can be converted from the time domain to the frequency domain through the Fourier transform method and represented as a two-dimensional array of frequency and time.

[0039]

[0040] Among them, s(n) represents the time domain signal; s i (n) is the data of the i-th frame, where n can range from 1 to 400; when Fourier transform is performed, S i (k) represents the kth complex coefficient of the i-th frame.

[0041] Step S1022: Calculate the power spectrum based on the audio spectrogram.

[0042] Specifically, the power spectrum represents the power distribution of an audio signal at different frequencies. For a time-domain signal, its power spectrum is a representation of the power of the signal at each frequency component. The power spectrum is calculated from the STFT results by square the amplitude of each frequency component to obtain the energy of the frequency component. This is shown below:

[0043]

[0044] Among them, P i (k) is the power spectrum of the i-th frame, S i (k) represents the kth complex coefficient of the i-th frame.

[0045] Step S1023 : construct a set of triangular filters according to the Mel frequency scale, map the power spectrum to the Mel frequency scale, and calculate the energy output by each Mel filter.

[0046] Specifically, the relationship between the Mel frequency and the normal frequency f is: M(f) = 1125ln(1+f / 700). A set of triangular filters is constructed based on the Mel frequency scale to perform weighted averaging on the power spectrum and map the power spectrum to the Mel frequency scale.

[0047] For example, Figure 4 As shown, about 20-40 (usually 26) triangular filter banks can be selected to filter the power spectrum estimate obtained in the previous step. The triangular filter bank consists of 26 (filter) vectors with a length of 257. Most of the 257 values ​​of each filter are 0, and only the frequency range to be collected is non-zero. The input 257-point signal will pass through 26 filters, so the energy of the signal passing through each filter can be calculated. The calculation result is as follows Figure 5 As shown, Figure 5 The figure shows the time-frequency diagram after the Mel filter is filtered. The horizontal axis represents time, the vertical axis represents frequency, and the brightness of the color represents the intensity information of the power spectrum. The brighter the color, the greater the intensity of the power spectrum. The triangular filter bank formula is as follows:

[0048]

[0049] Where,

[0050] Step S1024: Logarithmic compression is performed on the filter bank output to simulate the human ear's perception of sound intensity.

[0051] Specifically, the logarithm of each value output by the filter bank is taken to simulate the human ear's perception of sound intensity.

[0052] Exemplarily, the logarithm of each value output by the filter bank can be taken by the following formula:

[0053] LogMel=log(1+S)

[0054] Where LogMel represents the logarithm of the Mel spectrum, and S represents the energy value of the Mel spectrum.

[0055] In one embodiment, before Fourier transform is performed on the audio signal, pre-emphasis and windowing processing may be performed on the audio signal.

[0056] Specifically, pre-emphasis is applied to the sound signal to boost the energy of high frequencies. For example, this enhances the high-frequency, sharp sound of gangue crushing, making it more distinct from the low-frequency, dull sound of coal. After pre-emphasis, the sound signal can be windowed. This means that the long audio signal is divided into shorter time segments (frames), and each frame is multiplied by a window function. Windowing reduces spectral leakage and improves frequency resolution, smoothing the edges of each frame and ensuring that the audio features more accurately reflect the material properties of gangue and coal.

[0057] Vibration signals can directly reflect the physical state of mechanical structures (such as wear and cracks) and are sensitive to high-frequency shocks and transient events, making them suitable for analyzing high-frequency vibration energy distribution (such as wavelet packet decomposition). However, vibration signals are easily affected by environmental noise and are insensitive to low-frequency vibration or airborne acoustic characteristics. The audio Mel spectrum can simulate the human ear's auditory characteristics and has a strong ability to capture mid- and low-frequency acoustic features (such as friction and aerodynamic noise). It is suitable for processing continuous and stable signals, but the audio Mel spectrum may ignore high-frequency mechanical vibration details (such as ultrasound waves >20kHz). The combination of the two can cover the entire frequency band, from low-frequency acoustics to high-frequency vibrations, improving the comprehensiveness of coal gangue detection or scene recognition and enhancing noise robustness.

[0058] Step S103 : The extracted vibration signal features and audio mel spectrum features are used to form a feature vector, and the feature vector is input into a coal gangue recognition model to identify coal gangue.

[0059] Specifically, the extracted vibration signal features and audio mel-spectrogram features are collected and fused to construct a unified feature space. Gangue recognition models such as SVM (Support Vector Machine), XGBoost (eXtreme Gradient Boosting), CNN (Convolutional Neural Network), and MLP (Multilayer Perceptron) are selected.

[0060] For example, vibration signal features and audio mel-spectrogram features are extracted from the vibration signal data and audio signal data, respectively. For example, eight vibration features are extracted from the vibration signal: mean, variance, peak, standard deviation, maximum, minimum, kurtosis, and margin index. Thirteen audio mel-spectrogram coefficients are extracted from the audio signal. For each frame of the signal, the 13 audio mel-spectrogram coefficients and the eight vibration eigenvalues ​​are sequentially arranged and combined into a feature vector of length 13 + 8 = 23. The feature vectors composed of all frames constitute a feature space.

[0061] In one embodiment, a multi-layer perceptron can be selected as a coal gangue recognition model.

[0062] Specifically, the multilayer perceptron can include an input layer, two hidden layers, a ReLU activation function, and an output layer. The input layer constructs a feature space based on vibration signal features and audio mel-spectrogram features to configure the number of input layer nodes. The first hidden layer performs a nonlinear transformation on the input features. The first hidden layer can contain 64 nodes, and the activation function is ReLU. The second hidden layer extracts features from the output of the first hidden layer. The second hidden layer can contain 32 nodes, and the activation function is ReLU. The output layer outputs the probability of coal gangue classification, and the number of nodes in the output layer is configured based on the coal gangue classification.

[0063] For example, an acceleration sensor and an acoustic sensor are used to collect vibration and audio signal data during top coal caving. For example, signal data for five types of coal gangue are collected: 100% coal, 75% coal and 25% gangue, 50% coal and 50% gangue, 25% coal and 75% gangue, and 100% gangue. Each type of signal data has 1000 frames, and each frame is 25ms long.

[0064] Based on the above signal data, feature extraction is performed and the extracted features are combined to form feature vectors. All feature vectors and corresponding gangue classifications (a total of 5 categories) are divided into a training set (for example, accounting for 70% of the total data), a validation set (accounting for 15%), and a test set (accounting for 15%).

[0065] A multi-layer perceptron (MLP) was constructed as a gangue recognition model. The input layer had 23 nodes, corresponding to the dimension of the feature vector. Two hidden layers were set up, with the first hidden layer having 64 nodes and the second hidden layer having 32 nodes. Both used the ReLU activation function to introduce nonlinearity. The output layer had five nodes, corresponding to the number of gangue categories. Assuming there were five categories, the output layer would have five nodes and used the softmax activation function to output the probability of each gangue category.

[0066] The feature vectors of the training set are input into the neural network. Forward propagation is used to calculate the probability of each sample belonging to a different gangue category. This probability is then compared with the true gangue label and the cross-entropy loss function is calculated. Backpropagation is then used to calculate the gradient of the loss function with respect to the network parameters (weights and biases). Optimization algorithms such as stochastic gradient descent are used to update the parameters. This process is iterated repeatedly to gradually reduce the loss function. During training, the model's performance, such as accuracy, is regularly evaluated using the validation set. Hyperparameters (such as the learning rate) are adjusted based on the validation results.

[0067] After training is complete, the final model is evaluated using a test set to determine whether the model's performance meets requirements, including its accuracy in identifying gangue on previously unseen sound and vibration data. If performance is poor, further analysis can be conducted to determine factors such as insufficient feature extraction or an unreasonable model structure. Improvements and optimizations can then be made until the gangue recognition model is fully trained.

[0068] After the gangue recognition model is trained, the vibration signal data and audio signal data generated during the mining process are collected, and the signal data are subjected to feature extraction and feature vector construction. The feature vector is input into the gangue recognition model, and the ore is classified by the gangue recognition model to identify the gangue.

[0069] Corresponding to the above method provided by the present invention, the present invention also provides a device. Figure 6 FIG. 1 shows a schematic diagram of the structure of a coal gangue identification device based on audio and vibration provided in an embodiment of this specification. Figure 6 As shown, the device includes:

[0070] The data acquisition module 201 is used to collect vibration signals generated by coal and coal gangue falling and hitting the hydraulic support, and to obtain audio signals generated during the mining process.

[0071] The feature extraction module 202 is configured to extract vibration features based on the vibration signal and extract audio Mel-spectrogram features based on the audio signal.

[0072] The gangue identification module 203 is used to perform feature vector fusion on the extracted vibration signal features and audio mel spectrum features, and input the feature vector into a gangue identification model to identify gangue.

[0073] It should be readily apparent to those skilled in the art that, in combination with the units and algorithmic steps of the various examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0074] According to another embodiment, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the system realizes the combination of Figure 1 The method described.

[0075] According to another embodiment, there is also provided a computer readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 1 The method described.

[0076] Those skilled in the art will appreciate that, in one or more of the above examples, the functions described herein may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0077] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A coal gangue identification method based on audio and vibration, characterized in that: The method comprises: A vibration accelerometer is installed on the tail beam of the hydraulic support to collect vibration signals generated by the impact of falling coal and coal gangue on the hydraulic support; a microphone is installed on the base of the hydraulic support to obtain audio signals generated during mining; Extract vibration features based on the vibration signal, and extract audio Mel spectrum features based on the audio signal; The extracted vibration signal features and audio Mel spectrum features are used to form a feature vector, and the feature vector is input into a coal gangue recognition model to identify coal gangue.

2. The method according to claim 1, characterized in that The method for extracting audio mel spectrum features from an audio signal includes: Convert each frame of audio signal from the time domain to the frequency domain to obtain an audio spectrum; Calculate the power spectrum based on the audio spectrogram; Construct a set of triangular filters according to the Mel frequency scale, map the power spectrum to the Mel frequency scale, and calculate the energy of each Mel filter output; Logarithmically compress the filter bank output to simulate the human ear's perception of sound intensity.

3. The method according to claim 2, characterized in that Before Fourier transforming the audio signal, Pre-emphasize the sound signal to increase the energy of the high-frequency part; Windowing is performed on the pre-emphasized sound signal to make the edges of each frame signal smooth.

4. The method according to claim 1, wherein The vibration characteristics include: one or more of: mean, absolute mean, variance, standard deviation, root amplitude, root mean square value, peak value, maximum value, minimum value, waveform index, peak index, pulse index, margin index, skewness, and kurtosis.

5. The method according to claim 1, wherein A multilayer perceptron is used as a coal gangue recognition model, wherein the multilayer perceptron includes an input layer, two hidden layers, a ReLU activation function, and an output layer; wherein, The input layer configures the number of input layer nodes by forming a feature space based on vibration signal features and audio Mel spectrum features; the first hidden layer performs nonlinear transformation on the input features, the first hidden layer contains 64 nodes, and the activation function is ReLU; the second hidden layer performs feature extraction on the output of the first hidden layer, the second hidden layer contains 32 nodes, and the activation function is ReLU; the output layer is used to output the probability of coal gangue category, and the number of nodes in the output layer is configured according to the coal gangue category.

Images