Method for detecting abnormal sound of track inspection robot based on convolutional auto-encoder

Abstract

The invention provides a track inspection robot abnormal sound detection method based on a convolution auto-encoder, and the method comprises the steps: firstly collecting sound data of the normal work of coal mine transportation equipment in an inspection region through sound pickup equipment carried on a track inspection robot, and carrying out the environment noise reduction of the sound data through a noise reduction algorithm; secondly, extracting Mel-frequency cepstral coefficient features of the sound of the coal mine transportation equipment through a feature extraction algorithm, inputting normal audio features into a convolutional auto-encoder network for training to obtain reconstructed audio features, and calculating a mean square loss function of the input normal audio features and the reconstructed normal audio features to obtain a reconstruction threshold value; and finally, calculating a mean square loss function of other audio features input into the auto-encoder network, and detecting the abnormal sound of the coal mine transportation equipment in a mode of judging whether the mean square loss function exceeds a reconstruction threshold value or not. According to the detection method, the problem that enough abnormal samples are difficult to accumulate in a short time due to the fact that the abnormal sound of the coal mine transportation equipment has occasionality and diversity can be solved, and the abnormal sound of the coal mine transportation equipment can be detected by the track inspection robot under the condition that the abnormal samples are lacked; and the production safety problem caused by the abnormity of the coal mine transportation equipment in the coal mine production can be early warned in time.

Description

Technical Field

[0001] The invention relates to the technical field of coal mine monitoring, and in particular to an abnormal sound detection method of a track inspection robot based on a convolutional autoencoder. Background Art

[0002] In the coal mine production system, coal mine transportation equipment includes belt conveyor heads, belt conveyor rollers, pump room motors, etc. These devices will make sounds when running, and the sounds will change with the changes in the operating status. Manual inspections can judge the status of the equipment based on these sounds, but this is very dependent on the personal experience of the inspectors, and manual inspections alone cannot guarantee real-time monitoring of the equipment and cannot detect faults in time. Once an abnormal failure occurs in the equipment, especially large equipment, the losses may be very huge. Therefore, abnormal sound detection of coal mine transportation equipment is particularly important for coal mine production safety. The detection of abnormal sounds of coal mine transportation equipment can be greatly reduced by patrol robots. However, due to the sporadic and diverse abnormal sounds of coal mine transportation equipment, it is difficult to accumulate enough abnormal samples in a short period of time, and the robot cannot effectively identify abnormal sounds. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention proposes a method for detecting abnormal sounds of a track inspection robot based on a convolutional autoencoder, which overcomes the problem that it is difficult to accumulate enough abnormal samples in a short period of time due to the sporadic and diverse abnormal sounds of coal mine transportation equipment. The method enables the track inspection robot to complete the detection of abnormal sounds of coal mine transportation equipment in the absence of abnormal samples.

[0004] The present invention discloses a method for detecting abnormal sound of a track inspection robot based on a convolutional autoencoder, comprising the following steps:

[0005] S100: Collect audio data of normal operation of coal mine transportation equipment and perform noise reduction processing;

[0006] S200: extracting MFCC features of the denoised audio data;

[0007] S300: Constructing a convolutional autoencoder network for detecting abnormal sounds;

[0008] S400: training a convolutional autoencoder network of normal sound, obtaining a minimum reconstruction error through continuous optimization, and using the reconstruction error as a reconstruction threshold of normal sound;

[0009] S500: New sound data is input into the convolutional autoencoder network, and the reconstruction error obtained by the network is compared with the reconstruction threshold of normal sound to complete the detection of abnormal sound.

[0010] Preferably, in step S100, audio data of normal operation of coal mine transportation equipment in the inspection area is collected by a sound pickup device carried by the track inspection robot.

[0011] Further preferably, in step S100, the following steps are also included:

[0012] S110: converting the audio data containing noise from a time domain signal to a frequency domain signal by means of framing, windowing, and short-time Fourier transform, performing an initial noise estimation on the audio data of the frequency domain signal, and calculating a priori signal-to-noise ratio and a posteriori signal-to-noise ratio based on the estimated noise;

[0013] S120: Calculate likelihood ratio test, spectrum flatness and spectrum difference classification features, and determine normal audio to noise ratio probability based on these features, so as to determine whether the current audio is normal audio or noise;

[0014] S130: updating the noise estimation according to the calculated normal audio noise ratio probability, and then suppressing and denoising by Wiener filtering to obtain denoised frequency domain audio;

[0015] S140: converting the noise-suppressed audio from the frequency domain back to the time domain by means of short-time inverse Fourier transform, windowing, and overlap-addition, thereby obtaining the noise-reduced audio;

[0016] Among them, the audio y(t) containing noise is expressed as:

[0017] y(t)=x(t)+n(t)

[0018] Where: x(t) is the audio signal of coal mine transportation equipment operation, n(t) is the noise signal;

[0019] The relationship between y(t), x(t) and n(t) in the frequency domain can be expressed as:

[0020] Y(ω)=X(ω)+N(ω)

[0021] Where: Y(ω), X(ω) and N(ω) are the representations of y(t), x(t) and n(t) on the frequency spectrum respectively.

[0022] The calculation method for suppressing noise through Wiener filtering is:

[0023] X'(ω)=Y(ω)-N'(ω)

[0024] Where: X'(ω) is the pure audio signal, N'(ω) is the estimated noise signal.

[0025] More preferably, in step S200, for the audio data after noise reduction in step S100, the processing step includes the following steps:

[0026] S210: Use pre-emphasis to enhance the high frequency components of the audio, then perform frame division and windowing on the pre-emphasized audio, set the frame length to 25ms, and the frame shift to 10ms. To prevent signal distortion, the window function selects the Hamming window;

[0027] The method of improving high-frequency components by pre-emphasis can be expressed as:

[0028] a(p)=b(p)-q*b(p-1)

[0029] Where a(p) is the output of pre-emphasis; b(p) is the sampling value of the audio signal at the pth moment; q is the pre-emphasis coefficient, which is 0.97 in this patent;

[0030] S220: transform the audio from a time domain signal to a frequency domain signal by discrete Fourier transform, map the frequency domain signal of the audio to a Mel frequency, and filter it through M Mel filters, then calculate the logarithmic energy of the frequency band output by each filter, and obtain a feature of a certain dimension according to the energy in each Mel frequency band, wherein the dimension of the feature depends on the number of filters in the Mel filter bank;

[0031] S230: Perform cepstrum analysis on the logarithmic energy of the Mel filter through inverse Fourier transform to obtain Mel cepstrum coefficients, then calculate dynamic features through difference, merge them with the Mel cepstrum coefficients, and finally extract MFCC features of the audio.

[0032] More preferably, in step S300, the network of the convolutional autoencoder includes a convolutional layer and a deconvolutional layer, the convolutional layer acts as an encoder to reduce the dimension of the input audio MFCC features, and the deconvolutional layer acts as a decoder to reconstruct the MFCC features of the audio;

[0033] Assume that the extracted audio MFCC feature set is {x1,x2,...,x n} and each x i is a d-dimensional vector, that is, x i ∈R d ; In the encoder, for a feature vector x i ∈R d , is compressed into n dimensions after being input into the encoder; the method by which the encoder compresses audio features is:

[0034]

[0035] Among them, f is the sigmoid nonlinear activation function, x is the input of the encoder; θ is the parameter {W e ,b e},W eis the weight matrix of the encoder network, size is m×d, b e is the bias parameter of the encoder network, with a dimension of m, which is input into the encoder's high-dimensional vector x in this way i is compressed into a low-dimensional vector h i ;

[0036] In the decoder, x i The low-dimensional hidden representation h i Mapped to the original input space R d Above, the decoder maps audio features as follows:

[0037]

[0038] Among them, g is the sigmoid nonlinear activation function, h is the input of the decoder, and θ′ is the parameter of the decoder network {W d ,b d}.

[0039] More preferably, in step S400, the MFCC features {x1, x2, ..., x n} is input into the convolutional autoencoder. The autoencoder network maps the training set data to a low-dimensional subspace by encoding, and then regenerates the data by decoding to obtain {x1′, x2′, ..., x n ′}, in order to obtain the minimum reconstruction error MSELoss, it is necessary to optimize the parameters θ and θ′ in the autoencoder network. Finally, the MSELoss obtained by optimization is defined as the reconstruction threshold of this type of audio data.

[0040] More preferably, in step S500, when the training of the convolutional self-editing network is completed, the audio data when the training device is normally operating is re-input, and the reconstruction error at this time is calculated, so as to obtain the reconstruction threshold when the device is normally operating;

[0041] When the input of the convolutional self-editing network is the sound to be detected, its reconstruction error is obtained. When its reconstruction error is greater than the reconstruction threshold, it is an abnormal sound. When its reconstruction error is less than the reconstruction threshold, it is a normal sound, thereby completing the abnormal sound detection of coal mine transportation equipment.

[0042] The beneficial effects of the present invention are as follows: compared with the prior art, the present invention has the following advantages:

[0043] 1) The present invention is easy to use. It only needs to collect sound from the coal mine transportation equipment in the inspection area through the sound pickup device carried by the track inspection robot itself, so as to detect whether there is any abnormal sound in the coal mine transportation equipment.

[0044] 2) The audio noise reduction method used in the present invention can reduce the impact of background noise, which is helpful for identifying whether abnormal sounds occur when coal mine equipment is running.

[0045] 3) The abnormal sound detection method used in the present invention can realize the recognition of abnormal sound in the absence of abnormal sound samples of coal mine transportation equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a structural schematic diagram of the composition of the present invention;

[0047] Figure 2 It is a schematic diagram of the detection process of the present invention.

[0048] In the picture: 1. Track inspection robot; 2. Sound pickup equipment. DETAILED DESCRIPTION

[0049] The technical scheme in the embodiment of the present invention is described clearly and completely below in conjunction with the accompanying drawings in the embodiment of the present invention. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0050] like Figure 1 and Figure 2 As shown, this embodiment discloses a method for detecting abnormal sound of a rail inspection robot based on a convolutional autoencoder, comprising the following steps:

[0051] S100: performing noise reduction processing on the sound data collected when the coal mine transportation equipment works normally.

[0052] In this step, the sound pickup device 2 is first mounted on the track inspection robot 1, and the audio data of the normal operation of the coal mine transportation equipment in the inspection area is collected by the sound pickup device 2.

[0053] Specifically, the following steps are included:

[0054] S110: converting the noisy audio from the time domain to the frequency domain signal by means of framing, windowing and short-time Fourier transform, performing initial noise estimation on the frequency domain audio data, and calculating a priori signal-to-noise ratio and a posteriori signal-to-noise ratio based on the estimated noise;

[0055] S120: Calculate classification features such as likelihood ratio test, spectrum flatness, and spectrum difference, and determine the probability of normal audio to noise ratio based on these features, so as to determine whether the current audio is normal audio or noise;

[0056] S130: updating the noise estimation according to the calculated normal audio noise ratio probability, and then suppressing and denoising by Wiener filtering to obtain denoised frequency domain audio;

[0057] S:40: Finally, the noise suppressed audio is converted from the frequency domain back to the time domain by inverse short-time Fourier transform, windowing, and overlap-addition to obtain the denoised audio.

[0058] The audio y(t) containing noise can be expressed as,

[0059] y(t)=x(t)+n(t)

[0060] Where: x(t) is the audio signal of coal mine transportation equipment operation, and n(t) is the noise signal.

[0061] The relationship between y(t), x(t) and n(t) in the frequency domain can be expressed as:

[0062] Y(ω)=X(ω)+N(ω)

[0063] Where: Y(ω), X(ω) and N(ω) are the representations of y(t), x(t) and n(t) on the frequency spectrum respectively.

[0064] The calculation method for suppressing noise through Wiener filtering is:

[0065] X'(ω)=Y(ω)-N'(ω)

[0066] Where: X'(ω) is the pure audio signal, N'(ω) is the estimated noise signal.

[0067] S200: Extracting MFCC features of the denoised audio data.

[0068] In this step, the following steps are specifically included:

[0069] S210: Use pre-emphasis to enhance the high frequency components of the audio, then perform frame division and windowing on the pre-emphasized audio, set the frame length to 25ms, and the frame shift to 10ms. To prevent signal distortion, the window function selects the Hamming window;

[0070] S220: transform the audio from a time domain signal to a frequency domain signal by discrete Fourier transform, map the frequency domain signal of the audio to Mel frequency, filter it through M Mel filters, and then calculate the logarithmic energy of the frequency band output by each filter;

[0071] S230: According to the energy in each Mel frequency band, a feature of a certain dimension is obtained. The dimension of the feature depends on the number of filters in the Mel filter bank. Through inverse Fourier transform, the Mel filter logarithmic energy is subjected to cepstrum analysis to obtain Mel cepstrum coefficients, and then the dynamic features are calculated by difference, combined with the Mel cepstrum coefficients, and finally the MFCC features of the audio are extracted.

[0072] Among them, the method of enhancing high-frequency components by pre-emphasis can be expressed as:

[0073] a(p)=b(p)-q*b(p-1)

[0074] Wherein a(p) is the output of pre-emphasis; b(p) is the sampling value of the audio signal at the pth moment; q is the pre-emphasis coefficient, and in this embodiment, q is 0.97.

[0075] S300: Build a convolutional autoencoder network for detecting abnormal sounds.

[0076] In this step, the network of the convolutional autoencoder includes a convolutional layer and a deconvolutional layer, where the convolutional layer acts as an encoder to reduce the dimension of the input audio MFCC features, and the deconvolutional layer acts as a decoder to reconstruct the audio MFCC features. Assume that the extracted audio MFCC feature set is {x1, x2, ..., x n} and each x i is a d-dimensional vector, that is, x i ∈R d In the encoder, for a feature vector x i ∈R d , is compressed into n dimensions after being input into the encoder. The encoder compresses audio features as follows:

[0077]

[0078] Where f is the sigmoid nonlinear activation function, x is the input of the encoder; θ is the parameter {W e ,b e},W e is the weight matrix of the encoder network, size is m×d,b e is the bias parameter of the encoder network, with a dimension of m. In this way, the high-dimensional vector x input to the encoder i is compressed into a low-dimensional vector h i .

[0079] In the decoder, x needs to be i The low-dimensional hidden representation h i Mapped to the original input space R d The decoder maps audio features as follows:

[0080]

[0081] Where g is the sigmoid nonlinear activation function, h is the input of the decoder, and θ′ is the parameter of the decoder network {W d ,b d}.

[0082] S400: Train the convolutional autoencoder network of normal sound, obtain the minimum reconstruction error through continuous optimization, and use the reconstruction error as the reconstruction threshold of normal sound.

[0083] Specifically, the MFCC features of the audio {x1,x2,...,x n} is input into the convolutional autoencoder. The autoencoder network maps the training set data to a low-dimensional subspace by encoding, and then regenerates the data by decoding to obtain {x1′, x2′, ..., x n In order to obtain the minimum reconstruction error MSELoss, it is necessary to optimize the parameters θ and θ′ in the autoencoder network. Finally, the MSELoss obtained by optimization is defined as the reconstruction threshold of this type of audio data.

[0084] S500: New sound data is input into the convolutional autoencoder network, and the reconstruction error obtained by the network is compared with the reconstruction threshold of normal sound to complete the detection of abnormal sound.

[0085] Specifically, when the training of the convolutional self-editing network is completed, the audio data of the normal operation of the training equipment is re-input, and the reconstruction error at this time is calculated to obtain the reconstruction threshold when the equipment is operating normally. When the input of the convolutional self-editing network is the sound to be detected, its reconstruction error is obtained. When its reconstruction error is greater than the reconstruction threshold, it is an abnormal sound, and when its reconstruction error is less than the reconstruction threshold, it is a normal sound, thereby completing the abnormal sound detection of coal mine transportation equipment.

[0086] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Claims

1. A method for detecting abnormal sound of a track inspection robot based on a convolutional autoencoder, characterized in that: The following steps are involved: S100: Collect audio data of normal operation of coal mine transportation equipment and perform noise reduction processing; S200: extracting MFCC features of the denoised audio data; S300: Construct a convolutional autoencoder network for detecting abnormal sounds; S400: training a convolutional autoencoder network of normal sound, obtaining a minimum reconstruction error through continuous optimization, and using the reconstruction error as a reconstruction threshold of normal sound; S500: New sound data is input into the convolutional autoencoder network, and the reconstruction error obtained by the network is compared with the reconstruction threshold of normal sound to complete the detection of abnormal sound.

2. According to claim 1, a method for detecting abnormal sound of a rail inspection robot based on a convolutional autoencoder is characterized in that: In step S100, audio data of the normal operation of coal mine transportation equipment in the inspection area is collected by a sound pickup device carried by the track inspection robot.

3. The abnormal sound detection method of a rail inspection robot based on a convolutional autoencoder according to claim 2 is characterized in that: In step S100, the following steps are also included: S110: converting the audio data containing noise from a time domain signal to a frequency domain signal by means of framing, windowing, and short-time Fourier transform, performing an initial noise estimation on the audio data of the frequency domain signal, and calculating a priori signal-to-noise ratio and a posteriori signal-to-noise ratio based on the estimated noise; S120: Calculate likelihood ratio test, spectrum flatness and spectrum difference classification features, and determine normal audio to noise ratio probability based on these features, so as to determine whether the current audio is normal audio or noise; S130: updating the noise estimation according to the calculated normal audio noise ratio probability, and then suppressing and denoising by Wiener filtering to obtain denoised frequency domain audio; S140: converting the noise-suppressed audio from the frequency domain back to the time domain by means of short-time inverse Fourier transform, windowing, and overlap-addition, thereby obtaining the noise-reduced audio; Among them, the audio y(t) containing noise is expressed as: y(t)=x(t)+n(t) Where: x(t) is the audio signal of coal mine transportation equipment operation, n(t) is the noise signal; The relationship between y(t), x(t) and n(t) in the frequency domain can be expressed as: Y(ω)=X(ω)+N(ω) Where: Y(ω), X(ω) and N(ω) are the representations of y(t), x(t) and n(t) on the frequency spectrum respectively. The calculation method for suppressing noise through Wiener filtering is: X'(ω)=Y(ω)-N'(ω) Where: X'(ω) is the pure audio signal, N'(ω) is the estimated noise signal.

4. The abnormal sound detection method of a rail inspection robot based on a convolutional autoencoder according to claim 3 is characterized in that: In step S200, for the audio data after noise reduction in step S100, the processing step includes the following steps: S210: Use pre-emphasis to enhance the high frequency components of the audio, then perform frame division and windowing on the pre-emphasized audio, set the frame length to 25ms, and the frame shift to 10ms. To prevent signal distortion, the window function selects the Hamming window; The method of enhancing high-frequency components by pre-emphasis can be expressed as: a(p)=b(p)-q*b(p-1) Where a(p) is the output of pre-emphasis; b(p) is the sampling value of the audio signal at the pth moment; q is the pre-emphasis coefficient, which is 0.97 in this patent; S220: transform the audio from a time domain signal to a frequency domain signal by discrete Fourier transform, map the frequency domain signal of the audio to a Mel frequency, and filter it through M Mel filters, then calculate the logarithmic energy of the frequency band output by each filter, and obtain a feature of a certain dimension according to the energy in each Mel frequency band, wherein the dimension of the feature depends on the number of filters in the Mel filter bank; S230: Perform cepstrum analysis on the logarithmic energy of the Mel filter through inverse Fourier transform to obtain Mel cepstrum coefficients, then calculate dynamic features through difference, merge them with the Mel cepstrum coefficients, and finally extract MFCC features of the audio.

5. The abnormal sound detection method of a rail inspection robot based on a convolutional autoencoder according to claim 4 is characterized in that: In step S300, the network of the convolutional autoencoder includes a convolutional layer and a deconvolutional layer, the convolutional layer acts as an encoder to reduce the dimension of the input audio MFCC features, and the deconvolutional layer acts as a decoder to reconstruct the MFCC features of the audio; Assume that the extracted audio MFCC feature set is {x1,x2,...,x n } and each x i is a d-dimensional vector, that is, x i ∈R d ; In the encoder, for a feature vector x i ∈R d , is compressed into n dimensions after being input into the encoder; the method by which the encoder compresses audio features is: Among them, f is the sigmoid nonlinear activation function, x is the input of the encoder; θ is the parameter {W e ,b e },W e is the weight matrix of the encoder network, size is m×d, b e is the bias parameter of the encoder network, with a dimension of m, which is input into the encoder's high-dimensional vector x in this way i is compressed into a low-dimensional vector h i ; In the decoder, x i The low-dimensional hidden representation h i Mapped to the original input space R d Above, the decoder maps audio features as follows: Among them, g is the sigmoid nonlinear activation function, h is the input of the decoder, and θ′ is the parameter of the decoder network {W d ,b d }.

6. The abnormal sound detection method of a rail inspection robot based on a convolutional autoencoder according to claim 5 is characterized in that: In step S400, the MFCC features {x1, x2, ..., x n } is input into the convolutional autoencoder. The autoencoder network maps the training set data to a low-dimensional subspace by encoding, and then regenerates the data by decoding to obtain {x1′, x2′, ..., x n ′}, in order to obtain the minimum reconstruction error MSELoss, it is necessary to optimize the parameters θ and θ′ in the autoencoder network. Finally, the MSELoss obtained by optimization is defined as the reconstruction threshold of this type of audio data.

7. The abnormal sound detection method of a rail inspection robot based on a convolutional autoencoder according to claim 6 is characterized in that: In step S500, when the training of the convolutional self-editing network is completed, the audio data when the training device is operating normally is re-input, and the reconstruction error at this time is calculated, so as to obtain the reconstruction threshold when the device is operating normally; When the input of the convolutional self-editing network is the sound to be detected, its reconstruction error is obtained. When its reconstruction error is greater than the reconstruction threshold, it is an abnormal sound. When its reconstruction error is less than the reconstruction threshold, it is a normal sound, thereby completing the abnormal sound detection of coal mine transportation equipment.

Images