An abnormal sound detection method mounted on a patrol robot

Abstract

The application discloses an abnormal sound detection method carried on a patrol robot, comprising the following steps: 1, sound collection; 2, obtaining the time sequence of the sound, the attribute of the sampling frequency and the sample bit number of each sampling point, zero-padding the PCM to make the length of the PCM an integer power of 2 L, and normalizing the PCM; 3, calculating the energy of the sound; 4, performing short-time Fourier transform; 5, calculating a mel-frequency spectrum graph MelSpec , weighting through a mel filter, and finally performing logarithmic operation on the weighted filter output value to obtain MelSpec ; 6, obtaining a multi-feature fusion sound feature vector X; 7, constructing a data set, inputting the feature vector X obtained by splicing into a self-encoder for training, and judging whether it is an abnormal sound according to the relationship between the reconstruction error and a threshold value. The application can effectively avoid the problems of the prior art, such as the need for an abnormal sound sample, single feature and low recognition rate.

Description

Technical Field

[0001] This invention belongs to the field of abnormal sound detection technology, and in particular relates to an abnormal sound detection method mounted on a patrol robot. Background Technology

[0002] With the continuous development and advancement of artificial intelligence and robotics, the energy and power industry is evolving from high automation to high intelligence, leading to the widespread application of AI-powered inspection robots with path planning capabilities. To enable these robots to replace manual inspections and achieve the goals of reducing manpower, lowering costs, improving quality, and increasing efficiency, it is necessary to endow them with intelligent detection capabilities in both vision and hearing.

[0003] Existing anomalous sound detection algorithms typically extract features such as short-time energy, zero-crossing rate, Mel spectrum, and Fourier transform as sound information features for classification. The drawback of these methods is that they rely on anomalous sound samples, which are difficult to collect, and using a single feature extraction method leads to low recognition efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide an abnormal sound detection method mounted on a patrol robot to solve the above-mentioned technical problems.

[0005] To solve the above-mentioned technical problems, the specific technical solution of the abnormal sound detection method mounted on a patrol robot according to the present invention is as follows:

[0006] An abnormal sound detection method mounted on a patrol robot includes the following steps:

[0007] Step 1: The inspection robot runs along the planned route to the designated equipment to collect sound. Each equipment collects 10 seconds of continuous normal sound.

[0008] Step 2: Collect the sound file from the microphone array of the patrol robot and parse it to obtain the sound time series PCM, sampling frequency sr, and the number of bits per sample point. First, pad the PCM with zeros to make its length a power of 2 L, and then normalize the PCM.

[0009] Step 3: Calculate the energy of this sound segment based on the PCM data obtained in Step 2;

[0010] Step 4: Perform a short-time Fourier transform based on the PCM data obtained in Step 2;

[0011] Step 5: Calculate the Mel spectrogram MelSpec based on the spectrogram obtained in Step 4, construct a Mel filter, and weight the power spectrum of Sepc through the Mel filter. Finally, perform a logarithmic operation on the weighted filter output value to obtain MelSpec.

[0012] Step 6: Concatenate the one-dimensional vectors of the short-time energy, short-time zero-crossing rate, Fourier spectrum Spec and Mel spectrum MelSpec obtained in Steps 3 and 4 to obtain a multi-feature fused sound feature vector X.

[0013] Step 7: First, construct the dataset and input the concatenated feature vector X into the autoencoder for training. Determine whether the sound is abnormal based on the relationship between the reconstruction error and the threshold. If the reconstruction error is greater than the threshold, it is an abnormal sound; if the reconstruction error is less than the threshold, it is a normal sound.

[0014] Furthermore, the normalization formula for step 2 is as follows:

[0015]

[0016] Where PCM is the time-series data of sound, 0≤i<L, i is the index of the PCM array, L is the length of the PCM, and bits_per_sample is the number of bits per sample point.

[0017] Furthermore, the energy formula for the sound in step 3 is as follows:

[0018]

[0019] E[n] represents the energy of the nth frame, 0 ≤ n < (L - win_L) / win_Hop + 1, PCM n [k] is the k-th signal value in the n-th frame, where 0 ≤ k < win_L. The formula for calculating the zero-crossing rate of this sound is as follows:

[0020]

[0021] Where STZCR[n] represents the zero-crossing rate of the nth frame, 0≤n<(L-win_L) / win_Hop+1, PCM n [k] is the k-th signal value of the n-th frame, where 0 ≤ k < win_L.

[0022] Furthermore, the formula for performing the short-time Fourier transform in step 4 is as follows:

[0023] Spec = stft(PCM)

[0024] Where Spec is the calculated spectrum, the window size of the short-time Fourier transform is win_L, the step size is win_Hop, and the size of the spectrum is ((L-win_L) / win_Hop+1)*(win_L / 2+1).

[0025] Furthermore, the MelSpec calculation formula in step 5 is as follows:

[0026]

[0027] Where MF is the Mel filter bank, Max is the maximum value in the Spec, the size of MelSpec is n_mels*((L-win_L) / win_Hop+1), and n_mels is the number of Mel filters.

[0028] Furthermore, step 6 includes the following specific steps:

[0029] The short-time energy E and short-time zero-crossing rate STZCR calculated in step 3 have a size of 1*((L-win_L) / win_Hop+1). The spectrogram obtained in step 4 has a size of ((L-win_L) / win_Hop+1)*(win_L / 2+1). Transposing it gives a size of (win_L / 2+1)*((L-win_L) / win_Hop+1). Then, it is expanded into a one-dimensional vector. In step 5, the MelSpec is calculated to have a size of n_mels*((L-win_L) / win_Hop+1). It is expanded into a one-dimensional vector. The four feature vectors obtained are concatenated to obtain the sound feature vector X.

[0030] Furthermore, step 7 includes the following specific steps:

[0031] A set of sounds from motors during normal operation is collected and converted into feature vectors X, which are used as training samples. The training samples are then input into an autoencoder to train the model parameters. The encoding and decoding stages each consist of three fully connected layers. The relationship between the reconstruction error err and the threshold T is used to determine whether the sound is abnormal. If err > T, it is an abnormal sound; if err ≤ T, it is a normal sound.

[0032] The abnormal sound detection method mounted on a patrol robot according to the present invention has the following advantages: The method of the present invention can be used as an abnormal sound detection method mounted on a patrol robot, utilizing multiple sound features, which can effectively avoid the problems of existing technologies requiring abnormal sound samples and having low recognition rates due to single features. Even under conditions of scarce abnormal sound samples and complex equipment operating environments, it can still detect abnormal equipment running on the patrol route. Attached Figure Description

[0033] Figure 1 This is a flowchart of the present invention;

[0034] Figure 2 This is a schematic diagram of the self-encoder used in this invention. Detailed Implementation

[0035] To better understand the purpose, structure, and function of this invention, the following detailed description of an abnormal sound detection method mounted on a patrol robot, in conjunction with the accompanying drawings, is provided.

[0036] like Figure 1 As shown, an abnormal sound detection method mounted on a patrol robot according to the present invention includes the following steps:

[0037] Step 1: The inspection robot runs along the planned route to the designated equipment to collect sound. Each equipment collects 10 seconds of continuous normal sound.

[0038] Step 2: The audio files collected by the robot's microphone array are parsed to obtain the audio time series (PCM), sampling frequency (sr), and bit depth (bits_per_sample) for each sampling point. The PCM is first padded with zeros to make its length a power of 2 (L), and then normalized using the following formula:

[0039]

[0040] Where PCM is the time-series data of sound, 0≤i<L, i is the index of the PCM array, L is the length of the PCM, and bits_per_sample is the number of bits per sample point.

[0041] Step 3: Based on the PCM data obtained in Step 2, calculate the energy of this sound segment using the following formula:

[0042]

[0043] E[n] represents the energy of the nth frame, 0 ≤ n < (L - win_L) / win_Hop + 1, PCM n [k] is the k-th signal value in the n-th frame, where 0 ≤ k < win_L. The formula for calculating the zero-crossing rate of this sound is as follows:

[0044]

[0045] Where STZCR[n] represents the zero-crossing rate of the nth frame, 0≤n<(L-win_L) / win_Hop+1, PCM n [k] is the k-th signal value of the n-th frame, where 0 ≤ k < win_L.

[0046] Step 4: Based on the PCM data obtained in Step 2, perform a short-time Fourier transform, as shown in the following formula:

[0047] Spec = stft(PCM)

[0048] Where Spec is the calculated spectrum, the window size of the short-time Fourier transform is win_L, the step size is win_Hop, and the size of the spectrum is ((L-win_L) / win_Hop+1)*(win_L / 2+1).

[0049] Step 5: Calculate the Mel spectrogram (MelSpec) based on the spectrogram obtained in Step 4, construct a Mel filter, and weight the power spectrum of Sepc through the Mel filter. Finally, perform a logarithmic operation on the weighted filter output value to obtain MelSpec. The calculation formula is as follows:

[0050]

[0051] Where MF is the Mel filter bank, Max is the maximum value in the Spec, the size of MelSpec is n_mels*((L-win_L) / win_Hop+1), and n_mels is the number of Mel filters.

[0052] Step 6: The short-time energy E and short-time zero-crossing rate STZCR calculated in Step 3 have a size of 1*((L-win_L) / win_Hop+1). The spectrogram obtained in Step 4 has a size of ((L-win_L) / win_Hop+1)*(win_L / 2+1). Transposing this spectrogram gives a size of (win_L / 2+1)*((L-win_L) / win_Hop+1), which is then expanded into a one-dimensional vector. Step 5: The MelSpec is calculated to have a size of n_mels*((L-win_L) / win_Hop+1), which is then expanded into a one-dimensional vector. Concatenating the four feature vectors yields the sound feature vector X used in this invention.

[0053] Step 7: Collect a set of sounds from the motor during normal operation, convert them into feature vectors X, and use them as training samples. Input the training samples into the autoencoder to train the model parameters. The structure of the autoencoder is as follows: Figure 2 As shown, both the encoding and decoding stages consist of three fully connected layers. The relationship between the reconstruction error err and the threshold T determines whether the sound is abnormal; if err > T, it is an abnormal sound, and if err ≤ T, it is a normal sound.

[0054] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. An abnormal sound detection method mounted on a patrol robot, characterized in that, Includes the following steps: Step 1: The inspection robot runs along the planned route to the designated equipment to collect sound. Each equipment collects 10 seconds of continuous normal sound. Step 2: Collect the sound file from the microphone array of the patrol robot and parse it to obtain the sound time series PCM, sampling frequency sr, and the number of bits per sample point. First, pad the PCM with zeros to make its length a power of 2 L, and then normalize the PCM. Step 3: Calculate the energy of this sound segment based on the PCM data obtained in Step 2; Step 4: Perform a short-time Fourier transform based on the PCM data obtained in Step 2; Step 5: Calculate the Mel spectrogram MelSpec based on the spectrogram obtained in Step 4, construct a Mel filter, and weight the power spectrum of Sepc through the Mel filter. Finally, perform a logarithmic operation on the weighted filter output value to obtain MelSpec. Step 6: Concatenate the one-dimensional vectors of the short-time energy, short-time zero-crossing rate, Fourier spectrum Spec and Mel spectrum MelSpec obtained in Steps 3 and 4 to obtain a multi-feature fused sound feature vector X. Step 7: First, construct the dataset and input the concatenated feature vector X into the autoencoder for training. Determine whether the sound is abnormal based on the relationship between the reconstruction error and the threshold. If the reconstruction error is greater than the threshold, it is an abnormal sound; if the reconstruction error is less than the threshold, it is a normal sound.

2. The abnormal sound detection method according to claim 1, characterized in that, The normalization formula for step 2 is as follows: Where PCM is the time-series data of sound, 0≤i<L, i is the index of the PCM array, L is the length of the PCM, and bits_per_sample is the number of bits per sample point.

3. The abnormal sound detection method according to claim 1, characterized in that, The energy formula for the sound in step 3 is as follows: This represents the energy of the nth frame. , It is the k-th signal value of the n-th frame, where The formula for calculating the zero-crossing rate of this sound is as follows: in, This represents the zero-crossing rate of the nth frame. , It is the k-th signal value of the n-th frame, where win_L represents the window size of the short-time Fourier transform, and win_Hop represents the step size.

4. The abnormal sound detection method according to claim 1, characterized in that, The formula for performing the short-time Fourier transform in step 4 is as follows: Where Spec is the calculated spectrum, the window size of the short-time Fourier transform is win_L, and the step size is win_Hop, then the size of the spectrum is... .

5. The abnormal sound detection method according to claim 1, characterized in that, The MelSpec calculation formula in step 5 is as follows: Where MF is the Mel filter bank, Max is the maximum value in Spec, and the size of MelSpec is... win_L represents the window size of the short-time Fourier transform, win_Hop represents the step size, and n_mels is the number of Mel filters.

6. The abnormal sound detection method according to claim 5, characterized in that, Step 6 includes the following specific steps: The short-time energy calculated in step 3 and short-time zero crossing rate Size is 1* The size of the spectrum obtained in step 4 is If it is transposed, its size is Then expand it into a one-dimensional vector. Step 5 calculates and obtains the size of MelSpec. Expand it into a one-dimensional vector, and concatenate the four feature vectors to obtain the sound feature vector X.

7. The abnormal sound detection method according to claim 1, characterized in that, Step 7 includes the following specific steps: A set of sounds from motors during normal operation is collected and converted into feature vectors X, which are used as training samples. The training samples are then input into an autoencoder to train the model parameters. The encoding and decoding stages each consist of three fully connected layers. The relationship between the reconstruction error err and the threshold T is used to determine whether the sound is abnormal. If err > T, it is an abnormal sound; if err ≤ T, it is a normal sound.

Images