A method for locating equipment fault sound sources based on deep learning theory

Abstract

The present invention discloses a method for locating the sound source of equipment failure based on deep learning theory, comprising the following steps: collecting sound source signals through a microphone array MEMS sensor to obtain the original sound source signal of the fault point; performing Fourier transform and grayscale processing on the original sound source data to convert it into an STFT grayscale image, and inputting the STFT grayscale image into a trained sound source position neural network module to obtain the positioning information of the fault point; inputting the original sound source data into a trained sound pressure intensity neural network module to obtain the sound pressure intensity of the fault point; combining the positioning information of the fault point with the sound pressure intensity to judge the fault condition of the equipment. Based on the characteristics of the original sound source data, the present invention innovatively adopts a multi-task learning network architecture of a mutually independent sound source position neural network module + a sound pressure intensity neural network module, thereby simultaneously obtaining the positioning information and sound pressure intensity of the fault point, providing two-way support for the judgment of the equipment fault condition.

Description

Technical Field

[0001] The present invention relates to the technical field of sound source localization, and in particular to a method for localizing the sound source of equipment failure based on deep learning theory. Background Art

[0002] The sound source localization method is a commonly used equipment fault location method in the power industry and the industrial industry. The principle of this method is to analyze the microphone array to collect signals and estimate the wave direction of all sound sources in the scene relative to the microphone array. In recent years, thanks to the rapid development of artificial intelligence technology, many experts and scholars have begun to apply deep neural networks to sound source localization methods. For example, a Chinese patent with application number 202010916621.X discloses a method for localizing equipment fault sound sources. This method inputs the delayed summation beamforming imaging results into a trained dual-channel convolutional neural network to obtain the location information of the fault sound source point, thereby avoiding iterative calculations in the positioning process and improving the real-time nature of the positioning results and the accuracy of the positioning information.

[0003] This method uses the result of the delayed sum algorithm as input to the network, so it is necessary to perform multiple steps of Fourier transform, fast spectral kurtosis algorithm, and delayed sum algorithm on the original sound source signal. This has the following two problems: First, the original sound source signal contains sound source intensity information and phase information, but the sound source intensity information will be missing after Fourier transform, resulting in the algorithm only being able to obtain the location information of the sound source point, but unable to obtain the sound power intensity of the sound source point, so there are certain drawbacks; second, the rich information contained in the original sound source information after multiple steps of processing is missing, which directly affects the accuracy of the positioning result, and once the algorithm fails, it will affect the subsequent reasoning, and the reliability is low. In addition, the dual-channel convolutional neural network used in this method also has problems such as slow reasoning and more hardware storage. Summary of the invention

[0004] The present invention provides a method for locating the sound source of equipment faults based on deep learning theory, the main purpose of which is to solve the problems existing in the prior art.

[0005] The present invention adopts the following technical solution:

[0006] A method for locating equipment fault sound sources based on deep learning theory, characterized in that it includes the following steps:

[0007] S1. Acquire the sound source signal through the microphone array MEMS sensor to obtain the original sound source signal of the fault point;

[0008] S2, perform Fourier transform and grayscale processing on the original sound source data to convert it into an STFT grayscale image, and input the STFT grayscale image into a trained sound source location neural network module to obtain the location information of the fault point;

[0009] S3, inputting the original sound source data into the trained sound pressure intensity neural network module, thereby obtaining the sound pressure intensity of the fault point;

[0010] S4. Determine the fault condition of the equipment based on the location information of the fault point and the sound pressure intensity.

[0011] Further, in step S2, the sound source location neural network module is a RepVGG-B0 network

[0012] Furthermore, the training and reasoning processes of the RepVGG-B0 network have 5 stages, and the number of layers in each stage is [1, 4, 6, 16, 1] respectively.

[0013] Further, in step S3, the sound pressure intensity neural network module is a shallow one-dimensional convolutional neural network, which includes 3 one-dimensional convolutional layers and 4 fully connected layers.

[0014] Further, in step S2, the original sound source signal is converted into Converted into time-frequency image through short-time Fourier transform ;

[0015]

[0016] Where: N represents the number of sound sources, represents the length of the Hamming window, R represents the jump size between adjacent windows, Indicates the width of the Hamming window.

[0017] In steps S2 and S3, the sound source location neural network module and the sound pressure intensity neural network module are trained by the BP training method. First, the difference between the output sound source location and the actual sound source location is calculated. And the difference between the output sound pressure intensity and the actual sound pressure intensity , and then calculate the cost function according to formula (2) ,Finally, the sound source location neural network module and the sound pressure intensity neural network module are iteratively corrected according to the cost function;

[0018]

[0019] Where: α represents a hyperparameter that balances the magnitude of the loss function between the training sound source position and the sound pressure intensity.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. Based on the characteristics of the original sound source data, the present invention innovatively adopts a multi-task learning network architecture of a mutually independent sound source position neural network module + a sound pressure intensity neural network module, and uses the STFT grayscale image that has undergone Fourier transform and grayscale processing as the input of the sound source position neural network module, thereby regressing the position of the fault point using the phase feature; and uses the original sound source data as the input of the sound pressure intensity neural network module, thereby regressing the sound pressure intensity of the fault point using the amplitude information, thereby simultaneously obtaining the positioning information and sound pressure intensity of the fault point, providing strong support for the judgment of equipment fault conditions.

[0022] 2. The sound source location neural network module of the present invention uses the RepVGG-B0 network. Due to its small number of parameters and simple structure, the network can save hardware memory in the model reasoning process, facilitate hardware acceleration, and help the model to be better deployed on hardware. The sound pressure intensity neural network module uses a shallow one-dimensional convolutional neural network. The original sound source data is sampled and the amplitude features are extracted through three one-dimensional convolutional modules, and then four layers of fully connected layers are connected to regress and predict the sound pressure intensity of the sound source point. This corrects the shortcomings of using only RepVGG-B0 for model learning and effectively overcomes the problems existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the overall algorithm flow of the present invention.

[0024] Figure 2 Schematic diagram of data processing of STFT grayscale image in the present invention.

[0025] Figure 3 Schematic diagram of the network during the training and reasoning phases of RepVGG-B0 in the present invention.

[0026] Figure 4 Schematic diagram of the multi-task learning architecture in the present invention. DETAILED DESCRIPTION

[0027] The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0028] Reference Figures 1 to 4 , a method for locating equipment fault sound sources based on deep learning theory, comprising the following steps:

[0029] S1. Acquire the sound source signal through the microphone array MEMS sensor to obtain the original sound source signal of the fault point. In this step, the microphone array MEMS sensor can realize the acquisition of multi-channel sound source signals.

[0030] S2. Perform Fourier transform and grayscale processing on the original sound source data to convert it into an STFT grayscale image, and input the STFT grayscale image into a trained sound source location neural network module to obtain the location information of the fault point.

[0031] S21, STFT grayscale image data:

[0032] Based on research findings, the original sound source signal collected by the microphone array MEMS sensor contains significantly more amplitude information than phase information. Therefore, in order to facilitate the training of the deep learning model, the present invention first converts the original audio signal into a time-frequency image. Specifically, N spiral array microphones are located on the same two-dimensional plane, with a sampling frequency of fHz and a sampling time of t seconds. According to formula (1), the original sound source signal Converted into time-frequency image through short-time Fourier transform .

[0033]

[0034] Where N represents the number of sound sources. In this implementation, N=56. represents the length of the Hamming window, R represents the jump size between adjacent windows, Indicates the width of the Hamming window.

[0035] The reason why the original audio signal is converted into a time-frequency image is that the result of STFT contains rich phase information, so the sound source location neural network module can capture the phase changes between different images, thereby obtaining accurate fault location information. In addition, in order to remove unnecessary redundant information, the present invention further converts the STFT time-frequency image into an STFT grayscale image as the input of the sound source location neural network module. For details of the data processing process, see Figure 2 .

[0036] S22, sound source location neural network module:

[0037] The reparameterized VGG network is a simple network architecture with only the same type of operators (3×3 conv followed by ReLU). Compared with VGG, ResNet and DenseNet, RepVGG helps dedicated hardware save more computing resources and achieves higher speed calculations with fewer operators and a single convolutional layer. After research, it was found that RepVGG has a variety of specific network structures. Therefore, in order to select a network with a suitable number of parameters to extract the phase characteristics of the original sound source signal, the sound source position neural network module of the present invention is preferably a RepVGG-B0 network.

[0038] Specifically, the training and reasoning process of RepVGG-B0 in the present invention has 5 stages, and the number of layers in each stage is [1, 4, 6, 16, 1]. During training, RepVGG-B0 uses independent branches and 1×1 convolution branches to train, such as Figure 3 (a). After training, the model can remove independent branches and 1×1 convolution branches through a structured re-parameterization method. During inference, RepVGG-B0 can convert the training block into a single 3×3 conv layer, as shown in Figure 3 (b) shows the advantage of model conversion. The 3×3conv structure is more suitable for optimization on some computing libraries on GPU and CPU (such as NVDIA cuDNN and Intel MKL). And the fewer parameters after model pruning can save hardware storage, and a single module can speed up hardware computing.

[0039] S3. Input the original sound source data into the trained sound pressure intensity neural network module to obtain the sound pressure intensity of the fault point.

[0040] S31, original sound source data:

[0041] Although RepVGG-B0 has significant advantages in model simplification, the sound pressure intensity prediction task requires more amplitude information. However, the grayscale STFT grayscale image cannot extract good amplitude features. Therefore, in order to obtain more accurate fault location information through the sound pressure intensity neural network module, this method starts from the original audio data and corrects the shortcomings of using only RepVGG-B0 for model learning through the sound pressure intensity neural network module learning.

[0042] S32, sound pressure intensity neural network module:

[0043] like Figure 1 and Figure 4 As shown, in this implementation, the sound pressure intensity neural network module is preferably a shallow one-dimensional convolutional neural network, which includes 3 one-dimensional convolutional layers and 4 fully connected layers. Such a design can make the model more biased to focus on the representation of different features of the original sound source information, so that the model can be better generalized to new tasks.

[0044] S4. Determine the fault condition of the equipment by combining the location information of the fault point and the sound pressure intensity. The sound power value of the equipment can be obtained through the sound pressure intensity, which helps to determine the fault type and discharge strength. On this basis, combined with the location information of the fault point, it can ensure accurate determination of the fault condition of the equipment, thereby overcoming the shortcomings of the prior art of determining the fault condition of the equipment only through location information.

[0045] The following is a detailed introduction to the training process of the sound source location neural network module and the sound pressure intensity neural network module:

[0046] In steps S2 and S3, the sound source location neural network module and the sound pressure intensity neural network module are trained by the bp training method. Specifically, assuming that the data set for sound source localization training has R training samples, the data set is available To express. From the actual sound source position and sound pressure intensity It consists of two parts.

[0047] The present invention uses Represents each training sample The predicted result is obtained, and the difference between the output sound source position and the actual sound source position is calculated using the L2 loss of formula (3). , in order to optimize the output predicted value and true value.

[0048] (3)

[0049] In order to show the difference of sound pressure intensity, the present invention performs Use formula (4) to convert to decibel scale to calculate the loss function:

[0050] (4)

[0051] Generally speaking, the reference value of sound pressure in living air is , The unit is dB. The sound pressure level intensity (SPL) obtained by the above formula is During the training process, the SPL loss function uses L1 loss to obtain the difference between the output sound pressure intensity and the actual sound pressure intensity. . As shown in formula (5).

[0052] (5)

[0053] Combining formula (3) and formula (5), the cost function of the entire training process of this algorithm is

[0054]

[0055] in α It is a hyperparameter that balances the magnitude of the loss functions for training sound source position and sound pressure intensity. In the invention, α = 10 is set to adjust the two loss functions.

[0056] Finally, the sound source position neural network module and the sound pressure intensity neural network module are iteratively corrected according to the cost function.

[0057] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A method for locating equipment fault sound sources based on deep learning theory, characterized in that: The steps include: S1. Acquire the sound source signal through the microphone array MEMS sensor to obtain the original sound source signal of the fault point; S2, Fourier transform and grayscale processing are performed on the original sound source data to convert it into an STFT grayscale image, and the STFT grayscale image is input into a trained sound source position neural network module to obtain the location information of the fault point; the sound source position neural network module is a RepVGG-B0 network; S3, inputting the original sound source data into a trained sound pressure intensity neural network module to obtain the sound pressure intensity of the fault point; the sound pressure intensity neural network module is a shallow one-dimensional convolutional neural network, which includes 3 one-dimensional convolutional layers and 4 fully connected layers; In steps S2 and S3, the sound source location neural network module and the sound pressure intensity neural network module are trained by the BP training method. First, the difference between the output sound source location and the actual sound source location is calculated. And the difference between the output sound pressure intensity and the actual sound pressure intensity , and then calculate the cost function according to formula (2) ,Finally, the sound source location neural network module and the sound pressure intensity neural network module are iteratively corrected according to the cost function; Where: α represents a hyperparameter that balances the magnitude of the loss function between the training sound source position and the sound pressure intensity; S4. Determine the fault condition of the equipment based on the location information of the fault point and the sound pressure intensity.

2. The method for locating equipment fault sound sources based on deep learning theory as claimed in claim 1, characterized in that: The training and reasoning processes of the RepVGG-B0 network have 5 stages, and the number of layers in each stage is [1, 4, 6, 16, 1] respectively.

Images