Lightweight partial discharge ultrasonic identification network based on multi-scale feature fusion

Abstract

The invention discloses a lightweight partial discharge ultrasonic identification network based on multi-scale feature fusion. According to the technical scheme, the lightweight partial discharge ultrasonic identification network is characterized by comprising the following steps: step 1, sound signal acquisition; 2, calculating the Mel spectrum or the short-time Fourier transform spectrum of the sound signal, and taking the Mel spectrum or the short-time Fourier transform spectrum as the input of a neural network; step 3, performing multi-scale feature convolution on the input features in different dimensions, and projecting corresponding feature maps into the dimensions the same as the dimensions of the input features; 4, performing feature extraction on the projected feature map in the step 3 through a feature mask matrix, and calculating a corresponding weight matrix; and 5, carrying out weighted summation on the feature map obtained in the step 4 and the weight of the feature map to obtain an enhanced feature map. The core idea of the neural network structure provided by the invention is to perform signal-to-noise ratio enhancement on acoustic signal features extracted on a multi-dimensional scale based on the concept of a mask matrix.

Description

Technical Field

[0001] The present invention relates to the field of electrical technology, and in particular to a lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion. Background Art

[0002] Partial discharge (PD) is a fault that is widely found in electrical equipment. It is usually caused by defects in the equipment insulation structure and equipment aging. PD will intensify over time and may even cause major safety accidents such as power outages and fires. Therefore, the detection of partial discharge is an important measure for the safety maintenance of power systems.

[0003] Detecting discharge by ultrasonic signals generated by discharge is an emerging technology in recent years. It can not only determine the occurrence of discharge, but also locate the discharge and even determine the type of discharge, such as suspended discharge and surface discharge, by processing ultrasonic signals. In actual detection, deep learning technology has shown good application prospects for processing ultrasonic signals. Deep neural networks can extract relevant high-dimensional features from the original acoustic signal sequence or some artificially processed features, such as Mel Frequency Cepstral Coefficient (MFCC) and Short Time Fourier Transform (STFT) for judgment, and have shown good results in the problem of partial discharge type identification. However, in some detection scenarios with strong background noise, such as strong wind environment, bad weather or drone detection scenarios, the effect of the model may be significantly affected. Low signal-to-noise ratio and large outliers can cause neural network necrosis during training. In addition, the common network architectures such as Transformer and ResNet have large parameters, and their application on handheld terminals is still challenging. Therefore, designing a lightweight and deployable neural network structure is of great significance for using deep learning to detect discharge. Summary of the invention

[0004] The purpose of the present invention is to provide a lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion, which can improve the accuracy of partial discharge recognition under low signal-to-noise ratio conditions and minimize the number of model parameters and calculations.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion includes the following steps:

[0007] Step 1: Acoustic signal collection;

[0008] Step 2: Calculate the Mel spectrum or short-time Fourier transform spectrum of the sound signal and use it as the input of the neural network;

[0009] Step 3: Perform multi-scale feature convolution on the input features in different dimensions, and project the corresponding feature maps to the same dimension as the input features;

[0010] Step 4: Extract features from the feature map after projection in step 3 through the feature mask matrix, and calculate the corresponding weight matrix;

[0011] Step 5: Add the feature map obtained in step 4 and its weights to obtain the enhanced feature map.

[0012] Preferably, in the step 2, a short signal sequence is selected to calculate the Mel spectrum or short-time Fourier transform spectrum of the acoustic signal.

[0013] Preferably, the slice convolution of different dimensions and scales described in step three refers to slicing the input features into one-dimensional vectors in the time and frequency dimensions, performing long-time and short-time convolutions in the time dimension, the time dimension is determined by the convolution step size, performing convolution of a single scale in the frequency dimension, and splicing the convolution results at the original positions to obtain the corresponding feature maps.

[0014] Preferably, when projecting the feature map, for feature maps of different shapes, trainable left-multiplication and right-multiplication matrices are generated according to their shapes, and matrix multiplication is performed to project the feature map to the same dimension as the input matrix.

[0015] Preferably, in step 4, a trainable feature mask matrix is ​​set to interact with the feature map obtained in step 3 to update the feature map, and a corresponding weight matrix is ​​calculated, so that different feature maps are interacted with the same mask matrix after projection.

[0016] Preferably, in step five, convolutional neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, deep belief networks and long short-term memory networks are used for classification of the enhanced feature maps.

[0017] Preferably, in the step 3, the input feature is assumed to be a two-dimensional matrix S:

[0018]

[0019] S is sliced ​​in the horizontal time dimension and the vertical frequency dimension, and the time series vectors obtained by the horizontal slices are subjected to long-term and short-term one-dimensional convolution. The one-dimensional convolution results of multiple row vectors are spliced ​​into a two-dimensional matrix according to the positions of the original vectors. Similar operations are performed on the vertical slices, but convolutions of multiple scales are not performed. A two-dimensional matrix is ​​also spliced, which are denoted as A1, A2, and A3 respectively.

[0020] In step 3, the shapes of the two-dimensional matrices obtained by performing slice convolution and splicing on three different dimensions and scales are different. Trainable left and right multiplication matrices are generated according to their respective shapes. For the two-dimensional matrix A1 obtained by performing slice long-term convolution and splicing on the time dimension, the left multiplication projection matrix P is generated. 11 And right multiply the projection matrix P 12 Project A1 to the original input feature dimension through matrix multiplication to obtain feature map B1:

[0021] B1=P 11 *A1*P 12 ;

[0022] Furthermore, in step 4, a feature map B is generated. i The matrix Q of extracting features from (i=1,2,3):

[0023]

[0024] The matrix Q is called the mask matrix of the input features. It is combined with the feature map B in the following way i The corresponding weights are obtained by interaction:

[0025]

[0026]

[0027] Among them, B i,N Represents the feature map B corresponding to the i-th feature i The result of the Nth iteration, N is the number of cycles of the above feature extraction process, which is determined by debugging based on the effect of model training;

[0028] Preferably, in step 5, for each feature map B after N iterations, i,N The corresponding weight matrix K i,N , generate two trainable vectors to obtain the final weight value w i :

[0029]

[0030] Introducing the residual mechanism, we get the feature map S with enhanced signal-to-noise ratio e :

[0031]

[0032] For S obtained in step 5 e A simple two-dimensional convolutional neural network is used for classification, and the two-dimensional convolutional neural network is VGG-Net.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The core concept of the neural network structure proposed in the present invention is to enhance the signal-to-noise ratio of the acoustic signal features extracted on a multi-dimensional scale based on the concept of a mask matrix. In the operation, especially the transformation of the matrix shape has a certain complexity, but in fact, the network is specially optimized for the consideration of the amount of calculation and the amount of parameters, such as slice convolution and matrix projection. For common acoustic signal sampling rates and partial discharge detection scenarios, the number of parameters of the entire network is between 45k-100k, which is greatly reduced compared to the current common structures, such as ResNet18 / 34 (11.2M / 21.8M), VGG11 (132.9M), etc., and is suitable for deployment and use on some portable terminals. The present invention is particularly suitable for scenarios with low signal-to-noise ratio and non-stationary background noise. It can significantly improve the situation where networks containing multi-layer convolutional structures such as ResNet are necrotic during training due to abnormal input values, and extract high-dimensional features from spectra that are relatively close to different types of discharges. Considering some vibration or sound wave information accompanying partial discharge in common power equipment such as transformers, the accuracy of identifying partial discharge types is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of the present invention;

[0036] Figure 2 This is a schematic diagram of the structure of a neural network.

[0037] Figure 3 is a schematic diagram of the mask matrix. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Example 1

[0040] Please refer to Figures 1 to 3 ,,The present invention provides a technical solution: a lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion, such as Figure 1 As shown, the partial discharge classification by the neural network structure proposed by the present invention includes the following steps:

[0041] Step 1: Preliminary processing of scene sound signals: Calculate the Mel spectrum or STFT of the sound signal sequence collected by the microphone. Generally speaking, the longer the length of the sequence used for calculation, the larger the corresponding input feature data volume. Figure 1 The number of parameters required to train the neural network will also increase, and the real-time performance of the corresponding detection will decrease;

[0042] Step 2: Slice the input features and convolve them, concatenate the convolution results and project them to the dimension of the original input features. In fact, it has been proven in many studies that using the original sound signal as the input of the neural network does not produce ideal results. Therefore, on the basis of preprocessing, in order to retain the signal information as much as possible, the present invention chooses to use STFT or Mel spectrum as input. However, for partial discharge, considering its spectral characteristics, the two-dimensional correlation information can be ignored, and only the slice one-dimensional convolution method is used to extract features. This measure has a great effect on reducing the number of model parameters, especially on improving computational efficiency.

[0043] Step 3: query and extract the scale features of each dimension according to the steps described above. It should be pointed out that the above process is to train a matrix Q that can extract key information from the feature map. Q improves the signal-to-noise ratio of the feature map by acting on the feature map multiple times, and calculates the corresponding weight K of the feature map. The basic idea of ​​this method is similar to the mask-based deep learning denoising technology in the field of speech enhancement (SE). This technology trains a mask matrix of the same shape as the speech STFT to act on the STFT for denoising, and has shown superior performance in applications; a similar design is implemented in the deep neural network structure proposed in the present invention, and its expectation is that when the model training results are relatively ideal, the matrix Q, which is also a training parameter, will reflect the core characteristics of the spectrum of the local discharge acoustic signal to a certain extent;

[0044] like Figure 2 As shown in the figure, the Mel spectrum of one section of the partial discharge signal shows the characteristics of multiple amplitude modulation of partial discharge. Under the strong noise collected by the drone, this low-frequency component will be submerged by the noise and is difficult to remove using a bandpass filter or a comb filter. The trained query matrix Q shows a very strong correlation with this part of the amplitude modulation feature, proving that this structure can better improve the signal-to-noise ratio of the signal.

[0045] Example 2

[0046] refer to Figures 1 to 3 , a lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion, comprising the following steps:

[0047] Step 1: Acoustic signal collection;

[0048] Step 2: Calculate the Mel spectrum or short-time Fourier transform spectrum of the sound signal and use it as the input of the neural network;

[0049] Step 3: Perform multi-scale feature convolution on the input features in different dimensions, and project the corresponding feature maps to the same dimension as the input features;

[0050] Step 4: Extract features from the feature map after projection in step 3 through the feature mask matrix, and calculate the corresponding weight matrix;

[0051] Step 5: Add the feature map obtained in step 4 and its weights to obtain the enhanced feature map.

[0052] Preferably, in the step 2, a short signal sequence is selected to calculate the Mel spectrum or short-time Fourier transform spectrum of the acoustic signal; selecting a shorter signal sequence to calculate the STFT and taking a larger number of FFT points within an appropriate range can help improve the real-time performance of the detection on the one hand, and can also reduce the number of parameters and the amount of calculation of the neural network model to be trained on the other hand.

[0053] Among them, the slice convolution of different dimensions and scales described in the step three refers to slicing the input features into one-dimensional vectors in the time and frequency dimensions, performing long-time and short-time convolutions in the time dimension, the time dimension is determined by the convolution step size, performing a single-scale convolution in the frequency dimension, and splicing the convolution results at the original positions to obtain the corresponding feature maps.

[0054] When projecting the feature map, for feature maps of different shapes, trainable left-multiplication and right-multiplication matrices are generated according to their shapes, and matrix multiplication is performed to project the feature map to the same dimension as the input matrix.

[0055] Among them, in the step 4, a trainable feature mask matrix is ​​set to interact with the feature map obtained in step 3 to update the feature map, and the corresponding weight matrix is ​​calculated, so that different feature maps are interacted with the same mask matrix after projection.

[0056] Among them, in the step five, when classifying the enhanced feature map, convolutional neural network, feedforward neural network, convolutional neural network, recurrent neural network, deep belief network and long short-term memory network are used for classification.

[0057] In the step 3, the input feature is assumed to be a two-dimensional matrix S:

[0058]

[0059] S is sliced ​​in the horizontal time dimension and the vertical frequency dimension, and the time series vectors obtained by the horizontal slices are subjected to long-term and short-term one-dimensional convolution. The one-dimensional convolution results of multiple row vectors are spliced ​​into a two-dimensional matrix according to the positions of the original vectors. Similar operations are performed on the vertical slices, but convolutions of multiple scales are not performed. A two-dimensional matrix is ​​also spliced, which are denoted as A1, A2, and A3 respectively.

[0060] In step 3, the shapes of the two-dimensional matrices obtained by performing slice convolution and splicing on three different dimensions and scales are different. Trainable left and right multiplication matrices are generated according to their respective shapes. For the two-dimensional matrix A1 obtained by performing slice long-term convolution and splicing on the time dimension, the left multiplication projection matrix P is generated. 11 And right multiply the projection matrix P 12 Project A1 to the original input feature dimension through matrix multiplication to obtain feature map B1:

[0061] B1=P 11 *A1*P 12 ;

[0062] Furthermore, in step 4, a feature map B is generated. i The matrix Q of extracting features from (i=1,2,3):

[0063]

[0064] The matrix Q is called the mask matrix of the input features. It is combined with the feature map B in the following way i The corresponding weights are obtained by interaction:

[0065]

[0066] Among them, B i,N Represents the feature map B corresponding to the i-th feature i The result of the Nth iteration, N is the number of cycles of the above feature extraction process, which is determined by debugging based on the effect of model training;

[0067] Among them, in the step 5, for each feature map B after N iterations i,N The corresponding weight matrix K i,N , generate two trainable vectors to obtain the final weight value w i :

[0068]

[0069] The residual mechanism is introduced to obtain the feature map Se with enhanced signal-to-noise ratio:

[0070]

[0071] For S obtained in step 5 e A simple two-dimensional convolutional neural network is used for classification, and the two-dimensional convolutional neural network is VGG-Net.

[0072] Among them, the core concept of the neural network structure proposed in the present invention is to enhance the signal-to-noise ratio of the acoustic signal features extracted on a multi-dimensional scale based on the concept of a mask matrix. In the operation, especially in the transformation of the matrix shape, there is a certain complexity, but in fact, the network has been specially optimized for the consideration of the amount of calculation and the amount of parameters, such as slice convolution and matrix projection. For common acoustic signal sampling rates and partial discharge detection scenarios, the number of parameters of the entire network is between 45k-100k, which is greatly reduced compared to the current common structures, such as ResNet18 / 34 (11.2M / 21.8M), VGG11 (132.9M), etc., and is suitable for deployment and use on some portable terminals. The present invention is particularly suitable for scenarios with low signal-to-noise ratio and non-stationary background noise. It can significantly improve the situation where networks containing multi-layer convolutional structures such as ResNet are necrotic during the training process due to abnormal input values, and extract high-dimensional features from the frequency spectra that are relatively close to different types of discharges. Considering some vibration or sound wave information accompanying partial discharge in common power equipment such as transformers, the accuracy of identifying partial discharge types is improved.

[0073] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion, characterized by: The following steps are involved: Step 1: Acoustic signal collection; Step 2: Calculate the Mel spectrum or short-time Fourier transform spectrum of the sound signal and use it as the input of the neural network; Step 3: Perform multi-scale feature convolution on the input features in different dimensions, and project the corresponding feature maps to the same dimension as the input features; Step 4: Extract features from the feature map after projection in step 3 through the feature mask matrix, and calculate the corresponding weight matrix; Step 5: Add the feature map obtained in step 4 and its weights to obtain the enhanced feature map.

2. According to claim 1, a lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion is characterized by: In the step 2, a short signal sequence is selected to calculate the Mel spectrum or short-time Fourier transform spectrum of the acoustic signal.

3. The lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion according to claim 2 is characterized in that: The slice convolution of different dimensions and scales described in the step three refers to slicing the input features into one-dimensional vectors in the time and frequency dimensions, performing long-time and short-time convolutions in the time dimension, the time dimension is determined by the convolution step size, performing convolution of a single scale in the frequency dimension, and splicing the convolution results at the original positions to obtain the corresponding feature maps.

4. The lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion according to claim 3 is characterized in that: When projecting the feature map, for feature maps of different shapes, trainable left-multiplication and right-multiplication matrices are generated according to their shapes, and matrix multiplication is performed to project the feature map to the same dimension as the input matrix.

5. The lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion according to claim 1 is characterized in that: In the step 4, a trainable feature mask matrix is ​​set to interact with the feature map obtained in step 3 to update the feature map, and the corresponding weight matrix is ​​calculated, so that different feature maps are interacted with the same mask matrix after projection.

6. The lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion according to claim 1 is characterized in that: In the step 5, when classifying the enhanced feature map, convolutional neural network, feedforward neural network, convolutional neural network, recurrent neural network, deep belief network and long short-term memory network are used for classification.

7. The lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion according to claim 4 is characterized in that: In the step 3, the input feature is assumed to be a two-dimensional matrix S: S is sliced ​​in the horizontal time dimension and the vertical frequency dimension, and the time series vectors obtained by the horizontal slices are subjected to long-term and short-term one-dimensional convolution. The one-dimensional convolution results of multiple row vectors are spliced ​​into a two-dimensional matrix according to the positions of the original vectors. Similar operations are performed on the vertical slices, but convolutions of multiple scales are not performed. A two-dimensional matrix is ​​also spliced, which are denoted as A1, S2, and A3 respectively; In step 3, the shapes of the two-dimensional matrices obtained by performing slice convolution and splicing on three different dimensions and scales are different. Trainable left and right multiplication matrices are generated according to their respective shapes. For the two-dimensional matrix A1 obtained by performing slice long-term convolution and splicing on the time dimension, the left multiplication projection matrix P is generated. 11 And right multiply the projection matrix P 12 Project A1 to the original input feature dimension through matrix multiplication to obtain feature map B1: B1=P 11 *A1*P 12 ; Furthermore, in step 4, a feature map B is generated. i The matrix Q of extracting features from (i=1,2,3): The matrix Q is called the mask matrix of the input features. It is combined with the feature map B in the following way i The corresponding weights are obtained by interaction: Among them, B i,N Represents the feature map B corresponding to the i-th feature i The result of the Nth iteration, N is the number of cycles of the above feature extraction process, which is determined by debugging based on the effect of model training; 8. The lightweight partial discharge ultrasonic recognition network based on multi-scale feature fusion according to claim 7 is characterized in that: In the step 5, for each feature map B after N iterations, i,N The corresponding weight matrix K i,N , generate two trainable vectors to obtain the final weight value w i : Introducing the residual mechanism, we get the feature map S with enhanced signal-to-noise ratio e : For S obtained in step 5 e A simple two-dimensional convolutional neural network is used for classification, and the two-dimensional convolutional neural network is VGG-Net.

Images