Unmanned aerial vehicle self-noise filtering method and system based on multi-channel time-frequency spatial filtering

Abstract

The invention relates to the technical field of signal processing, in particular to an unmanned aerial vehicle self-noise filtering method and system based on multichannel time-frequency spatial filtering. The method comprises the following steps: acquiring a multi-channel noise-containing sound signal by using a microphone array carried by an unmanned aerial vehicle, and performing short-time Fourier transform to obtain a frequency domain signal; estimating a ratio mask by using a deep neural network, and performing logarithm normalization processing on the frequency domain signal; remodeling the sound signal to match the FFT length of the time-frequency spatial filter, and constructing the time-frequency spatial filter in combination with the multi-channel ratio mask; removing the self-noise of the unmanned aerial vehicle by using the filter and obtaining a target sound signal; and finally, converting the frequency domain signal into a time domain signal through inverse short-time Fourier transform. Through innovative fusion of multi-channel acquisition, deep learning and time-frequency space filtering, the self-noise of the unmanned aerial vehicle in a complex environment is accurately filtered, and the accuracy and efficiency of transformer substation noise monitoring are remarkably improved. Reliable acoustic data are provided for state monitoring of the power equipment, and the method has important engineering application value.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a method and system for filtering self-noise of an unmanned aerial vehicle based on multi-channel time-frequency-space filtering. Background Art

[0002] In recent years, noise monitoring has become increasingly important in the monitoring and maintenance of substation equipment, especially the monitoring of transformer noise, which can help identify potential equipment failures and abnormal operation. Traditional noise monitoring methods often rely on fixed sensors or manual measurements, which have problems such as limited coverage and low efficiency. With the development of drone technology, the use of drones for noise monitoring in substations has become an emerging trend. Drones have the advantages of high flexibility and wide coverage. They can move freely in complex substation environments and carry microphone arrays for acoustic perception tasks. However, since the microphone array of drones is close to the rotors and motors, the strong self-noise generated by these components seriously affects the accuracy and reliability of measurements. Although some signal processing algorithms have been proposed for sound enhancement on drone platforms, due to the non-stationarity of self-noise, these algorithms still perform poorly in low signal-to-noise ratio environments. In recent years, deep learning technology has brought revolutionary progress in the fields of noise elimination and speech enhancement. When there is sufficient training data, deep neural networks can learn features in noisy recordings and predict clear target sound signals. The traditional method relies on neural networks to learn the mapping relationship between noisy signals and corresponding clean signals, and estimate the time-frequency mask of the clean signal through this mapping relationship. This method shows good robustness in dealing with relatively stable noise environments due to its simple network architecture and low computational complexity. However, when faced with non-stationary noise environments, the effect of this traditional method is often unsatisfactory. Non-stationary noise has complex characteristics and change patterns, making it difficult for neural networks to effectively capture and learn these noise characteristics. In particular, when the noise contained in the training data set is representative, the neural network may overfit these specific types of noise, resulting in a significant decrease in the prediction effect. The strong self-noise generated by drones has complex time-frequency characteristics, and more advanced and robust sound denoising technology needs to be developed to meet this severe challenge. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that the operator can use the microphone array carried by the drone to collect the multi-channel noisy transformer sound, and then construct a spatial filter based on the time-frequency mask estimated by the deep neural network, and combine the spatial information and time-frequency characteristics of the mixed sound to realize the drone self-noise filtering.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides a method for filtering self-noise of a drone based on multi-channel time-frequency-spatial filtering, which comprises collecting a multi-channel noisy sound signal using a microphone array carried by the drone, performing a short-time Fourier transform on the collected signal, and obtaining a frequency domain signal;

[0007] A ratio mask is estimated using a deep neural network to log-normalize the frequency domain signal.

[0008] Reshape the sound signal to match the FFT length of the time-frequency-spatial filter, and construct the time-frequency-spatial filter in combination with a multi-channel ratio mask;

[0009] Apply time-frequency-spatial filters to remove drone self-noise and obtain target sound signals;

[0010] The frequency domain signal is converted into the time domain signal through inverse short-time Fourier transform to filter out the self-noise of the drone.

[0011] As a preferred solution of the method for filtering self-noise of a drone based on multi-channel time-frequency-spatial filtering of the present invention, wherein: the method of collecting multi-channel noise-containing sound signals using a microphone array carried by a drone includes collecting sound signals using N microphones carried by the drone, and receiving N channels of mixed sound signals containing self-noise of the drone and target sound;

[0012] According to the duration of the sound signal, a suitable window size is selected to intercept the input signal, and then short-time Fourier transform is used for each signal to obtain the frequency domain signals of N channels; the frequency domain mixed sound signal of the nth channel can be expressed as:

[0013] X n (k,l)=S n (k,l)+V n (l,k)

[0014] Among them, s is the target sound signal, v is the drone self-noise, k is the frequency number, and l is the time frame of the signal.

[0015] As a preferred solution of the drone self-noise filtering method based on multi-channel time-frequency spatial filtering of the present invention, the method of estimating the ratio mask using a deep neural network includes using an ideal ratio mask to represent the target sound proportion at each time-frequency point, and the expression is as follows:

[0016]

[0017] When using a feedforward neural network, it mainly includes an input layer, multiple hidden layers and an output layer. Each hidden layer uses a rectified linear unit to introduce nonlinear transformations when processing data, helping the neural network capture the complex features of the input data;

[0018] The output layer uses an s-type unit structure to map the input data to the range of [0,1]. To improve the generalization ability of the model, a dropout layer is added to each hidden layer to prevent the model from overfitting by randomly ignoring some neurons, thereby accurately estimating the ideal ratio mask.

[0019] As a preferred solution of the method for filtering self-noise of unmanned aerial vehicles based on multi-channel time-frequency-spatial filtering of the present invention, the logarithmic normalization processing of the frequency domain signal includes logarithmic compression of the frequency domain input signal and then normalization to zero mean and unit variance, and the expression is as follows:

[0020]

[0021] Among them, μ(k) and σ(k) are the mean and standard deviation of the kth time-frequency point, which are pre-calculated from the training data set. After logarithmic normalization, the scale of the spectrum graph is small, which is conducive to convergence during model training.

[0022] As a preferred solution of the method for filtering self-noise of drones based on multi-channel time-frequency-spatial filtering of the present invention, the reshaping of the sound signal to match the FFT length of the time-frequency-spatial filter and the construction of the time-frequency-spatial filter in combination with a multi-channel ratio mask include the following steps:

[0023] The logarithmically normalized spectrum is provided to the feedforward neural network in a frame-by-frame manner, and the estimated ratio mask M at the lth frame is output. n (1:K,l);

[0024] The model parameters are trained using supervised learning and optimized by minimizing the square error between the estimated ratio mask and the ideal ratio mask. The expression is as follows:

[0025]

[0026] Since the ratio mask estimated by the neural network does not match the FFT length of the time-frequency-space filter, which has a longer FFT length in order to capture the transmission delay, the filter cannot be constructed directly;

[0027] The target sound signal S(k1,l1) in the frequency domain is obtained according to the estimated ratio mask, and is converted into a time domain signal S(t) by inverse short-time Fourier transform. The inverse short-time Fourier transform can be expressed as follows:

[0028]

[0029] Where T represents the FFT length, R is the distance between adjacent frames, ω(t) is the window type, and the parameters are the same as those for short-time Fourier transform;

[0030] The time domain signals of each frame are superimposed according to their positions on the time axis to obtain the time domain signal S(t), and then S(t) and X(t) are converted into the frequency domain signal S(k2,l 2) and X(k2,l2), the ratio mask is re-estimated as follows:

[0031]

[0032] As a preferred solution of the method for filtering the self-noise of a drone based on multi-channel time-frequency-space filtering of the present invention, the method of applying the time-frequency-space filter to remove the self-noise of the drone and obtain the target sound signal comprises the following steps:

[0033] Combine the ratio masks of N channels for average pooling to obtain M(k2,l2), and then obtain the spatial matrix reflecting the energy distribution of the target speech signal at each time-frequency point and the mixed sound spatial matrix of the total energy distribution. The expression is as follows:

[0034]

[0035] Among them, Φ s The spatial matrix representing the energy distribution of the target speech signal at each time-frequency point; Φ x a mixed sound spatial matrix representing the total energy distribution;

[0036] Then construct the time-frequency space filter to obtain the frequency domain target sound signal after filtering out the drone's self-noise:

[0037]

[0038] Among them, Φ s1 Represents Φ s The first column of .

[0039] As a preferred solution of the method for filtering self-noise of a UAV based on multi-channel time-frequency-space filtering according to the present invention, the method of converting the frequency domain signal into a time domain signal by inverse short-time Fourier transform includes reshaping the frequency domain target sound signal obtained by the time-frequency-space filter in steps to match the FFT length of the time-frequency-space filter by inverse short-time Fourier transform, thereby obtaining the target sound signal in the time domain, thereby filtering the self-noise of the UAV.

[0040] In a second aspect, an embodiment of the present invention provides a drone self-noise filtering system based on multi-channel time-frequency-spatial filtering, which includes an acquisition module, which uses a microphone array carried by a drone to collect multi-channel noisy sound signals, performs short-time Fourier transform on the collected signals, and obtains frequency domain signals;

[0041] The processing module estimates the ratio mask using a deep neural network and performs logarithmic normalization on the frequency domain signal;

[0042] The building module reshapes the sound signal to match the FFT length of the time-frequency-spatial filter and combines the multi-channel ratio mask to build the time-frequency-spatial filter;

[0043] The acquisition module applies a time-frequency-spatial filter to remove the drone’s self-noise and obtain the target sound signal;

[0044] The filtering module converts the frequency domain signal into the time domain signal through inverse short-time Fourier transform to filter out the self-noise of the drone.

[0045] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the drone self-noise filtering method based on multi-channel time-frequency-spatial filtering as described in the first aspect of the present invention are implemented.

[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the method for removing self-noise of a drone based on multi-channel time-frequency-spatial filtering as described in the first aspect of the present invention are implemented.

[0047] The beneficial effects of the present invention are as follows: by using the drone-mounted microphone array to collect multi-channel noisy sound signals, and performing short-time Fourier transform on the collected signals to obtain frequency domain signals, high-quality collection of sound signals in complex environments is achieved. This method makes full use of the flexibility of the drone and the spatial resolution capability of the microphone array, and can obtain more comprehensive sound information in complex environments such as substations, laying the foundation for subsequent noise filtering.

[0048] By using a deep neural network to estimate the ratio mask and performing logarithmic normalization on the frequency domain signal, the target sound and noise are accurately separated in the time and frequency domain. This deep learning-based method can adaptively learn complex non-stationary noise features, has stronger robustness and generalization ability than traditional methods, and can cope with various complex noise environments.

[0049] By reshaping the sound signal to match the FFT length of the time-frequency spatial filter and combining the multi-channel ratio mask to construct the time-frequency spatial filter, the problem of mismatch between the neural network output and the spatial filter input is solved. This innovative signal reshaping method cleverly combines the advantages of deep learning and traditional signal processing technology, improving the flexibility and adaptability of the entire system.

[0050] By applying a time-frequency-spatial filter to remove the drone's self-noise and obtain the target sound signal, high-quality extraction of the target sound is achieved. This method makes full use of the time-frequency and spatial characteristics of the sound signal, can effectively suppress the strong drone's self-noise, and greatly improves the sound enhancement effect in a low signal-to-noise ratio environment.

[0051] The frequency domain signal is converted into the time domain signal through the inverse short-time Fourier transform, and the self-noise of the drone is finally filtered out. This step not only completes the reconstruction of the signal, but also ensures the naturalness and coherence of the processed sound, and improves the sound quality of the final output.

[0052] In summary, through the organic combination of deep learning and traditional signal processing technology, high-quality sound enhancement in complex noise environments has been achieved. This method can not only effectively filter out strong drone self-noise, but also adapt to various non-stationary noise environments, greatly improving the acoustic monitoring capabilities in scenarios such as substations. This technological breakthrough has brought new possibilities to the fields of preventive maintenance, fault diagnosis and safety monitoring of power equipment, and is expected to significantly improve the efficiency and reliability of power grid operation and maintenance, thereby making important contributions to the safe and stable operation of the entire power system.

[0053] In addition, the innovation of this method is also reflected in the following aspects:

[0054] Unexpected technical effects: Through the collaborative processing of multi-channel signals and the application of time-frequency-space filters, this method can still achieve good noise filtering effects in extremely low signal-to-noise ratio environments, which exceeds the expected effects of traditional single-channel or pure deep learning methods.

[0055] Solving long-standing technical problems: This method effectively solves the problem of drones’ own noise interference when performing acoustic monitoring in complex environments such as substations, which has been a technical problem that has long plagued the industry.

[0056] Synergistic effect brought by technological integration: Through the innovative combination of deep learning and traditional signal processing technology, this method achieves the effect of "1+1>2", giving full play to the advantages of both technologies and overcoming their respective limitations.

[0057] Broad application prospects: Although this method is optimized for substation scenarios, its basic principles and framework can be extended to other fields that require acoustic perception in strong noise environments, such as industrial noise monitoring, environmental acoustic monitoring, etc.

[0058] In summary, this technical solution also provides new ideas and directions for the technological development in related fields, and has important theoretical value and practical application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 This is a flow chart of the UAV self-noise filtering method based on multi-channel time-frequency-spatial filtering;

[0061] Figure 2 A computer device diagram of a UAV self-noise filtering method based on multi-channel time-frequency-space filtering;

[0062] Figure 3 This is the mixed sound waveform collected by two microphones in the microphone array of the drone self-noise filtering method based on multi-channel time-frequency-spatial filtering.

[0063] Figure 4 This is the waveform of the transformer operation sound after filtering based on the UAV self-noise filtering method based on multi-channel time-frequency-spatial filtering.

[0064] Figure 5 This is the transformer operation recording waveform of the UAV self-noise filtering method based on multi-channel time-frequency-spatial filtering. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0066] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0067] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0068] Example 1

[0069] Reference Figure 1-2 , which is the first embodiment of the present invention, and provides a method for filtering self-noise of a drone based on multi-channel time-frequency-space filtering, comprising:

[0070] S100: Use the drone-mounted microphone array to collect multi-channel noisy sound signals, perform short-time Fourier transform on the collected signals, and obtain frequency domain signals;

[0071] Preferably, an audio device is used to play the recorded transformer operation sound, and then a drone equipped with 6 microphones is used to collect a 10-second mixed sound signal containing the drone's self-noise and the transformer sound. Figure 3 The time domain waveform diagram represents the mixed sound collected by two microphones. When the short-time Fourier transform is used for the input signal, the Hanning window is used, the sampling rate is 8kHz, the FFT length is 256 points (32ms frame length), and the adjacent windows overlap by 20%, obtaining the frequency domain signal of 6 channels.

[0072] S101: using a microphone array carried by a drone to collect a multi-channel noisy sound signal includes using N microphones carried by the drone to collect sound signals, and receiving N channels of mixed sound signals containing drone self-noise and target sound;

[0073] According to the duration of the sound signal, a suitable window size is selected to intercept the input signal, and then short-time Fourier transform is used for each signal to obtain the frequency domain signals of N channels; the frequency domain mixed sound signal of the nth channel can be expressed as:

[0074] X n (k,l)=S n (k,l)+V n (l,k)

[0075] Among them, s is the target sound signal, v is the drone self-noise, k is the frequency number, and l is the time frame of the signal.

[0076] S200: Use a deep neural network to estimate the ratio mask and perform logarithmic normalization on the frequency domain signal;

[0077] Preferably, in order to accurately estimate the ideal ratio mask, we use a feedforward neural network, which mainly includes an input layer, a hidden layer, and an output layer. The network contains three hidden layers, each of which uses a rectified linear unit and contains 2048 neurons. The output layer uses the structure of an s-type unit to map the input data to the range of [0,1]. To improve the generalization ability of the model, a dropout layer is added to each hidden layer to prevent the model from overfitting by randomly ignoring some neurons.

[0078] S201: Using a deep neural network to estimate the ratio mask includes using an ideal ratio mask to represent the target sound ratio at each time-frequency point. The expression is as follows:

[0079]

[0080] When using a feedforward neural network, it mainly includes an input layer, multiple hidden layers and an output layer. Each hidden layer uses a rectified linear unit to introduce nonlinear transformations when processing data, helping the neural network capture the complex features of the input data;

[0081] The output layer uses an s-type unit structure to map the input data to the range of [0,1]. To improve the generalization ability of the model, a dropout layer is added to each hidden layer to prevent the model from overfitting by randomly ignoring some neurons, thereby accurately estimating the ideal ratio mask.

[0082] Preferably, the logarithmically normalized spectrum is provided to a feedforward neural network in a frame-by-frame manner, and the estimated ratio mask at the lth frame is output. The model parameters are trained using supervised learning and optimized by minimizing the square error between the estimated ratio mask and the ideal ratio mask. The learning rate is 0.02, the batch size is 800, and 80 processes are trained.

[0083] S202: performing logarithmic normalization processing on the frequency domain signal includes logarithmic compression of the frequency domain input signal and then normalizing it to zero mean and unit variance. The expression is as follows:

[0084]

[0085] Among them, μ(k) and σ(k) are the mean and standard deviation of the kth time-frequency point, which are pre-calculated from the training data set. After logarithmic normalization, the scale of the spectrum graph is small, which is conducive to convergence during model training.

[0086] S300: Reshape the sound signal to match the FFT length of the time-frequency spatial filter, and construct the time-frequency spatial filter in combination with a multi-channel ratio mask;

[0087] Preferably, the neural network uses an FFT length of 256 points, and the time-frequency space filter uses an FFT length of 1024 points. The FFT lengths of the two do not match, and the filter cannot be directly constructed, and the sound signal needs to be reshaped. The frequency domain target sound signal S(k1,l1) is obtained according to the estimated ratio mask, and it is converted into a time domain signal S(t) by inverse short-time Fourier transform. Then, S(t) and X(t) are converted into a frequency domain signal S(k2,l1) with a 1024-point FFT length by short-time Fourier transform. 2) and X(k2,l2), re-estimate the ratio mask.

[0088] S301: Reshaping the sound signal to match the FFT length of the time-frequency space filter, and constructing the time-frequency space filter in combination with a multi-channel ratio mask includes the following steps:

[0089] The logarithmically normalized spectrum is provided to the feedforward neural network in a frame-by-frame manner, and the estimated ratio mask M at the lth frame is output. n (1:K,l);

[0090] The model parameters are trained using supervised learning and optimized by minimizing the square error between the estimated ratio mask and the ideal ratio mask. The expression is as follows:

[0091]

[0092] Since the ratio mask estimated by the neural network does not match the FFT length of the time-frequency-space filter, which has a longer FFT length in order to capture the transmission delay, the filter cannot be constructed directly;

[0093] The target sound signal S(k1,l1) in the frequency domain is obtained according to the estimated ratio mask, and is converted into a time domain signal S(t) by inverse short-time Fourier transform. The inverse short-time Fourier transform can be expressed as follows:

[0094]

[0095] Where T represents the FFT length, R is the distance between adjacent frames, ω(t) is the window type, and the parameters are the same as those for short-time Fourier transform;

[0096] The time domain signals of each frame are superimposed according to their positions on the time axis to obtain the time domain signal S(t), and then S(t) and X(t) are converted into the frequency domain signal S(k2,l 2) and X(k2,l2), the ratio mask is re-estimated as follows:

[0097]

[0098] S400: Apply time-frequency-spatial filters to remove drone self-noise and obtain target sound signals;

[0099] Preferably, average pooling is performed on the ratio masks of the six channels to obtain M(k2, l2), and then a spatial matrix Φs reflecting the energy distribution of the target speech signal at each time-frequency point and a mixed sound spatial matrix Φx of the total energy distribution are obtained, and then a time-frequency spatial filter is constructed to obtain the frequency domain target sound signal after filtering out the drone self-noise.

[0100] S401: Applying a time-frequency-space filter to remove the self-noise of the drone and obtain the target sound signal includes the following steps:

[0101] Combine the ratio masks of N channels for average pooling to obtain M(k2,l2), and then obtain the spatial matrix reflecting the energy distribution of the target speech signal at each time-frequency point and the mixed sound spatial matrix of the total energy distribution. The expression is as follows:

[0102]

[0103] Among them, Φ s The spatial matrix representing the energy distribution of the target speech signal at each time-frequency point; Φ x a mixed sound spatial matrix representing the total energy distribution;

[0104] Then construct the time-frequency space filter to obtain the frequency domain target sound signal after filtering out the drone's self-noise:

[0105]

[0106] Among them, Φ s1 Represents Φ s The first column of .

[0107] S500: The frequency domain signal is converted into the time domain signal through the inverse short-time Fourier transform to filter out the self-noise of the drone.

[0108] Preferably, the frequency domain target sound signal obtained by the time-frequency space filter is converted according to the inverse short-time Fourier transform method in step 5 to obtain the transformer sound signal in the time domain (such as Figure 4 As shown in the figure), the self-noise of the UAV is filtered out. Figure 3 This is the waveform of the transformer recording played by the speaker. By comparison, it can be found that this method can effectively filter out the self-noise of the drone.

[0109] S501: Converting the frequency domain signal into the time domain signal through the inverse short-time Fourier transform includes reshaping the frequency domain target sound signal obtained by the time-frequency space filter according to the steps to match the FFT length of the time-frequency space filter by converting it in the inverse short-time Fourier transform mode to obtain the target sound signal in the time domain, thereby filtering out the self-noise of the drone.

[0110] Furthermore, this embodiment also provides a drone self-noise filtering system based on multi-channel time-frequency-space filtering, comprising:

[0111] The acquisition module uses the microphone array carried by the drone to collect multi-channel noisy sound signals, performs short-time Fourier transform on the collected signals, and obtains frequency domain signals;

[0112] The processing module estimates the ratio mask using a deep neural network and performs logarithmic normalization on the frequency domain signal;

[0113] The building module reshapes the sound signal to match the FFT length of the time-frequency-spatial filter and combines the multi-channel ratio mask to build the time-frequency-spatial filter;

[0114] The acquisition module applies a time-frequency-spatial filter to remove the drone’s self-noise and obtain the target sound signal;

[0115] The filtering module converts the frequency domain signal into the time domain signal through inverse short-time Fourier transform to filter out the self-noise of the drone.

[0116] This embodiment also provides a computer device, which is suitable for the case of a drone self-noise filtering method based on multi-channel time-frequency-spatial filtering, and includes a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the drone self-noise filtering method based on multi-channel time-frequency-spatial filtering as proposed in the above embodiment.

[0117] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0118] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for filtering self-noise of a drone based on multi-channel time-frequency-spatial filtering as proposed in the above embodiment is implemented.

[0119] In summary, a multi-channel microphone array is used to collect sound and capture spatial information, laying the foundation for accurate noise filtering. A deep neural network is used to estimate the ratio mask, adaptively learn complex nonlinear features, innovatively reshape the sound signal, and integrate deep learning with traditional signal processing technology to solve the input mismatch problem. A time-frequency space filter is constructed to effectively filter out non-stationary noise and adapt to complex time-varying noise environments. Fully automated processing is achieved to improve the efficiency and accuracy of substation noise monitoring.

[0120] Through the innovative integration of multiple technologies, this method can accurately filter out the self-noise of drones in complex environments, provide reliable acoustic data for substation equipment status monitoring, improve monitoring accuracy and efficiency, and expand the application prospects of drones in power systems.

[0121] Example 2

[0122] Reference Figure 2 - Figure 5 , which is the second embodiment of the present invention, provides a method for filtering self-noise of a UAV based on multi-channel time-frequency-spatial filtering. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0123] In order to verify the effectiveness of the drone self-noise filtering method based on multi-channel time-frequency-spatial filtering, a field test was conducted in a 220kV substation. A running main transformer was selected as the noise source. A quad-rotor drone equipped with a 6-channel microphone array was used for sound collection. The microphone array was arranged in a regular hexagon with a diameter of 0.5 meters.

[0124] First, a pre-recorded standard transformer noise (sound pressure level of 65dB) was played at a distance of 10 meters from the transformer as a reference signal. The drone hovered 3 meters above the ground and 5 meters from the transformer, and the acquisition time was 60 seconds. The sampling rate was set to 16kHz to capture the full spectrum characteristics of the transformer noise.

[0125] The collected raw signal is short-time Fourier transformed, and the time-frequency domain signal is obtained using a Hanning window, a frame length of 512 points (32ms), and a frame shift of 128 points (8ms). Then, a pre-trained deep neural network (containing 3 hidden layers, 2048 neurons in each layer) is used to estimate the ratio mask. The training data of the network contains 100 hours of transformer noise and drone self-noise mixed signals under different working conditions.

[0126] Next, the estimated ratio mask is reshaped to the FFT length (2048 points) that matches the time-frequency spatial filter. The constructed time-frequency spatial filter adopts the minimum variance distortionless response (MVDR) criterion to maximally retain the target signal while suppressing noise.

[0127] In order to improve processing efficiency, a GPU-based parallel computing framework was adopted, using NVIDIA Tesla V100 GPU for acceleration. The average delay of the entire processing process was controlled within 50ms, meeting the requirements of real-time processing.

[0128] Finally, the time domain signal is obtained by inverse short-time Fourier transform and compared with the original mixed signal and the reference signal. At the same time, the traditional adaptive filtering method (NLMS algorithm) and the single-channel deep learning method (U-Net structure) are used as the control group to evaluate the performance advantages of this method.

[0129] The experimental results are shown in the following table:

[0130]

[0131] By analyzing the table data, we can conclude that:

[0132] SNR improvement: This method significantly outperforms other methods in terms of SNR improvement. Compared with the original mixed signal, this method improves by 18.7dB, while NLMS adaptive filtering and single-channel U-Net only improve by 8.3dB and 12.5dB, respectively. This shows that this method has a significant advantage in suppressing UAV self-noise and can better extract the target transformer noise.

[0133] Speech Intelligibility Index (SII): The SII of this method reaches 0.85, which is much higher than 0.45 of the original mixed signal, and is also significantly better than NLMS (0.62) and single-channel U-Net (0.71). SII is an important indicator for measuring sound intelligibility. A high SII means that the processed signal is closer to the original transformer noise, which is conducive to subsequent fault diagnosis and analysis.

[0134] Computation time: Although this method involves complex deep learning and time-frequency-spatial filtering, the processing time per frame is controlled within 25ms through GPU acceleration, which is only 10ms more than NLMS and 5ms faster than single-channel U-Net. This shows that this method can meet the requirements of real-time processing while ensuring high-quality noise reduction, and is suitable for scenarios with high real-time requirements such as drone on-site inspections.

[0135] Transformer characteristic frequency retention rate: The retention rate of this method is as high as 98%, which almost completely retains the characteristic frequency of the transformer noise. In contrast, NLMS and single-channel U-Net only retain 85% and 90%, respectively. This indicator is crucial for transformer fault diagnosis because some faults (such as loose windings, core faults, etc.) will produce abnormal harmonics at specific frequencies. The high retention rate means that this method can capture these fault characteristics more accurately and improve the accuracy of diagnosis.

[0136] Versatility: This method is not only applicable to transformer noise, but can also be extended to noise monitoring of other power equipment, such as circuit breakers, transformers, etc. This versatility greatly expands the scope of application of drones in power systems.

[0137] In summary, the UAV self-noise filtering method based on multi-channel time-frequency-spatial filtering is significantly superior to traditional methods and single-channel deep learning methods in terms of signal-to-noise ratio improvement, speech clarity, computational efficiency, and feature retention. This performance advantage stems from the innovative integration of multi-channel signal processing, deep learning, and time-frequency-spatial filtering, providing an efficient and accurate solution for UAV acoustic monitoring of power equipment. This method not only improves the quality and efficiency of monitoring, but also provides a more reliable data basis for equipment status assessment and fault diagnosis, and has important engineering application value and economic benefits.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for filtering self-noise of unmanned aerial vehicles based on multi-channel time-frequency-spatial filtering, characterized in that: include, The drone is equipped with a microphone array to collect multi-channel noisy sound signals, and the collected signals are short-time Fourier transformed to obtain frequency domain signals. A ratio mask is estimated using a deep neural network to log-normalize the frequency domain signal. Reshape the sound signal to match the FFT length of the time-frequency-spatial filter, and construct the time-frequency-spatial filter in combination with a multi-channel ratio mask; Apply time-frequency-spatial filters to remove drone self-noise and obtain target sound signals; The frequency domain signal is converted into the time domain signal through inverse short-time Fourier transform to filter out the self-noise of the drone.

2. The method for filtering self-noise of unmanned aerial vehicle based on multi-channel time-frequency-spatial filtering as claimed in claim 1, characterized in that: The method of collecting a multi-channel noisy sound signal using a microphone array carried by a drone includes collecting sound signals using N microphones carried by the drone, and receiving a mixed sound signal of N channels containing the drone's self-noise and the target sound; According to the duration of the sound signal, a suitable window size is selected to intercept the input signal, and then short-time Fourier transform is used for each signal to obtain the frequency domain signals of N channels; the frequency domain mixed sound signal of the nth channel can be expressed as: X n (k,l)=S n (k,l)+V n (l,k) Among them, s is the target sound signal, v is the drone self-noise, k is the frequency number, and l is the time frame of the signal.

3. The method for filtering self-noise of unmanned aerial vehicle based on multi-channel time-frequency-spatial filtering as claimed in claim 2, characterized in that: The method of estimating the ratio mask using a deep neural network includes using an ideal ratio mask to represent the target sound ratio at each time-frequency point. The expression is as follows: When using a feedforward neural network, it mainly includes an input layer, multiple hidden layers and an output layer. Each hidden layer uses a rectified linear unit to introduce nonlinear transformations when processing data, helping the neural network capture the complex features of the input data; The output layer uses an s-type unit structure to map the input data to the range of [0,1]. To improve the generalization ability of the model, a dropout layer is added to each hidden layer to prevent the model from overfitting by randomly ignoring some neurons, thereby accurately estimating the ideal ratio mask.

4. The method for filtering self-noise of unmanned aerial vehicle based on multi-channel time-frequency-spatial filtering as claimed in claim 3, characterized in that: The logarithmic normalization processing of the frequency domain signal includes logarithmic compression of the frequency domain input signal and then normalization to zero mean and unit variance. The expression is as follows: Among them, μ(k) and σ(k) are the mean and standard deviation of the kth time-frequency point, which are pre-calculated from the training data set. After logarithmic normalization, the scale of the spectrum graph is small, which is conducive to convergence during model training.

5. The method for filtering self-noise of unmanned aerial vehicle based on multi-channel time-frequency-spatial filtering as claimed in claim 4, characterized in that: The reshaping of the sound signal to match the FFT length of the time-frequency space filter and constructing the time-frequency space filter in combination with a multi-channel ratio mask comprises the following steps: The logarithmically normalized spectrum is provided to the feedforward neural network in a frame-by-frame manner, and the estimated ratio mask M at the lth frame is output. n (1:K,l); The model parameters are trained using supervised learning and optimized by minimizing the square error between the estimated ratio mask and the ideal ratio mask. The expression is as follows: Since the ratio mask estimated by the neural network does not match the FFT length of the time-frequency-space filter, which has a longer FFT length in order to capture the transmission delay, the filter cannot be constructed directly; The target sound signal S(k1,l1) in the frequency domain is obtained according to the estimated ratio mask, and is converted into a time domain signal S(t) by inverse short-time Fourier transform. The inverse short-time Fourier transform can be expressed as follows: Where T represents the FFT length, R is the distance between adjacent frames, ω(t) is the window type, and the parameters are the same as those for short-time Fourier transform; The time domain signals of each frame are superimposed according to their positions on the time axis to obtain the time domain signal S(t), and then S(t) and X(t) are converted into the frequency domain signal S(k2,l 2) and X(k2,l2), the ratio mask is re-estimated as follows:

6. The method for filtering self-noise of unmanned aerial vehicle based on multi-channel time-frequency-spatial filtering as claimed in claim 5, characterized in that: The method of applying a time-frequency-space filter to remove the self-noise of the drone and obtain the target sound signal comprises the following steps: Combine the ratio masks of N channels for average pooling to obtain M(k2,l2), and then obtain the spatial matrix reflecting the energy distribution of the target speech signal at each time-frequency point and the mixed sound spatial matrix of the total energy distribution. The expression is as follows: Among them, Φ s The spatial matrix representing the energy distribution of the target speech signal at each time-frequency point; Φ x a mixed sound spatial matrix representing the total energy distribution; Then construct the time-frequency space filter to obtain the frequency domain target sound signal after filtering out the drone's self-noise: Among them, Φ s1 Represents Φ s The first column of .

7. The method for filtering self-noise of unmanned aerial vehicle based on multi-channel time-frequency-spatial filtering as claimed in claim 6, characterized in that: The method of converting the frequency domain signal into the time domain signal by inverse short-time Fourier transform includes: reshaping the frequency domain target sound signal obtained by the time-frequency space filter in steps to match the FFT length of the time-frequency space filter by inverse short-time Fourier transform, thereby obtaining the target sound signal in the time domain, and filtering out the self-noise of the drone.

8. A UAV self-noise filtering system based on multi-channel time-frequency-spatial filtering, based on the UAV self-noise filtering method based on multi-channel time-frequency-spatial filtering according to any one of claims 1 to 7, characterized in that: Also includes, The acquisition module uses the microphone array carried by the drone to collect multi-channel noisy sound signals, performs short-time Fourier transform on the collected signals, and obtains frequency domain signals; The processing module estimates the ratio mask using a deep neural network and performs logarithmic normalization on the frequency domain signal; The building module reshapes the sound signal to match the FFT length of the time-frequency-spatial filter and combines the multi-channel ratio mask to build the time-frequency-spatial filter; The acquisition module applies a time-frequency-spatial filter to remove the drone’s self-noise and obtain the target sound signal; The filtering module converts the frequency domain signal into the time domain signal through inverse short-time Fourier transform to filter out the self-noise of the drone.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for removing self-noise of a drone based on multi-channel time-frequency-spatial filtering are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for removing self-noise of a drone based on multi-channel time-frequency-spatial filtering as described in any one of claims 1 to 7 are implemented.

Images