Method and system for enhancing target sound collected by unmanned aerial vehicle

Abstract

The invention discloses a method and a system for enhancing target sound collected by an unmanned aerial vehicle, and the method comprises the steps: obtaining a first target transformer sound mixed signal through a sound obtaining device array carried by the unmanned aerial vehicle, and carrying out the preprocessing of the first target transformer sound mixed signal, the first target transformer sound mixed signal at least comprises a target sound signal; performing signal separation on the pre-processed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and obtaining an aligned third target transformer sound mixed signal in combination with a preset first-stage arrangement alignment strategy and a second-stage arrangement alignment strategy; and filtering the third target transformer sound mixed signal according to a preset filtering strategy to obtain a target sound signal. According to the method and the device, the definition and the recognizability of the target sound signal are improved, and the method and the device have relatively high reliability and stability in practical application.

Description

Technical Field

[0001] The present invention relates to the technical field of drone target sound enhancement, and in particular to a drone target sound enhancement method and system. Background Art

[0002] In the daily inspection of power transformers, the collection and analysis of sound signals is an important means to identify internal faults of transformers. Traditional ground microphone arrays cannot accurately capture the subtle sounds inside the transformer due to the long distance and large interference. With the advancement of multi-rotor drone technology, it is possible to collect transformer sound signals at close range by using drone-mounted microphone arrays. However, drones will generate significant self-noise during flight, mainly from the mechanical vibration and aerodynamic noise of their rotors. Since this self-noise has a wide frequency range, high intensity, and overlaps with the frequency range of the target sound, it is a huge challenge to achieve high-precision sound collection and sound source localization on drones.

[0003] Traditional noise suppression methods face many challenges when dealing with drone self-noise. First, the rotor noise of drones is high-intensity, wide-bandwidth, and has obvious non-stationarity, which makes it difficult for conventional noise suppression techniques to effectively handle. Usually, traditional methods rely on simple filtering or adaptive filtering techniques, which perform well in low-noise environments, but often seem powerless when faced with the complex noise generated by drone rotors. In addition, during the flight of a drone, its noise will also change significantly with changes in position and attitude, further increasing the difficulty of noise suppression.

[0004] In complex industrial environments such as transformer testing, there are many types of background noise sources with varying intensities, such as electromagnetic noise from high-voltage currents and mechanical noise from equipment operation, which pose a huge challenge to the extraction of target sounds. Traditional methods are difficult to effectively distinguish these background noises from target sounds, resulting in a large amount of noise components remaining in the extracted signals, thereby reducing the reliability and accuracy of the detection results.

[0005] Therefore, in complex industrial environments, especially in transformer detection scenarios, there is an urgent need for an innovative framework that can comprehensively use multiple signal processing technologies to effectively reduce the drone's own noise while enhancing the target sound signal. This framework is not only required to be able to adapt to complex noise characteristics, but also to have high real-time processing capabilities to cope with the noise changes of drones in dynamic environments. Through this comprehensive signal processing framework, the quality of drone sound collection can be significantly improved, providing reliable sound signal support for the accurate detection of transformer internal faults. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the above existing problems, the present invention is proposed.

[0008] Therefore, the present invention provides a method and system for enhancing target sound collected by a drone, which can solve the problems mentioned in the background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] In a first aspect, the present invention provides a method for enhancing target sound collected by an unmanned aerial vehicle, comprising:

[0011] Acquire a first target transformer sound mixed signal through a sound acquisition device array carried by the unmanned aerial vehicle, and pre-process the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal;

[0012] Performing signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and acquiring an aligned third target transformer sound mixed signal in combination with a preset first-stage alignment strategy and a second-stage alignment strategy;

[0013] The third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain a target sound signal.

[0014] As a preferred solution of the target sound enhancement method for drone acquisition of the present invention, the step of obtaining the first target transformer sound mixed signal and preprocessing the first target transformer sound mixed signal comprises:

[0015] The mixed signal of the first target transformer sound collected by the M sound acquisition device array carried by the UAV within the transformer target distance range is expressed as X(t)=[x1(t),x2(t),…,x M (t)] T ;

[0016] The first target transformer sound mixed signal is preprocessed, and the preprocessing includes high-pass filtering and normalization, removing the signal with a frequency lower than the first frequency threshold in the first target transformer sound mixed signal, and stabilizing the signal amplitude. The mathematical model of the first target transformer sound mixed signal is:

[0017] X(t)=A·S(t)+N(t)

[0018] Among them, A is the mixing coefficient matrix, S(t) is the target sound signal, and N(t) is the self-noise signal of the drone, that is, all the first target transformer sound mixed signals can be obtained by combining the mixing coefficient matrix, the target sound signal and the self-noise signal of the drone.

[0019] As a preferred solution of the target sound enhancement method collected by the drone of the present invention, the signal separation of the preprocessed first target transformer sound mixed signal in combination with the preset blind source separation strategy includes:

[0020] Use the independent component analysis method to separate the signal on each frequency band and solve the separation matrix W to obtain the separated signal:

[0021] Y(t)=W·X(t)

[0022] The separation result Y(t) contains several signal components, and the signal components include the target sound signal and the self-noise signal of the drone.

[0023] As a preferred solution of the target sound enhancement method collected by the drone of the present invention, the method of combining the preset first-stage arrangement and alignment strategy and the second-stage arrangement and alignment strategy to obtain the aligned third target transformer sound mixed signal includes:

[0024] The first stage alignment strategy includes spatial filtering based on time-frequency signal processing, using the time-frequency spatial filtering algorithm to spatially enhance the sound in the direction of the target signal, and defining the spatial filter h(θ) as:

[0025]

[0026] Where a(θ) is the directional vector of the sound acquisition device array, θ represents the angle parameter, which is used to represent the direction or position of the sound source, and a H (θ) represents the conjugate transposed form of the directivity vector of the sound acquisition device array at a given direction θ.

[0027] As a preferred solution of the target sound enhancement method collected by the drone of the present invention, the method of combining the preset first-stage arrangement and alignment strategy and the second-stage arrangement and alignment strategy to obtain the aligned third target transformer sound mixed signal also includes:

[0028] The second stage alignment strategy includes cross-frequency alignment, using the signal output by the time-frequency spatial filtering algorithm as a reference to align the frequency components separated by independent components across frequencies. The aligned signals are verified by a clustering algorithm and grouped based on time activity:

[0029] C i ={f j ∣Activity(y ij (t))>τ}

[0030] Among them, C t represents the classification set at time t, which contains all frequency components that meet the activity threshold condition, f j represents the jth frequency component, Activity(y ij (t)) represents the activity value of the jth frequency component received by the i-th sound acquisition device at time t, τ represents the activity threshold, and when Activity(y ij When (t))>τ, f j Join C t middle.

[0031] As a preferred solution of the target sound enhancement method collected by the drone of the present invention, the third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain the target sound signal, including: using an improved time-frequency mask to further suppress the noise in the arranged aligned signal, and performing single-channel spectrum post-filtering on the suppressed arranged aligned signal.

[0032] As a preferred solution of the target sound enhancement method for drone acquisition of the present invention, wherein: filtering the third target transformer sound mixed signal according to a preset filtering strategy to obtain the target sound signal also includes:

[0033] Perform inverse short-time Fourier transform on the frequency domain signal after single-channel spectrum post-filtering to reconstruct the time domain signal;

[0034] The signal after reconstructing the time domain signal processing is smoothed by using overlapping windowing method to eliminate the discontinuity of the time domain signal and output the smoothed time domain signal.

[0035] In a second aspect, the present invention provides a drone target sound enhancement system, comprising:

[0036] A sound source acquisition and processing module, used to acquire a first target transformer sound mixed signal through a sound acquisition device array carried by the UAV, and pre-process the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal;

[0037] A signal separation and recombining module, used to perform signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and to obtain an aligned third target transformer sound mixed signal in combination with a preset first-stage alignment strategy and a second-stage alignment strategy;

[0038] The filtering module is used to filter the third target transformer sound mixed signal according to a preset filtering strategy to obtain a target sound signal.

[0039] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method when executing the computer program.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method described above when executed by a processor.

[0041] Compared with the prior art, the invention has the following beneficial effects: the invention proposes a method and system for enhancing target sound collected by a drone, wherein a first target transformer sound mixed signal is obtained through a sound acquisition device array carried by the drone, and the first target transformer sound mixed signal is preprocessed, wherein the first target transformer sound mixed signal at least includes a target sound signal; the first target transformer sound mixed signal after preprocessing is separated by combining a preset blind source separation strategy, and a third target transformer sound mixed signal that has been arranged and aligned is obtained by combining a preset first-stage arrangement and alignment strategy and a preset second-stage arrangement and alignment strategy; the third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain a target sound signal. By combining an independent component analysis method and a preset blind source separation strategy, the target sound signal and the drone's self-noise signal can be effectively separated, thereby improving the purity of the target sound signal. By using a spatial filtering technology based on time-frequency signal processing, the sound in the direction of the target signal is enhanced, and the signal-to-noise ratio of the target sound signal is further improved. By using a cross-frequency arrangement and alignment strategy, the frequency components after the independent component separation are aligned, ensuring the coherence of the target sound signal at each frequency, thereby improving the comprehensibility of the signal. The use of improved time-frequency masking technology further suppresses the noise in the aligned signal, making the target sound signal clearer. The overlapping windowing method is used for smoothing, eliminating the discontinuity of the time domain signal, making the reconstructed time domain signal smoother, and improving the audibility of the target sound signal. The system structure of the present invention is simple, easy to implement, and has high practical value and promotion prospects.

[0042] In summary, the method and system for enhancing target sound collected by drones of the present invention not only improve the clarity and recognizability of target sound signals, but also have high reliability and stability in practical applications. The technical solution of the present invention can be widely used in the target sound collection and processing of drones in complex noise environments, and has important application value and market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0044] Figure 1 A method flow chart of a method and system for enhancing target sound collected by a drone provided by one embodiment of the present invention;

[0045] Figure 2 A flowchart of a method and system for enhancing target sound collected by a drone provided by one embodiment of the present invention;

[0046] Figure 3 An internal structural diagram of a computer device of a method and system for enhancing target sound collected by a drone provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0048] Example 1

[0049] Reference Figure 1-Figure 3 , which is the first embodiment of the present invention, provides a method and system for enhancing target sound collected by a drone, including:

[0050] Before describing the embodiments of the present application in detail, some related concepts are first explained for the sake of clarity.

[0051] Blind source separation: Blind source separation is a signal processing technique that aims to recover the original, unknown signal source from multiple mixed signals. In this case, it is unknown how the signals are mixed, nor is the exact characteristics of the original signal. Blind source separation technology does not require knowledge of the specific details of the mixing process, but relies on the statistical properties of the signals themselves and their independence from each other for separation. Blind source separation is used to separate the target sound signal and the drone's own noise signal from the mixed sound signal collected by the drone. The independent component analysis (ICA) method is used here. ICA is a common blind source separation technique that assumes that different source signals are statistically independent and tries to find a set of transformation matrices to make the mixed signal as close to the original source signal as possible.

[0052] Time-Frequency Masking: Time-frequency masking is a technique used in signal processing that selectively enhances or suppresses a portion of a signal in the time-frequency domain. Time-frequency masking is commonly used in noise suppression and speech enhancement scenarios. It works by creating a mask by comparing the amplitude of the target signal with the background noise, and then applying this mask to the mixed signal to enhance the target signal and reduce the effects of the noise. Time-frequency masking is used to further suppress the noise component in the already aligned signal. This can be achieved by calculating the relative strength of the target signal and the noise signal at each time-frequency point, and then adjusting the amplitude of the signal based on this ratio.

[0053] Inverse Short-Time Fourier Transform: Inverse Short-Time Fourier Transform is the inverse transform of Short-Time Fourier Transform (STFT), which is used to convert frequency domain signals back to time domain signals. In signal processing, signals are often converted to frequency domain for various processing, such as filtering, denoising, etc., and then the processed frequency domain signal is converted back to time domain signal using ISTFT for further analysis or playback. Inverse Short-Time Fourier Transform is used to convert the signal processed in frequency domain back to time domain signal in order to reconstruct the enhanced original target sound signal.

[0054] There are some problems in the related technology, such as poor acquisition and enhancement of target sound signals in complex noise environments, which leads to reduced clarity and recognizability of target sound signals. In addition, existing technical solutions often require complex algorithms and a large amount of computing resources when processing signals, which limits their promotion and application in practical applications.

[0055] This application provides a method that can effectively solve the above-mentioned problems. Next, we will combine multiple embodiments to explain in detail how to implement the method for enhancing the target sound collected by the drone;

[0056] Figure 1A method flow chart of a method and system for enhancing target sound collected by a drone is shown, including:

[0057] S101, obtaining a first target transformer sound mixed signal through a sound acquisition device array carried by a drone, and preprocessing the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal;

[0058] In an embodiment of the present application, obtaining a first target transformer sound mixed signal and preprocessing the first target transformer sound mixed signal includes:

[0059] The mixed signal of the first target transformer sound collected by the M sound acquisition device array carried by the UAV within the transformer target distance range is expressed as X(t)=[x1(t),x2(t),…,x M (t)] T ;

[0060] The first target transformer sound mixed signal is preprocessed, and the preprocessing includes high-pass filtering and normalization, removing the signal with a frequency lower than the first frequency threshold in the first target transformer sound mixed signal, and stabilizing the signal amplitude. The mathematical model of the first target transformer sound mixed signal is:

[0061] X(t)=A·S(t)+N(t)

[0062] Among them, A is the mixing coefficient matrix, S(t) is the target sound signal, and N(t) is the self-noise signal of the drone, that is, all the first target transformer sound mixed signals can be obtained by combining the mixing coefficient matrix, the target sound signal and the self-noise signal of the drone.

[0063] In the embodiments of the present application, the sound acquisition device uses a microphone. Relevant technicians can use other types of sound acquisition devices, such as ultrasonic sensors, laser microphones, etc., to adapt to different collection environments and requirements.

[0064] In the embodiment of the present application, the determination of the target distance range is mainly to ensure that the drone can receive a sufficiently strong target sound signal while minimizing noise interference from far away. This usually involves two main factors: the strength of the sound source and the sensitivity of the drone microphone array. The following method is used in this application to obtain the target distance range:

[0065] Step 1: Measure the sound source intensity. Use a sound level meter or other measuring tool to determine the sound intensity (sound pressure level) emitted by the target transformer. Record the maximum sound pressure level P of the sound source. max .

[0066] Step 2: Determine the sensitivity of the microphone array. Check the technical specifications of the microphone array on the drone to obtain its sensitivity threshold S. threshold .

[0067] Step 3: Calculate the maximum allowable distance. Calculate the maximum allowable distance D from the sound source to the microphone array based on the free space propagation loss formula. max :

[0068]

[0069] Step 4: Determine the actual operating range, which should be smaller than D max Considering the maneuverability and safety of UAVs, D is generally used. max A certain proportion of the actual operating range D op :D op =α·D max , where 0<α<1, in this application, α is 0.8, and choosing α=0.8 means the actual operating distance D op will be less than the maximum allowable distance D max 80% of the total distance. This helps ensure that even in the worst case, the signal strength from the sound source to the microphone array is still strong enough to ensure good signal quality. As the distance decreases, the strength of the sound source signal relative to the background noise increases, thereby improving the signal-to-noise ratio (SNR). A higher SNR is beneficial for subsequent sound processing and enhancement. Retaining a certain safety margin in actual operation can reduce the safety risks caused by the drone being too close to the sound source due to unexpected situations.

[0070] In an embodiment of the present application, the first frequency threshold is determined by analyzing the spectrum of the target sound signal using a spectrum analysis tool to determine the main frequency range of the target sound signal. For example, if the target sound signal is mainly concentrated above 100 Hz, 100 Hz can be selected as the reference frequency. Considering the spectrum of the drone's self-noise, the spectrum of the drone's self-noise is analyzed to determine the main noise frequency range. If the drone's self-noise is mainly concentrated below 50 Hz, then selecting a frequency higher than 50 Hz as the threshold can effectively filter out most of the self-noise. Step 3: Comprehensively consider and set the threshold, and combine the above analysis results to select a frequency that can effectively filter out low-frequency noise without affecting the target sound signal as the first frequency threshold. For example, if the main frequency of the target sound signal is above 100 Hz, and the drone's self-noise is mainly concentrated below 50 Hz, a frequency of 60 Hz or higher can be selected as the first frequency threshold.

[0071] It should be noted that the benefit of step S101 is that it can effectively separate the target sound signal from the mixed signal and reduce the influence of the drone's own noise. Through high-pass filtering and normalization processing, it can ensure that the frequency components of the target sound signal are retained while suppressing low-frequency noise, thereby improving the clarity and recognizability of the signal.

[0072] S102, performing signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and obtaining an aligned third target transformer sound mixed signal in combination with a preset first-stage alignment strategy and a second-stage alignment strategy;

[0073] In an embodiment of the present application, performing signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy includes:

[0074] Use the independent component analysis method to separate the signal on each frequency band and solve the separation matrix W to obtain the separated signal:

[0075] Y(t)=W·X(t)

[0076] The separation result Y(t) contains several signal components, and the signal components include the target sound signal and the self-noise signal of the drone.

[0077] In the embodiment of the present application, combining the preset first-stage alignment strategy and the second-stage alignment strategy to obtain the aligned third target transformer sound mixed signal includes:

[0078] The first stage alignment strategy includes spatial filtering based on time-frequency signal processing. The time-frequency spatial filtering algorithm is used to spatially enhance the sound in the direction of the target signal. The spatial filter h(θ) is defined as:

[0079]

[0080] Where a(θ) is the directional vector of the sound acquisition device array, θ represents the angle parameter, which is used to represent the direction or position of the sound source, and a H (θ) represents the conjugate transposed form of the directivity vector of the sound acquisition device array at a given direction θ.

[0081] In the embodiment of the present application, combining the preset first-stage alignment strategy and the second-stage alignment strategy to obtain the aligned third target transformer sound mixed signal also includes:

[0082] The second stage alignment strategy includes cross-frequency alignment. The signal output by the time-frequency spatial filtering algorithm is used as a reference to align the frequency components separated by independent components across frequencies. The aligned signals are verified by the clustering algorithm and grouped based on time activity:

[0083] C i ={f j ∣Activity(y ij (t))>τ}

[0084] Among them, C t represents the classification set at time t, which contains all frequency components that meet the activity threshold condition, f j represents the jth frequency component, Activity(y ij (t)) represents the activity value of the jth frequency component received by the i-th sound acquisition device at time t, τ represents the activity threshold, and when Activity(y ij When (t))>τ, f j Join C t middle.

[0085] It should be noted that the benefit of S102 is that it can effectively separate the target sound signal from the mixed signal and reduce the influence of the drone's own noise. By combining the preset blind source separation strategy and the arrangement alignment strategy, it can ensure that the frequency components of the target sound signal are retained while suppressing low-frequency noise, thereby improving the clarity and recognizability of the signal.

[0086] S103, filtering the third target transformer sound mixed signal according to a preset filtering strategy to obtain a target sound signal.

[0087] In an embodiment of the present application, the third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain the target sound signal, including: using an improved time-frequency mask to further suppress noise in the arranged aligned signal, and performing single-channel spectrum post-filtering on the suppressed arranged aligned signal.

[0088] In the embodiment of the present application, an improved time-frequency mask is used to further suppress the noise in the aligned signal. The calculation of the improved time-frequency mask M(t,f) needs to consider the relative strength of the target signal and the noise signal at each time-frequency point. Some additional factors are introduced to optimize the calculation of the mask: the strongest component of the aligned signal s″(t,f) is used as the estimate of the target signal. Subtract the target signal estimate from the aligned signal s″(t,f) to get the noise signal estimate A weighting factor β is introduced to balance the contribution of the target signal and the noise signal. A smoothing factor δ is introduced to avoid zero division errors and numerical instability.

[0089] The calculation of the improved time-frequency mask M(t,f) can be expressed as:

[0090]

[0091] in, is the estimated amplitude of the target signal at time t and frequency f. is the estimated amplitude of the noise signal at time t and frequency f. β is a constant used to balance the contribution of the target signal and the noise signal, and its value is 1. δ is a small positive number used to avoid zero division errors, and its value is 10 -10 or smaller.

[0092] Once the improved time-frequency mask M(t,f) is obtained, it can be applied to the aligned signal s''(t) to further suppress the noise. This can be done with a simple multiplication operation:

[0093] s''''(t,f)=M(t,f)·s''(t,f)

[0094] Among them, s''''(t,f) is the signal after noise suppression, and M(t,f) corresponds to the mask value at a specific time point t and frequency point f. The improved mask can effectively filter out self-noise and retain the target sound.

[0095] Furthermore, the signal after arrangement is subjected to single-channel spectral post-filtering to further improve the signal quality through the form of Wiener filter:

[0096]

[0097] The filtered signal is s'''''′(t,f):

[0098] s'''''(t,f)=s''''(t,f)·H(f)

[0099] It should be noted that by improving the time-frequency mask and single-channel spectrum post-filtering, the purity and recognizability of the target sound signal can be further improved. The introduction of the Wiener filter can effectively suppress the residual noise while retaining the important features of the target sound signal. Through this filtering strategy, it can be ensured that the target sound signal can still be accurately extracted and recognized in a complex background noise environment.

[0100] In the embodiment of the present application, an improved Wiener filter is designed to distinguish non-speech segments from speech segments. The specific expression is as follows:

[0101]

[0102] in, It is an estimate of the clean speech spectrum. α is an adjustable coefficient used to adjust the degree of noise suppression by the filter. When VAD(k) = 0, that is, in the non-speech segment, the gain is directly set to 0 to avoid filtering the noise. The noise spectrum update rule in spectral subtraction is adopted, for example:

[0103]

[0104] Where X(f,k) is the observed spectrum of the current frame. β is the attenuation factor, which is usually small to smooth the noise spectrum estimate.

[0105] In the embodiment of the present application, filtering the third target transformer sound mixed signal according to a preset filtering strategy, and obtaining the target sound signal further includes:

[0106] Perform inverse short-time Fourier transform on the frequency domain signal after single-channel spectrum post-filtering to reconstruct the time domain signal;

[0107] The signal after reconstructing the time domain signal processing is smoothed by using overlapping windowing method to eliminate the discontinuity of the time domain signal and output the smoothed time domain signal.

[0108] In the embodiment of the present application, the processed frequency domain signal is subjected to an inverse short-time Fourier transform to reconstruct the time domain signal:

[0109]

[0110] Among them, S m (f,k) is the frequency domain signal of the mth channel, and Δt is the time frame interval.

[0111] The processed signal is smoothed using the overlapping windowing method to eliminate the discontinuity of the time domain signal:

[0112]

[0113] Here, w(t) is the window function used for smoothing.

[0114] It should be noted that the window function can also select a Hamming window, a Hanning window or other suitable window functions to ensure a smooth transition of the signal in the time domain. By selecting a suitable window function, the Gibbs phenomenon in the reconstructed signal, that is, the ringing effect that may occur at the signal mutation point, can be effectively reduced.

[0115] After reconstructing the time domain signal, the purpose of smoothing is to eliminate any discontinuities introduced by frequency domain filtering, thereby obtaining a smoother and more continuous time domain signal. The smoothed signal can better reflect the real sound signal and reduce the distortion introduced by signal processing.

[0116] Finally, the smoothed time domain signal can be used for further analysis and processing. For example, the signal can be used in a sound recognition algorithm to identify and classify the target sound signal. In addition, the signal can also be used for sound signal monitoring and diagnosis, such as real-time monitoring of equipment operating status in an industrial environment.

[0117] In summary, by combining the preset blind source separation strategy, the arrangement alignment strategy, the improved time-frequency mask and the single-channel spectrum post-filtering, as well as the inverse short-time Fourier transform and the overlapping windowing method, the embodiment of the present application provides an efficient and accurate method for extracting and identifying the target sound signal. This method can not only effectively suppress noise, but also retain the important features of the target sound signal, thereby improving the clarity and recognizability of the signal. This has important practical application value for extracting and identifying sound signals in a complex background noise environment.

[0118] like Figure 2 As shown, the entire workflow diagram of this application is very detailed through Figure 2 First, the input multi-channel speech signal (x1, x2, ..., x M ) are subjected to time-frequency transformation, such as short-time Fourier transform (STFT). Time-frequency transformation converts time domain signals into frequency domain signals for subsequent processing. The signals after time-frequency transformation are sent to the independent component analysis module to separate potential independent source signals. Independent component analysis is a blind source separation technique that can decompose a mixed signal into a series of independent source signals. In this example, Y1,Y2,...,Y M , which are independent stars after ICA processing. Then, the time-frequency filter will be based on a certain directional parameter θ d The independent components are filtered by the time-frequency filter (which may be an angle or direction). This filter may be used to increase the sound signal from a specific direction, such as the sound from the target speaker. The independent components after time-frequency filtering are combined in a certain order to form a new signal This combination may take into account the correlation between channels or other factors. Finally, each independent component is sent to a single-channel noise suppression module, which may use some algorithms (such as Wiener filter) to further reduce the noise. The noise suppression results for each channel are All single-channel noise suppression results are combined to generate the final output signals Y1, Y2, ..., Y M .

[0119] In summary, the present invention proposes a method for enhancing target sound collected by a drone, wherein a first target transformer sound mixed signal is obtained through an array of sound acquisition devices carried by the drone, and the first target transformer sound mixed signal is preprocessed, wherein the first target transformer sound mixed signal at least includes a target sound signal; the preprocessed first target transformer sound mixed signal is signal separated in combination with a preset blind source separation strategy, and the aligned third target transformer sound mixed signal is obtained in combination with a preset first-stage arrangement alignment strategy and a second-stage arrangement alignment strategy; the third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain the target sound signal. By combining the independent component analysis method and the preset blind source separation strategy, the target sound signal and the drone's self-noise signal can be effectively separated, thereby improving the purity of the target sound signal. By using the spatial filtering technology based on time-frequency signal processing, the sound in the direction of the target signal is enhanced, and the signal-to-noise ratio of the target sound signal is further improved. By using the cross-frequency arrangement alignment strategy, the frequency components after the independent component separation are aligned to ensure the coherence of the target sound signal at each frequency, thereby improving the comprehensibility of the signal. The use of improved time-frequency masking technology further suppresses the noise in the aligned signal, making the target sound signal clearer. The overlapping windowing method is used for smoothing, eliminating the discontinuity of the time domain signal, making the reconstructed time domain signal smoother, and improving the audibility of the target sound signal. The system structure of the present invention is simple, easy to implement, and has high practical value and promotion prospects.

[0120] In summary, the method and system for enhancing target sound collected by drones of the present invention not only improve the clarity and recognizability of target sound signals, but also have high reliability and stability in practical applications. The technical solution of the present invention can be widely used in the target sound collection and processing of drones in complex noise environments, and has important application value and market prospects.

[0121] This embodiment also provides a drone target sound enhancement system, including:

[0122] A sound source acquisition and processing module, used to acquire a first target transformer sound mixed signal through a sound acquisition device array carried by the UAV, and pre-process the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal;

[0123] A signal separation and recombining module is used to perform signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and to obtain an aligned third target transformer sound mixed signal in combination with a preset first-stage alignment strategy and a second-stage alignment strategy;

[0124] The filtering module is used to filter the third target transformer sound mixed signal according to a preset filtering strategy to obtain a target sound signal.

[0125] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0126] This embodiment also provides a computer device, which may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 3 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for enhancing the sound of a target collected by a drone is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0127] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0128] Acquire a first target transformer sound mixed signal through a sound acquisition device array carried by the UAV, and pre-process the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal;

[0129] Combine the preset blind source separation strategy to perform signal separation on the preprocessed first target transformer sound mixed signal, and combine the preset first-stage alignment strategy and the second-stage alignment strategy to obtain the aligned third target transformer sound mixed signal;

[0130] The third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain a target sound signal.

[0131] Example 2

[0132] Reference Figure 1-Figure 3, which is an embodiment of the present invention, provides a method and system for enhancing target sound collected by a drone. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0133] The mixed signal collected by the microphone array (e.g., M microphones) carried by the drone near the transformer is expressed as X(t) = [x1(t), x2(t), …, x M (t)] T The mixed signal is preprocessed, including high-pass filtering and normalization, to remove low-frequency interference and stabilize the signal amplitude.

[0134] The independent component analysis method is used to separate the signal on each frequency band, and the separation matrix W is solved to obtain the separated signal.

[0135] Since independent component analysis may produce arrangement ambiguity problems in different frequency bands, firstly, based on spatial filtering of time-frequency signal processing, the time-frequency spatial filtering algorithm is used to spatially enhance the sound in the direction of the target signal. Secondly, cross-frequency alignment is performed, and the signal output by the time-frequency spatial filtering algorithm is used as a reference to align the frequency components separated by independent components across frequencies. The aligned signal is verified by the clustering algorithm.

[0136] The noise in the aligned signal is further suppressed using an improved time-frequency masker.

[0137] The arranged signal is subjected to single-channel spectral post-filtering and the signal quality is further improved by using a Wiener filter.

[0138] The processed frequency domain signal is subjected to inverse short-time Fourier transform to reconstruct the time domain signal. The processed signal is smoothed using overlapping windowing method to eliminate the discontinuity of the time domain signal.

[0139] Finally, a smoothed time domain signal is output, which has enhanced target sound characteristics and is suitable for applications in complex industrial environments.

[0140] During a certain experiment, the experimental results are shown in Table 1;

[0141] Table 1: Experimental results.

[0142]

[0143]

[0144] By further analyzing the experimental results (Table 1), the performance of the drone acquisition target sound enhancement system can be more comprehensively evaluated. The experiment shows that the permutation error rate of the initial blind source separation algorithm is 63.0%. After blind source separation and spatial filtering pre-permutation processing, the error rate is reduced to 13.2%, and further reduced to 10.8% after noise suppression. In addition, the single-channel noise filtering process improves the signal-to-noise ratio of the method by 5.7dB by further suppressing static noise, and finally reaches 15.1dB, although the distortion is slightly increased from 3.5 to 4.2. In terms of overall speech intelligibility score, the single-channel noise filtering process performs best, reaching 0.73, followed by 0.72 for single-channel noise suppression, while the remaining two processes are 0.66 and 0.57 respectively. These results show that the proposed system not only has significant advantages in signal-to-noise ratio and permutation error rate, but also shows excellent performance in enhancing the clarity of target sound, which fully verifies the practical application effect of the system in complex drone noise environment.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0146] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0150] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0151] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for enhancing target sound collected by an unmanned aerial vehicle, characterized in that: include: Acquire a first target transformer sound mixed signal through a sound acquisition device array carried by the unmanned aerial vehicle, and pre-process the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal; Performing signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and acquiring an aligned third target transformer sound mixed signal in combination with a preset first-stage alignment strategy and a second-stage alignment strategy; The third target transformer sound mixed signal is filtered according to a preset filtering strategy to obtain a target sound signal.

2. The method for enhancing target sound collected by an unmanned aerial vehicle according to claim 1, characterized in that: The step of obtaining a first target transformer sound mixed signal and preprocessing the first target transformer sound mixed signal comprises: The mixed signal of the first target transformer sound collected by the M sound acquisition device array carried by the UAV within the transformer target distance range is expressed as X(t)=[x1(t),x2(t),…,x M (t)] T ; The first target transformer sound mixed signal is preprocessed, and the preprocessing includes high-pass filtering and normalization, removing the signal with a frequency lower than the first frequency threshold in the first target transformer sound mixed signal, and stabilizing the signal amplitude. The mathematical model of the first target transformer sound mixed signal is: X(t)=A·S(t)+N(t) Among them, A is the mixing coefficient matrix, S(t) is the target sound signal, and N(t) is the self-noise signal of the drone, that is, all the first target transformer sound mixed signals can be obtained by combining the mixing coefficient matrix, the target sound signal and the self-noise signal of the drone.

3. The method for enhancing target sound collected by an unmanned aerial vehicle according to claim 2, characterized in that: The performing signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy comprises: Use the independent component analysis method to separate the signal on each frequency band and solve the separation matrix W to obtain the separated signal: Y(t)=W·X(t) The separation result Y(t) contains several signal components, and the signal components include the target sound signal and the self-noise signal of the drone.

4. The method for enhancing target sound collected by an unmanned aerial vehicle as claimed in claim 3, characterized in that: The step of combining the preset first-stage alignment strategy and the second-stage alignment strategy to obtain the aligned third target transformer sound mixed signal includes: The first stage alignment strategy includes spatial filtering based on time-frequency signal processing, using the time-frequency spatial filtering algorithm to spatially enhance the sound in the direction of the target signal, and defining the spatial filter h(θ) as: Where a(θ) is the directional vector of the sound acquisition device array, θ represents the angle parameter, which is used to indicate the direction or position of the sound source, and a H (θ) represents the conjugate transposed form of the directivity vector of the sound acquisition device array at a given direction θ.

5. The method for enhancing target sound collected by a drone as claimed in claim 4, characterized in that: The step of combining the preset first-stage alignment strategy and the second-stage alignment strategy to obtain the aligned third target transformer sound mixed signal also includes: The second stage alignment strategy includes cross-frequency alignment, using the signal output by the time-frequency spatial filtering algorithm as a reference to align the frequency components separated by independent components across frequencies. The aligned signals are verified by a clustering algorithm and grouped based on time activity: C i ={f j ∣Activity(y ij (t))>τ} Among them, C t represents the classification set at time t, which contains all frequency components that meet the activity threshold condition, f j represents the jth frequency component, Activity(y ij (t)) represents the activity value of the jth frequency component received by the i-th sound acquisition device at time t, τ represents the activity threshold, and when Activity(y ij When (t))>τ, f j Join C t middle.

6. The method for enhancing target sound collected by a drone as claimed in claim 5, characterized in that: The filtering of the third target transformer sound mixed signal according to a preset filtering strategy to obtain a target sound signal includes: further suppressing noise in the arranged aligned signal by using an improved time-frequency mask, and performing single-channel spectrum post-filtering on the suppressed arranged aligned signal.

7. The method for enhancing target sound collected by a drone as claimed in claim 6, characterized in that: The filtering the third target transformer sound mixed signal according to the preset filtering strategy to obtain the target sound signal also includes: Perform inverse short-time Fourier transform on the frequency domain signal after single-channel spectrum post-filtering to reconstruct the time domain signal; The signal after reconstructing the time domain signal processing is smoothed by using overlapping windowing method to eliminate the discontinuity of the time domain signal and output the smoothed time domain signal.

8. A drone target sound enhancement system, characterized in that: include: A sound source acquisition and processing module, used to acquire a first target transformer sound mixed signal through a sound acquisition device array carried by the UAV, and pre-process the first target transformer sound mixed signal, wherein the first target transformer sound mixed signal at least includes a target sound signal; A signal separation and recombining module, used to perform signal separation on the preprocessed first target transformer sound mixed signal in combination with a preset blind source separation strategy, and to obtain an aligned third target transformer sound mixed signal in combination with a preset first-stage alignment strategy and a second-stage alignment strategy; The filtering module is used to filter the third target transformer sound mixed signal according to a preset filtering strategy to obtain a target sound signal.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images