An unmanned aerial vehicle acoustic identification method and device

Abstract

The application provides a method and device for acoustic identification of a UAV, wherein a UAV audio signal is collected, and a plurality of signal frames are obtained by frame division; the power spectrum density of each signal frame is calculated, and the envelope of the power spectrum density of each signal frame is determined according to the power spectrum density of each signal frame; the peak value of the envelope of the power spectrum density of each signal frame is filtered to determine the frequency of the retained prominent peak value; the first feature vector of the UAV audio signal is determined according to the standard preset frequency range of each UAV based on the frequency of the prominent peak value of each signal frame; the convolutional neural network model is trained based on the first feature vector of the UAV audio signal; and the trained convolutional neural network model is used to identify the audio signal to be identified. The above method is used to obtain the performance parameters of the battery. The above method is used to effectively identify the UAV audio signal, and improve the accuracy and effectiveness of the identification result of the UAV.

Description

Technical Field

[0001] This invention relates to the field of signal processing, and more specifically, to a method and apparatus for acoustic recognition of unmanned aerial vehicles (UAVs). Background Technology

[0002] With the continuous development of technology, drones are increasingly widely used in various fields, such as security, environmental monitoring, remote sensing, rescue, and infrastructure inspection. As technology advances, the barriers to drone use are constantly decreasing. While this brings convenience, it also leads to problems such as unauthorized drone flights, security threats, noise pollution, and privacy violations, resulting in negative impacts on society.

[0003] To address these issues, effective countermeasures against drones are needed. This will not only help reduce the negative impacts of drone use but also ensure their standardized use and promote the orderly development of the drone industry. Identification is a crucial element in drone countermeasure systems. Only by accurately identifying threatening drones can effective countermeasures be taken.

[0004] Drone detection and identification encompasses various identification technologies, including vision, acoustics, radar, and radio frequency (RF). Vision-based identification technologies are susceptible to factors such as obstruction, lighting conditions, and object blur. Radar-based identification technologies struggle to detect drones due to their small size and limited radar cross-section. RF-based identification technologies fail to detect drones flying along pre-defined routes (without emitting RF signals).

[0005] Acoustic-based drone identification technology can operate in all weather conditions (such as fog, rain, or low-light environments), exhibiting strong environmental adaptability. Furthermore, sound waves can bypass obstacles, overcoming terrain occlusion or building obstruction to some extent, enabling the detection of drones that are difficult for visual or radar systems to detect. Compared to complex systems like radar or lidar, acoustic identification systems typically have lower equipment costs, making them suitable for large-scale deployments or resource-constrained applications.

[0006] Drone sounds are generally categorized into aerodynamic sounds and mechanical sounds. Aerodynamic sounds originate from the rotor cutting through the air and the friction between the drone and the air, and are typically located in the low-frequency range of the sound signal, attenuating more slowly. Mechanical sounds originate from the motor and transmission components, and are typically located in the high-frequency range of the sound signal, attenuating more quickly. Overall, the acoustic signal generated by a drone when hovering or flying is a harmonic signal, and this unique property of drone sound can be used to effectively distinguish it from ambient noise.

[0007] Traditional harmonic analysis directly divides the frequency domain after Fourier transform into different frequency bands, each corresponding to a higher harmonic, and then determines the frequency value corresponding to the maximum amplitude within each band. This approach faces several problems: 1. In real-world environments, there are spectral lines of other frequencies near the center frequencies of each higher harmonic, and these sidelobes can influence the selection of the harmonic center frequency. 2. The range of the divided frequency bands is limited. If the center frequencies of adjacent higher harmonics of a drone differ by 400Hz, then the frequency band range cannot exceed 800Hz. The output values ​​of high-frequency bands, whether from the drone or noise, have limited differences. 3. The divided frequency bands are affected by the harmonic frequency values. Different drone models have different harmonic frequency values, meaning that the divided frequency bands are only applicable to one type of drone and are ineffective for other models. Therefore, existing drone detection and recognition methods cannot effectively identify drone audio signals, reducing the accuracy and effectiveness of drone identification results. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide an acoustic recognition method and apparatus for unmanned aerial vehicles (UAVs) to effectively recognize UAV audio signals and improve the accuracy and effectiveness of UAV recognition results.

[0009] In a first aspect, embodiments of this application provide an acoustic recognition method for unmanned aerial vehicles (UAVs), the method comprising:

[0010] Collect the drone audio signal and divide the drone audio signal into frames to obtain several signal frames;

[0011] Calculate the power spectral density of each signal frame, and determine the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame;

[0012] The peak values ​​of the power spectral density envelope of each signal frame are detected and filtered to determine the frequency of the prominent peak values ​​retained after filtering of each signal frame.

[0013] Based on the standard preset frequency range of each UAV, the first feature vector of the UAV audio signal is determined based on the frequency of the prominent peak of each signal frame;

[0014] The convolutional neural network model is trained based on the first feature vector of the drone's audio signal.

[0015] The trained convolutional neural network model is used to identify the audio signal to be identified.

[0016] Optionally, calculating the power spectral density of each signal frame includes:

[0017] Windowing is applied to each signal frame using a window function;

[0018] Performing a Fast Fourier Transform on each windowed signal frame yields the frequency domain signal of each frame:

[0019] Based on the number of points in the Fast Fourier Transform, the power spectral density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0020] Optionally, determining the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame includes:

[0021] The power spectral density of each signal frame is smoothed to obtain the envelope of the power spectral density of each signal frame.

[0022] Optionally, determining the first feature vector of the UAV audio signal based on the frequency of prominent peaks in each signal frame, according to the standard preset frequency range of each UAV, includes:

[0023] For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone.

[0024] The second feature vector of the UAV in each signal frame is constructed based on the frequency of the prominent peak of each signal frame within the standard preset frequency range of the UAV.

[0025] The first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

[0026] Optionally, generating the first feature vector of the UAV audio signal based on the second feature vector of each UAV in each signal frame includes:

[0027] For each signal frame, the second feature vectors of each UAV in that signal frame are integrated to obtain the third feature vector of each signal frame;

[0028] The first feature vector of the UAV audio signal is obtained by integrating the third feature vectors of each signal frame.

[0029] Secondly, embodiments of this application provide an acoustic recognition device for unmanned aerial vehicles (UAVs), the device comprising:

[0030] The signal acquisition module is used to acquire the audio signal of the UAV and divide the UAV audio signal into frames to obtain several signal frames.

[0031] The signal processing module is used to calculate the power spectral density of each signal frame and determine the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame.

[0032] The signal filtering module is used to detect and filter the peak values ​​of the power spectral density envelope of each signal frame, and determine the frequency of the prominent peak values ​​retained after filtering each signal frame.

[0033] The feature vector determination module is used to determine the first feature vector of the UAV audio signal based on the frequency of the prominent peak of each signal frame, according to the standard preset frequency range of each UAV.

[0034] The model training module is used to train the convolutional neural network model based on the first feature vector of the UAV audio signal;

[0035] The signal recognition module is used to recognize the audio signal to be recognized using a trained convolutional neural network model.

[0036] Optionally, calculating the power spectral density of each signal frame includes:

[0037] Windowing is applied to each signal frame using a window function;

[0038] Performing a Fast Fourier Transform on each windowed signal frame yields the frequency domain signal of each frame:

[0039] Based on the number of points in the Fast Fourier Transform, the power spectral density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0040] Optionally, determining the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame includes:

[0041] The power spectral density of each signal frame is smoothed to obtain the envelope of the power spectral density of each signal frame.

[0042] Optionally, determining the first feature vector of the UAV audio signal based on the frequency of prominent peaks in each signal frame, according to the standard preset frequency range of each UAV, includes:

[0043] For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone.

[0044] The second feature vector of the UAV in each signal frame is constructed based on the frequency of the prominent peak of each signal frame within the standard preset frequency range of the UAV.

[0045] The first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

[0046] Optionally, generating the first feature vector of the UAV audio signal based on the second feature vector of each UAV in each signal frame includes:

[0047] For each signal frame, the second feature vectors of each UAV in that signal frame are integrated to obtain the third feature vector of each signal frame;

[0048] The first feature vector of the UAV audio signal is obtained by integrating the third feature vectors of each signal frame.

[0049] The technical solution provided in this application includes, but is not limited to, the following beneficial effects:

[0050] The UAV acoustic recognition method provided in this application optimizes the classical harmonic analysis method based on improved harmonic analysis and convolutional neural networks. First, the classical harmonic analysis method does not consider sidelobe interference, directly selecting the frequency value corresponding to the peak with the largest amplitude within a selected frequency range. However, in reality, many sidelobes appear near the center frequencies of each higher harmonic, interfering with the selection of the harmonic center frequency. To suppress sidelobe interference, this application reduces the frequency resolution and extracts the power spectral density envelope, making the center frequencies of each higher harmonic more prominent. Second, the classical harmonic analysis method requires pre-setting multiple frequency ranges, each corresponding to a higher harmonic. However, the frequency values ​​between adjacent higher harmonics are not significantly different, resulting in a limited range of frequency ranges; the frequency range within each range cannot exceed 800Hz. This application, by first filtering out prominent peaks and then further filtering to see if they are near higher harmonics, avoids the need for selecting specific frequency ranges. Furthermore, traditional post-processing often uses machine learning models, such as Support Vector Machines (SVM). The samples processed in traditional methods are one-dimensional feature vectors, which can only reflect changes within a short time period and cannot reflect feature change trends over a longer period. This application, however, constructs a two-dimensional feature matrix from the one-dimensional feature vectors extracted from continuous frame signals. Each sample is a two-dimensional feature matrix, which can reflect feature change trends over a longer period. Furthermore, classical harmonic analysis methods can only extract features from a single type of UAV, and cannot extract features from different UAV models simultaneously. This is because the frequency range is pre-defined and only applicable to one type of UAV. By filtering and selecting from sufficiently prominent peak points, features from different types of UAVs can be extracted. Moreover, convolutional neural networks are chosen for subsequent data processing, avoiding the limitation of traditional machine learning models that can only process one-dimensional feature vectors. This reduces the false recognition rate, effectively achieves UAV acoustic recognition, and improves the accuracy and effectiveness of UAV acoustic recognition results.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart of an acoustic recognition method for unmanned aerial vehicles provided in Embodiment 1 of the present invention is shown;

[0054] Figure 2 A flowchart of a power spectral density calculation method provided in Embodiment 1 of the present invention is shown;

[0055] Figure 3 A flowchart of the first feature vector determination method provided in Embodiment 1 of the present invention is shown;

[0056] Figure 4 A flowchart of a specific first feature vector determination method provided in Embodiment 1 of the present invention is shown;

[0057] Figure 5 This diagram illustrates the workflow of an acoustic recognition system for unmanned aerial vehicles (UAVs) provided in Embodiment 1 of the present invention.

[0058] Figure 6 A schematic diagram showing the features of the first type of UAV provided in Embodiment 1 of the present invention is shown;

[0059] Figure 7 A schematic diagram showing the features of the second type of UAV provided in Embodiment 1 of the present invention is shown;

[0060] Figure 8 A schematic diagram of an environmental noise characteristic provided in Embodiment 1 of the present invention is shown;

[0061] Figure 9 A schematic diagram of the experimental audio of the first type of UAV provided in Embodiment 1 of the present invention is shown;

[0062] Figure 10 A schematic diagram of the first type of UAV audio prediction provided in Embodiment 1 of the present invention is shown;

[0063] Figure 11 A schematic diagram of the experimental audio of the second type of UAV provided in Embodiment 1 of the present invention is shown;

[0064] Figure 12 A schematic diagram of the second type of UAV audio prediction provided in Embodiment 1 of the present invention is shown;

[0065] Figure 13A schematic diagram of the experimental audio of an unmanned aerial vehicle (UAV) with environmental noise provided in Embodiment 1 of the present invention is shown;

[0066] Figure 14 A schematic diagram of the UAV predictive audio for environmental noise provided in Embodiment 1 of the present invention is shown;

[0067] Figure 15 A schematic diagram of the structure of an acoustic recognition device for unmanned aerial vehicles provided in Embodiment 2 of the present invention is shown. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0069] Example 1

[0070] To facilitate understanding of this application, the following is combined with... Figure 1 The flowchart of the acoustic recognition method for unmanned aerial vehicles provided in Embodiment 1 of the present invention illustrates the embodiment of this application in detail.

[0071] See Figure 1 As shown, Figure 1 A flowchart of an acoustic recognition method for unmanned aerial vehicles (UAVs) according to Embodiment 1 of the present invention is shown, wherein the method includes steps S101 to S106:

[0072] S101: Collect the drone audio signal and divide the drone audio signal into frames to obtain several signal frames.

[0073] Specifically, the harmonic signal x(t) in a real-world environment can be described by the following mathematical formula:

[0074]

[0075] Where f1 represents the fundamental frequency. A represents the phase of the nth higher harmonic component. n The value represents the amplitude of the nth higher harmonic component, where N is the total number of higher harmonics, n(t) is the noise, and t represents time.

[0076] The drone's audio signal is divided into multiple frames, each with a duration of T. f Seconds, number of intra-frame samples N f The calculation is as follows:

[0077] N f =round(T f ×f s )

[0078] Among them, f s It is the sampling frequency, and round(·) is the rounding function.

[0079] For a drone audio signal with length L samples, the total number of frames F can be obtained:

[0080]

[0081] in, This is the floor function.

[0082] Signal x of each frame n (t) is extracted from the drone's audio signal:

[0083] x n (t)=x[(n-1)×N f +1: n×N f n = 1, 2, ..., F

[0084] S102: Calculate the power spectral density of each signal frame, and determine the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame.

[0085] Specifically, the Welch method is used to calculate the power spectral density. To obtain a smoother spectral curve, a low-pass filter is used to adjust the power spectral density P. xx (f) is smoothed to obtain the envelope E(f).

[0086] S103: Detect and filter the peak values ​​of the power spectral density envelope of each signal frame to determine the frequency of the prominent peak values ​​retained after filtering each signal frame.

[0087] Specifically, the envelope spectral peak is found using a peak search function, and it is determined whether the envelope spectral peak is prominent enough. If not, the prominent peak is discarded; if so, the frequency corresponding to the envelope spectral peak at this time is recorded, which is the frequency of the prominent peak retained after filtering.

[0088] Furthermore, peak p is detected in the envelope E(f). i And its corresponding frequency fi.

[0089] p i =E(fi)E(fi) is a local maximum

[0090] Detection peak p i Adjacent valley values ​​v i-1 and v i+1 .

[0091]

[0092] Filter out peaks using the following expression, retaining the most prominent peaks:

[0093]

[0094] Map the filtered peak frequency fj to a predefined frequency range matrix. u is the number of frequency range matrices.

[0095] S104: Based on the standard preset frequency range of each UAV, determine the first feature vector of the UAV audio signal based on the frequency of the prominent peak of each signal frame.

[0096] Specifically, because the distance between the drone and the microphone data acquisition device is not fixed in real-world applications, the amplitude of each higher harmonic is not a fixed value and will vary with distance. The frequency values ​​of each higher harmonic are relatively fixed and do not change significantly with distance; therefore, the extracted feature information should include frequency rather than amplitude.

[0097] For each UAV, several higher harmonic frequencies are acquired. A floating range for each higher harmonic frequency is set as the standard preset frequency range for each UAV. Values ​​are assigned to vector variables based on whether the frequency of the prominent peak of each signal frame falls within the standard preset frequency range for each UAV, resulting in a one-dimensional feature vector for each UAV in different signal frames. Then, the one-dimensional feature vectors of each UAV in the same signal frame are concatenated to obtain a one-dimensional feature vector for each signal frame. Finally, the one-dimensional feature vectors of consecutive signal frames are concatenated to obtain a two-dimensional feature vector of the UAV audio signal, denoted as the second feature vector.

[0098] S105: Train the convolutional neural network model based on the first feature vector of the drone's audio signal.

[0099] Specifically, the first feature vector of the drone's audio signal is used as a sample to train the convolutional neural network model.

[0100] In this application, each sample is an X∈R 10×u A two-dimensional matrix.

[0101] Convolutional layers are the core component of CNNs, used to extract features from the input data. Convolutional layers perform convolution operations by sliding filters (kernels) across the input data, generating feature maps. Each kernel is responsible for capturing different features of the image, such as edges and textures.

[0102] The first convolutional layer has a 3×3 kernel (filter) and a total of 8 filters. The weight matrix for each filter is as follows: (The k-th filter), with bias as The output Y of the convolution operation 1 The corresponding formula is:

[0103]

[0104] Where i and are the position indices of the output matrix, and k is the filter index. m and n are the indices of the convolution kernel (or filter) on the input matrix. They represent the row and column index positions of the convolution kernel. The output dimension remains unchanged (padding with zeros), so Y 1 ∈R 10×u×8 .

[0105] Batch normalization is used to normalize the output of convolutional layers to accelerate training. The normalized output is Z. 1 Then we have:

[0106]

[0107]

[0108]

[0109]

[0110] Where, γ k And is β k The learnable parameter, ∈, is a constant to prevent division by zero. μ k It is the mean of the k-th channel. H is the variance of the k-th channel, and H and W are the spatial dimensions.

[0111] The ReLU activation function sets negative values ​​to 0, and the formula is:

[0112]

[0113] Activated output A 1 ∈R 10×u×8 .

[0114] The pooling layer halves the output size by pooling operations from A. 1Extract the maximum value from each 2×2 region. Output P using pooling. 1 The formula is:

[0115]

[0116] The output dimension is P 1 ∈R 5×u / 2×8 .

[0117] The second convolutional layer operates similarly to the first, with the kernel size remaining 3×3, but the number of filters increases to 16. Let the input be P. 1 Then the output Y after convolution 2 The formula is:

[0118]

[0119] Output dimension Y 2 ∈R 5×u / 2×16 Batch normalization and ReLU activation are also applied to Y. 2 Generate output A 2 ∈R 5 ×u / 2×16 .

[0120] In the fully connected layer, the convolution output is flattened into a vector vec(A) 2 The dimension is 5×u / 2×16=40u. The output of the fully connected layer is:

[0121] F = W 3 ·vec(A 2 )+b 3

[0122] Among them, W 3 ∈R 3×40u It is the weight matrix, b 3 ∈R 3 It is the bias vector.

[0123] The Softmax function is used to convert the output of a fully connected layer into a probability distribution, as shown in the formula:

[0124]

[0125] in C is the predicted probability of class i, and C = 3 is the number of classes.

[0126] The Softmax function ensures that the sum of all output probabilities is 1, that is:

[0127]

[0128] It can be used to determine the predicted category of an input sample; typically, the category with the highest probability is selected as the final score.

[0129]

[0130] S106: Use the trained convolutional neural network model to identify the audio signal to be identified.

[0131] Specifically, after training, when acoustic recognition of a drone is required, the following steps are taken: First, the audio signal to be recognized is acquired and segmented into frames. The power spectral density (PSD) of each frame is calculated, and its envelope is determined. Peak values ​​within the PSD envelope are detected and filtered to determine the frequencies of the prominent peaks retained after filtering. Based on the standard preset frequency range for each drone, the fourth feature vector of the audio signal is determined using the frequencies of the prominent peaks in each frame. This fourth feature vector is then input into a trained convolutional neural network model to obtain the signal category of the audio signal, enabling its recognition. The recognition result indicates which drone emitted the audio signal or whether it is ambient noise.

[0132] In an optional implementation, see Figure 2 As shown, Figure 2 The flowchart of a power spectral density calculation method provided in Embodiment 1 of the present invention is shown, wherein the calculation of the power spectral density of each signal frame includes steps S201 to S203:

[0133] S201: Window each signal frame using a window function.

[0134] S202: Perform a Fast Fourier Transform on each signal frame after windowing to obtain the frequency domain signal of each signal frame.

[0135] S203: Based on the number of points in the Fast Fourier Transform, the power spectral density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0136] Specifically, firstly, a window function ω(t) is used for each frame x. n (t) Perform windowing processing to obtain the processed signal frame x′ n (t):

[0137] x′ n (t)=x n (t)·ω(t)

[0138] Perform a Fast Fourier Transform (FFT) on the windowed signal to obtain the frequency domain signal x.n (f):

[0139]

[0140] P is the number of FFT points. As P decreases, the frequency resolution decreases. f is the frequency, k is the frequency index, and j is the imaginary unit.

[0141] Calculate the power spectral density for each frame.

[0142]

[0143] In an optional implementation, determining the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame includes:

[0144] The power spectral density of each signal frame is smoothed to obtain the envelope of the power spectral density of each signal frame.

[0145] Specifically, a low-pass filter is used to filter the power spectral density P. xx (f) is smoothed to obtain the envelope E(f).

[0146] A low-pass filter H(f) has a cutoff frequency of f. c .

[0147]

[0148] Among them, b i and a i These are the coefficients of the filter, and z is the complex frequency variable. M is the order of the numerator polynomial, N is the order of the denominator polynomial, and b0, b1, b... M These are the preset molecular coefficients, a1, and a. N These are the preset denominator coefficients.

[0149] To eliminate phase distortion, a bidirectional filter, the filtfilt(·) function, is used.

[0150] E(f) = filtfilt(b, a, P) xx (f))

[0151] Where b is the numerator coefficient and a is the denominator coefficient.

[0152] In an optional implementation, see Figure 3 As shown, Figure 3 The flowchart of the first feature vector determination method provided in Embodiment 1 of the present invention is shown, wherein the first feature vector of the UAV audio signal is determined based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each UAV, including steps S301 to S303:

[0153] S301: For each UAV, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the UAV.

[0154] Specifically, for each signal frame, it is determined whether the frequency of the prominent peak of the signal frame is near the j-th higher harmonic frequency of the UAV, that is, whether it falls within the j-th standard preset frequency range.

[0155] S302: Construct a second feature vector for the UAV in each signal frame based on the frequency of the prominent peak of each signal frame within the standard preset frequency range of the UAV.

[0156] Specifically, if yes, the frequency of the prominent peak of the signal frame is placed in the j-th dimension blank variable; otherwise, the value of the j-th dimension blank variable is set to 0. Then, it continues to determine whether the frequency of the prominent peak of the signal frame is near the (j+1)-th higher harmonic frequency of the UAV and whether it falls within the (j+1)-th standard preset frequency range. If yes, the frequency of the prominent peak of the signal frame is placed in the (j+1)-th dimension blank variable; otherwise, the value of the (j+1)-th dimension blank variable is set to 0, until the determination of whether it falls within all standard preset frequency ranges is completed, and then the second feature vector of the UAV in each signal frame is generated.

[0157] Furthermore, check whether the peak frequency falls within a certain frequency range:

[0158]

[0159] The feature vector y extracted from each frame j For [y1, y2, ... y u ].

[0160] S303: Generate the first feature vector of the UAV audio signal based on the second feature vector of each UAV in each signal frame.

[0161] Specifically, the first feature vector of the drone audio signal is obtained by concatenating the second feature vectors of each drone in each signal frame.

[0162] In an optional implementation, see Figure 4 As shown, Figure 4 The flowchart of a specific first feature vector determination method provided in Embodiment 1 of the present invention is shown, wherein the step of generating the first feature vector of the UAV audio signal based on the second feature vector of each UAV in each signal frame includes steps S401 to S402:

[0163] S401: For each signal frame, integrate the second feature vectors of each UAV in that signal frame to obtain the third feature vector of each signal frame.

[0164] Specifically, when the second feature vector of each UAV in the signal frame is a horizontal vector, the second feature vectors of each UAV in the signal frame are concatenated horizontally to obtain the third feature vector of each signal frame. When the second feature vector of each UAV in the signal frame is a vertical vector, the second feature vectors of each UAV in the signal frame are concatenated vertically to obtain the third feature vector of each signal frame.

[0165] S402: Integrate the third feature vectors of each signal frame to obtain the first feature vector of the UAV audio signal.

[0166] Specifically, when the third feature vector of each signal frame is a horizontal vector, the third feature vectors of each signal frame are vertically concatenated to obtain the first feature vector of the UAV audio signal. When the third feature vector of each signal frame is a vertical vector, the third feature vectors of each signal frame are horizontally concatenated to obtain the first feature vector of the UAV audio signal.

[0167] For example, the feature vectors extracted from 10 consecutive frames can be combined into a two-dimensional feature matrix:

[0168]

[0169] The drone acoustic recognition method provided in this application is applied to the processor of a drone acoustic recognition system, which also includes a microphone collector for acquiring drone audio signals. See also... Figure 5 As shown, Figure 5 The diagram illustrates the workflow of an acoustic recognition system for unmanned aerial vehicles (UAVs) according to Embodiment 1 of the present invention. The system acquires a signal x(t), performs frame segmentation to reduce frequency resolution, calculates the power spectral density using a window function (Hamming window) and the Welch method, extracts the envelope of the power spectral density, and uses a peak search function to find a potential envelope peak. The system then determines whether the envelope peak is sufficiently prominent. If not, the data is discarded; otherwise, the frequency corresponding to the envelope peak is recorded. For each UAV, the system determines whether the frequency is near the j-th higher harmonic frequency of UAV A. If so, the current frequency f is recorded. i The frequency is placed into the j-th dimension blank variable, and then it is determined whether the frequency is near the (j+1)-th higher harmonic frequency of UAV A, i.e., j = j + 1; otherwise, the value of the j-th dimension blank variable is set to 0, forming a one-dimensional feature vector of UAV A. Similarly, it is determined whether the frequency is near the j-th higher harmonic frequency of UAV B, and if so, the frequency f at this time is set to 0. iThe value is placed into the j-th dimension blank variable, and then it is determined whether the frequency is near the (j+1)-th higher harmonic frequency of UAV B, i.e., j = j+1; otherwise, the value of the j-th dimension blank variable is set to 0, forming a one-dimensional feature vector of UAV B. After combining the one-dimensional feature vectors of UAV A and UAV B into a one-dimensional feature vector, 50 consecutive frames are combined into a two-dimensional feature vector, which is then input into a CNN classification model for training and application to obtain the output result.

[0170] In the process of applying the UAV acoustic recognition method provided in this application in practice, see [link / reference]. Figure 6 As shown, Figure 6 A schematic diagram illustrating the features of a first type of UAV provided in Embodiment 1 of the present invention is shown, wherein the horizontal axis represents the sample ordinal number and the vertical axis represents the frequency. See also Figure 7 As shown, Figure 7 A schematic diagram illustrating the features of a second type of UAV provided in Embodiment 1 of the present invention is shown, wherein the horizontal axis represents the sample ordinal number, and the vertical axis represents the frequency. See also... Figure 8 As shown, Figure 8 A schematic diagram of an environmental noise characteristic provided in Embodiment 1 of the present invention is shown, wherein the horizontal axis represents the sample ordinal number and the vertical axis represents the frequency. Figure 6 , Figure 7 and Figure 8 It can be observed that the extracted acoustic characteristics of different drone models are different, and all are significantly different from ambient noise. (See also...) Figure 9 As shown, Figure 9 A schematic diagram of the experimental audio of the first type of UAV provided in Embodiment 1 of the present invention is shown. The diagram illustrates the experimental audio of the first type of UAV first hovering and then flying to a height of 70m. See also... Figure 10 As shown, Figure 10 This diagram illustrates the first type of UAV audio prediction provided in Embodiment 1 of the present invention. The diagram shows the process of the model classifying and recognizing audio after CNN training. Figure 9 The audio in the data is categorized into three classes: Category 1 represents ambient noise, Category 2 represents the first type of drone, and Category 3 represents the second type of drone. From... Figure 10 It can be observed that whether hovering or flying at 70m, the model can be identified as the first type of drone. See also Figure 11 As shown, Figure 11 A schematic diagram of the experimental audio of the second type of UAV provided in Embodiment 1 of the present invention is shown, wherein the diagram illustrates the experimental audio of the second type of UAV hovering. See also Figure 12 As shown, Figure 12This diagram illustrates the second type of UAV audio prediction provided in Embodiment 1 of the present invention. The diagram shows the process of the model classifying and recognizing audio after CNN training. Figure 11 The audio in the data is categorized into three classes: Category 1 represents ambient noise, Category 2 represents the first type of drone, and Category 3 represents the second type of drone. From... Figure 12 The model can be observed to identify it as the second type of drone. (See also...) Figure 13 As shown, Figure 13 A schematic diagram of the experimental audio of an unmanned aerial vehicle (UAV) simulating environmental noise, as provided in Embodiment 1 of the present invention, is shown, illustrating the experimental audio waveform of the environmental noise. (See also...) Figure 14 As shown, Figure 14 This diagram illustrates the prediction of drone audio based on environmental noise provided in Embodiment 1 of the present invention. The diagram shows the process of classifying and recognizing the model after training the CNN. Figure 14 The audio in the data is categorized into three classes: Category 1 represents ambient noise, Category 2 represents the first type of drone, and Category 3 represents the second type of drone. From... Figure 14 It can be observed that the model can identify environmental noise.

[0171] Example 2

[0172] Embodiment 2 of the present invention provides an acoustic recognition device for unmanned aerial vehicles (UAVs), see [link to embodiment 2]. Figure 15 As shown, Figure 15 This diagram illustrates the structure of an acoustic recognition device for unmanned aerial vehicles (UAVs) according to Embodiment 2 of the present invention, wherein the device includes:

[0173] The signal acquisition module 1501 is used to acquire the audio signal of the UAV and divide the UAV audio signal into frames to obtain several signal frames.

[0174] The signal processing module 1502 is used to calculate the power spectral density of each signal frame and determine the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame.

[0175] The signal filtering module 1503 is used to detect and filter the peak value of the envelope of the power spectral density of each signal frame, and determine the frequency of the prominent peak value retained after filtering each signal frame.

[0176] The feature vector determination module 1504 is used to determine the first feature vector of the UAV audio signal based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each UAV.

[0177] Model training module 1505 is used to train a convolutional neural network model based on the first feature vector of the UAV audio signal;

[0178] The signal recognition module 1506 is used to recognize the audio signal to be recognized using a trained convolutional neural network model.

[0179] In an optional implementation, calculating the power spectral density of each signal frame includes:

[0180] Windowing is applied to each signal frame using a window function;

[0181] Performing a Fast Fourier Transform on each windowed signal frame yields the frequency domain signal of each frame:

[0182] Based on the number of points in the Fast Fourier Transform, the power spectral density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0183] In an optional implementation, determining the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame includes:

[0184] The power spectral density of each signal frame is smoothed to obtain the envelope of the power spectral density of each signal frame.

[0185] In an optional implementation, determining the first feature vector of the UAV audio signal based on the frequency of prominent peaks in each signal frame, according to a standard preset frequency range for each UAV, includes:

[0186] For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone.

[0187] The second feature vector of the UAV in each signal frame is constructed based on the frequency of the prominent peak of each signal frame within the standard preset frequency range of the UAV.

[0188] The first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

[0189] In an optional implementation, generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes:

[0190] For each signal frame, the second feature vectors of each UAV in that signal frame are integrated to obtain the third feature vector of each signal frame;

[0191] The first feature vector of the UAV audio signal is obtained by integrating the third feature vectors of each signal frame.

[0192] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0193] The drone acoustic recognition device provided in this embodiment of the invention can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this embodiment of the invention are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiments can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0194] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] In addition, the functional units in the embodiments provided by the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0197] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0199] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An acoustic recognition method for unmanned aerial vehicles (UAVs), characterized in that, The method includes: Collect the drone audio signal and divide the drone audio signal into frames to obtain several signal frames; Calculate the power spectral density of each signal frame, and determine the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame; The peak values ​​of the power spectral density envelope of each signal frame are detected and filtered to determine the frequency of the prominent peak values ​​retained after filtering of each signal frame. Based on the standard preset frequency range of each UAV, the first feature vector of the UAV audio signal is determined based on the frequency of the prominent peak of each signal frame; The convolutional neural network model is trained based on the first feature vector of the drone's audio signal. The trained convolutional neural network model is used to identify the audio signal to be identified.

2. The method according to claim 1, characterized in that, The calculation of the power spectral density of each signal frame includes: Windowing is applied to each signal frame using a window function; Performing a Fast Fourier Transform on each windowed signal frame yields the frequency domain signal of each frame: Based on the number of points in the Fast Fourier Transform, the power spectral density of each signal frame is determined according to the frequency domain signal of each signal frame.

3. The method according to claim 1, characterized in that, The step of determining the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame includes: The power spectral density of each signal frame is smoothed to obtain the envelope of the power spectral density of each signal frame.

4. The method according to claim 1, characterized in that, The step of determining the first feature vector of the drone audio signal based on the frequency of prominent peaks in each signal frame, according to the standard preset frequency range of each drone, includes: For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone. The second feature vector of the UAV in each signal frame is constructed based on the frequency of the prominent peak of each signal frame within the standard preset frequency range of the UAV. The first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

5. The method according to claim 4, characterized in that, The process of generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes: For each signal frame, the second feature vectors of each UAV in that signal frame are integrated to obtain the third feature vector of each signal frame; The first feature vector of the UAV audio signal is obtained by integrating the third feature vectors of each signal frame.

6. An acoustic recognition device for unmanned aerial vehicles (UAVs), characterized in that, The device includes: The signal acquisition module is used to acquire the audio signal of the UAV and divide the UAV audio signal into frames to obtain several signal frames. The signal processing module is used to calculate the power spectral density of each signal frame and determine the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame. The signal filtering module is used to detect and filter the peak values ​​of the power spectral density envelope of each signal frame, and determine the frequency of the prominent peak values ​​retained after filtering each signal frame. The feature vector determination module is used to determine the first feature vector of the UAV audio signal based on the frequency of the prominent peak of each signal frame, according to the standard preset frequency range of each UAV. The model training module is used to train the convolutional neural network model based on the first feature vector of the UAV audio signal; The signal recognition module is used to recognize the audio signal to be recognized using a trained convolutional neural network model.

7. The apparatus according to claim 6, characterized in that, The calculation of the power spectral density of each signal frame includes: Windowing is applied to each signal frame using a window function; Performing a Fast Fourier Transform on each windowed signal frame yields the frequency domain signal of each frame: Based on the number of points in the Fast Fourier Transform, the power spectral density of each signal frame is determined according to the frequency domain signal of each signal frame.

8. The apparatus according to claim 6, characterized in that, The step of determining the envelope of the power spectral density of each signal frame based on the power spectral density of each signal frame includes: The power spectral density of each signal frame is smoothed to obtain the envelope of the power spectral density of each signal frame.

9. The apparatus according to claim 6, characterized in that, The step of determining the first feature vector of the drone audio signal based on the frequency of prominent peaks in each signal frame, according to the standard preset frequency range of each drone, includes: For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone. The second feature vector of the UAV in each signal frame is constructed based on the frequency of the prominent peak of each signal frame within the standard preset frequency range of the UAV. The first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

10. The apparatus according to claim 9, characterized in that, The process of generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes: For each signal frame, the second feature vectors of each UAV in that signal frame are integrated to obtain the third feature vector of each signal frame; The first feature vector of the UAV audio signal is obtained by integrating the third feature vectors of each signal frame.

Images