Unmanned aerial vehicle acoustic identification method and device

Abstract

The invention provides an unmanned aerial vehicle acoustic recognition method and device, and the method comprises the steps: collecting an audio signal of an unmanned aerial vehicle, and carrying out the framing of the audio signal to obtain a plurality of signal frames; calculating the power spectral density of each signal frame, and determining the envelope of the power spectral density of each signal frame according to the power spectral density of each signal frame; filtering the envelope peak value of the power spectral density of each signal frame to determine the frequency of the reserved prominent peak value; according to the standard preset frequency range of each unmanned aerial vehicle, determining a first feature vector of the audio signal of the unmanned aerial vehicle based on the frequency of the prominent peak value of each signal frame; performing model training on a convolutional neural network model based on the first feature vector of the unmanned aerial vehicle audio signal; and identifying the to-be-identified audio signal by using the trained convolutional neural network model. By adopting the method, the performance parameters of the battery are obtained. By adopting the method, the audio signal of the unmanned aerial vehicle is effectively identified, and the accuracy and effectiveness of an unmanned aerial vehicle ascending identification result are improved.

Description

Technical Field

[0001] The present invention relates to the field of signal processing, and in particular to a method and device for acoustic recognition of unmanned aerial vehicles. Background Art

[0002] With the continuous development of science and technology, drones are increasingly used in various fields, such as security, environmental testing, remote sensing, rescue, infrastructure inspection, etc. With the continuous advancement of science and technology, the threshold for using drones is constantly lowered, which brings convenience to people, but also causes problems such as illegal flying, security threats, noise pollution, and privacy infringement, which has brought negative impacts to society.

[0003] In order to deal with these problems, it is necessary to study effective drone countermeasures. This will not only help reduce the negative impact of drone use, but also ensure the standardized use of drones and promote the orderly development of the drone industry. In the drone countermeasure system, identification is a crucial link. Only by accurately identifying threatening drones can effective countermeasures be taken.

[0004] The detection and identification of drones includes recognition technologies in various fields such as vision, acoustics, radar, and radio frequency. Recognition technology based on vision is easily affected by factors such as obstacles, light illumination, and blurred objects; recognition technology based on radar cannot easily detect drones due to their small size and small radar cross-sectional area; recognition technology based on radio frequency signals cannot detect intruding drones when they fly along a pre-set route (without emitting radio frequency signals).

[0005] Acoustic-based drone identification technology can achieve all-weather operation (such as haze, rainy days or low-light environments) and has strong environmental adaptability. In addition, sound waves can bypass obstacles and, to a certain extent, overcome the problem of terrain occlusion or building obstruction, and detect drones that are difficult to capture by visual or radar systems. Compared with complex systems such as radar or lidar, the equipment cost of acoustic identification systems is usually lower, which is suitable for large-scale deployment or resource-constrained application scenarios.

[0006] Drone sounds are usually divided into aerodynamic sounds and mechanical sounds. The former originates from the rotor cutting the air and the friction between the body and the air. It is usually distributed in the low-frequency part of the sound signal and decays slowly. The latter originates from the motor and transmission parts. It is usually distributed in the high-frequency part of the sound signal and decays quickly. In general, the acoustic signal generated by a drone when hovering or flying is a harmonic signal, which can be effectively distinguished from environmental noise by utilizing this unique property of drone sound.

[0007] Traditional harmonic analysis directly divides different frequency bands in the frequency domain after Fourier transformation. Each frequency band directly corresponds to a high-order harmonic, and then determines the frequency value corresponding to the maximum amplitude in each frequency band. Faced with the following problems: 1. In the actual environment, there will be spectral lines of other frequencies near the center frequency of each high-order harmonic, and these side lobes will affect the selection of the center frequency of the harmonic. 2. The range of the divided frequency band is limited. If the center frequency of adjacent high-order harmonics of the drone differs by 400Hz, then the frequency band range cannot exceed 800Hz. The numerical difference between the high-frequency band output of the drone and the noise is limited. 3. The divided frequency band is affected by the value of the harmonic frequency. The harmonic frequency values ​​of different models of drones are different, resulting in the divided frequency band can only be used for one type of drone, and it does not work for the recognition of other models of drones. It can be seen that the existing drone detection and recognition methods cannot effectively identify unmanned audio signals, which will reduce the accuracy and effectiveness of drone recognition results. Summary of the invention

[0008] In view of this, the purpose of the present invention is to provide a drone acoustic recognition method and device to effectively recognize drone audio signals and improve the accuracy and effectiveness of drone acoustic recognition results.

[0009] In a first aspect, an embodiment of the present application provides a method for acoustic identification of a drone, the method comprising:

[0010] Collecting a drone audio signal, and dividing the drone audio signal into frames to obtain a plurality of signal frames;

[0011] Calculating the power spectrum density of each signal frame, and determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame;

[0012] Detect and filter the peak value of the envelope of the power spectrum density of each signal frame to determine the frequency of the prominent peak value retained after filtering each signal frame;

[0013] Determining a first eigenvector of the drone audio signal based on the frequency of a prominent peak of each signal frame according to a standard preset frequency range of each drone;

[0014] Performing model training on a convolutional neural network model based on a first eigenvector of the drone audio signal;

[0015] The trained convolutional neural network model is used to identify the audio signal to be identified.

[0016] Optionally, calculating the power spectrum density of each signal frame includes:

[0017] Perform windowing processing on each signal frame using a window function;

[0018] Perform fast Fourier transform on each signal frame after windowing to obtain the frequency domain signal of each signal frame:

[0019] Based on the number of fast Fourier transform points, the power spectrum density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0020] Optionally, determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame includes:

[0021] The power spectrum density of each signal frame is smoothed to obtain the envelope of the power spectrum density of each signal frame.

[0022] Optionally, the determining of the first feature vector of the drone audio signal based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each drone includes:

[0023] For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone;

[0024] Constructing a second feature vector of the drone in each signal frame based on the frequency of the prominent peak of each signal frame within a standard preset frequency range of the drone;

[0025] A first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

[0026] Optionally, the generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes:

[0027] For each signal frame, the second eigenvectors of each drone in the signal frame are integrated to obtain the third eigenvector of each signal frame;

[0028] The third eigenvectors of each signal frame are integrated to obtain the first eigenvector of the drone audio signal.

[0029] In a second aspect, an embodiment of the present application provides a drone acoustic recognition device, the device comprising:

[0030] A signal acquisition module, used for acquiring the drone audio signal, and dividing the drone audio signal into frames to obtain a plurality of signal frames;

[0031] A signal processing module, used to calculate the power spectrum density of each signal frame, and determine the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame;

[0032] A signal filtering module, used to detect and filter the peak value of the envelope of the power spectrum density of each signal frame, and determine the frequency of the prominent peak value retained after filtering each signal frame;

[0033] A feature vector determination module, configured to determine a first feature vector of the drone audio signal based on the frequency of a prominent peak of each signal frame according to a standard preset frequency range of each drone;

[0034] A model training module, used for performing model training on a convolutional neural network model based on a first feature vector of the drone audio signal;

[0035] The signal recognition module is used to recognize the audio signal to be recognized using the trained convolutional neural network model.

[0036] Optionally, calculating the power spectrum density of each signal frame includes:

[0037] Perform windowing processing on each signal frame using a window function;

[0038] Perform fast Fourier transform on each signal frame after windowing to obtain the frequency domain signal of each signal frame:

[0039] Based on the number of fast Fourier transform points, the power spectrum density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0040] Optionally, determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame includes:

[0041] The power spectrum density of each signal frame is smoothed to obtain the envelope of the power spectrum density of each signal frame.

[0042] Optionally, the determining of the first feature vector of the drone audio signal based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each drone includes:

[0043] For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone;

[0044] Constructing a second feature vector of the drone in each signal frame based on the frequency of the prominent peak of each signal frame within a standard preset frequency range of the drone;

[0045] A first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

[0046] Optionally, the generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes:

[0047] For each signal frame, the second eigenvectors of each drone in the signal frame are integrated to obtain the third eigenvector of each signal frame;

[0048] The third eigenvectors of each signal frame are integrated to obtain the first eigenvector of the drone audio signal.

[0049] The technical solution provided by this application includes but is not limited to the following beneficial effects:

[0050] The drone acoustic identification method provided by the present application optimizes the classical harmonic analysis method based on improved harmonic analysis and convolutional neural network. First, the classical harmonic analysis method does not take into account the sidelobe interference problem, and directly selects the frequency value corresponding to the peak with the largest amplitude in the selected frequency interval. In actual situations, more sidelobes will appear near the center frequency of each high-order harmonic, and this sidelobe will interfere with the selection of the center frequency of the harmonic. In order to suppress the sidelobe interference, the present application suppresses the sidelobes by reducing the frequency resolution and extracting the envelope of the power spectrum density, so that the center frequency of each high-order harmonic is more prominent. Secondly, the classical harmonic analysis method needs to set multiple frequency intervals in advance, and each frequency interval corresponds to a high-order harmonic. However, the frequency values ​​between adjacent high-order harmonics are not much different, which leads to a limited range of frequency intervals, and the upper and lower frequency ranges of each frequency interval cannot differ by more than 800Hz. The present application first screens out the prominent peaks, and then further screens whether they are near the high-order harmonics. This can avoid the selected frequency interval. Furthermore, traditional post-data processing selects machine learning models such as support vector machines (SVM). The sample it processes is a one-dimensional feature vector, which can only reflect changes in a short period of time and cannot reflect the trend of feature changes in a longer period of time. However, the present application forms a two-dimensional feature matrix by combining the one-dimensional feature vectors extracted from continuous frame signals. Each sample is a two-dimensional feature matrix that can reflect the trend of feature changes over a longer period of time. At the same time, the classic harmonic analysis method can only extract the features of a single model of drone, and cannot extract the features of different models of drones at the same time. This is because the frequency range is pre-set and can only be used for one model of drone. By filtering and selecting from sufficiently prominent peak points, the features of different types of drones can be extracted. In addition, the convolutional neural network is selected in the later data processing to avoid the disadvantage that the traditional machine learning model can only process one-dimensional feature vectors, which can reduce the misrecognition rate, effectively realize drone acoustic recognition, and improve the accuracy and effectiveness of drone recognition results.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 A flowchart of a drone acoustic identification method provided by Embodiment 1 of the present invention is shown;

[0054] Figure 2 A flow chart of a method for calculating power spectral density provided by the first embodiment of the present invention is shown;

[0055] Figure 3 A flow chart of a method for determining a first feature vector provided in the first embodiment of the present invention is shown;

[0056] Figure 4 A flowchart of a specific method for determining a first characteristic vector provided in the first embodiment of the present invention is shown;

[0057] Figure 5 A flowchart of a drone acoustic identification system provided in Embodiment 1 of the present invention is shown;

[0058] Figure 6 A schematic diagram showing features of a first type of drone provided by Embodiment 1 of the present invention;

[0059] Figure 7 A schematic diagram showing features of a second type of drone provided in the first embodiment of the present invention;

[0060] Figure 8 A schematic diagram showing an environmental noise feature provided by the first embodiment of the present invention is shown;

[0061] Fig. 9 A schematic diagram of the first type of drone experimental audio provided by the first embodiment of the present invention is shown;

[0062] Fig.10 A schematic diagram showing the predicted audio of a first type of drone provided in the first embodiment of the present invention is shown;

[0063] Fig.11 A schematic diagram of the experimental audio of a second type of drone provided in the first embodiment of the present invention is shown;

[0064] Fig.12 A schematic diagram showing the predicted audio of a second type of drone provided in the first embodiment of the present invention is shown;

[0065] Fig.13A schematic diagram of the experimental audio of a drone with ambient noise provided by the first embodiment of the present invention is shown;

[0066] Fig.14 A schematic diagram of the predicted audio of a drone of environmental noise provided by the first embodiment of the present invention is shown;

[0067] Fig.15 A schematic structural diagram of a drone acoustic identification device provided in Embodiment 2 of the present invention is shown. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0069] Embodiment 1

[0070] To facilitate understanding of this application, Figure 1 The flowchart of a drone acoustic identification method provided in the first embodiment of the present invention is shown to describe the contents of the first embodiment of the present application in detail.

[0071] See also Figure 1 As shown, Figure 1 A flowchart of a drone acoustic identification method provided in Embodiment 1 of the present invention is shown, wherein the method comprises steps S101 to S106:

[0072] S101: Collecting a drone audio signal, and dividing the drone audio signal into frames to obtain a plurality of signal frames.

[0073] Specifically, the harmonic signal x(t) model in the actual environment can be described by the following mathematical formula:

[0074]

[0075] Where f1 represents the fundamental frequency, Indicates the phase of the nth higher harmonic component, A n It represents the amplitude of the nth harmonic component, N is the total number of harmonics, n(t) is the noise, and t represents time.

[0076] Divide the drone audio signal into multiple frames, each frame has a duration of T f seconds, the number of samples in a frame is N f The calculation is as follows:

[0077] N f =round(T f ×f s )

[0078] Among them, f s is the sampling frequency and round(·) is the rounding function.

[0079] For a drone audio signal with a length of L samples, the total number of frames F can be obtained:

[0080]

[0081] in, is the floor function.

[0082] The signal x of each frame n (t) is extracted from the drone audio signal:

[0083] x n (t) = x[(n-1) × N f +1:n×N f ]n=1,2,…,F

[0084] S102: Calculate the power spectrum density of each signal frame, and determine the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame.

[0085] Specifically, the Welch method is used to calculate the power spectral density. In order to obtain a smoother spectrum curve, a low-pass filter is used to calculate the power spectral density P xx (f) is smoothed to obtain the envelope E(f).

[0086] S103: Detect and filter the peak value of the envelope of the power spectrum density of each signal frame to determine the frequency of the prominent peak value retained after filtering each signal frame.

[0087] Specifically, the envelope spectrum peak is found through the peak search function to determine whether the envelope spectrum peak is prominent enough. If not, the prominent peak is discarded; if yes, the frequency corresponding to the envelope spectrum peak at this time is recorded, that is, the frequency of the prominent peak retained after filtering.

[0088] Furthermore, the peak value p is detected in the envelope E(f). i and its corresponding frequency fi.

[0089] p i =E(fi)E(fi) is a local maximum

[0090] Detection peak p i Adjacent valley value v i-1 and v i+1 .

[0091]

[0092] Filter the peaks by the following expression and keep the significant peaks:

[0093]

[0094] Map the filtered peak frequency fj to a predefined frequency range matrix u is the number of frequency range matrices.

[0095] S104: According to a standard preset frequency range of each drone, a first feature vector of the drone audio signal is determined based on the frequency of a prominent peak of each signal frame.

[0096] Specifically, because the distance between the drone and the microphone collector is not fixed in the actual application environment, the amplitude of each high-order harmonic is not a fixed value, and will change with the change of distance. The frequency value of each high-order harmonic is relatively fixed and basically does not change with the change of distance factors, so the extracted feature information should include frequency rather than amplitude.

[0097] For each drone, obtain several higher harmonic frequencies of the drone, set the floating range of each higher harmonic frequency with each higher harmonic frequency as the center, and use it as the standard preset frequency range of each drone. Assign values ​​to each vector variable according to whether the frequency of the prominent peak of each signal frame falls within the standard preset frequency range of each drone, and obtain the one-dimensional feature vector of each drone in different signal frames. Then, concatenate the one-dimensional feature vectors of each drone in the same signal frame to obtain the one-dimensional feature vector of each signal frame. Finally, concatenate the one-dimensional feature vectors of each consecutive signal frame to obtain the two-dimensional feature vector of the drone audio signal, which is recorded as the second feature vector.

[0098] S105: Performing model training on a convolutional neural network model based on the first eigenvector of the drone audio signal.

[0099] Specifically, the first eigenvector of the drone audio signal is used as a sample for training the convolutional neural network to perform model training on the training convolutional neural network model.

[0100] In this application, each sample is an X∈R 10×u A two-dimensional matrix.

[0101] The convolution layer is the core component of CNN, which is used to extract the features of input data. The convolution layer slides the filter (convolution kernel) on the input data, performs convolution operation, and generates a feature map. Each convolution kernel is responsible for capturing different features of the image, such as edges, textures, etc.

[0102] The dimension of the convolution kernel (filter) of the first convolution layer is 3×3, with a total of 8 filters, and the weight matrix of each filter is (kth filter), with a bias of The output Y of the convolution operation 1 The corresponding formula is:

[0103]

[0104] Among them, i and , are the position indexes of the output matrix, and k is the index of the filter. m and n are the indexes of the convolution kernel (or filter) on the input matrix. They represent the index positions of the rows and columns of the convolution kernel, and the output dimension remains unchanged (using 0 padding), so Y 1 ∈R 10×u×8 .

[0105] Batch normalization is used to normalize the output of the convolutional layer to speed up training. The normalized output is Z 1 , then:

[0106]

[0107]

[0108]

[0109]

[0110] Among them, γ k and is β k The learnable parameter, ∈, is a constant to prevent division by zero. k is the mean of the kth channel, is the variance of the kth channel, and H and W are the spatial dimensions.

[0111] The ReLU activation function sets negative values ​​to 0. The formula is:

[0112]

[0113] Output A after activation 1 ∈R 10×u×8 .

[0114] The pooling layer halves the size of the output, 1Take the maximum value of each 2×2 region. Pooling output P 1 The formula is:

[0115]

[0116] The output dimension is P 1 ∈R 5×u / 2×8 .

[0117] The operation of the second convolutional layer is similar to the first layer. The convolution kernel size is still 3×3, but the number of filters is increased to 16. Let the input be P 1 , then the output Y after convolution 2 The formula is:

[0118]

[0119] Output dimension Y 2 ∈R 5×u / 2×16 , batch normalization and ReLU activation are also applied to Y 2 , generating output A 2 ∈R 5 ×u / 2×16 .

[0120] In the fully connected layer, the convolution output is flattened into a vector vec(A 2 ), the dimension is 5×u / 2×16=40u. The output of the fully connected layer is:

[0121] F=W 3 vec(A 2 )+b 3

[0122] Among them, W 3 ∈R 3×40u is the weight matrix, b 3 ∈R 3 is the bias vector.

[0123] The Softmax function is used to convert the output of the fully connected layer into a probability distribution. The formula is:

[0124]

[0125] in is the predicted probability of the i-th category, and C=3 is the number of categories.

[0126] The Softmax function ensures that the sum of all output probabilities is 1, that is:

[0127]

[0128] It can be used to determine the predicted category of the input sample, usually selecting the category with the highest probability as the final classification result:

[0129]

[0130] S106: Using the trained convolutional neural network model to recognize the audio signal to be recognized.

[0131] Specifically, after the training is completed, when the drone needs to be acoustically identified, first collect the audio signal to be identified, divide the audio signal to be identified into frames to obtain a number of signal frames to be identified; calculate the power spectrum density of each signal frame to be identified, and determine the envelope of the power spectrum density of each signal frame to be identified according to the power spectrum density of each signal frame to be identified; detect and filter the peak of the envelope of the power spectrum density of each signal frame to be identified, and determine the frequency of the prominent peak retained after filtering each signal frame to be identified; according to the standard preset frequency range of each drone, determine the fourth eigenvector of the audio signal to be identified based on the frequency of the prominent peak of each signal frame to be identified; input the fourth eigenvector of the audio signal to be identified into the trained convolutional neural network model to obtain the signal category of the audio signal to be identified, so as to identify the audio signal to be identified. The recognition result is which drone the audio signal to be identified specifically contains or whether it is environmental noise.

[0132] In an alternative embodiment, see Figure 2 As shown, Figure 2 A flowchart of a method for calculating power spectrum density provided by the first embodiment of the present invention is shown, wherein the calculation of the power spectrum density of each signal frame includes steps S201 to S203:

[0133] S201: Perform windowing processing on each signal frame using a window function.

[0134] S202: Performing fast Fourier transform on each signal frame after the windowing process to obtain a frequency domain signal of each signal frame.

[0135] S203: Based on the number of fast Fourier transform points, the power spectrum density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0136] Specifically, we first use the window function ω(t) to represent each frame x n (t) Windowing is performed to obtain the processed signal frame x′ n (t):

[0137] x′ n (t) = x n (t)·ω(t)

[0138] Perform fast Fourier transform (FFT) on the windowed signal to obtain the frequency domain signal xn (f):

[0139]

[0140] P is the number of FFT points. When P decreases, the frequency resolution decreases. f is the frequency, k is the frequency index, and j is the imaginary unit.

[0141] Calculate the power spectral density of each frame

[0142]

[0143] In an optional implementation manner, determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame includes:

[0144] The power spectrum density of each signal frame is smoothed to obtain the envelope of the power spectrum density of each signal frame.

[0145] Specifically, a low-pass filter is used to filter the power spectral density P xx (f) is smoothed to obtain the envelope E(f).

[0146] Low-pass filter H(f), with a cutoff frequency of f c .

[0147]

[0148] Among them, b i and a i is the coefficient of the filter, z is the complex frequency variable. M is the order of the numerator polynomial, N is the order of the denominator polynomial, b0, b1, b M are the preset numerator coefficient, a1, a N are the preset denominator coefficients respectively.

[0149] To eliminate phase distortion, a bilateral filtering filtfilt(·) function is used.

[0150] E(f) = filtfilt(b, a, P xx (f))

[0151] Among them, b is the numerator coefficient and a is the denominator coefficient.

[0152] In an alternative embodiment, see Figure 3 As shown, Figure 3 The flowchart of the method for determining the first eigenvector provided by the first embodiment of the present invention is shown, wherein the first eigenvector of the drone audio signal is determined based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each drone, including steps S301 to S303:

[0153] S301: For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone.

[0154] Specifically, for each signal frame, it is determined whether the frequency of the prominent peak of the signal frame is near the j-th higher harmonic frequency of the drone, that is, whether it falls within the j-th standard preset frequency range.

[0155] S302: Constructing a second feature vector of the drone in each signal frame based on the frequency of the prominent peak of each signal frame within a standard preset frequency range of the drone.

[0156] Specifically, if yes, the frequency of the prominent peak of the signal frame is placed in the j-th dimension blank variable; if no, the value of the j-th dimension blank variable is set to 0. Then continue to judge whether the frequency of the prominent peak of the signal frame is near the j+1th high harmonic frequency of the drone and whether it falls into the j+1th standard preset frequency range. If yes, the frequency of the prominent peak of the signal frame is placed in the j+1th dimension blank variable; if no, the value of the j+1th dimension blank variable is set to 0, until the second eigenvector of the drone in each signal frame is generated after the judgment of whether it falls into all standard preset frequency ranges is completed.

[0157] Furthermore, check whether the peak frequency falls within a certain frequency range:

[0158]

[0159] The feature vector y extracted from each frame j is [y1,y2,…y u ].

[0160] S303: Generate a first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame.

[0161] Specifically, the second eigenvectors of each drone in each signal frame are concatenated to obtain the first eigenvector of the drone audio signal.

[0162] In an alternative embodiment, see Figure 4 As shown, Figure 4 A flowchart of a specific first feature vector determination method provided in Embodiment 1 of the present invention is shown, wherein the first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame, including steps S401 to S402:

[0163] S401: For each signal frame, the second eigenvectors of each UAV in the signal frame are integrated to obtain the third eigenvector of each signal frame.

[0164] Specifically, when the second eigenvector of each drone in the signal frame is a horizontal vector, the second eigenvector of each drone in the signal frame is horizontally spliced ​​to obtain the third eigenvector of each signal frame. When the second eigenvector of each drone in the signal frame is a vertical vector, the second eigenvector of each drone in the signal frame is vertically spliced ​​to obtain the third eigenvector of each signal frame.

[0165] S402: Integrate the third eigenvectors of each signal frame to obtain a first eigenvector of the drone audio signal.

[0166] Specifically, when the third eigenvector of each signal frame is a horizontal vector, the third eigenvectors of each signal frame are vertically spliced ​​to obtain the first eigenvector of the drone audio signal. When the third eigenvector of each signal frame is a vertical vector, the third eigenvectors of each signal frame are horizontally spliced ​​to obtain the first eigenvector of the drone audio signal.

[0167] For example, the feature vectors extracted from 10 consecutive frames are combined into a two-dimensional feature matrix:

[0168]

[0169] The drone acoustic recognition method provided in the present application is applied to a processor in a drone acoustic recognition system, and the system also includes a microphone collector, which is used to collect drone audio signals. Figure 5 As shown, Figure 5 The flowchart of the working process of the drone acoustic identification system provided by the first embodiment of the present invention is shown, wherein the signal x(t) is collected, and then the signal is framed to reduce the frequency resolution, and then the window function (Hamming window) is selected to use the Welch method to calculate the power spectrum density, and then the envelope of the power spectrum density is extracted, and the suspected envelope spectrum peak is found through the peak search function to determine whether the envelope spectrum peak is prominent enough. Otherwise, the data is discarded. If so, the frequency corresponding to the envelope spectrum peak at this time is recorded. For each drone, it is determined whether the frequency is near the j-th higher harmonic frequency of drone A. If so, the frequency f at this time is i Put it into the j-th dimension blank variable, and then continue to judge whether the frequency is near the j+1th harmonic frequency of drone A, that is, execute j=j+1; if not, set the value of the j-th dimension blank variable to 0 to form a one-dimensional feature vector of drone A. Similarly, judge whether the frequency is near the j-th harmonic frequency of drone B. If so, set the frequency f at this time iPut it into the j-th dimension blank variable, and then continue to determine whether the frequency is near the j+1th higher harmonic frequency of drone B, that is, execute j=j+1; if not, set the value of the j-th dimension blank variable to 0 to form a one-dimensional feature vector of drone B. After combining the one-dimensional feature vector of drone A and the one-dimensional feature vector of drone B into a one-dimensional feature vector, 50 consecutive frames are combined into a two-dimensional feature vector, which is input into the CNN classification model for training and application to obtain the output result.

[0170] In the process of applying the drone acoustic recognition method provided by this application, see Figure 6 As shown, Figure 6 The schematic diagram shows the characteristics of the first type of drone provided by the first embodiment of the present invention, wherein the horizontal axis in the figure represents the sample sequence number and the vertical axis represents the frequency. Figure 7 As shown, Figure 7 The schematic diagram shows the characteristics of the second type of drone provided by the first embodiment of the present invention, wherein the horizontal axis in the figure represents the sample sequence number and the vertical axis represents the frequency. Figure 8 As shown, Figure 8 A schematic diagram of an environmental noise feature provided by the first embodiment of the present invention is shown, wherein the horizontal axis in the figure represents the sample sequence number and the vertical axis represents the frequency. Figure 6 , Figure 7 and Figure 8 It can be observed that the extracted acoustic features of different drone models are different and are significantly different from the ambient noise. Fig. 9 As shown, Fig. 9 The first embodiment of the present invention provides a schematic diagram of the experimental audio of the first type of drone provided in the first embodiment of the present invention, wherein the first type of drone first hovers and then flies to 70m. Fig.10 As shown, Fig.10 A schematic diagram of the first type of drone audio prediction provided by the first embodiment of the present invention is shown, wherein the figure shows that after the CNN is trained, the model is allowed to classify and identify Fig. 9 In the audio, category 1 represents ambient noise, category 2 represents the first type of drone, and category 3 represents the second type of drone. Fig.10 It can be observed that whether it is hovering or flying up to 70m, the model can identify it as the first type of drone. Fig.11 As shown, Fig.11 A schematic diagram of the experimental audio of the second type of drone provided in the first embodiment of the present invention is shown, wherein the figure shows the experimental audio of the second type of drone hovering. Fig.12 As shown, Fig.12A schematic diagram of the second type of drone audio prediction provided by the first embodiment of the present invention is shown, wherein the figure shows that after the CNN is trained, the model is used to classify and identify Fig.11 In the audio, category 1 represents ambient noise, category 2 represents the first type of drone, and category 3 represents the second type of drone. Fig.12 It can be observed that the model can identify the second type of drone. Fig.13 As shown, Fig.13 The figure shows a schematic diagram of the experimental audio of the drone with environmental noise provided by the first embodiment of the present invention, wherein the figure shows the experimental audio of environmental noise. Fig.14 As shown, Fig.14 A schematic diagram of the drone audio prediction of environmental noise provided by the first embodiment of the present invention is shown, wherein the figure shows that after the CNN is trained, the model is used to classify and identify Fig.14 In the audio, category 1 represents ambient noise, category 2 represents the first type of drone, and category 3 represents the second type of drone. Fig.14 It can be observed that the model can identify the environmental noise.

[0171] Embodiment 2

[0172] Embodiment 2 of the present invention provides a drone acoustic recognition device, see Fig.15 As shown, Fig.15 The structure diagram of a drone acoustic identification device provided in the second embodiment of the present invention is shown, wherein the device comprises:

[0173] The signal acquisition module 1501 is used to collect the drone audio signal and divide the drone audio signal into frames to obtain a plurality of signal frames;

[0174] The signal processing module 1502 is used to calculate the power spectrum density of each signal frame, and determine the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame;

[0175] The signal filtering module 1503 is used to detect and filter the peak value of the envelope of the power spectrum density of each signal frame, and determine the frequency of the prominent peak value retained after filtering each signal frame;

[0176] A feature vector determination module 1504 is used to determine a first feature vector of the drone audio signal based on the frequency of the prominent peak of each signal frame according to a standard preset frequency range of each drone;

[0177] A model training module 1505 is used to perform model training on a convolutional neural network model based on the first feature vector of the drone audio signal;

[0178] The signal recognition module 1506 is used to recognize the audio signal to be recognized by using the trained convolutional neural network model.

[0179] In an optional implementation, the calculating the power spectrum density of each signal frame includes:

[0180] Perform windowing processing on each signal frame using a window function;

[0181] Perform fast Fourier transform on each signal frame after windowing to obtain the frequency domain signal of each signal frame:

[0182] Based on the number of fast Fourier transform points, the power spectrum density of each signal frame is determined according to the frequency domain signal of each signal frame.

[0183] In an optional implementation manner, determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame includes:

[0184] The power spectrum density of each signal frame is smoothed to obtain the envelope of the power spectrum density of each signal frame.

[0185] In an optional embodiment, the first feature vector of the drone audio signal is determined based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each drone, including:

[0186] For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone;

[0187] Constructing a second feature vector of the drone in each signal frame based on the frequency of the prominent peak of each signal frame within a standard preset frequency range of the drone;

[0188] A first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

[0189] In an optional embodiment, the generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes:

[0190] For each signal frame, the second eigenvectors of each drone in the signal frame are integrated to obtain the third eigenvector of each signal frame;

[0191] The third eigenvectors of each signal frame are integrated to obtain the first eigenvector of the drone audio signal.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0193] The drone acoustic identification device provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0194] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0195] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0197] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0198] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0199] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features thereof; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A drone acoustic identification method, characterized in that: The method comprises: Collecting a drone audio signal, and dividing the drone audio signal into frames to obtain a plurality of signal frames; Calculating the power spectrum density of each signal frame, and determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame; Detect and filter the peak value of the envelope of the power spectrum density of each signal frame to determine the frequency of the prominent peak value retained after filtering each signal frame; Determining a first eigenvector of the drone audio signal based on the frequency of a prominent peak of each signal frame according to a standard preset frequency range of each drone; Performing model training on a convolutional neural network model based on a first eigenvector of the drone audio signal; The trained convolutional neural network model is used to identify the audio signal to be identified.

2. The method according to claim 1, characterized in that The calculating the power spectrum density of each signal frame comprises: Perform windowing processing on each signal frame using a window function; Perform fast Fourier transform on each signal frame after windowing to obtain the frequency domain signal of each signal frame: Based on the number of fast Fourier transform points, the power spectrum density of each signal frame is determined according to the frequency domain signal of each signal frame.

3. The method according to claim 1, characterized in that Determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame includes: The power spectrum density of each signal frame is smoothed to obtain the envelope of the power spectrum density of each signal frame.

4. The method according to claim 1, characterized in that: The method of determining the first feature vector of the drone audio signal based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each drone includes: For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone; Constructing a second feature vector of the drone in each signal frame based on the frequency of the prominent peak of each signal frame within a standard preset frequency range of the drone; A first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

5. The method according to claim 4, characterized in that The generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes: For each signal frame, the second eigenvectors of each drone in the signal frame are integrated to obtain the third eigenvector of each signal frame; The third eigenvectors of each signal frame are integrated to obtain the first eigenvector of the drone audio signal.

6. A drone acoustic identification device, characterized in that: The device comprises: A signal acquisition module, used for acquiring the drone audio signal, and dividing the drone audio signal into frames to obtain a plurality of signal frames; A signal processing module, used to calculate the power spectrum density of each signal frame, and determine the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame; A signal filtering module, used to detect and filter the peak value of the envelope of the power spectrum density of each signal frame, and determine the frequency of the prominent peak value retained after filtering each signal frame; A feature vector determination module, configured to determine a first feature vector of the drone audio signal based on the frequency of a prominent peak of each signal frame according to a standard preset frequency range of each drone; A model training module, used for performing model training on a convolutional neural network model based on a first feature vector of the drone audio signal; The signal recognition module is used to recognize the audio signal to be recognized using the trained convolutional neural network model.

7. The device according to claim 6, characterized in that The calculating the power spectrum density of each signal frame comprises: Perform windowing processing on each signal frame using a window function; Perform fast Fourier transform on each signal frame after windowing to obtain the frequency domain signal of each signal frame: Based on the number of fast Fourier transform points, the power spectrum density of each signal frame is determined according to the frequency domain signal of each signal frame.

8. The device according to claim 6, characterized in that Determining the envelope of the power spectrum density of each signal frame according to the power spectrum density of each signal frame includes: The power spectrum density of each signal frame is smoothed to obtain the envelope of the power spectrum density of each signal frame.

9. The device according to claim 6, characterized in that The method of determining the first feature vector of the drone audio signal based on the frequency of the prominent peak of each signal frame according to the standard preset frequency range of each drone includes: For each drone, determine in turn whether the frequency of the prominent peak of each signal frame is within the standard preset frequency range of the drone; Constructing a second feature vector of the drone in each signal frame based on the frequency of the prominent peak of each signal frame within a standard preset frequency range of the drone; A first feature vector of the drone audio signal is generated based on the second feature vector of each drone in each signal frame.

10. The device according to claim 9, characterized in that The generating the first feature vector of the drone audio signal based on the second feature vector of each drone in each signal frame includes: For each signal frame, the second eigenvectors of each drone in the signal frame are integrated to obtain the third eigenvector of each signal frame; The third eigenvectors of each signal frame are integrated to obtain the first eigenvector of the drone audio signal.

Images