Unmanned aerial vehicle sound recognition method and device

Abstract

The invention provides an unmanned aerial vehicle sound recognition method and device, and the method comprises the steps: obtaining original unmanned aerial vehicle sound data, and carrying out the data expansion of the original unmanned aerial vehicle sound data, and obtaining optimized unmanned aerial vehicle sound data; performing feature extraction on the optimized unmanned aerial vehicle sound data by using different feature extraction algorithms to obtain sound sub-features corresponding to the feature extraction algorithms; performing feature fusion on the sound sub-features corresponding to the feature extraction algorithms to obtain a target sound feature; training an unmanned aerial vehicle sound recognition model based on the target sound features; and performing sound recognition on the unmanned aerial vehicle by using the trained unmanned aerial vehicle sound recognition model. By adopting the scheme, the accuracy of unmanned aerial vehicle sound recognition is improved, and the reliability and effectiveness of an unmanned aerial vehicle sound recognition result are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method and device for identifying sound of unmanned aerial vehicles. Background Art

[0002] With the rapid development and popularization of drone technology, drones are increasingly used in various fields. However, the widespread use of drones has also brought about a series of security and privacy issues. In order to effectively monitor and manage drones, drone sound recognition technology has emerged. This technology detects and identifies drones by identifying and analyzing the sound characteristics generated by drones during flight.

[0003] In the prior art, when performing drone sound recognition, model training often relies on limited drone sound data samples, which may lead to poor performance of the model in identifying drones of different models and in different flight states, especially the lack of sound data under different environmental noise conditions, which limits the generalization ability of the model. At the same time, the traditional feature extraction method is limited to single feature extraction, which will lead to the inability to fully capture the key information in the drone sound, thereby reducing the accuracy of drone sound recognition and making drone sound recognition unreliable. Summary of the invention

[0004] In view of this, the object of the present invention is to provide a drone sound recognition method and device to improve the accuracy of drone sound recognition and ensure the reliability and effectiveness of drone sound recognition results.

[0005] In a first aspect, an embodiment of the present application provides a method for drone sound recognition, the method comprising:

[0006] Acquire original drone sound data, and perform data expansion on the original drone sound data to obtain optimized drone sound data;

[0007] Using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm;

[0008] The sound sub-features corresponding to each feature extraction algorithm are fused to obtain the target sound feature;

[0009] Training a drone sound recognition model based on the target sound features;

[0010] Use the trained drone sound recognition model to identify the drone sound.

[0011] Optionally, the using a trained drone sound recognition model to perform sound recognition on the drone includes:

[0012] Obtain the sound data of the drone to be identified;

[0013] Extracting features of the drone sound data to be identified to obtain sound features to be identified;

[0014] The sound features to be identified are input into a trained drone sound recognition model to achieve drone sound recognition.

[0015] Optionally, performing data expansion on the original drone sound data to obtain optimized drone sound data includes:

[0016] The sound volume of the original drone sound data is adjusted,

[0017] and / or adding noise of different intensities to the original drone sound data,

[0018] And / or performing time perturbation on the original drone sound data to obtain the optimized drone sound data.

[0019] Optionally, the extracting features of the optimized drone sound data using different feature extraction algorithms to obtain sound sub-features corresponding to each feature extraction algorithm includes:

[0020] Performing feature extraction on the optimized drone sound data based on Mel-frequency cepstral coefficients to obtain a first sound sub-feature;

[0021] The optimized drone sound data is subjected to feature extraction based on gamma frequency cepstral coefficients to obtain a second sound sub-feature.

[0022] Optionally, the step of fusing the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature includes:

[0023] Determine the most informative target sub-feature among the sound sub-features corresponding to each feature extraction algorithm;

[0024] The target sound feature is obtained by performing feature fusion on the target sub-features in each sound sub-feature using a preset feature fusion algorithm.

[0025] In a second aspect, an embodiment of the present application provides a drone sound recognition device, the device comprising:

[0026] A drone sound data acquisition module is used to acquire original drone sound data and perform data expansion on the original drone sound data to obtain optimized drone sound data;

[0027] A sound feature extraction module, used to use different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm;

[0028] The sound feature fusion module is used to fuse the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature;

[0029] A model training base module, used for training a drone sound recognition model based on the target sound features;

[0030] The drone sound recognition module is used to recognize the sound of drones using the trained drone sound recognition model.

[0031] Optionally, the using a trained drone sound recognition model to perform sound recognition on the drone includes:

[0032] Obtain the sound data of the drone to be identified;

[0033] Extracting features of the drone sound data to be identified to obtain sound features to be identified;

[0034] The sound features to be identified are input into a trained drone sound recognition model to achieve drone sound recognition.

[0035] Optionally, performing data expansion on the original drone sound data to obtain optimized drone sound data includes:

[0036] The sound volume of the original drone sound data is adjusted,

[0037] and / or adding noise of different intensities to the original drone sound data,

[0038] And / or performing time perturbation on the original drone sound data to obtain the optimized drone sound data.

[0039] Optionally, the extracting features of the optimized drone sound data using different feature extraction algorithms to obtain sound sub-features corresponding to each feature extraction algorithm includes:

[0040] Performing feature extraction on the optimized drone sound data based on Mel-frequency cepstral coefficients to obtain a first sound sub-feature;

[0041] The optimized drone sound data is subjected to feature extraction based on gamma frequency cepstral coefficients to obtain a second sound sub-feature.

[0042] Optionally, the step of fusing the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature includes:

[0043] Determine the most informative target sub-feature among the sound sub-features corresponding to each feature extraction algorithm;

[0044] The target sound feature is obtained by performing feature fusion on the target sub-features in each sound sub-feature using a preset feature fusion algorithm.

[0045] The technical solution provided by this application includes but is not limited to the following beneficial effects:

[0046] This application generates more diverse drone sound data through data expansion technology (such as volume adjustment, adding noise, time disturbance, etc.), which can more comprehensively reflect the sound characteristics of drones in different flight states, improve the generalization ability of the model, and thus improve the accuracy of recognition. And by using different feature extraction algorithms to process the sound data, a variety of sound sub-features can be extracted. Through feature fusion technology, these sound sub-features are combined and optimized to form more representative target sound features, further improving the accuracy and robustness of recognition. In addition, by adding sound data under different environmental noise conditions during the training process, the adaptability of the model to high noise environments can be improved. Finally, using the trained drone sound recognition module for drone sound recognition can improve the accuracy of drone sound recognition and ensure the reliability and effectiveness of drone sound recognition results.

[0047] To summarize, the technical solution of the present application effectively solves the problems of data limitations, insufficient feature extraction, low model training efficiency and poor environmental adaptability existing in the existing drone sound recognition technology through improvements in data expansion and optimization, multi-feature fusion, improved model training efficiency and enhanced environmental adaptability, thereby improving the accuracy and robustness of drone sound recognition.

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 A flowchart of a drone sound recognition method provided by Embodiment 1 of the present invention is shown;

[0051] Figure 2 A flowchart of a specific drone sound recognition method provided by Embodiment 1 of the present invention is shown;

[0052] Figure 3A flow chart of a method for extending sound data provided by the first embodiment of the present invention is shown;

[0053] Figure 4 A flow chart of a second sound data expansion method provided by the first embodiment of the present invention is shown;

[0054] Figure 5 A flow chart of a sound feature method based on Mel-cepstral coefficients provided in the first embodiment of the present invention is shown;

[0055] Figure 6 A flow chart of a sound feature method based on gamma frequency cepstral coefficients provided in the first embodiment of the present invention is shown;

[0056] Figure 7 A flow chart of a sound feature fusion method provided in Embodiment 1 of the present invention is shown;

[0057] Figure 8 A schematic diagram of a residual unit structure provided by the first embodiment of the present invention is shown;

[0058] Fig. 9 A schematic diagram showing a second residual unit structure provided by the first embodiment of the present invention is shown;

[0059] Fig.10 A schematic diagram showing a drone sound recognition model structure provided by the first embodiment of the present invention is shown;

[0060] Fig.11 A schematic diagram showing a practical application of a drone sound recognition method provided by the first embodiment of the present invention;

[0061] Fig.12 A schematic diagram of the structure of a drone sound recognition device provided by Embodiment 2 of the present invention is shown. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0063] Embodiment 1

[0064] To facilitate understanding of this application, Figure 1 The flowchart of a drone sound recognition method provided by the first embodiment of the present invention is shown to describe the first embodiment of the present application in detail.

[0065] See also Figure 1 As shown, Figure 1 A flowchart of a drone sound recognition method provided by Embodiment 1 of the present invention is shown, wherein the method comprises steps S101 to S105:

[0066] S101: Acquire original drone sound data, and perform data expansion on the original drone sound data to obtain optimized drone sound data.

[0067] Specifically, the sound data generated by the drone in various environments is collected, which may include sounds of various states such as takeoff, flight, landing, and failure. In order to improve the generalization ability of the model, it is usually necessary to expand the original sound data, including data enhancement (such as adding noise, changing speed, changing pitch, etc.) and increasing the amount of data (such as copying, rotating, flipping, etc., although these are more used for image data, sound data can also be processed similarly, such as changing pitch and speed, etc.).

[0068] S102: Using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm.

[0069] Specifically, different feature extraction algorithms are used to extract different sound characteristics from the sound data, such as Mel-frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), chrominance features, etc. Each feature extraction algorithm generates a set of specific sound features, which can be regarded as different representations or "sub-features" of the sound data.

[0070] S103: Fusing the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature.

[0071] Specifically, the sound sub-features obtained by different feature extraction algorithms are merged to create a more comprehensive sound feature set. This can be achieved through simple splicing, weighted averaging, principal component analysis (PCA), etc. The fused feature set is the target sound feature, which contains multiple representations of sound data and helps improve the model's ability to recognize drone sounds.

[0072] S104: Training a drone sound recognition model based on the target sound features.

[0073] Specifically, a drone sound recognition model is trained using the target sound features as input. This model can be a deep learning model (such as a convolutional neural network CNN, a recurrent neural network RNN, or a combination thereof) or other machine learning models. The model learns to map the target sound features to the corresponding drone sound categories (such as takeoff, flight, landing, etc.), thereby realizing the recognition of drone sounds.

[0074] S105: Use the trained drone sound recognition model to perform sound recognition on the drone.

[0075] Specifically, after training is completed, the trained model can be used to identify new drone sound data. The model will output a probability distribution, indicating the probability that the input sound belongs to each category. Based on the probability distribution, the category with the highest probability can be selected as the recognition result, or a threshold can be set according to the needs of the application scenario to determine whether the recognition is successful.

[0076] In an alternative embodiment, see Figure 2 As shown, Figure 2 A flowchart of a specific drone sound recognition method provided by the first embodiment of the present invention is shown, wherein the method of using a trained drone sound recognition model to perform sound recognition on a drone includes steps S201 to S203:

[0077] S201: Acquire the sound data of the drone to be identified.

[0078] Specifically, collect and obtain drone sound data to be identified. This data may come from real-time recordings, pre-recorded audio files, or drone sound samples obtained from other data sources. Ensure that the format and quality of the sound data to be identified are consistent with the data used to train the model to avoid recognition errors caused by data format mismatch or quality issues.

[0079] S202: Extracting features of the drone sound data to be identified to obtain features of the sound to be identified.

[0080] Specifically, after obtaining the drone sound data to be identified, the sound features need to be extracted using the same feature extraction algorithm used when training the model. These feature extraction algorithms can be Mel Frequency Cepstral Coefficients (MFCC), Linear Prediction Cepstral Coefficients (LPCC), etc., depending on the feature extraction method used when training the model. The feature extraction process needs to maintain consistency and accuracy to ensure that the sound features to be identified are comparable to the sound features used when training the model.

[0081] S203: Inputting the sound features to be recognized into a trained drone sound recognition model to realize drone sound recognition.

[0082] Specifically, the extracted sound features to be identified are input into the trained drone sound recognition model. The model will calculate based on the input sound features and output a probability distribution, indicating the possibility that the input sound belongs to each drone sound category. Based on the output of the model, the category with the highest probability can be selected as the recognition result. If a threshold is set, the threshold can also be used to determine whether the recognition is successful. Before inputting features into the model, preprocessing may be required, such as data normalization, feature scaling, etc., to ensure that the model can correctly process the input data. In addition, attention should also be paid to the real-time and accuracy of the model to meet the needs of actual application scenarios.

[0083] In an optional embodiment, the step of performing data expansion on the original drone sound data to obtain optimized drone sound data includes:

[0084] The sound volume of the original drone sound data is adjusted.

[0085] For details, see Figure 3 As shown, Figure 3 A flow chart of a sound data expansion method provided in Embodiment 1 of the present invention is shown, wherein the loudness of the original drone sound is adjusted to obtain drone sounds of different loudnesses, and the volume of the original sound data is increased or decreased as needed. This adjustment can be linear or nonlinear based on a certain algorithm.

[0086] And / or adding noise of different intensities to the original drone sound data.

[0087] For details, see Figure 4 As shown, Figure 4 The flowchart of the second sound data expansion method provided by the first embodiment of the present invention is shown, wherein different intensities of noise are injected into the collected drone sound to generate drone sounds injected with different noises to simulate a complex environment. Different types of noise (such as white noise, pink noise, etc.) are selected from the noise library, and their intensities are adjusted as needed, and then mixed with the original sound data.

[0088] And / or performing time perturbation on the original drone sound data to obtain the optimized drone sound data.

[0089] Specifically, the original drone sound data is time-stretched by adjusting the playback time of the sound data without changing the sound pitch.

[0090] Through these three methods, various drone sound data are generated and a drone sound database is established.

[0091] In an optional embodiment, the method of using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm includes:

[0092] The optimized drone sound data is subjected to feature extraction based on the Mel-frequency cepstral coefficients to obtain a first sound sub-feature.

[0093] Specifically, the Mel frequency cepstral coefficient Mel(f) is the Mel Frequency Cepstral Coefficient. The sensitivity of the human ear to sound is proportional to the logarithm of the frequency. The Mel scale uses this feature, and its relationship with frequency is as follows:

[0094]

[0095] f is the actual frequency of the recorded sound signal, and the above formula converts the actual frequency into Mel frequency.

[0096] The mel-spectorgram obtained by Fourier transform and triangular filtering is also a way to analyze sound signals. The difference between the two is that MFCC performs logarithm and discrete cosine transform (DCT). The main task of DCT is to reduce the feature dimension of the sound and achieve the effect of removing redundancy.

[0097] See also Figure 5 As shown, Figure 5 The flowchart of a sound feature method based on Mel-frequency cepstral coefficients provided by the first embodiment of the present invention is shown, wherein a sound signal is first input and then pre-emphasized. The essence of pre-emphasis is to pass the speech signal through a high-pass filter:

[0098] H(z)=1-μz -1

[0099] This formula represents a high-pass filter H(z), whose time domain representation is y(n)=x(n)-μ*x(n-1), where the weighted version of the previous input x(n-1) is subtracted from the current input x(n) to get y(n). This is a filtering process, and z is a complex variable.

[0100] Where μ is a constant, usually 0.97. Pre-emphasis enhances the low-frequency part of the input signal power spectrum, avoiding signal distortion caused by excessive low frequency and weak high frequency. The processed signal S1(n) is:

[0101] S1(n)=s(n)-μs(n-1)

[0102] n is the number of sample points of the sound signal. The mathematical representation of the sound signal is a 1*n matrix form, where n=time*sampling rate, and s(n) is the original audio signal.

[0103] The pre-emphasized data is then framed and windowed.

[0104] The specific process of framing is: combine several sampling points into one frame, the frame length is generally 128 to 512, and each frame time is about 20 to 30ms. In order to avoid too large a difference between two adjacent frames, an overlap is usually reserved between the two frames, and the overlap rate is usually 50%.

[0105] The specific processing of adding Hamming window is as follows: If no processing is performed, the two ends of each frame signal will be discontinuous, and the Gibbs phenomenon will appear after Fourier expansion. Adding Hamming window can make the signals at both ends of the frame gradually weaken to 0, ensuring the continuity of the two frame signals. If S2(n,i) is defined as the i-th frame audio signal, the windowing operation can be expressed as:

[0106] S(n) represents a sound signal, S2(n,i) represents the audio signal of the i-th frame after framing (framing means dividing a sound into n small segments), and S2 represents the next frame signal.

[0107]

[0108] Among them, the a value determines the filtering effect of the Hamming window, and a is usually taken as 0.46. N is the frame length and also the width of the Hamming window.

[0109] Then, the windowed signal is subjected to FFT Fourier transform to obtain the Fourier frequency signal spectrum F(k,i).

[0110]

[0111] j is the imaginary unit and F(k,i) is the kth sample represented in the frequency domain.

[0112] The Fourier spectrum is input into a set of Mel-scaled triangular filter banks, also known as Mel filter banks. After filtering, the signal is logarithmic and logarithmic operations are performed to simulate human hearing. The main purpose of adding this filter is to smooth the spectrum, eliminate harmonics, highlight the resonance peaks of the original sound, and also reduce the amount of calculation. The logarithmic energy s(m) output by each filter bank is:

[0113]

[0114] m is the mth filter, and there are M filters in total.

[0115] Where M is the number of filters, H m (k) is the frequency response of the triangular filter.

[0116] Finally, discrete cosine transform (DCT) is performed. The function of DCT is to losslessly reduce the dimension of the filtered signal and remove redundant signals in the sound. The MFCC used for sound recognition generally takes the first 13 bits. Let f(k,l) be a two-dimensional discrete function, k,l=0,1,…,N-1, F(x,y) is the returned cepstrum signal, then the calculation formula is:

[0117]

[0118] Among them, F(x,y) represents the coefficient matrix of discrete cosine transform (representing the cepstrum signal),

[0119]

[0120] The optimized drone sound data is subjected to feature extraction based on gamma frequency cepstral coefficients to obtain a second sound sub-feature.

[0121] For details, see Figure 6 As shown, Figure 6 The flowchart of a sound feature method based on gamma frequency cepstral coefficients provided by the first embodiment of the present invention is shown, wherein the feature extraction process based on GFCC gamma frequency cepstral coefficients includes inputting sound signals, pre-emphasis processing, framing and windowing processing, and Fourier transform processing. For details, please refer to the corresponding step description of feature extraction based on Mel cepstral coefficients mentioned above. After Fourier transform, the following steps are also included:

[0122] Step 1: filter the speech signal through a Gammatone filter bank;

[0123] Step 2, performing logarithmic operation on the filtered signal to obtain a logarithmic energy spectrum;

[0124] Step 3, calculating the inverse Fourier transform of the logarithmic energy spectrum to obtain the cepstrum;

[0125] Step 4: Extract characteristic coefficients, namely GFCC, from the cepstrum.

[0126] In an alternative embodiment, see Figure 7 As shown, Figure 7 A flow chart of a sound feature fusion method provided in the first embodiment of the present invention is shown, wherein the sound sub-features corresponding to each feature extraction algorithm are fused to obtain the target sound feature, including steps S701 to S702:

[0127] S701: Determine the most informative target sub-feature among the sound sub-features corresponding to each feature extraction algorithm.

[0128] Specifically, the sound sub-features corresponding to each feature extraction algorithm are extracted independently, and before fusion, the most informative part of each feature is selected.

[0129] For the Mel frequency cepstral coefficients and the gamma frequency cepstral coefficients, select the first 13 coefficients respectively;

[0130] S702: Using a preset feature fusion algorithm, feature fusion is performed on target sub-features in each sound sub-feature to obtain the target sound feature.

[0131] Specifically, the preset feature fusion algorithms include concatenation, weighted sum, principal component analysis (PCA), linear discriminant analysis (LDA), etc. The selection of these methods depends on the specific requirements of the task, the nature of the data, and the limitations of computing resources.

[0132] The drone sound recognition method provided by the present application also includes constructing a drone sound recognition model. When constructing the drone sound recognition model, the present application takes into account that the drone sound features in complex environments are not obvious, so the features are further extracted by increasing the number of network layers. At the same time, in order to solve the problem of model degradation and gradient disappearance as the number of network layers increases, residual blocks are introduced into the model. By introducing residual blocks, each of which contains a residual path and an identity mapping path, cross-layer connection is achieved.

[0133] The residual path uses a series of convolutional layers and activation functions to extract features, while the identity mapping path simplifies the network structure, allowing the input to skip certain layers and be added to the output, ensuring that the gradient is directly and effectively transferred during back propagation. By stacking these residual blocks, ResNet is able to build deep networks without losing performance. The end of the network usually includes global average pooling and fully connected layers for aggregating features and performing the final drone sound recognition task.

[0134] The mathematical expression of the convolution layer in the residual convolutional neural network in this application is:

[0135]

[0136] Where s(i,j): the element in the i-th row and j-th column of the output feature map.

[0137] X: Input matrix (feature matrix).

[0138] W: convolution kernel matrix (convolution filter).

[0139] m,n: Index in the convolution kernel, used to traverse each element of the convolution kernel.

[0140] x(i+m,j+n): The element in the input matrix that is aligned with the convolution kernel.

[0141] w(m,n): weights in the convolution kernel matrix.

[0142] The size of the feature map after the convolution layer operation is calculated by the following specific formula:

[0143]

[0144] Where H and W are the height and width of the input feature map respectively.

[0145] P: Padding, which is the number of zero padding layers added to the edges of the input feature map.

[0146] FH and FW: are the sizes of the convolution kernel in height and width respectively.

[0147] S: stride, that is, the interval at which the convolution kernel slides.

[0148] OH and OW: are the height and width of the output feature map, respectively.

[0149] ReLu activation function:

[0150]

[0151] Softmax function:

[0152]

[0153] Among them, Softmax(x i ) represents the value of the i-th element in the input vector after the Softmax function, that is, the probability prediction of the i-th category; x i represents the raw score (linear output) of the i-th element in the input vector, x j represents the raw score linear output of the jth element in the input vector; N represents the dimension of the input vector, that is, the number of categories; e represents the base of the natural logarithm. The Softmax function is often used in the output layer of a neural network to convert the model output into a probability prediction for each category, thereby performing multi-category classification.

[0154] Fully connected layer: Each node in the fully connected layer is connected to all nodes in the previous layer to combine the features extracted previously. Due to its fully connected nature, the fully connected layer generally has the most parameters. At the end of the convolutional neural network, there are often one or two fully connected layers. The fully connected layer generally converts the two-dimensional feature map output by the convolution into a one-dimensional vector. The expression of the output layer is:

[0155] z (n) =W(n) ·a (n-1) +b (n)

[0156]

[0157] z (n) is the linear output of the nth layer, W (n) The weight matrix of the nth layer, a (n-1) is the activation output of layer n-1, b (n) Bias vector for the nth layer.

[0158] in is the predicted output of the model, and outputfunction is the activation function of the output layer.

[0159] Backpropagation and Gradient Descent: To train the network, we use the backpropagation algorithm to calculate the gradient of the loss function with respect to each parameter, and use gradient descent (or other optimization algorithms) to update the weights and biases:

[0160]

[0161] Where η is the learning rate, L represents the loss function, is the partial derivative of the loss function L with respect to the parameter W, where W represents the weight of the i-th parameter in the model, W new is the updated i-th parameter weight.

[0162] The residual path of the residual unit structure of ResNet-18 consists of two convolutional layers followed by a batch normalization layer and an activation layer. Its cross-layer connection path includes two methods. Figure 8 As shown, Figure 8 FIG. 4 is a schematic diagram showing a residual unit structure provided by the first embodiment of the present invention, wherein the input x1 is outputted intactly. Fig. 9 As shown, Fig. 9 The schematic diagram of the second residual unit structure provided by the first embodiment of the present invention is shown, wherein the output needs to be consistent with the output of the F(x1) path through a 1*1 convolutional layer kernel. Except for the above differences, the other modules are the same, for details, please refer to Fig.10 The final network structure used is shown in Figure 1. Fig.10 As shown, Fig.10A schematic diagram of a drone sound recognition model structure provided by the first embodiment of the present invention is shown, wherein in the drone sound recognition model, first, the data feature matrix is ​​connected through the input layer, the data is input into the 7*7 convolution layer, and the 7x7 convolution kernel is used to extract the features of the input image. Then the output of the convolution layer is normalized to improve the training speed and stability, and then the size of the feature map is reduced by taking the maximum value of the local area in the maximum pooling layer, reducing the amount of calculation while retaining important features. Next, a 3x3 convolution kernel is used to further extract features, batch normalization is performed, and nonlinearity is introduced using the ReLu activation function so that the model can learn more complex features, and then batch normalization is performed, and the above steps are iterated multiple times. Then, the output is kept consistent in shape with the output of the F(x1) path through the 1*1 convolution layer kernel, and the nonlinearity is introduced using the ReLu activation function, and then the above steps are returned and iterated at least 3 times, and then the output is output to the average pooling layer and the fully connected layer, and finally the required data is output. The two 3*3 convolutional layers contained in the figure, with batch normalization and ReLu activation function in the middle, are the residual blocks described in this application. The residual blocks add the input directly to the output through jump connections to solve the gradient vanishing problem in deep networks.

[0163] In practical applications, the drone sound recognition method provided by this application can be found in Fig.11 As shown, Fig.11 A schematic diagram of the practical application of a drone sound recognition method provided by the first embodiment of the present invention is shown, wherein drone sound data is first collected. To establish a drone sound recognition system, a high-precision microphone will be used to record the sounds of various types of drones at different distances (0-200 meters) and different flight attitudes, including take-off, approach, distance, turn, hover, rise and fall, etc., to ensure that each model of drone has a balanced sample. At the same time, an equal amount of non-drone environmental background noise is collected to enhance the model's ability to distinguish. After the recording is completed, the drone sound data is enhanced, including adjusting the volume of the sound, adding noise of different intensities, and performing time disturbance in three ways to generate more diverse drone sound data and build a drone sound database. Some samples are selected from the drone sound database for MFCC feature extraction and GFCC feature extraction, and then multi-feature fusion is performed. All sound data will be subjected to unified sampling rate adjustment, gain compensation and normalization processing to standardize the data format. In order to meet the input requirements of the neural network model, these pre-processed sound signals will be divided into small samples of 3 seconds, thereby forming a high-quality, diverse and structured sound data set, which provides a basis for training an accurate drone sound recognition model.

[0164] The ResNet network model is trained using the fused and processed sound features. Specifically, a residual neural network-based model architecture is constructed using the Python programming language to process sound data. Next, the model is trained by writing Python code, including loading and preprocessing data, compiling the model, executing the training process, and evaluating the model performance. This process uses the deep learning library PyTorch to define the network layer, loss function, optimizer, and iterate the model on the training data to optimize its parameters. Finally, a neural network model that can accurately identify drone sounds is trained.

[0165] After obtaining the trained drone sound recognition model, real-time sound data was collected in a variety of complex environments, including busy streets, noisy construction sites, crowded parks, and weather conditions with large interference, heavy rain and strong winds, by using high-precision data acquisition equipment. The original sound data of these scenes is used as the basis of the recognition system. After data collection, a unified preprocessing is performed on the data to ensure the quality of the data. Subsequently, the feature extraction algorithm is used to extract sound features from the preprocessed data, which are then input into the trained model for recognition and output the recognition results, thus completing the entire process of drone sound recognition.

[0166] In the above practical application process, data enhancement (including adjusting the volume of the sound, adding noise of different intensities, and performing time perturbation to generate more diverse drone sound data) is used to increase the drone sound database to solve the problem that the drone sound data is less due to the difficulty of obtaining aircraft. A drone sound database under various environments is established. By using microphone collection and data enhancement, the drone sound database contains drone sounds in various states and noise backgrounds. A residual neural network (ResNet) is built. Since the drone sound features in complex environments are not obvious, the features are further extracted by increasing the number of network layers. However, as the number of network layers increases, model degradation and gradient disappearance problems will occur. Therefore, a residual neural network is established to deal with the model degradation and gradient disappearance problems in deep network training. When extracting sound features, the drone sound features under multi-noise background sounds are not obvious. The MFCC and GFCC feature extraction methods are used to perform feature fusion to improve the generalization ability of the model so that the model can be applied to complex environments with multiple noises. Finally, during identification, a microphone array is used to collect the sounds of surrounding drones and adopt real-time identification. Compared with a single microphone, the use of a microphone array can obtain drone sounds at a longer distance, which can expand the range of drone sound recognition. Real-time recognition can quickly respond to identified illegal drones.

[0167] Embodiment 2

[0168] See also Fig.12 As shown, Fig.12 The structure diagram of a drone sound recognition device provided by the second embodiment of the present invention is shown, wherein the device comprises:

[0169] The drone sound data acquisition module 1201 is used to acquire original drone sound data and perform data expansion on the original drone sound data to obtain optimized drone sound data;

[0170] The sound feature extraction module 1202 is used to use different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm;

[0171] The sound feature fusion module 1203 is used to fuse the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature;

[0172] A model training base module 1204 is used to train a drone sound recognition model based on the target sound features;

[0173] The drone sound recognition module 1205 is used to perform sound recognition on the drone using the trained drone sound recognition model.

[0174] In an optional embodiment, the method of using a trained drone sound recognition model to perform sound recognition on a drone includes:

[0175] Obtain the sound data of the drone to be identified;

[0176] Extracting features of the drone sound data to be identified to obtain sound features to be identified;

[0177] The sound features to be identified are input into a trained drone sound recognition model to achieve drone sound recognition.

[0178] In an optional embodiment, the step of performing data expansion on the original drone sound data to obtain optimized drone sound data includes:

[0179] The sound volume of the original drone sound data is adjusted,

[0180] and / or adding noise of different intensities to the original drone sound data,

[0181] And / or performing time perturbation on the original drone sound data to obtain the optimized drone sound data.

[0182] In an optional embodiment, the method of using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm includes:

[0183] Performing feature extraction on the optimized drone sound data based on Mel-frequency cepstral coefficients to obtain a first sound sub-feature;

[0184] The optimized drone sound data is subjected to feature extraction based on gamma frequency cepstral coefficients to obtain a second sound sub-feature.

[0185] In an optional implementation, the step of fusing the sound sub-features corresponding to the feature extraction algorithms to obtain the target sound feature includes:

[0186] Determine the most informative target sub-feature among the sound sub-features corresponding to each feature extraction algorithm;

[0187] The target sound feature is obtained by performing feature fusion on the target sub-features in each sound sub-feature using a preset feature fusion algorithm.

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0189] The drone sound recognition device provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0190] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0193] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0194] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0195] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A drone sound recognition method, characterized in that: The method comprises: Acquire original drone sound data, and perform data expansion on the original drone sound data to obtain optimized drone sound data; Using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm; The sound sub-features corresponding to each feature extraction algorithm are fused to obtain the target sound feature; Training a drone sound recognition model based on the target sound features; Use the trained drone sound recognition model to recognize the drone sound.

2. The method according to claim 1, characterized in that The method of using the trained drone sound recognition model to perform sound recognition on the drone includes: Obtain the sound data of the drone to be identified; Extracting features of the drone sound data to be identified to obtain features of the sound to be identified; The sound features to be identified are input into a trained drone sound recognition model to achieve drone sound recognition.

3. The method according to claim 1, characterized in that The step of performing data expansion on the original drone sound data to obtain optimized drone sound data includes: The sound volume of the original drone sound data is adjusted, and / or adding noise of different intensities to the original drone sound data, And / or performing time perturbation on the original drone sound data to obtain the optimized drone sound data.

4. The method according to claim 1, characterized in that: The method of using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm includes: Performing feature extraction on the optimized drone sound data based on Mel-frequency cepstral coefficients to obtain a first sound sub-feature; The optimized drone sound data is subjected to feature extraction based on gamma frequency cepstral coefficients to obtain a second sound sub-feature.

5. The method according to claim 1, characterized in that The step of fusing the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature includes: Determine the most informative target sub-feature among the sound sub-features corresponding to each feature extraction algorithm; The target sound feature is obtained by performing feature fusion on the target sub-features in each sound sub-feature using a preset feature fusion algorithm.

6. A drone sound recognition device, characterized in that: The device comprises: A drone sound data acquisition module is used to acquire original drone sound data and perform data expansion on the original drone sound data to obtain optimized drone sound data; A sound feature extraction module, used to use different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm; The sound feature fusion module is used to fuse the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature; A model training base module, used for training a drone sound recognition model based on the target sound features; The drone sound recognition module is used to recognize the sound of drones using the trained drone sound recognition model.

7. The device according to claim 6, characterized in that The method of using the trained drone sound recognition model to perform sound recognition on the drone includes: Obtain the sound data of the drone to be identified; Extracting features of the drone sound data to be identified to obtain sound features to be identified; The sound features to be identified are input into a trained drone sound recognition model to achieve drone sound recognition.

8. The device according to claim 6, characterized in that The step of performing data expansion on the original drone sound data to obtain optimized drone sound data includes: The sound volume of the original drone sound data is adjusted, and / or adding noise of different intensities to the original drone sound data, And / or performing time perturbation on the original drone sound data to obtain the optimized drone sound data.

9. The device according to claim 6, characterized in that The method of using different feature extraction algorithms to extract features from the optimized drone sound data to obtain sound sub-features corresponding to each feature extraction algorithm includes: Performing feature extraction on the optimized drone sound data based on Mel-frequency cepstral coefficients to obtain a first sound sub-feature; The optimized drone sound data is subjected to feature extraction based on gamma frequency cepstral coefficients to obtain a second sound sub-feature.

10. The device according to claim 6, characterized in that The step of fusing the sound sub-features corresponding to each feature extraction algorithm to obtain the target sound feature includes: Determine the most informative target sub-feature among the sound sub-features corresponding to each feature extraction algorithm; The target sound feature is obtained by performing feature fusion on the target sub-features in each sound sub-feature using a preset feature fusion algorithm.

Images