Unmanned aerial vehicle sound recognition method and device, electronic equipment and storage medium

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for drone sound recognition. Its features include: preprocessing and performing wavelet transform on the acquired raw drone sound to determine a smoothed drone signal; extracting features from the smoothed drone signal using Mel-frequency cepstral coefficients to determine sound signal features; performing feature reduction and combination on the sound signal features using principal component analysis and linear discriminant analysis to determine model-extracted features; and identifying the target based on the model-extracted features using a pre-trained drone recognition model to determine target identification information. This enables drone identification and localization, improves the accuracy and real-time performance of drone sound recognition, and enhances the accurate identification and classification of drone sounds.

Description

Technical Field

[0001] This invention relates to the field of drone sound detection technology, and more particularly to drone sound recognition methods, devices, electronic devices, and storage media. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are aircraft managed by a control station and can be remotely controlled or fly autonomously. With advancements in UAV technology, they have been widely applied in various fields, with both commercial and personal UAVs in use. While UAV technology has brought many conveniences, the current regulatory measures are not yet perfect, resulting in unrestricted flight ranges and frequent incidents of illegal UAV flights. Restricting UAV flights requires the ability to detect and identify UAVs quickly and promptly, then locate and track them, limiting their flight range and preventing illegal flights. Current technologies can detect UAVs by sound detection to identify their presence in the surrounding environment. However, background noise from urban and traffic environments makes it difficult to identify and separate human and machine sounds. Furthermore, the varied and complex characteristics of UAV sounds increase the complexity and difficulty of sound recognition. Summary of the Invention

[0003] This invention provides a method, apparatus, electronic device, and storage medium for drone sound recognition, in order to solve the technical problem of the inability to efficiently identify and detect drones through their sounds.

[0004] According to one aspect of the present invention, a method for drone sound recognition is provided, comprising:

[0005] The collected raw drone sound is preprocessed and subjected to wavelet transform to determine the smoothed drone signal;

[0006] Mel frequency cepstral coefficients are extracted to extract features from the smoothed UAV signal to determine the sound signal characteristics;

[0007] The sound signal features are reduced in dimensionality and combined using principal component analysis and linear discriminant analysis to determine the features to be extracted by the model.

[0008] The target identification information is determined by extracting features based on a pre-trained drone identification model.

[0009] According to another aspect of the present invention, a drone sound recognition device is provided, characterized in that it comprises:

[0010] The collected raw drone sound is preprocessed and subjected to wavelet transform to determine the smoothed drone signal;

[0011] Mel frequency cepstral coefficients are extracted to extract features from the smoothed UAV signal to determine the sound signal characteristics;

[0012] The sound signal features are reduced in dimensionality and combined using principal component analysis and linear discriminant analysis to determine the features to be extracted by the model.

[0013] The target identification information is determined by extracting features based on a pre-trained drone identification model.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the drone sound recognition method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the drone sound recognition method according to any embodiment of the present invention.

[0019] The technical solution of this invention preprocesses and performs wavelet transform on the collected raw drone sound to determine the smoothed drone signal. Signal preprocessing improves the signal quality of the raw drone sound, while wavelet transform removes high-frequency noise, smoothing the sound signal and improving processing efficiency and accuracy. Mel-frequency cepstral coefficients are then extracted from the smoothed drone signal to determine its features. These cepstral coefficients effectively characterize the sound signal based on both frequency and time domain features, improving feature recognition accuracy and drone identification accuracy. Principal component analysis and linear discriminant analysis are then used to further refine the signal. The analysis method performs feature dimensionality reduction and combination on the sound signal features to determine the model for feature extraction. Principal component analysis and linear discriminant analysis improve the expressive power of sound features, reduce the feature dimensionality of sound features, and improve the accuracy and efficiency of recognition. A pre-trained UAV recognition model is used to extract features for recognition, determining target identification information. This UAV recognition model efficiently processes sound features, thereby achieving UAV identification and localization, improving the accuracy and real-time performance of UAV sound recognition, and solving the technical problem of inefficiently identifying and detecting UAVs through sound. This achieves the technical effect of accurate identification and classification of UAV sound.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a drone sound recognition method is provided as an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of a drone recognition model provided in an embodiment of the present invention;

[0024] Figure 3 A flowchart of another drone sound recognition method provided in an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of a multi-head self-attention network provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of a convolutional network disclosed in an embodiment of the present invention;

[0027] Figure 6 This is a schematic diagram of the structure of a drone sound recognition device provided in an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Figure 1 This invention provides a flowchart of a drone sound recognition method. This embodiment is applicable to various situations where the method can be executed by a drone sound recognition device. This drone sound recognition device can be implemented in hardware and / or software and can be configured in an electronic device. For example... Figure 1 As shown, the method includes:

[0032] S110. Preprocess and perform wavelet transform on the collected raw drone sound to determine the smooth signal of the drone.

[0033] The raw sound of the drone can be the sound signal emitted by the drone during flight. It should be noted that the raw sound of the drone is collected through a set sound sensor, which collects the sound signal, converts it into an electrical signal, and records and stores it.

[0034] Optionally, the raw sound of a drone typically includes the basic components and properties of the sound signal, such as amplitude, frequency, wavelength, period, phase, velocity, sound pressure level, harmonics, and noise.

[0035] Optionally, after acquiring the original sound of the drone, the original sound of the drone is preprocessed. The preprocessing includes pre-emphasis, adaptive framing, and windowing. Pre-emphasis can be used to enhance the high-frequency part of the sound signal, improve the signal-to-noise ratio and reduce signal loss during transmission, and effectively avoid the influence of noise. Adaptive framing can segment the sound signal and adjust the sound signal to improve the feature analysis effect of the sound signal. Windowing can be used to reduce the spectral edge effect of the sound signal.

[0036] Optionally, pre-emphasis is achieved by selecting and adjusting the pre-emphasis coefficient based on the characteristics of the audio signal, thus processing the audio signal with different pre-emphasis coefficients. For example, if the audio signal has a high frequency component, a larger pre-emphasis coefficient can be selected; if the audio signal has a low frequency component or a high noise level, a smaller pre-emphasis coefficient can be selected; the selection range of the pre-emphasis coefficient is typically between 0.9 and 1.0.

[0037] Optionally, adaptive framing can segment the audio signal into multiple audio signal segments. The adaptive framing process is as follows: First, the frequency distribution or energy distribution of the signal at different time periods is obtained using spectrum analysis and / or energy calculation methods. Based on the frequency distribution or energy distribution of the signal, the frame length is adaptively adjusted, thereby segmenting the audio signal into frames.

[0038] Optionally, after adaptively framing the audio signal, to prevent discontinuities at both ends of the framing, windowing is applied to the segmented audio signal. By analyzing the characteristics of the audio signal, the frequency distribution, energy distribution, and time-varying properties after framing are determined, and a corresponding window length and adjustment strategy are defined, thus setting the window length. In another implementation, the signal characteristics of the audio signal can be monitored in real time, and the window length is dynamically adjusted according to the adjustment strategy based on these characteristics. This dynamically adjusted window length is then applied during the audio sample windowing process. The adjustment strategy can be that a shorter window can be used if the audio signal is unstable, and a longer window can be used if the audio signal is relatively stable.

[0039] Optionally, the sound signal is decomposed using wavelet decomposition. This is done by using a series of wavelet basis functions with different scales and shifts to decompose the sound signal, transforming it into the time-frequency domain. The decomposed signal is then thresholded, and wavelet reconstruction is performed on the thresholded signal. The signal is then transformed from the time-frequency domain back to the time domain using the same wavelet basis functions as in the wavelet decomposition. For example, the formula used in wavelet decomposition is shown below:

[0040]

[0041] Where x(t) is the input time-domain signal, ψ a,b (t) is the wavelet basis function, a and b are the scaling and translation parameters. W(a,b) is the time-frequency domain signal obtained by wavelet decomposition, where t is the time variable and dt is the time step.

[0042] Wavelet reconstruction is the inverse transform of wavelet decomposition, which can be used to recover signals from the time-frequency domain to the time domain, as shown in the following formula:

[0043] Among them, C ψ These are the normalized coefficients of the wavelet basis functions, while da and db are the coefficients of the wavelet transform.

[0044] Specifically, the original sound of the drone is pre-emphasized, adaptively framed, and windowed to obtain a pre-processed sound signal. Wavelet transform is then performed on the sound signal to determine the smoothed signal of the drone.

[0045] S120. Perform Mel frequency cepstral coefficient extraction feature extraction on the smoothed signal of the UAV to determine the sound signal features.

[0046] Mel frequency cepstral coefficients can be used as a method for feature extraction of sound signals.

[0047] Optionally, Mel frequency cepstral coefficients are typically used to pre-emphasize the smoothed UAV signal, followed by framing and windowing. Each frame is then subjected to a Fast Fourier Transform (FFT) to convert the time-domain signal to the frequency-domain signal, yielding a spectrum. A set of nonlinear Mel filters is applied to this spectrum, with the filter density gradually varying from low to high frequencies to simulate human sound perception. The energy output of each Mel filter bank is logarithmically transformed for subsequent cepstral analysis. A Discrete Cosine Transform (DCT) is then applied to the logarithmic energy to obtain 13-dimensional MFCC (Mel Frequency Cepstral Coefficients) characteristic parameters. These 13-dimensional MFCC characteristic parameters are used as features of the audio signal. These audio signal features can be used to represent the spectral characteristics of the audio signal.

[0048] For example, Mel frequency cepstral coefficients are obtained in the Mel frequency domain, which can better describe sound characteristics compared to the linear frequency domain. The relationship between Mel frequency and Hz frequency is shown in the following formula:

[0049] Where f is the frequency, and the unit is Hz.

[0050] Optionally, in this invention, since the Mel frequency cepstral coefficients are affected by environmental noise, in order to improve the accuracy of feature extraction, feature fusion can be performed by combining MFCC with GFCC (Gammatone Frequency Cepstral Coefficients) to improve the robustness of the sound features. During the fusion extraction process, MFCC and GFCC coefficients can be concatenated or fused in parallel to obtain new sound signal features.

[0051] For example, the time-domain expression of the Gammatone filter of GFCC is shown below:

[0052] Where t is time, φ is phase, fc is center frequency, k is a constant, p is filter order, and b is filter bandwidth;

[0053] Furthermore, the formula for calculating the filter bandwidth b is as follows:

[0054] Where ERB(fc) is the equivalent rectangular bandwidth of the Gammatone filter; ERB(fc) is calculated as follows: E RB (f c ) = 24.7 + 0.108f c .

[0055] Specifically, the smoothed signal from the UAV is subjected to Mel-frequency cepstral coefficient extraction to extract features and determine the characteristics of the sound signal.

[0056] S130. The sound signal features are reduced in dimensionality and combined using principal component analysis and linear discriminant analysis to determine the features to be extracted by the model.

[0057] Principal component analysis can be used to reduce the dimensionality of features and extract the main features of sound signals.

[0058] Optionally, the sound signal features are centered according to the feature dimension to obtain centered data. The covariance matrix is ​​calculated based on the centered data, and the covariance matrix is ​​decomposed to obtain eigenvalues ​​and eigenvectors. The eigenvectors are arranged in descending order of eigenvalues. The eigenvectors corresponding to the number of principal components are selected as principal components. The selected principal component matrix is ​​projected onto the centered data to obtain the dimensionality-reduced principal component features.

[0059] Optionally, the formula used to calculate the covariance matrix is ​​as follows:

[0060] Wherein, the covariance matrix can be the covariance matrix, and X is the characteristic matrix. is the mean vector of the features, and n is the number of samples.

[0061] Among them, linear discriminant analysis can be used to reduce the dimensionality of features and distinguish different feature categories.

[0062] Optionally, after obtaining the sound signal features, the sound signal features are standardized, and the intra-class scatter matrix is ​​calculated based on the standardized data. That is, by calculating the intra-class scatter matrix for each category, the intra-class scatter matrices of all categories are summed to obtain the total intra-class scatter matrix, and the inter-class scatter matrix is ​​calculated. Based on the intra-class scatter matrix and the inter-class scatter matrix, the generalized eigenvalues ​​are solved to perform eigenvalue decomposition, obtaining eigenvalues ​​and eigenvectors. Then, the eigenvector corresponding to the largest eigenvalue is selected as the linear discriminant vector, and the sound signal features are projected into the selected linear discriminant vector space to obtain the dimensionality-reduced linear discriminant analysis features.

[0063] Optionally, the following formula is used to calculate the within-class scatter matrix:

[0064] The formula used to calculate the inter-class scatter matrix is ​​as follows:

[0065] Where c is the number of categories, D i It is the set of samples of the i-th class, N i μ is the number of samples in the i-th class. i is the mean vector of the i-th class of samples, and μ is the mean vector of all samples.

[0066] Optionally, when combining the sound signal features using principal component analysis and linear discriminant analysis, principal component features are first obtained using principal component analysis, and then linear discriminant analysis is applied to the principal component features to obtain the model-extracted features.

[0067] Optionally, when combining the sound signal features using principal component analysis and linear discriminant analysis, the sound signal features are dimensionality reduced using both methods to obtain principal component features and linear discriminant analysis features. The principal component features and linear discriminant analysis features are then combined to obtain the model-extracted features.

[0068] Among them, the features extracted by the model can be the sound signal features used as input to the drone recognition model for feature recognition.

[0069] Specifically, principal component analysis and linear discriminant analysis are used to reduce the dimensionality of sound signal features and combine them to determine the features to be extracted by the model.

[0070] S140. The target identification information is determined by extracting features based on the pre-trained UAV identification model.

[0071] The drone recognition model can be a pre-trained neural network model for recognizing drone sounds. It should be noted that the drone recognition model can be a neural network model built based on convolutional neural networks and self-attention mechanisms. The drone recognition model consists of a first feedforward network, a multi-head self-attention network, a convolutional network, a second feedforward network, and an output network.

[0072] For example, Figure 2 This is a schematic diagram of the structure of a drone recognition model provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the drone recognition model consists of a first feedforward network, which is connected to a multi-head self-attention network via residual connections. The multi-head self-attention network is connected to a convolutional network via residual connections. The convolutional network is connected to a second feedforward network via residual connections. The second feedforward network is connected to an output network via residual connections.

[0073] The target identification information can be drone information corresponding to the original sound of the drone. For example, the target identification information can be the location and category of the drone.

[0074] Specifically, after obtaining the features extracted by the model, the extracted features are input into a pre-trained UAV recognition model to determine the target recognition information.

[0075] The technical solution of this invention preprocesses and performs wavelet transform on the collected raw drone sound to determine the smoothed drone signal. Signal preprocessing improves the signal quality of the raw drone sound, while wavelet transform removes high-frequency noise, smoothing the sound signal and improving processing efficiency and accuracy. Mel-frequency cepstral coefficients are then extracted from the smoothed drone signal to determine its features. These cepstral coefficients effectively characterize the sound signal based on both frequency and time domain features, improving feature recognition accuracy and drone identification accuracy. Principal component analysis and linear discriminant analysis are then used to further refine the signal. The analysis method performs feature dimensionality reduction and combination on the sound signal features to determine the model for feature extraction. Principal component analysis and linear discriminant analysis improve the expressive power of sound features, reduce the feature dimensionality of sound features, and improve the accuracy and efficiency of recognition. A pre-trained UAV recognition model is used to extract features for recognition, determining target identification information. This UAV recognition model efficiently processes sound features, thereby achieving UAV identification and localization, improving the accuracy and real-time performance of UAV sound recognition, and solving the technical problem of inefficiently identifying and detecting UAVs through sound. This achieves the technical effect of accurate identification and classification of UAV sound.

[0076] Figure 3 This is a flowchart of another drone sound recognition method provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiments is that this is a specific method for drone recognition networks to identify and locate sound features. Figure 3 As shown, the method includes:

[0077] S310. Preprocess and perform wavelet transform on the collected raw drone sound to determine the smooth signal of the drone.

[0078] S320. Perform Mel frequency cepstral coefficient extraction feature extraction on the smoothed signal of the UAV to determine the sound signal features.

[0079] S330. The sound signal features are reduced in dimensionality and combined using principal component analysis and linear discriminant analysis to determine the features to be extracted by the model.

[0080] Optionally, in this embodiment of the invention, after obtaining the model-extracted features, a short-time Fourier transform is applied to the model-extracted features to optimize them.

[0081] S340. The features extracted from the model are processed and residuals are connected through the first feedforward network to determine the feedforward network features.

[0082] The first feedforward network can be used to improve the learning and expressive capabilities of the UAV recognition model.

[0083] Specifically, in the UAV recognition model, residual connections are designed in the first feedforward network. These residual connections are used to improve the stability and performance of the UAV recognition model. By performing residual connections between the output of the feature-processed features extracted by the model and the feature-processed features of the first feedforward network, feedforward network features are obtained. These feedforward network features can be the feature information output by the first feedforward network, and they can improve the recognition accuracy of the UAV recognition model.

[0084] Optionally, in another optional embodiment of the present invention, the step of performing feature processing and residual connections on the features extracted from the model through the first feedforward network to determine the feedforward network features includes:

[0085] The first feedforward feature is determined by performing feature transformation and activation on the features extracted from the model through the first feedforward network; the first feedforward feature and the features extracted from the model are then residually connected to determine the feedforward network feature.

[0086] The first feedforward network includes a layer normalization unit, two linear transformations, and a non-linear activation function. The linear transformations convert the input features into output features through a weight matrix. The non-linear activation function introduces non-linearity into the neural network, improving its learning efficiency. The layer normalization unit helps mitigate gradient problems, such as vanishing or exploding gradients. For example, the linear transformations can be set as fully connected layers.

[0087] The first feedforward feature can be the feature obtained by the linear transformation of the feature extraction process of the model and the feature activation of the first feedforward network.

[0088] Specifically, the model extracts features by performing feature transformation and activation function calculations through the first feedforward network to obtain the first feedforward features. The first feedforward features and the model extracts features are then residually connected to determine the feedforward network features.

[0089] For example, the first feedforward network can be represented by an FFN, denoted by x. i This indicates that the model extracts features, using The characteristics of the feedforward network are represented by the following formula for calculating the first feedforward network:

[0090]

[0091] S350. The feedforward network features are processed by self-attention through the multi-head self-attention network to determine the self-attention features.

[0092] The multi-head self-attention network (MDI) can be a neural network structure with a multi-head self-attention mechanism. The MDI includes layer normalization units, multi-head self-attention units, and regularization units. Each multi-head self-attention unit can divide the input data into multiple subspaces and independently perform self-attention operations in each subspace. Each subspace independently learns different attention weights. The MDI also introduces positional encoding, allowing each subspace to learn the relative relationships between different positions, capturing temporal information in sound features, understanding the temporal correlations in sound features, and effectively improving the accuracy of sound recognition. For example... Figure 4 This is a schematic diagram of the structure of a multi-head self-attention network provided in an embodiment of the present invention; as shown below. Figure 4 As shown, the multi-head self-attention module consists of a layer normalization unit, a multi-head self-attention unit, and a regularization unit, with the layer normalization unit connected to the multi-head self-attention unit and the multi-head self-attention unit connected to the regularization unit.

[0093] The regularization unit can randomly discard some of the neuron's output to prevent overfitting.

[0094] Among them, self-attention features can be the feature information output by the self-attention network.

[0095] Specifically, a multi-head self-attention network is used to perform self-attention feature processing on the feedforward network features to determine the self-attention features.

[0096] Optionally, in another optional embodiment of the present invention, the step of performing self-attention feature processing on the feedforward network features through the multi-head self-attention network to determine the self-attention features includes:

[0097] The feedforward network features are processed by the layer normalization unit to determine the self-attention input features; the self-attention input features are processed by the multi-head self-attention unit in combination with relative position encoding to determine the self-attention output features; the self-attention output features are regularized by the regularization unit to determine the first regularized feature; and the feedforward network features and the first regularized feature are residually connected to determine the self-attention feature.

[0098] Among them, the self-attention input features can be the input features obtained by the layer normalization unit performing feature processing on the feedforward network features.

[0099] Among them, the self-attention output features can be the feature information output by the multi-head self-attention unit.

[0100] Specifically, the self-attention input features are determined by performing feature processing on the feedforward network features through layer normalization units; the self-attention output features are determined by processing the self-attention input features through multi-head self-attention units combined with relative position encoding; the self-attention output features are regularized through regularization units to determine the first regularization feature; and the feedforward network features and the first regularization feature are residually connected to determine the self-attention feature. For example, MHSA is represented as a multi-head self-attention network, x' i Represented as a self-attention feature, the calculation formula for a multi-head self-attention network is:

[0101]

[0102] S360. The self-attention features are processed by the convolutional network to determine the deep convolutional features.

[0103] In this context, deep convolutional features can be the convolutional feature information output by the convolutional network. It should be noted that the convolutional network may include layer normalization units, a first activation function, a one-dimensional convolutional unit, a batch normalization unit, a second activation function, and a regularization unit. The first activation function can be the GLU activation function, the second activation function can be the Swish activation function, and the one-dimensional convolutional unit may include two one-dimensional convolutional layers. In the convolutional network, the self-attention features are sequentially processed by layer normalization units, pointwise convolution, first activation function calculation, convolution with two one-dimensional convolutional layers, layer normalization units, Swish activation function calculation, pointwise convolution, and regularization unit calculation, achieving deep convolution of the self-attention features.

[0104] Specifically, deep convolution features are determined by performing deep convolution feature processing on the self-attention features through a convolutional network.

[0105] Optionally, in another optional embodiment of the present invention, the step of performing deep convolution feature processing on the self-attention features through the convolutional network to determine the deep convolution features includes:

[0106] The self-attention features are processed by the layer normalization unit to determine the convolution input features; the convolution output features are determined by channel number projection and depthwise convolution of the convolution input features using the first activation function and a one-dimensional convolution unit; the convolution output features are processed by the batch normalization unit and the second activation function to determine the standardized features; the standardized features are regularized by the regularization unit to determine the second regularized features; and the self-attention features and the second regularized features are residually concatenated to determine the depthwise convolution features.

[0107] The input features of the convolution can be obtained by the layer normalization unit performing feature processing based on the self-attention features.

[0108] The convolutional output features can be obtained by performing depthwise convolution on the convolutional input features. Optionally, the convolutional input features are projected onto the number of channels using a first activation function, and then subjected to pointwise convolution with a spread factor of 2. Finally, they are output as convolutional output features through depthwise convolution of two one-dimensional convolutional layers.

[0109] The standardized features can be the feature information output by the batch normalization unit and the second activation function based on the convolution output features. Optionally, after obtaining the convolution output features, the convolution output features are input into the batch normalization unit for feature processing, and the activation of the second activation function is performed. Then, through pointwise convolution processing with an expansion coefficient of 2, the standardized features are obtained.

[0110] The second regularization feature can be the feature information obtained by the regularization unit performing regularization processing on the standardized feature.

[0111] Specifically, the self-attention features are processed by a layer normalization unit to determine the convolution input features. These input features are then projected onto the channel number using a first activation function and subjected to pointwise convolution with a spread factor of 2. A depthwise convolution of two one-dimensional convolutional layers is then performed to output the convolution output features. These output features are then input into a batch normalization unit for further processing and activation calculation using a second activation function. A pointwise convolution with a spread factor of 2 is then performed to obtain standardized features. These standardized features are then regularized by a regularization unit to determine the second regularized features. Finally, the self-attention features and the second regularized features are residually concatenated to determine the depthwise convolution features. For example, Conv represents the convolution module, and x″ i The formula for calculating the depthwise convolutional features is shown below:

[0112] x″ i =x′ i +Conv(x′ i )

[0113] Figure 5 This is a schematic diagram of the structure of a convolutional network disclosed in an embodiment of the present invention, as shown below. Figure 5 As shown, the convolutional network consists of a layer normalization unit connected to pointwise convolution processing, then connected to the first activation function, followed by two one-dimensional convolutional layers, then connected to the batch normalization unit and the second activation function, then connected to pointwise convolution processing, and finally connected to the regularization unit.

[0114] S370. The deep convolutional features are transformed by the second feedforward network to determine the features to be output.

[0115] The output features can be the feature information obtained by the second feedforward network through feature transformation based on the deep convolution features.

[0116] The second feedforward network can be used to improve the accuracy of the output features of the UAV recognition model.

[0117] Specifically, in the UAV recognition model, residual connections are also designed in the second feedforward network. The deep convolutional features are transformed through the second feedforward network to obtain the output features.

[0118] Optionally, in another optional embodiment of the present invention, the step of performing feature processing and residual connections on the features extracted from the model through the first feedforward network to determine the feedforward network features includes:

[0119] The second feedforward network is used to transform and activate the feature to be output to determine the second feedforward feature; the second feedforward feature and the feature to be output are then residually connected to determine the feature to be output.

[0120] The second feedforward network includes a layer normalization unit, two linear transformations, and a non-linear activation function. The linear transformations convert the input features into output features through a weight matrix. The non-linear activation function introduces non-linearity into the neural network, improving its learning efficiency. The layer normalization unit helps mitigate gradient problems, such as vanishing or exploding gradients. For example, the linear transformations can be set as fully connected layers.

[0121] The second feedforward feature can be the feature obtained by the linear transformation of the feature extraction process of the feedforward network and the feature activation.

[0122] Specifically, the model extracts features by performing feature transformation and activation function calculations through a second feedforward network to obtain second feedforward features. The second feedforward features and the model extracts features are then residually connected to determine the features to be output.

[0123] S380. The output network is used to process the features to be output to determine the target recognition information.

[0124] Specifically, the output network performs feature processing on the output features to determine the target recognition information. For example, FNN2 is used as the second feedforward network, Layernorm represents the output network, and the output network consists of layer normalization units. i Represented as target recognition information, the calculation formulas for the output network and the second feedforward network are as follows:

[0125]

[0126] The technical solution of this invention uses a pre-trained UAV recognition model to extract features for identification and determine target identification information. The UAV recognition model can efficiently process sound features, thereby realizing the identification and positioning of UAVs, improving the accuracy and real-time performance of UAV sound recognition, and achieving the technical effect of accurate identification and classification of UAV sounds.

[0127] Figure 6 This is a schematic diagram of the structure of a drone sound recognition device provided in an embodiment of the present invention. Figure 6 As shown, the device includes: a preprocessing module 610, a feature extraction module 620, a feature optimization module 630, and a feature recognition module 640; wherein,

[0128] The preprocessing module 610 is used to preprocess and perform wavelet transform on the acquired raw drone sound to determine the smoothed drone signal;

[0129] Feature extraction module 620 is used to extract features from the smoothed UAV signal by Mel frequency cepstral coefficient extraction to determine the sound signal features;

[0130] The feature optimization module 630 is used to perform feature dimensionality reduction and combination on the sound signal features through principal component analysis and linear discriminant analysis to determine the features to be extracted by the model.

[0131] The feature recognition module 640 is used to extract features based on a pre-trained UAV recognition model to identify the target and determine the target recognition information.

[0132] The technical solution of this invention preprocesses and performs wavelet transform on the collected raw drone sound to determine the smoothed drone signal. Signal preprocessing improves the signal quality of the raw drone sound, while wavelet transform removes high-frequency noise, smoothing the sound signal and improving processing efficiency and accuracy. Mel-frequency cepstral coefficients are then extracted from the smoothed drone signal to determine its features. These cepstral coefficients effectively characterize the sound signal based on both frequency and time domain features, improving feature recognition accuracy and drone identification accuracy. Principal component analysis and linear discriminant analysis are then used to further refine the signal. The analysis method performs feature dimensionality reduction and combination on the sound signal features to determine the model for feature extraction. Principal component analysis and linear discriminant analysis improve the expressive power of sound features, reduce the feature dimensionality of sound features, and improve the accuracy and efficiency of recognition. A pre-trained UAV recognition model is used to extract features for recognition, determining target identification information. This UAV recognition model efficiently processes sound features, thereby achieving UAV identification and localization, improving the accuracy and real-time performance of UAV sound recognition, and solving the technical problem of inefficiently identifying and detecting UAVs through sound. This achieves the technical effect of accurate identification and classification of UAV sound.

[0133] Optionally, the feature recognition module is specifically used for:

[0134] The first feedforward network is used to extract features from the model, perform feature processing and residual connections, and determine the feedforward network features.

[0135] The feedforward network features are processed by self-attention through the multi-head self-attention network to determine the self-attention features.

[0136] The self-attention features are processed by deep convolutional feature processing through the convolutional network to determine the deep convolutional features;

[0137] The second feedforward network is used to perform feature transformation on the deep convolutional features to determine the features to be output.

[0138] The output network performs feature processing on the features to be output to determine the target recognition information.

[0139] Optionally, the feature recognition module is further configured to:

[0140] The first feedforward feature is determined by extracting features from the model through the first feedforward network, performing feature transformation and activation;

[0141] The first feedforward feature and the model-extracted feature are residually connected to determine the feedforward network features.

[0142] Optionally, the feature recognition module is further configured to:

[0143] The feedforward network features are processed by the layer normalization unit to determine the self-attention input features.

[0144] The self-attention input features are processed by the multi-head self-attention unit in combination with relative position encoding to determine the self-attention output features;

[0145] The self-attention output feature is regularized using the regularization unit to determine the first regularized feature;

[0146] The feedforward network features and the first regularized features are residually connected to determine the self-attention features.

[0147] Optionally, the feature recognition module is further configured to:

[0148] The self-attention features are processed by the layer normalization unit to determine the convolution input features;

[0149] The convolutional output features are determined by performing channel number projection and depth convolution on the convolutional input features using the first activation function and a one-dimensional convolutional unit.

[0150] The convolutional output features are processed by the batch normalization unit and the second activation function to determine the standardized features.

[0151] The standardized feature is regularized using the regularization unit to determine the second regularized feature;

[0152] The self-attention feature and the second regularization feature are residually connected to determine the deep convolution feature.

[0153] Optionally, the feature recognition module is further configured to:

[0154] The second feedforward feature is determined by performing feature transformation and activation on the feature to be output through the second feedforward network.

[0155] The second feedforward feature and the feature to be output are residually concatenated to determine the target recognition information.

[0156] The drone sound recognition device provided in this embodiment of the invention can execute the drone sound recognition method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0157] Figure 7A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their patterns are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0158] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0159] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer grids such as the Internet and / or various telecommunications grids.

[0160] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as drone sound recognition methods.

[0161] In some embodiments, the drone sound recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the drone sound recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the drone sound recognition method by any other suitable means (e.g., by means of firmware).

[0162] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0163] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the patterns / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0164] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0165] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0166] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or grid browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication grid). Examples of communication grids include local area networks (LANs), wide area networks (WANs), blockchain grids, and the Internet.

[0167] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact through a communication mesh. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0168] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0169] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the steps of the drone sound recognition method provided in any embodiment of the present invention. The method includes:

[0170] The collected raw drone sound is preprocessed and subjected to wavelet transform to determine the smoothed drone signal;

[0171] Mel frequency cepstral coefficients are extracted to extract features from the smoothed UAV signal to determine the sound signal characteristics;

[0172] The sound signal features are reduced in dimensionality and combined using principal component analysis and linear discriminant analysis to determine the features to be extracted by the model.

[0173] The target identification information is determined by extracting features based on a pre-trained drone identification model.

[0174] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0175] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0176] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0177] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of mesh, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0178] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a grid of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0179] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0180] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for drone sound recognition, the method comprising: include: The collected raw drone sound is preprocessed and subjected to wavelet transform to determine the smoothed drone signal; wherein, the wavelet transform is to decompose the sound signal into wavelet, convert the sound signal into the time-frequency domain, perform threshold processing on the decomposed signal, and then perform wavelet reconstruction on the threshold-processed signal. The threshold-processed signal is then converted from the time-frequency domain to the time domain by using the same wavelet basis function as in the wavelet decomposition. The sound signal features are determined by extracting features from the smoothed signal of the UAV using Mel frequency cepstral coefficients. Principal component analysis (PCA) is used to reduce the dimensionality of the sound signal features and extract the main features to obtain principal component features. Linear discriminant analysis (LDA) is then used to further reduce the dimensionality of the sound signal features and distinguish different feature categories to obtain LDA features. The principal component features and LDA features are then combined to obtain the model-extracted features. The target identification information is determined by extracting features from a pre-trained UAV identification model, including: the UAV identification model comprising a first feedforward network, a multi-head self-attention network, a convolutional network, a second feedforward network, and an output network; feature processing and residual connections are performed on the features extracted from the model through the first feedforward network to determine feedforward network features; feature processing is performed on the feedforward network features through the multi-head self-attention network to determine self-attention features; deep convolutional feature processing is performed on the self-attention features through the convolutional network to determine deep convolutional features; feature transformation is performed on the deep convolutional features through the second feedforward network to determine the output features; and feature processing is performed on the output features through the output network to determine the target identification information. 2.The UAV sound recognition method of claim 1, wherein, The step of extracting features from the model through the first feedforward network, performing feature processing and residual connections, and determining the feedforward network features includes: The first feedforward feature is determined by extracting features from the model through the first feedforward network, performing feature transformation and activation; The first feedforward feature and the model-extracted feature are residually connected to determine the feedforward network features. 3.The UAV sound recognition method of claim 1, wherein, The multi-head self-attention network includes a layer normalization unit, a multi-head self-attention unit, and a regularization unit; the step of performing feature processing on the feedforward network features through the multi-head self-attention network to determine self-attention features includes: The feedforward network features are processed by the layer normalization unit to determine the self-attention input features. The self-attention input features are processed by the multi-head self-attention unit in combination with relative position encoding to determine the self-attention output features; The self-attention output feature is regularized using the regularization unit to determine the first regularized feature; The feedforward network features and the first regularized features are residually connected to determine the self-attention features. 4.The UAV sound recognition method of claim 1, wherein, The convolutional network includes: layer normalization units, a first activation function, a one-dimensional convolutional unit, a batch normalization unit, a second activation function, and a regularization unit; the step of performing deep convolutional feature processing on the self-attention features through the convolutional network to determine the deep convolutional features includes: The self-attention features are processed by the layer normalization unit to determine the convolution input features; The convolutional output features are determined by performing channel number projection and depth convolution on the convolutional input features using the first activation function and a one-dimensional convolutional unit. The convolutional output features are processed by the batch normalization unit and the second activation function to determine the standardized features. The standardized feature is regularized using the regularization unit to determine the second regularized feature; The self-attention feature and the second regularization feature are residually connected to determine the deep convolution feature.

5. A drone sound recognition apparatus, comprising: include: The preprocessing module is used to preprocess and perform wavelet transform on the acquired raw sound of the drone to determine the smooth signal of the drone; wherein, the wavelet transform is to decompose the sound signal into wavelet, convert the sound signal into the time-frequency domain, perform threshold processing on the decomposed signal, and then perform wavelet reconstruction on the threshold-processed signal. By using the same wavelet basis function as in the wavelet decomposition, the threshold-processed signal is converted from the time-frequency domain to the time domain. The feature extraction module is used to extract features from the smoothed signal of the UAV using Mel-frequency cepstral coefficients to determine the characteristics of the sound signal; The feature optimization module is used to reduce the dimensionality of the sound signal features using principal component analysis (PCA) to extract the main features of the sound signal features and obtain principal component features; to reduce the dimensionality of the sound signal features using linear discriminant analysis (LDA) to distinguish different feature categories and obtain LDA features; and to combine the principal component features and the LDA features to obtain the model-extracted features. The feature recognition module is used to extract features based on a pre-trained UAV recognition model to identify the target and determine the target recognition information; wherein, the UAV recognition model includes a first feedforward network, a multi-head self-attention network, a convolutional network, a second feedforward network, and an output network; The feature recognition module is specifically used for: The first feedforward network is used to extract features from the model, perform feature processing and residual connections, and determine the feedforward network features. The self-attention features are determined by performing feature processing on the feedforward network features through the multi-head self-attention network. The self-attention features are processed by deep convolutional feature processing through the convolutional network to determine the deep convolutional features; The second feedforward network is used to perform feature transformation on the deep convolutional features to determine the features to be output. The output network performs feature processing on the features to be output to determine the target recognition information.

6. An electronic device, comprising: The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the drone sound recognition method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the drone sound recognition method according to any one of claims 1-4.

Images