Unmanned aerial vehicle sound recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses an unmanned aerial vehicle sound recognition method and device, electronic equipment and a storage medium. The method is characterized by comprising the following steps: carrying out preprocessing and wavelet transformation on collected unmanned aerial vehicle original sound, and determining an unmanned aerial vehicle smooth signal; mel-frequency cepstral coefficient feature extraction is carried out on the unmanned aerial vehicle smooth signal, and sound signal features are determined; performing feature dimensionality reduction and combination on the sound signal features through a principal component analysis method and a linear discriminant analysis method, and determining model extraction features; and performing identification based on the model extraction features through a pre-trained unmanned aerial vehicle identification model, and determining target identification information. The unmanned aerial vehicle is identified and positioned, the accuracy and real-time performance of unmanned aerial vehicle sound identification are improved, and the accurate identification and classification of the unmanned aerial vehicle sound are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone sound detection, and in particular to a drone sound recognition method, device, electronic equipment and storage medium. Background Art

[0002] A drone is an aircraft managed by a control station that can be remotely controlled or fly autonomously. With the advancement of drone technology, drones have been widely used in various fields, and various commercial and personal drones have been used in society. Although drone technology has gained many conveniences, the regulatory measures for drones are not yet perfect, resulting in the flight range of drones not being restricted, and frequent incidents of illegal drone flights. For restrictions on drone flights, it is necessary to be able to detect and identify drones in a timely and rapid manner, and then locate and track drones, limit the flight range of drones, and prevent drones from flying illegally. In the prior art, when detecting drones, it is possible to detect the sound of drones to identify whether there are drones around, but the background noise from cities, traffic and other environments makes it difficult to identify and separate human-machine sounds, and the characteristics of drone sounds are varied and complex, which increases the complexity and difficulty of sound recognition. Summary of the invention

[0003] The present invention provides a drone sound recognition method, device, electronic device and storage medium to solve the technical problem that drones cannot be efficiently recognized and detected through drone sounds.

[0004] According to one aspect of the present invention, a method for identifying drone sound is provided, comprising:

[0005] Preprocess and wavelet transform the collected original drone sound to determine the drone smooth signal;

[0006] Perform Mel frequency cepstral coefficient extraction on the smooth signal of the drone to determine the sound signal characteristics;

[0007] Performing feature dimension reduction and combining of the sound signal features by principal component analysis and linear discriminant analysis to determine model extraction features;

[0008] The pre-trained UAV recognition model is used to extract features based on the model for recognition, thereby determining target recognition information.

[0009] According to another aspect of the present invention, there is provided a drone sound recognition device, characterized in that it comprises:

[0010] Preprocess and wavelet transform the collected original drone sound to determine the drone smooth signal;

[0011] Perform Mel frequency cepstral coefficient extraction on the smooth signal of the drone to determine the sound signal characteristics;

[0012] Performing feature dimension reduction and combining of the sound signal features by principal component analysis and linear discriminant analysis to determine model extraction features;

[0013] The pre-trained UAV recognition model is used to extract features based on the model for recognition, thereby determining target recognition information.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drone sound recognition method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the drone sound recognition method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention determines the drone smooth signal by preprocessing and wavelet transforming the collected original drone sound. The signal quality of the drone original sound can be improved through signal preprocessing. The high-frequency noise in the sound signal is removed through wavelet transform, the sound signal is smoothed, and the processing efficiency and accuracy of the sound are improved. The Mel-frequency cepstral coefficients are extracted from the drone smooth signal to determine the sound signal characteristics. The Mel-frequency cepstral coefficients can be used to simultaneously characterize the characteristics of the sound signal according to the frequency domain and time domain characteristics in the sound signal, thereby improving the accuracy of feature recognition and the accuracy of drone recognition. The principal component analysis method and linear discriminant The analysis method performs feature dimensionality reduction and combination on the sound signal features to determine the model extraction features. The principal component analysis method and the linear discriminant analysis method can improve the expressiveness of the sound features, reduce the feature dimensions of the sound features, and improve the accuracy and efficiency of recognition. The pre-trained drone recognition model is used to extract features based on the model to perform recognition and determine target recognition information. The drone recognition model can efficiently process the sound features, thereby realizing the recognition and positioning of the drone, improving the accuracy and real-time performance of the drone sound recognition, solving the technical problem of being unable to efficiently identify and detect drones through drone sounds, and achieving the technical effect of accurately identifying and classifying drone sounds.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a drone sound recognition method is provided for an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the structure of a drone identification model provided by an embodiment of the present invention;

[0024] Figure 3 A flowchart of another drone sound recognition method provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the structure of a multi-head self-attention network provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the structure of a convolutional network disclosed in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of the structure of a drone sound recognition device provided by an embodiment of the present invention;

[0028] Figure 7 A schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Figure 1 A flowchart of a drone sound recognition method is provided for an embodiment of the present invention. This embodiment is applicable to situations where the method can be performed by a drone sound recognition device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0032] S110, preprocessing and wavelet transforming the collected original sound of the drone to determine a smooth signal of the drone.

[0033] The original sound of the drone can be a sound signal collected from the drone during flight. It should be noted that the original sound of the drone is collected by a set sound sensor, which collects the sound signal and converts it into an electrical signal for recording and storage.

[0034] Optionally, the original sound of the drone generally includes the basic composition and properties of the sound signal, and the original sound of the drone includes amplitude, frequency, wavelength, period, phase, speed, sound pressure level, harmonics and noise.

[0035] Optionally, after obtaining the original sound of the drone, the original sound of the drone is preprocessed. The preprocessing methods include pre-emphasis, adaptive framing and windowing. Pre-emphasis can be used to enhance the high-frequency part of the sound signal. Pre-emphasis improves the signal-to-noise ratio and reduces the loss during signal transmission, which can effectively avoid the influence of noise; adaptive framing can divide the sound signal and adjust the sound signal to improve the characteristic analysis effect of the sound signal; windowing can be used to reduce the spectral edge effect of the sound signal.

[0036] Optionally, pre-emphasis is to select and adjust the pre-emphasis coefficient according to the characteristics of the sound signal, and process the sound signal through different pre-emphasis coefficients. For example, if there are more high-frequency components in the sound signal, a larger pre-emphasis coefficient can be selected; if there are fewer high-frequency components in the sound signal or the noise level is high, a smaller pre-emphasis coefficient can be selected; the selection range of the pre-emphasis coefficient is usually between 0.9 and 1.0.

[0037] Optionally, adaptive framing can cut and divide the sound signal into multiple sound signal segments. The process of adaptive framing is as follows: first, use spectrum analysis and / or energy calculation methods to obtain the frequency distribution or energy distribution of the signal in different time periods, and adaptively adjust the frame length according to the frequency distribution or energy distribution of the signal, and then frame the sound signal.

[0038] Optionally, after the sound signal is adaptively framed, in order to prevent discontinuity of the sound signal at both ends after framing, the sound signal after framing is windowed, and the characteristics of the sound signal after framing, such as frequency distribution, energy distribution, and time-varying properties, are determined by analyzing the characteristics of the sound signal, and then the corresponding window length and window length adjustment strategy are defined, and then the window length is set. In another embodiment, the signal characteristics of the sound signal can be monitored in real time, and the window length can be dynamically adjusted according to the adjustment strategy based on the signal characteristics of the sound signal, and the dynamically adjusted window length can be applied during the windowing process of the sound sample. Among them, the adjustment strategy can be that if the sound signal is not stable, a shorter window can be used, and if the sound signal is relatively stable, a longer window can be used.

[0039] Optionally, the sound signal is subjected to wavelet decomposition, the sound signal is decomposed by using a series of wavelet basis functions with different scales and displacements, the sound signal is converted to the time-frequency domain, the decomposed signal is subjected to threshold processing, the threshold-processed signal is subjected to wavelet reconstruction, and the signal is converted from the time-frequency domain to the time domain by using the same wavelet basis function as that used in the wavelet decomposition. Exemplary: The formula used in the wavelet decomposition is as follows:

[0040]

[0041] Among them, x(t) is the input time domain signal, ψ a,b (t) is the wavelet basis function, a, b are the scale parameter and translation parameter. W(a,b) is the time-frequency domain signal obtained by wavelet decomposition, t is the time variable and dt is the time step.

[0042] Wavelet reconstruction is the inverse transform of wavelet decomposition, which can be used to restore the signal in the time-frequency domain to the time domain. The formula is as follows:

[0043] Among them, C ψ are the normalized coefficients of the wavelet basis function, and da and db are the coefficients of the wavelet transform.

[0044] Specifically, pre-emphasis, adaptive framing and windowing are performed on the collected original sound of the drone to obtain a pre-processed sound signal, and wavelet transform is performed on the sound signal to determine a smooth signal of the drone.

[0045] S120, performing Mel-frequency cepstral coefficient extraction on the drone smooth signal to determine the sound signal characteristics.

[0046] Among them, the Mel-frequency cepstral coefficient can be an extraction method for extracting features of sound signals.

[0047] Optionally, the Mel frequency cepstral coefficients are usually used to pre-emphasize the drone smooth signal, and perform frame division and windowing, and perform fast Fourier transform on each frame of the generated signal, convert the time domain signal into a frequency domain signal, obtain a spectrum, and apply a set of nonlinear Mel filters in the spectrum. The density of this set of Mel filters gradually changes from low frequency to high frequency to simulate the human ear's perception of sound, and the energy output by each Mel filter group is logarithmically transformed for subsequent cepstral analysis; and the logarithmic energy is discrete cosine transformed to obtain the converted 13-dimensional MFCC (Mel Frequency Cepstral Coefficients, Mel frequency cepstral coefficients) feature parameters, and the 13-dimensional MFCC feature parameters are used as sound signal features. Sound signal features can be used to represent the spectral characteristics of sound signals.

[0048] For example, the Mel frequency cepstral coefficients are obtained in the Mel frequency domain, which can better describe the sound characteristics than the linear frequency domain. The relationship between the Mel frequency and the Hz frequency is as follows:

[0049] Where f is the frequency in hz.

[0050] Optionally, in the present invention, since the Mel-frequency cepstral coefficients may be affected by environmental noise, in order to improve the accuracy of feature extraction, feature fusion of extracted features can be performed by combining MFCC with GFCC (Gammatone Frequency Cepstral Coefficients), which can improve the robustness of sound features; in the fusion extraction process, the MFCC and GFCC coefficients can be selected to be spliced ​​in series or fused in parallel to obtain new sound signal features.

[0051] For example, the time domain expression of the Gammatone filter of GFCC is as follows:

[0052] Where t is time, φ is phase, fc is center frequency, k is a constant, p is filter order, and b is filter bandwidth;

[0053] Furthermore, the calculation formula of the bandwidth b of the filter is as follows:

[0054] Where ERB(fc) is the equivalent rectangular bandwidth of the gammatone filter; ERB(fc) is calculated as follows: RB (f c )=24.7+0.108f c .

[0055] Specifically, the Mel-frequency cepstral coefficients are extracted from the drone smooth signal to determine the sound signal characteristics.

[0056] S130, performing feature dimension reduction and combination on the sound signal features by using a principal component analysis method and a linear discriminant analysis method to determine a model extraction feature.

[0057] Among them, the principal component analysis method can be used to reduce the dimension of features and extract the main features of sound signal characteristics.

[0058] Optionally, centralized data is obtained by centering the sound signal features according to the feature dimension, the covariance matrix is ​​calculated based on the centralized data, the covariance matrix is ​​eigen-decomposed to obtain eigenvalues ​​and eigenvectors, the eigenvectors are arranged in descending order of the eigenvalues, and the eigenvectors corresponding to the number of the first principal components are selected as principal components. The selected principal component matrix is ​​projected onto the centralized data to obtain the principal component features after dimensionality reduction.

[0059] Optionally, when calculating the covariance matrix, the calculation formula used is as follows:

[0060] Among them, Covariance Matrix can be the covariance matrix, X is the feature matrix, is the mean vector of the features, and n is the number of samples.

[0061] Among them, the linear discriminant analysis method can be used to reduce the dimension of features and distinguish different feature categories.

[0062] Optionally, after obtaining the sound signal features, the sound signal features are standardized, and the intra-class scatter matrix is ​​calculated based on the standardized data, that is, the intra-class scatter matrix is ​​calculated for each category, the intra-class scatter matrices of all categories are summed up to obtain a total intra-class scatter matrix, and the inter-class scatter matrix is ​​calculated. The generalized eigenvalues ​​are solved based on the intra-class scatter matrix and the inter-class scatter matrix to perform eigenvalue decomposition to obtain eigenvalues ​​and eigenvectors, and then the eigenvector corresponding to the maximum eigenvalue is selected as the linear discriminant vector, and the sound signal features are projected into the selected linear discriminant vector space to obtain the linear discriminant analysis features after dimensionality reduction.

[0063] Optionally, when calculating the intra-class scatter matrix, the calculation formula used is as follows:

[0064] When calculating the inter-class scatter matrix, the calculation formula used is as follows:

[0065] Where c is the number of categories, D i is the i-th sample set, N i is the number of samples in the i-th category, μ i is the mean vector of the i-th class of samples, and μ is the mean vector of all samples.

[0066] Optionally, when combining the sound signal features through the principal component analysis method and the linear discriminant analysis method, the principal component features are first obtained through the principal component analysis method, and then the linear discriminant analysis method is applied to the principal component features to obtain the model extraction features.

[0067] Optionally, when the sound signal features are combined by the principal component analysis method and the linear discriminant analysis method, feature dimension reduction is performed on the sound signal features by the principal component analysis method and the linear discriminant analysis method respectively to obtain principal component features and linear discriminant analysis features, and the principal component features and the linear discriminant analysis features are combined to obtain model extraction features.

[0068] Among them, the model extraction feature can be a sound signal feature used to input into the drone recognition model for feature recognition.

[0069] Specifically, the principal component analysis method and the linear discriminant analysis method are used to reduce the dimension of the sound signal features and combine them to determine the model extraction features.

[0070] S140, extracting features based on the model through a pre-trained drone recognition model for recognition, and determining target recognition information.

[0071] The drone recognition model may be a neural network model pre-trained for drone sound recognition. It should be noted that the drone recognition model may be a neural network model based on a convolutional neural network and a self-attention mechanism. The drone recognition model is composed of a first feedforward network, a multi-head self-attention network, a convolutional network, a second feedforward network, and an output network.

[0072] For example, Figure 2 A schematic diagram of a drone identification model provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the drone recognition model consists of a first feedforward network, which is connected to a multi-head self-attention network through a residual connection, the multi-head self-attention network is connected to a convolutional network through a residual connection, the convolutional network is connected to the second feedforward network through a residual, and the second feedforward network is connected to the output network through a residual.

[0073] The target identification information may be the drone information corresponding to the original drone sound. For example, the target identification information may be the location of the drone and the category of the drone.

[0074] Specifically, after obtaining the model extraction features, the model extraction features are input into a pre-trained drone recognition model to determine the target recognition information.

[0075] The technical solution of the embodiment of the present invention determines the drone smooth signal by preprocessing and wavelet transforming the collected original drone sound. The signal quality of the drone original sound can be improved through signal preprocessing. The high-frequency noise in the sound signal is removed through wavelet transform, the sound signal is smoothed, and the processing efficiency and accuracy of the sound are improved. The Mel-frequency cepstral coefficients are extracted from the drone smooth signal to determine the sound signal characteristics. The Mel-frequency cepstral coefficients can be used to simultaneously characterize the characteristics of the sound signal according to the frequency domain and time domain characteristics in the sound signal, thereby improving the accuracy of feature recognition and the accuracy of drone recognition. The principal component analysis method and linear discriminant The analysis method performs feature dimensionality reduction and combination on the sound signal features to determine the model extraction features. The principal component analysis method and the linear discriminant analysis method can improve the expressiveness of the sound features, reduce the feature dimensions of the sound features, and improve the accuracy and efficiency of recognition. The pre-trained drone recognition model is used to extract features based on the model to perform recognition and determine target recognition information. The drone recognition model can efficiently process the sound features, thereby realizing the recognition and positioning of the drone, improving the accuracy and real-time performance of the drone sound recognition, solving the technical problem of being unable to efficiently identify and detect drones through drone sounds, and achieving the technical effect of accurately identifying and classifying drone sounds.

[0076] Figure 3 This is a flowchart of another drone sound recognition method provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for the drone recognition network to identify and locate sound features. Figure 3 As shown, the method includes:

[0077] S310, preprocessing and wavelet transforming the collected original sound of the drone to determine a smooth signal of the drone.

[0078] S320, performing Mel-frequency cepstral coefficient extraction on the drone smooth signal to determine the sound signal characteristics.

[0079] S330, performing feature dimension reduction and combination on the sound signal features by using a principal component analysis method and a linear discriminant analysis method to determine a model extraction feature.

[0080] Optionally, in an embodiment of the present invention, after obtaining the model extraction features, short-time Fourier transform is used on the model extraction features to optimize the model extraction features.

[0081] S340: Perform feature processing and residual connection on the features extracted from the model through the first feedforward network to determine feedforward network features.

[0082] Among them, the first feedforward network can be used to improve the learning ability and expression ability of the drone recognition model.

[0083] Specifically, in the drone recognition model, a residual connection is also designed in the first feedforward network, and the residual connection is used to improve the stability and model performance of the drone recognition model, and the feedforward network features are obtained by performing a residual connection between the output of the model extraction features after the first feedforward network performs feature processing and the model extraction features. The feedforward network features may be feature information output by the first feedforward network, and the feedforward network features can improve the recognition accuracy of the drone recognition model.

[0084] Optionally, in another optional embodiment of the present invention, performing feature processing and residual connection on the model extracted features through the first feedforward network to determine the feedforward network features includes:

[0085] The model extraction feature is transformed and activated through the first feedforward network to determine the first feedforward feature; the first feedforward feature and the model extraction feature are residually connected to determine the feedforward network feature.

[0086] The first feedforward network includes a layer normalization unit, two linear changes and a nonlinear activation function; the linear change can convert the input features into output features through the weight matrix; the nonlinear activation function can introduce nonlinearity to the neural network and improve the learning efficiency of the neural network; the layer normalization unit can be used to improve the gradient problem of the neural network, that is, it helps to alleviate the gradient disappearance or gradient explosion problem. Exemplarily, the linear change can be set as a fully connected layer.

[0087] The first feedforward feature may be a feature obtained by linearly changing the feature extraction process of the model and activating the feature through the first feedforward network.

[0088] Specifically, the model extracts features by performing feature changes and activation calculations of activation functions through a first feedforward network to obtain first feedforward features, and the first feedforward features and the model extracts features are residually connected to determine the feedforward network features.

[0089] Exemplarily, the first feedforward network can be represented by FFN, with x i Represents the model extraction features, using Represents the feedforward network characteristics. The calculation formula of the first feedforward network is as follows:

[0090]

[0091] S350, performing feature processing by self-attention on the feedforward network features through the multi-head self-attention network to determine the self-attention features.

[0092] Among them, the multi-head self-attention network can be a neural network structure with a multi-head self-attention mechanism; the multi-head self-attention network has a layer normalization unit, a multi-head self-attention unit and a regularization unit; wherein the multi-head self-attention unit can divide the input data into multiple subspaces, and independently perform self-attention operations in each subspace, each subspace independently learns different attention weights, and the multi-head self-attention unit also introduces the same position encoding, which can allow each subspace to learn the relative relationship between different positions, can capture the timing information in the sound features, understand the time correlation in the sound features, and effectively improve the accuracy of sound recognition. Exemplary, Figure 4 A schematic diagram of the structure of a multi-head self-attention network provided by an embodiment of the present invention; Figure 4 As shown, the multi-head self-attention module is composed of a layer normalization unit, a multi-head self-attention unit and a regularization unit in sequence. The layer normalization unit is connected to the multi-head self-attention unit, and the multi-head self-attention unit is connected to the regularization unit.

[0093] Among them, the regularization unit can randomly discard the output of some neurons to prevent output overfitting.

[0094] Among them, the self-attention feature can be the feature information output by the self-attention network.

[0095] Specifically, the self-attention feature is processed on the feedforward network features through a multi-head self-attention network to determine the self-attention feature.

[0096] Optionally, in another optional embodiment of the present invention, the step of performing feature processing on the feedforward network features by performing self-attention on the feedforward network features through the multi-head self-attention network to determine the self-attention features includes:

[0097] The feedforward network features are feature processed by the layer normalization unit to determine the self-attention input features; the self-attention input features are processed by the multi-head self-attention unit in combination with relative position encoding to determine the self-attention output features; the self-attention output features are regularized by the regularization unit to determine the first regularized features; the feedforward network features and the first regularized features are residually connected to determine the self-attention features.

[0098] Among them, the self-attention input feature can be an input feature obtained by the layer normalization unit performing feature processing on the feedforward network feature.

[0099] Among them, the self-attention output feature can be the feature information output by the multi-head self-attention unit.

[0100] Specifically, the feedforward network features are processed by the layer normalization unit to determine the self-attention input features; the self-attention input features are processed by the multi-head self-attention unit in combination with the relative position encoding to determine the self-attention output features; the self-attention output features are regularized by the regularization unit to determine the first regularized features; the feedforward network features and the first regularized features are residually connected to determine the self-attention features. Exemplarily, MHSA is represented as a multi-head self-attention network, x' i Expressed as self-attention features, the calculation formula of the multi-head self-attention network is:

[0101]

[0102] S360: Perform deep convolution feature processing on the self-attention feature through the convolution network to determine the deep convolution feature.

[0103] Among them, the deep convolution feature can be the convolution feature information output by the convolution network. It should be noted that the convolution network can include a layer normalization unit, a first activation function, a one-dimensional convolution unit, a batch normalization unit, a second activation function and a regularization unit; the first activation function can be a GLU activation function, the second activation function can be a Swish activation function, and the one-dimensional convolution unit can include two one-dimensional convolution layers. In the convolution network, the self-attention feature is sequentially processed by the layer normalization unit, point-by-point convolution, the first activation function calculation, two one-dimensional convolution layers convolution, layer normalization unit processing, Swish activation function calculation, point-by-point convolution processing and regularization unit calculation to achieve deep convolution of the self-attention feature.

[0104] Specifically, the self-attention features are processed by deep convolutional feature through a convolutional network to determine the deep convolutional features.

[0105] Optionally, in another optional embodiment of the present invention, performing deep convolution feature processing on the self-attention feature through the convolution network to determine the deep convolution feature includes:

[0106] The self-attention feature is feature processed by the layer normalization unit to determine the convolution input feature; the convolution input feature is projected by the number of channels and deep convolution by the first activation function and the one-dimensional convolution unit to determine the convolution output feature; the convolution output feature is feature processed by the batch normalization unit and the second activation function to determine the standardized feature; the standardized feature is regularized by the regularization unit to determine the second regularized feature; the self-attention feature and the second regularized feature are residually connected to determine the deep convolution feature.

[0107] Among them, the convolution input feature can be the feature information obtained by the layer normalization unit through feature processing based on the self-attention feature.

[0108] The convolution output feature can be feature information obtained by performing deep convolution based on the convolution input feature. Optionally, the convolution input feature is projected by the number of channels through a first activation function, and a point-by-point convolution with an expansion factor of 2 is performed, and a deep convolution is performed through two one-dimensional convolution layers to output the convolution output feature.

[0109] Among them, the standardized features can be feature information output by the batch normalization unit and the second activation function based on the convolution output features for feature processing. Optionally, after obtaining the convolution output features, the convolution output features are input into the batch normalization unit for feature processing, and the activation calculation of the second activation function is performed, and then the standardized features are obtained by point-by-point convolution processing with an expansion factor of 2.

[0110] The second regularized feature may be feature information obtained by regularizing the standardized feature by the regularization unit.

[0111] Specifically, the self-attention feature is processed by the layer normalization unit to determine the convolution input feature, the convolution input feature is projected by the first activation function, and a point-by-point convolution with an expansion factor of 2 is performed, and the convolution output feature is output through a depth convolution of two one-dimensional convolution layers, the convolution output feature is input into the batch normalization unit for feature processing, and an activation calculation of the second activation function is performed, and then a point-by-point convolution with an expansion factor of 2 is performed to obtain a standardized feature, the standardized feature is regularized by the regularization unit to determine the second regularized feature, the self-attention feature and the second regularized feature are residually connected to determine the deep convolution feature. Exemplarily, Conv represents the convolution module, x″ i Represents the deep convolution feature. The calculation formula of the convolution module is as follows:

[0112] x″ i = x′ i +Conv(x′ i )

[0113] Figure 5 A schematic diagram of the structure of a convolutional network disclosed in an embodiment of the present invention is shown in FIG. Figure 5 As shown in the figure, the convolutional network consists of a layer normalization unit connected to a point-by-point convolution process, and connected to the first activation function, and then through two one-dimensional convolutional layers, connected to a batch normalization unit and a second activation function, and then connected to a point-by-point convolution process, and finally connected to a regularization unit.

[0114] S370: Perform feature changes on the deep convolution features through the second feed-forward network to determine features to be output.

[0115] Among them, the feature to be output can be feature information obtained by the second feedforward network through feature change based on the deep convolution feature.

[0116] Among them, the second feedforward network can be used to improve the accuracy of the output features of the drone recognition model.

[0117] Specifically, in the drone recognition model, a residual connection is also designed in the second feed-forward network, and the deep convolution features are transformed through the second feed-forward network to obtain the features to be output.

[0118] Optionally, in another optional embodiment of the present invention, performing feature processing and residual connection on the model extracted features through the first feedforward network to determine the feedforward network features includes:

[0119] The feature to be output is transformed and activated by the second feedforward network to determine the second feedforward feature; the second feedforward feature and the feature to be output are residually connected to determine the feature to be output.

[0120] The second feedforward network includes a layer normalization unit, two linear changes and a nonlinear activation function; the linear change can convert the input features into output features through the weight matrix; the nonlinear activation function can introduce nonlinearity to the neural network and improve the learning efficiency of the neural network; the layer normalization unit can be used to improve the gradient problem of the neural network, that is, it helps to alleviate the gradient disappearance or gradient explosion problem. Exemplarily, the linear change can be set as a fully connected layer.

[0121] Among them, the second feedforward feature can be the feature obtained by linearly changing the feature extraction process of the model and activating the feature.

[0122] Specifically, the model extracts features through a second feedforward network to perform feature changes and activation calculations of activation functions to obtain second feedforward features, and the second feedforward features and the model extracts features are residually connected to determine features to be output.

[0123] S380: Perform feature processing on the features to be output through the output network to determine the target recognition information.

[0124] Specifically, the output network performs feature processing on the output features to determine the target recognition information. For example, FNN2 is represented as the second feedforward network, Layernorm is used to represent the output network, and the output network is a layer normalization unit. i Expressed as target recognition information, the calculation formula of the output network and the second feedforward network is as follows:

[0125]

[0126] The technical solution of the embodiment of the present invention uses a pre-trained drone recognition model to extract features based on the model for recognition, and determines target recognition information. The drone recognition model can efficiently process sound features, thereby realizing the recognition and positioning of the drone, improving the accuracy and real-time performance of drone sound recognition, and achieving the technical effect of accurate recognition and classification of drone sounds.

[0127] Figure 6 The following is a schematic diagram of the structure of a drone sound recognition device provided by an embodiment of the present invention. Figure 6 As shown, the device includes: a preprocessing module 610, a feature extraction module 620, a feature optimization module 630 and a feature recognition module 640; wherein,

[0128] The preprocessing module 610 is used to preprocess and wavelet transform the collected original sound of the drone to determine the drone smooth signal;

[0129] A feature extraction module 620 is used to extract Mel frequency cepstral coefficients of the drone smooth signal to determine the sound signal characteristics;

[0130] A feature optimization module 630 is used to reduce and combine the features of the sound signal by a principal component analysis method and a linear discriminant analysis method to determine a model extraction feature;

[0131] The feature recognition module 640 is used to extract features based on the model through a pre-trained drone recognition model for recognition and determine target recognition information.

[0132] The technical solution of the embodiment of the present invention determines the drone smooth signal by preprocessing and wavelet transforming the collected original drone sound. The signal quality of the drone original sound can be improved through signal preprocessing. The high-frequency noise in the sound signal is removed through wavelet transform, the sound signal is smoothed, and the processing efficiency and accuracy of the sound are improved. The Mel-frequency cepstral coefficients are extracted from the drone smooth signal to determine the sound signal characteristics. The Mel-frequency cepstral coefficients can be used to simultaneously characterize the characteristics of the sound signal according to the frequency domain and time domain characteristics in the sound signal, thereby improving the accuracy of feature recognition and the accuracy of drone recognition. The principal component analysis method and linear discriminant The analysis method performs feature dimensionality reduction and combination on the sound signal features to determine the model extraction features. The principal component analysis method and the linear discriminant analysis method can improve the expressiveness of the sound features, reduce the feature dimensions of the sound features, and improve the accuracy and efficiency of recognition. The pre-trained drone recognition model is used to extract features based on the model to perform recognition and determine target recognition information. The drone recognition model can efficiently process the sound features, thereby realizing the recognition and positioning of the drone, improving the accuracy and real-time performance of the drone sound recognition, solving the technical problem of being unable to efficiently identify and detect drones through drone sounds, and achieving the technical effect of accurately identifying and classifying drone sounds.

[0133] Optionally, the feature recognition module is specifically used for:

[0134] Performing feature processing and residual connection on the features extracted from the model through the first feedforward network to determine feedforward network features;

[0135] Performing feature processing on the feedforward network features by performing self-attention on the features of the feedforward network through the multi-head self-attention network to determine the self-attention features;

[0136] Performing deep convolution feature processing on the self-attention feature through the convolution network to determine a deep convolution feature;

[0137] Performing feature changes on the deep convolutional features through the second feedforward network to determine features to be output;

[0138] The output network performs feature processing on the features to be output to determine the target recognition information.

[0139] Optionally, the feature recognition module is further used to:

[0140] Performing feature transformation and activation on the model extraction features through the first feedforward network to determine first feedforward features;

[0141] The first feedforward feature and the model extraction feature are residually connected to determine the feedforward network feature.

[0142] Optionally, the feature recognition module is further used to:

[0143] Performing feature processing on the feedforward network features through the layer normalization unit to determine self-attention input features;

[0144] Processing the self-attention input features by the multi-head self-attention unit in combination with relative position encoding to determine a self-attention output feature;

[0145] Regularizing the self-attention output feature by the regularization unit to determine a first regularized feature;

[0146] The feedforward network feature and the first regularization feature are residually connected to determine the self-attention feature.

[0147] Optionally, the feature recognition module is further used to:

[0148] Performing feature processing on the self-attention feature through the layer normalization unit to determine a convolution input feature;

[0149] Performing channel number projection and depth convolution on the convolution input feature through the first activation function and the one-dimensional convolution unit to determine the convolution output feature;

[0150] Performing feature processing on the convolution output features through the batch normalization unit and the second activation function to determine standardized features;

[0151] Performing regularization processing on the standardized feature by the regularization unit to determine a second regularized feature;

[0152] The self-attention feature and the second regularization feature are residually connected to determine the deep convolution feature.

[0153] Optionally, the feature recognition module is further used to:

[0154] Performing feature transformation and activation on the feature to be output by the second feedforward network to determine a second feedforward feature;

[0155] Perform a residual connection on the second feedforward feature and the feature to be output to determine the target recognition information.

[0156] The drone sound recognition device provided in the embodiment of the present invention can execute the drone sound recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0157] Figure 7A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0158] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0159] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer grid such as the Internet and / or various telecommunication grids.

[0160] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the drone sound recognition method.

[0161] In some embodiments, the drone sound recognition method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the drone sound recognition method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the drone sound recognition method in any other appropriate manner (e.g., by means of firmware).

[0162] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0163] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the modes / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0164] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0165] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0166] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a grid browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication grid). Examples of communication grids include: a local area network (LAN), a wide area network (WAN), a blockchain grid, and the Internet.

[0167] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication grid. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0168] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0169] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the drone sound recognition method provided by any embodiment of the present invention are implemented. The method includes:

[0170] Preprocess and wavelet transform the collected original drone sound to determine the drone smooth signal;

[0171] Perform Mel frequency cepstral coefficient extraction on the smooth signal of the drone to determine the sound signal characteristics;

[0172] Performing feature dimension reduction and combining of the sound signal features by principal component analysis and linear discriminant analysis to determine model extraction features;

[0173] The pre-trained UAV recognition model is used to extract features based on the model for recognition, thereby determining target recognition information.

[0174] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0175] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0176] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0177] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0178] It should be understood by those skilled in the art that the above modules or steps of the present invention can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a grid composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0179] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0180] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A drone sound recognition method, characterized in that: include: Preprocess and wavelet transform the collected original drone sound to determine the drone smooth signal; Perform Mel frequency cepstral coefficient extraction on the smooth signal of the drone to determine the sound signal characteristics; Performing feature dimension reduction and combining of the sound signal features by principal component analysis and linear discriminant analysis to determine model extraction features; The pre-trained UAV recognition model is used to extract features based on the model for recognition, thereby determining target recognition information.

2. The method according to claim 1, characterized in that The drone recognition model includes a first feedforward network, a multi-head self-attention network, a convolutional network, a second feedforward network and an output network.

3. The method according to claim 2, characterized in that The pre-trained drone recognition model is used to extract features based on the model for recognition and positioning, including: Performing feature processing and residual connection on the features extracted from the model through the first feedforward network to determine feedforward network features; Performing feature processing on the feedforward network features by performing self-attention on the features of the feedforward network through the multi-head self-attention network to determine the self-attention features; Performing deep convolution feature processing on the self-attention feature through the convolution network to determine a deep convolution feature; Performing feature changes on the deep convolutional features through the second feedforward network to determine features to be output; The output network performs feature processing on the features to be output to determine the target recognition information.

4. The method according to claim 3, characterized in that: The step of performing feature processing and residual connection on the model extracted features through the first feedforward network to determine the feedforward network features includes: Performing feature transformation and activation on the model extraction features through the first feedforward network to determine first feedforward features; The first feedforward feature and the model extraction feature are residually connected to determine the feedforward network feature.

5. The method according to claim 3, characterized in that: The multi-head self-attention network includes a layer normalization unit, a multi-head self-attention unit and a regularization unit; the self-attention of the feedforward network features is processed by the multi-head self-attention network to determine the self-attention features, including: Performing feature processing on the feedforward network features through the layer normalization unit to determine self-attention input features; Processing the self-attention input features by the multi-head self-attention unit in combination with relative position encoding to determine a self-attention output feature; Regularizing the self-attention output feature by the regularization unit to determine a first regularized feature; The feedforward network feature and the first regularization feature are residually connected to determine the self-attention feature.

6. The method according to claim 3, characterized in that The convolutional network includes: a layer normalization unit, a first activation function, a one-dimensional convolution unit, a batch normalization unit, a second activation function and a regularization unit; the self-attention feature is subjected to deep convolution feature processing by the convolutional network to determine the deep convolution feature, including: Performing feature processing on the self-attention feature through the layer normalization unit to determine a convolution input feature; Performing channel number projection and depth convolution on the convolution input feature through the first activation function and the one-dimensional convolution unit to determine the convolution output feature; Performing feature processing on the convolution output features through the batch normalization unit and the second activation function to determine standardized features; Performing regularization processing on the standardized feature by the regularization unit to determine a second regularized feature; The self-attention feature and the second regularization feature are residually connected to determine the deep convolution feature.

7. The method according to claim 3, characterized in that The step of performing feature processing on the feature to be output through the output network to determine the target recognition information includes: Performing feature transformation and activation on the feature to be output by the second feedforward network to determine a second feedforward feature; Perform a residual connection on the second feedforward feature and the feature to be output to determine the target recognition information.

8. A drone sound recognition device, characterized in that: include: The preprocessing module is used to preprocess and wavelet transform the collected original sound of the drone to determine the smooth signal of the drone; A feature extraction module, used for performing Mel frequency cepstral coefficient extraction on the smooth signal of the drone to determine the characteristics of the sound signal; A feature optimization module, used to perform feature dimension reduction and combination on the sound signal features by using a principal component analysis method and a linear discriminant analysis method to determine model extraction features; The feature recognition module is used to extract features based on the model through a pre-trained drone recognition model for recognition and determine target recognition information.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drone sound recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the drone sound recognition method according to any one of claims 1 to 7 when executed.

Images