Unmanned aerial vehicle sound source positioning method and device

Abstract

The invention provides an unmanned aerial vehicle sound source positioning method and device, and the method comprises the steps: obtaining historical sound data of a target unmanned aerial vehicle; a model training set and a model test set are constructed based on the historical sound data, the historical sound data in the model training set are training samples, and the historical sound data in the model test set are test samples; performing feature extraction on each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample; training and testing the initial sound source localization model by using the first sound feature of each training sample and the second sound feature of each test sample to obtain an unmanned aerial vehicle sound source localization model; and carrying out sound source positioning on the target unmanned aerial vehicle by using the unmanned aerial vehicle sound source positioning model. By adopting the method, it is ensured that unmanned aerial vehicle sound source positioning has universality and universality, and meanwhile the reliability and accuracy of unmanned aerial vehicle sound source positioning are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method and device for locating a sound source of an unmanned aerial vehicle. Background Art

[0002] In recent years, with the rapid rise of the low-altitude economy and the continuous development of drone technology, drones have become an indispensable tool in various fields, covering multiple application scenarios such as logistics distribution, agricultural plant protection, film and television shooting, and environmental monitoring. In order to locate and track drones, it is necessary to locate the sound source of the drone.

[0003] In the prior art, common sound source localization methods mainly include a positioning method based on controllable beamforming with maximum output power, a positioning method based on high-resolution spectrum estimation, and a positioning method based on arrival delay difference estimation. However, it was found in the study that the above methods are highly dependent on the environment and are greatly affected by environmental noise. They can only accurately locate the sound source of the drone in a noise-free environment, which will reduce the versatility and universality of the drone sound source positioning. If the drone is currently in an environment with high noise, then if the above method is used to locate the sound source, the drone sound source positioning will be wrong due to the interference of environmental noise, thereby reducing the reliability and accuracy of the drone sound source positioning. Summary of the invention

[0004] In view of this, the object of the present invention is to provide a method and device for locating the sound source of a drone, so as to ensure that the sound source positioning of the drone has universality and applicability, while improving the reliability and accuracy of the sound source positioning of the drone.

[0005] In a first aspect, an embodiment of the present application provides a method for locating a drone sound source, the method comprising:

[0006] Obtain historical sound data of the target drone;

[0007] Constructing a model training set and a model test set based on the historical sound data, wherein the historical sound data in the model training set is a training sample, and the historical sound data in the model test set is a test sample;

[0008] Performing feature extraction on each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample;

[0009] The initial sound source localization model is trained and tested respectively using the first sound feature of each training sample and the second sound feature of each test sample to obtain a UAV sound source localization model;

[0010] The UAV sound source localization model is used to locate the sound source of the target UAV.

[0011] Optionally, the using the drone sound source localization model to perform sound source localization on the target drone includes:

[0012] Get the current sound data of the target drone;

[0013] Performing feature extraction on the current sound data to obtain a third sound feature of the current sound data;

[0014] Based on the third sound feature, the sound source position of the target drone is determined using the drone sound source localization model.

[0015] Optionally, obtaining historical sound data of the target UAV includes:

[0016] Setting a uniform microphone array, wherein the uniform microphone array comprises a plurality of microphones;

[0017] The microphones are used to collect historical sound data of the target UAV.

[0018] Optionally, the extracting features of each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample includes:

[0019] A generalized cross-correlation-phase transform weighted algorithm is used to extract features from each training sample and each test sample, respectively, to obtain a first sound feature of each training sample and a second sound feature of each test sample, wherein the first sound feature and the second sound feature are arrival time differences of the historical sound data at each microphone.

[0020] Optionally, the method further comprises:

[0021] Parameters are set based on the ResNet-18 network architecture to build the sound source localization model;

[0022] The parameter settings include:

[0023] The first layer of convolution uses three 3*3 convolution kernels;

[0024] The pooling layer uses a 2*2 maximum pooling layer only after the first convolutional layer;

[0025] Use 8 residual blocks, and the two convolutional layers in the residual blocks use three 3*3*3 convolution kernels with edge padding of 1;

[0026] The activation function uses the Swish activation function;

[0027] The mean square error loss function and Adam optimizer are used.

[0028] In a second aspect, an embodiment of the present application provides a drone sound source localization device, the device comprising:

[0029] A historical sound data acquisition module is used to acquire historical sound data of the target UAV;

[0030] A sample set construction module, used to construct a model training set and a model test set based on the historical sound data, wherein the historical sound data in the model training set is a training sample, and the historical sound data in the model test set is a test sample;

[0031] A first feature extraction module, used to extract features from each training sample in the model training set and each test sample in the model test set, to obtain a first sound feature of each training sample and a second sound feature of each test sample;

[0032] A sound source localization model determination module is used to train and test the initial sound source localization model using the first sound feature of each training sample and the second sound feature of each test sample to obtain a drone sound source localization model;

[0033] The sound source localization module is used to locate the sound source of the target UAV using the UAV sound source localization model.

[0034] Optionally, the using the drone sound source localization model to perform sound source localization on the target drone includes:

[0035] Get the current sound data of the target drone;

[0036] Performing feature extraction on the current sound data to obtain a third sound feature of the current sound data;

[0037] Based on the third sound feature, the sound source position of the target drone is determined using the drone sound source localization model.

[0038] Optionally, obtaining historical sound data of the target UAV includes:

[0039] Setting a uniform microphone array, wherein the uniform microphone array comprises a plurality of microphones;

[0040] The microphones are used to collect historical sound data of the target UAV.

[0041] Optionally, the extracting features of each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample includes:

[0042] A generalized cross-correlation-phase transform weighted algorithm is used to extract features from each training sample and each test sample, respectively, to obtain a first sound feature of each training sample and a second sound feature of each test sample, wherein the first sound feature and the second sound feature are arrival time differences of the historical sound data at each microphone.

[0043] Optionally, the device further comprises:

[0044] A sound source localization model building module is used to set parameters based on the ResNet-18 network architecture to build the sound source localization model;

[0045] The parameter settings include:

[0046] The first layer of convolution uses three 3*3 convolution kernels;

[0047] The pooling layer uses a 2*2 maximum pooling layer only after the first convolutional layer;

[0048] Use 8 residual blocks, and the two convolutional layers in the residual blocks use three 3*3*3 convolution kernels with edge padding of 1;

[0049] The activation function uses the Swish activation function;

[0050] The mean square error loss function and Adam optimizer are used.

[0051] The technical solution provided by this application includes but is not limited to the following beneficial effects:

[0052] The present application constructs a training set and a test set by using the historical sound data of the target drone, and extracts features from these data, so as to train a sound source localization model optimized for the drone sound features. Compared with the traditional sound source localization method, this method can more accurately identify the sound features of the drone, thereby improving the accuracy of positioning. Since the training data covers sound samples in different environments and different flight states, the trained sound source localization model has a stronger generalization ability. This means that even in the case of large noise or complex environment, the model can maintain a high positioning accuracy, enhancing the versatility and universality of drone sound source localization. In addition, by separating the model training set and the test set, and strictly testing and verifying the model, it can be ensured that the trained sound source localization model has stable and reliable performance in practical applications. This reduces the positioning error rate caused by environmental noise interference and improves the reliability of drone sound source localization. Compared with the traditional sound source localization method, which needs to rely on high-precision external equipment (such as sensor arrays, microphone arrays, etc.) to achieve high-precision positioning, the drone sound source localization method of the present application mainly relies on the processing of sound data and model training, reducing the degree of dependence on external equipment, thereby reducing costs and improving flexibility.

[0053] In summary, the above-mentioned UAV sound source positioning method and device have brought significant beneficial effects in improving positioning accuracy, enhancing environmental adaptability, improving positioning reliability and reducing dependence on external equipment. It can ensure that the UAV sound source positioning is universal and universal, while improving the reliability and accuracy of UAV sound source positioning.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A flow chart of a method for locating a sound source of a drone provided in Embodiment 1 of the present invention is shown;

[0057] Figure 2 A flow chart of a model training method provided in Embodiment 1 of the present invention is shown;

[0058] Figure 3A flowchart of a specific drone sound source localization method provided in Embodiment 1 of the present invention is shown;

[0059] Figure 4 A flowchart of a method for acquiring historical sound data provided by Embodiment 1 of the present invention is shown;

[0060] Figure 5 A schematic diagram showing a method for determining sound features provided by Embodiment 1 of the present invention is shown;

[0061] Figure 6 A schematic diagram of the structure of a convolutional neural network provided in the first embodiment of the present invention is shown;

[0062] Figure 7 A schematic diagram of the structure of a drone sound source localization model provided by the first embodiment of the present invention is shown;

[0063] Figure 8 A schematic diagram showing a drone sound source localization result provided by the first embodiment of the present invention is shown;

[0064] Fig. 9 A schematic diagram showing a second drone sound source localization result provided by the first embodiment of the present invention is shown;

[0065] Fig.10 A schematic diagram showing a third drone sound source localization result provided by the first embodiment of the present invention is shown;

[0066] Fig.11 A schematic structural diagram of a drone sound source localization device provided in Embodiment 2 of the present invention is shown. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0068] Embodiment 1

[0069] To facilitate understanding of this application, Figure 1The contents described in the flowchart of a charging pile software update method provided in the first embodiment of the present invention are described in detail in the first embodiment of the present application.

[0070] See also Figure 1 As shown, Figure 1 A flowchart of a method for locating a sound source of a drone provided in the first embodiment of the present invention is shown, wherein the method comprises steps S101 to S104:

[0071] S101: Acquire historical sound data of the target UAV.

[0072] Specifically, the sound data generated by the drone in the past is collected. This data comes from the drone's flight records, test environment or other related scenarios. The collected sound data will be used for subsequent model training and testing.

[0073] S102: constructing a model training set and a model test set based on the historical sound data, wherein the historical sound data in the model training set are training samples, and the historical sound data in the model test set are test samples.

[0074] Specifically, the collected historical sound data is divided into two parts: one part is used as a model training set to train the sound source localization model; the other part is used as a model test set to test the performance of the trained model. The division of the training set and the test set usually follows a certain ratio, such as 70% for training and 30% for testing.

[0075] S103: Perform feature extraction on each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample.

[0076] Specifically, feature extraction is performed on each sound sample in the training set and the test set to obtain respective sound features, which include the frequency, amplitude, duration, etc. of the sound.

[0077] S104: Using the first sound feature of each training sample and the second sound feature of each test sample to train and test the initial sound source localization model, respectively, to obtain a drone sound source localization model.

[0078] Specifically, the extracted sound features are used to train and test the initial sound source localization model. The training process is to minimize the prediction error by adjusting the parameters of the model, while the testing process is to evaluate the performance of the model on unseen data. Through this step, a sound source localization model optimized for the target drone sound can be obtained. Figure 2 As shown, Figure 2A flow chart of a model training method provided in Embodiment 1 of the present invention is shown, wherein the drone sound is collected by a microphone array to obtain a microphone array signal, which is divided into a test set and a training set, and GCC-PHAT is used to extract features from the data in the test set and the training set to obtain a test feature vector and a training feature vector, respectively. The test feature vector and the training feature vector are used to train and test the ResNet model to obtain a drone sound source azimuth positioning model.

[0079] S105: Using the UAV sound source localization model to locate the sound source of the target UAV.

[0080] Specifically, the trained UAV sound source localization model is used to locate the sound source of the target UAV, which specifically involves applying the model to real-time or recorded sound data and outputting the location information of the sound source for applications such as monitoring the flight trajectory of the UAV and detecting abnormal sounds.

[0081] In an alternative embodiment, see Figure 3 As shown, Figure 3 A flowchart of a specific UAV sound source localization method provided in Embodiment 1 of the present invention is shown, wherein the method of localizing the sound source of the target UAV by using the UAV sound source localization model comprises steps S301 to S303:

[0082] S301: Acquire current sound data of the target UAV.

[0083] Specifically, the sound data currently generated by the target drone can be collected through a microphone, a sound sensor or other audio recording equipment. The collected current sound data will be used for subsequent sound source positioning analysis.

[0084] S302: Extract features from the current sound data to obtain a third sound feature of the current sound data.

[0085] Specifically, it is necessary to perform feature extraction on the current sound data to obtain features similar to the sound features used when training the model. These features may include the frequency, amplitude, duration, spectral characteristics, etc. of the sound. The purpose of feature extraction is to convert the original sound data into a format that the model can understand and process. Through this step, the third sound feature of the current sound data is obtained.

[0086] S303: Based on the third sound feature, determine the sound source position of the target drone using the drone sound source localization model.

[0087] Specifically, the extracted third sound feature is input into the trained drone sound source localization model. The model will analyze and calculate based on the input sound feature, and finally output the sound source location of the target drone. This location information may be a specific coordinate point, direction angle, or distance relative to a reference point, etc., to achieve sound source localization.

[0088] In an alternative embodiment, see Figure 4 As shown, Figure 4 A flowchart of a method for acquiring historical sound data provided by the first embodiment of the present invention is shown, wherein the method for acquiring historical sound data of a target UAV comprises steps S401 to S402:

[0089] S401: Setting a uniform microphone array, wherein the uniform microphone array includes a plurality of microphones.

[0090] Specifically, a uniform circular microphone array is set. Assume that there are M microphones on the uniform circular microphone array that are evenly distributed on a circle with a radius of R, and there are N far-field drone sound source signals with a wavelength of λ that propagate to the uniform circular microphone array.

[0091] S402: Collect historical sound data of the target UAV using each microphone.

[0092] Specifically, the radius of the first microphone array element is used as the reference baseline. The angle between the projection line of the target drone's sound source incident signal on the xoy plane and the reference baseline is the azimuth angle θ. The angle between the incident direction of the drone sound source signal and the axis is the pitch angle φ. The range of the azimuth angle is θ∈[0,2π), and the range of the pitch angle is φ∈[0,π / 2). The sound signal received by each microphone on the circular microphone array can be expressed as:

[0093]

[0094] x M (t) represents the data received by the Mth microphone at time t, S N (t) represents the sound source of the Nth UAV at time t, n M (t) represents the noise received by the Mth microphone at time t.

[0095] The above formula can be expressed as:

[0096] x(t)=A C s(t)+n(t) (2)

[0097] Among them, x(t) is the M×1-dimensional output data vector; s(t) is the N×1-dimensional far-field drone sound source signal; n(t) is the noise, and the noise is additive white Gaussian noise; A Cis the array flow matrix;

[0098] A C =[a1(ω0),a2(ω0),…,a N (ω0)];

[0099] a n (ω0), n=1,2,…,N is the steering vector, expressed as:

[0100]

[0101] Where ω0 is the angular frequency of the signal, ω0=2πf0=2πc / λ, τ mn is the time delay from the nth signal to the Mth array element, c is the speed of sound, f0 is the sound frequency, and λ is the wavelength of sound.

[0102] The drone sound source is from (θ n ,φ n ) direction propagates to the array, the azimuth angle θ with the m-th microphone m=1,2,…,M mn It can be expressed as:

[0103] θ mn =θ n -2π(m-1) / M (4)

[0104] Among them, θ n is the azimuth angle of the Nth UAV sound source signal arriving at the array, φ n is the pitch angle at which the Nth UAV sound source signal reaches the array.

[0105] Assume that the noise between each independent channel is independent of each other and obeys Gaussian distribution. The noise on each channel is unrelated to the signal source. R is the radius of the circular microphone array, and the arrival time difference τ between the drone sound source signal reaching the mth microphone and the arrival time difference τ at the center of the array mn It can be expressed as:

[0106]

[0107] In an optional implementation, the extracting features of each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample includes:

[0108] A generalized cross-correlation-phase transform weighted algorithm is used to extract features from each training sample and each test sample, respectively, to obtain a first sound feature of each training sample and a second sound feature of each test sample, wherein the first sound feature and the second sound feature are arrival time differences of the historical sound data at each microphone.

[0109] For details, see Figure 5 As shown, Figure 5 A schematic diagram of a sound feature determination method provided in Embodiment 1 of the present invention is shown, wherein the reference microphone array element and other microphone array elements in the microphone array are taken as microphone 1 and microphone 2, and the signal x1(t) received by microphone 1 and the signal x2(t) received by microphone 2 can be expressed as:

[0110] x1(t)=a1(ω)s(t)+n1(t) (6)

[0111] x2(t)=a2(ω)s(t-τ 12 )+n2(t) (7)

[0112] s(t) is the source signal, n1(t) is the noise, τ 12 That is, it is the time delay between the signal reaching the two microphones, and a1(ω) and a2(ω) are the steering vectors with angular frequency ω corresponding to microphone 1 and microphone 2.

[0113] Model R using cross-correlation to calculate time delay x1x2 :

[0114] R x1x2 =E{x1(t)x2(t-τ 12 )} (8)

[0115] Substituting the signal into the model we get:

[0116] R x1x2 =E{(a1(ω0s(t)+n1(t))(a2(ω)s(t-τ 12 )+n2(t))} (9)

[0117] Assuming that s(t) is uncorrelated with n1 and n2, then:

[0118] R x1x2 =R ss (τ-τ 12 )+R n1n2 (τ) (10)

[0119] Assuming that n1 and n2 are uncorrelated Gaussian white noise, the above formula can be written as:

[0120] R x1x2 =R ss (τ-τ 12 ) (11)

[0121] E represents the mathematical expectation of the calculated data, R ss is the signal covariance matrix, R n1n2is the noise covariance matrix.

[0122] According to the properties of the cross-correlation function, when τ = τ 12 , R x1x2 The maximum value is the time delay between the two microphones. After FFT Fourier transform, the cross-correlation function R x1x2 (τ) and the cross power spectrum G x1x2 The relationship between (ω) can be expressed as:

[0123]

[0124] x1(ω) and x2(ω) are the frequency domain representations of the sound data received by microphone 1 and microphone 2 respectively, and * represents the transpose of the matrix.

[0125] From the above signal model, we can see that in the actual environment, noise is inevitable, which makes the peak of the cross-correlation function unclear and reduces the algorithm performance. Therefore, it is necessary to weight the cross-power spectrum in the frequency domain to suppress noise interference. Then, through the IFFT inverse Fourier transform, we get the cross-correlation function:

[0126]

[0127] φ 12 (ω) is the frequency domain weighting function.

[0128] The commonly used weighting function is phase transform PHAT weighting, which is expressed as follows:

[0129]

[0130] In an optional embodiment, the method further comprises:

[0131] Parameters are set based on the ResNet-18 network architecture to build the sound source localization model.

[0132] Specifically, the sound source localization model is a convolutional neural network, which is used to achieve the mapping from input to the sound source position. It is a deep learning model or a multi-layer perceptron similar to an artificial neural network.

[0133] See also Figure 6 As shown, Figure 6 The figure shows a schematic diagram of the structure of a convolutional neural network provided in the first embodiment of the present invention, wherein the convolutional neural network has four types of layers:

[0134] (1) Convolutional layer: The convolutional layer is the core hierarchical structure in the convolutional neural network. Its basic function is to extract the local features of the input data through the convolution operation, and then perform the next step of processing and analysis on the features. The convolution operation uses a convolution kernel to scan the input data and generate the corresponding feature matrix. It multiplies and accumulates the convolution kernel and the input data element by element to obtain a new value, which is placed in the corresponding position of the feature matrix. In order to control the size of the feature matrix and the receptive field and the adequacy of feature extraction, the convolution layer includes edge padding and step size definition. Edge padding is to add additional values ​​around the input data matrix so that the input data can be fully utilized and the size of the feature matrix can be maintained. The step size definition determines the distance the convolution kernel moves on the input data.

[0135] The convolution operation process includes convolution and activation operations. The convolution kernel is used for operation, which is similar to the filter of two-dimensional signal. The convolution operation is expressed as:

[0136]

[0137] in, represents the jth feature matrix of the lth layer in the network, M j is the size of the feature matrix of layer l, represents the convolution kernel, is the bias of the j-th feature matrix of the l-th layer, * is the two-dimensional convolution operation, and f(·) is the activation function.

[0138] In order to reasonably set the size and step length of the convolution kernel when building the network structure, it is necessary to know the data size of the input and output of each convolution operation. The length and width relationship between the input and output of the convolution operation satisfies:

[0139]

[0140] W1, H1 represent the length and width of the input; W2, H2 represent the length and width of the output; FW, FH represent the length and width of the convolution kernel; P represents edge padding, and S represents the step size of the sliding window.

[0141] The activation function contained in the convolution layer can introduce nonlinear factors to neurons, allowing the neural network to approximate nonlinear functions and improve the expression ability of deep neural networks. This application uses a 3*3 convolution layer to receive input data.

[0142] The Swish activation function used in this application has good performance in most cases. The mathematical expression of the Swish activation function f(a) is as follows, where a is the input data of the activation function.

[0143]

[0144] (2) Pooling layer: Pooling is equivalent to performing a dimensionality reduction operation within a certain range, so that the model can extract more representative features. It reduces the input size of the next layer, reduces the amount of calculation and the number of parameters; the pooling operation is the model's judgment of whether certain features exist, which can prevent overfitting to a certain extent. Taking the maximum value or average value within a given window range, thereby reducing the data dimension, can be divided into maximum pooling and average pooling, expressed as:

[0145]

[0146] Where n×n represents the size of a given window; M j Represents each value in the input matrix.

[0147] (3) Fully connected layer: The fully connected layer is a standard neural network layer that plays the role of a "classifier" in the entire convolutional neural network. It can convert high dimensions into low dimensions, or convert low dimensions into high dimensions, while retaining useful information. It flattens the output vector (or matrix) of the previous convolutional layer and pooling layer into a vector. i The calculation is expressed as:

[0148]

[0149] y represents a vector flattened from the output vector (or matrix) of the previous convolution, or the output of the previous fully connected layer, ω i represents the weight, b i Indicates the bias.

[0150] (4) BN layer: Normalizes each batch of data to accelerate convergence, solve the problem of gradient vanishing and gradient exploding, and eliminates the need to carefully set weight initialization.

[0151] The steps are mainly divided into 4 steps:

[0152] Calculate the mean of each training batch of data.

[0153] Calculate the variance of each training batch of data.

[0154] The training data of this batch is normalized using the calculated mean and variance so that the data of this batch satisfies the 0-1 distribution. At the same time, ε is introduced to avoid the divisor being 0.

[0155] Scaling and shifting: normalizing the data Multiply by γ to adjust the value, add β to increase the offset to get the output y i , while γ is the scale factor and β is the translation factor. It will be restricted to the normal distribution, which will reduce the network's expressive ability to a certain extent, so the parameters γ and β are introduced; γ and β are learned when training the network.

[0156] Input: x i ,X={x1,x2,x3...x n},γ,β,i∈[1,n],n is the total number of data to be processed

[0157] Output: y i =BN γ,β (x i )(twenty two)

[0158] Calculate the average:

[0159] Calculate the variance:

[0160] standardization

[0161] Introduce γ and β:

[0162] Among them, BN γ,β (x i ) represents the overall output expression function of the BN layer.

[0163] After using the calculated mean and variance to subtract the mean and divide the variance of the batch of data, the data is moved to the central area, and the gradient of most activation functions in this area is the largest or has a gradient. This is an effective way to combat gradient vanishing. The input data of each convolutional layer is normalized, so the data can always be distributed in areas that are sensitive to changes, which is equivalent to not having to consider changes in data distribution, making training more efficient. At the same time, the initialization settings before network training have less impact on the learning effect, and the initial weights do not need to be set carefully. For the input value of a batch of data, whether the weight is w or the scaled weight kw, k is the scaling factor, σ is the variance, It is the mean of the input data weights. The values ​​after the BN layer processing are the same, and the ones used for scaling will be eliminated during normalization.

[0164]

[0165] Among them, see Figure 7 As shown, Figure 7A structural schematic diagram of a drone sound source localization model provided by Embodiment 1 of the present invention is shown, wherein the parameter settings in the drone sound source localization model include: the first convolution layer uses three 3*3 convolution kernels for receiving data input; the pooling layer has only one 2*2 maximum pooling layer after the first convolution layer; 8 residual blocks are used, with a residual block*8 set, and the two convolution layers in the residual block use three 3*3*3 convolution kernels, with an edge padding of 1; the activation function uses the Swish activation function; the mean square error loss function and the Adam optimizer are used, and finally the data is output through the fully connected layer.

[0166] Specifically, the improvement is based on the ResNet-18 network architecture of the convolutional neural network, which mainly includes the residual block. The residual block has two convolutional layers with the same number of output channels, and each convolutional layer is connected to a BN layer and an activation function. The characteristic of the residual block is that the input can pass through the cross-layer data path, skipping these two convolution operations, and directly adding the input of the entire residual block before the activation function of the final output, so that the newly added network layer is at least not worse than the original one, and the number of layers can be steadily deepened to improve the performance of the model. At the same time, it can also avoid the problem of gradient disappearance to a certain extent.

[0167] The drone sound source localization method provided in this application is used to locate the drone sound source in different signal-to-noise ratio environments. Figure 8 As shown, Figure 8 A schematic diagram of the sound source localization result of a drone provided by the first embodiment of the present invention is shown, wherein the signal-to-noise ratio in the test environment is -3dB, the signal incident azimuth angle (X-axis value) is 9° and 29° respectively, and the corresponding spatial spectrum intensity (Y-axis value) is 0.1346dB and 0.2109dB. Fig. 9 As shown, Fig. 9 The figure shows the second UAV sound source localization result provided by the first embodiment of the present invention, wherein the signal-to-noise ratio in the test environment is 2 dB, the signal incident azimuth angle (X-axis value) is 14° and 30° respectively, and the corresponding spatial spectrum intensity (Y-axis value) is 0.425 dB and 0.3896 dB. Fig.10 As shown, Fig.10 The figure shows the third type of drone sound source localization result provided by the first embodiment of the present invention, wherein the signal-to-noise ratio in the test environment is 10 dB, the signal incident azimuth angle (X-axis value) is 20° and 30° respectively, and the corresponding spatial spectrum intensity (Y-axis value) is 1.025 dB and 0.87 dB. Figure 8 , Fig. 9 and Fig.10 In FIG. 5 , the horizontal axis (X axis) represents the incident angle Theta, and the vertical axis (Y axis) represents the spatial spectrum Spatial Spectrum.

[0168] Embodiment 2

[0169] See Figure 11. Fig.11 The schematic diagram of the structure of a drone sound source localization device provided by the second embodiment of the present invention is shown, wherein the device comprises:

[0170] The historical sound data acquisition module 1101 is used to acquire the historical sound data of the target UAV;

[0171] A sample set construction module 1102 is used to construct a model training set and a model test set based on the historical sound data, wherein the historical sound data in the model training set is a training sample, and the historical sound data in the model test set is a test sample;

[0172] A first feature extraction module 1103 is used to extract features from each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample;

[0173] The sound source localization model determination module 1104 is used to train and test the initial sound source localization model using the first sound feature of each training sample and the second sound feature of each test sample to obtain the drone sound source localization model;

[0174] The sound source localization module 1105 is used to locate the sound source of the target UAV using the UAV sound source localization model.

[0175] In an optional embodiment, the using the drone sound source localization model to locate the sound source of the target drone includes:

[0176] Get the current sound data of the target drone;

[0177] Performing feature extraction on the current sound data to obtain a third sound feature of the current sound data;

[0178] Based on the third sound feature, the sound source position of the target drone is determined using the drone sound source localization model.

[0179] In an optional embodiment, the obtaining of historical sound data of the target UAV includes:

[0180] Setting a uniform microphone array, wherein the uniform microphone array comprises a plurality of microphones;

[0181] The microphones are used to collect historical sound data of the target UAV.

[0182] In an optional implementation, the extracting features of each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample includes:

[0183] A generalized cross-correlation-phase transform weighted algorithm is used to extract features from each training sample and each test sample, respectively, to obtain a first sound feature of each training sample and a second sound feature of each test sample, wherein the first sound feature and the second sound feature are arrival time differences of the historical sound data at each microphone.

[0184] In an optional embodiment, the device further comprises:

[0185] A sound source localization model building module is used to set parameters based on the ResNet-18 network architecture to build the sound source localization model;

[0186] The parameter settings include:

[0187] The first layer of convolution uses three 3*3 convolution kernels;

[0188] The pooling layer uses a 2*2 maximum pooling layer only after the first convolutional layer;

[0189] Use 8 residual blocks, and the two convolutional layers in the residual blocks use three 3*3*3 convolution kernels with edge padding of 1;

[0190] The activation function uses the Swish activation function;

[0191] The mean square error loss function and Adam optimizer are used.

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0193] The drone sound source localization device provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0194] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0195] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0197] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0198] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0199] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features thereof; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for locating a drone sound source, characterized in that: The method comprises: Obtain historical sound data of the target drone; Constructing a model training set and a model test set based on the historical sound data, wherein the historical sound data in the model training set is a training sample, and the historical sound data in the model test set is a test sample; Performing feature extraction on each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample; The initial sound source localization model is trained and tested respectively using the first sound feature of each training sample and the second sound feature of each test sample to obtain a UAV sound source localization model; The UAV sound source localization model is used to locate the sound source of the target UAV.

2. The method according to claim 1, characterized in that The method of using the UAV sound source localization model to locate the sound source of the target UAV includes: Get the current sound data of the target drone; Performing feature extraction on the current sound data to obtain a third sound feature of the current sound data; Based on the third sound feature, the sound source position of the target drone is determined using the drone sound source localization model.

3. The method according to claim 1, characterized in that The obtaining of historical sound data of the target UAV includes: Setting a uniform microphone array, wherein the uniform microphone array comprises a plurality of microphones; The microphones are used to collect historical sound data of the target UAV.

4. The method according to claim 3, characterized in that The extracting features of each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample includes: A generalized cross-correlation-phase transform weighted algorithm is used to extract features from each training sample and each test sample, respectively, to obtain a first sound feature of each training sample and a second sound feature of each test sample, wherein the first sound feature and the second sound feature are arrival time differences of the historical sound data at each microphone.

5. The method according to claim 1, characterized in that The method further comprises: Parameters are set based on the ResNet-18 network architecture to build the sound source localization model; The parameter settings include: The first layer of convolution uses three 3*3 convolution kernels; The pooling layer uses a 2*2 maximum pooling layer only after the first convolutional layer; Use 8 residual blocks, and the two convolutional layers in the residual blocks use three 3*3*3 convolution kernels with edge padding of 1; The activation function uses the Swish activation function; The mean square error loss function and Adam optimizer are used.

6. A drone sound source localization device, characterized in that: The device comprises: A historical sound data acquisition module is used to acquire historical sound data of the target UAV; A sample set construction module, used to construct a model training set and a model test set based on the historical sound data, wherein the historical sound data in the model training set is a training sample, and the historical sound data in the model test set is a test sample; A first feature extraction module, used to extract features from each training sample in the model training set and each test sample in the model test set, to obtain a first sound feature of each training sample and a second sound feature of each test sample; A sound source localization model determination module is used to train and test the initial sound source localization model using the first sound feature of each training sample and the second sound feature of each test sample to obtain a drone sound source localization model; The sound source localization module is used to locate the sound source of the target UAV using the UAV sound source localization model.

7. The device according to claim 6, characterized in that The method of using the UAV sound source localization model to locate the sound source of the target UAV includes: Get the current sound data of the target drone; Performing feature extraction on the current sound data to obtain a third sound feature of the current sound data; Based on the third sound feature, the sound source position of the target drone is determined using the drone sound source localization model.

8. The device according to claim 6, characterized in that The obtaining of historical sound data of the target UAV includes: Setting a uniform microphone array, wherein the uniform microphone array comprises a plurality of microphones; The microphones are used to collect historical sound data of the target UAV.

9. The device according to claim 6, characterized in that The extracting features of each training sample in the model training set and each test sample in the model test set to obtain a first sound feature of each training sample and a second sound feature of each test sample includes: A generalized cross-correlation-phase transform weighted algorithm is used to extract features from each training sample and each test sample, respectively, to obtain a first sound feature of each training sample and a second sound feature of each test sample, wherein the first sound feature and the second sound feature are arrival time differences of the historical sound data at each microphone.

10. The device according to claim 6, characterized in that The device also includes: A sound source localization model building module is used to set parameters based on the ResNet-18 network architecture to build the sound source localization model; The parameter settings include: The first layer of convolution uses three 3*3 convolution kernels; The pooling layer uses a 2*2 maximum pooling layer only after the first convolutional layer; Use 8 residual blocks, and the two convolutional layers in the residual blocks use three 3*3*3 convolution kernels with edge padding of 1; The activation function uses the Swish activation function; The mean square error loss function and Adam optimizer are used.

Images