Method and device for realizing sound source localization by carrying microphone array on unmanned aerial vehicle

Abstract

The invention discloses a method and a device for realizing sound source localization by using a microphone array carried by an unmanned aerial vehicle. The method comprises the following steps: S1, acquiring a multi-channel audio signal which is measured by the microphone array carried by the unmanned aerial vehicle and contains flight noise of the unmanned aerial vehicle; s2, extracting a beam forming diagram of the multi-channel audio signal by using a conventional beam forming technology; and S3, inputting the beam forming diagram into the trained neural network model to obtain a predicted sound source position and sound source intensity. The invention aims to overcome harsh sound array conditions (limited array aperture and relatively small array element spacing) and complex acoustic environment (extremely low signal-to-noise ratio and unmanned aerial vehicle self-noise non-stationary characteristics) on the unmanned aerial vehicle, and realizes efficient and real-time sound source localization of a high-frequency sound source by the microphone array carried by the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention belongs to the field of sound source positioning methods, and specifically relates to a method and device for realizing sound source positioning by utilizing a microphone array carried by an unmanned aerial vehicle. Background Art

[0002] UAVs are equipped with microphone arrays, which play an important role in monitoring abnormal sounds in the air, search and rescue, human-computer interaction, etc. However, unlike handheld microphone arrays, the noise sources of UAV microphone arrays are more diverse, including self-noise, such as propeller motor noise, blade rotation, etc., as well as external noise, including wind noise in hovering or flying state, and noise in the surrounding environment.

[0003] Sound source localization is the problem of estimating the position information of one or more sound sources relative to a reference point based on multi-channel collected sound signals. This reference point is usually related to the position of the microphone array. In most cases, sound source localization refers to the estimation of the direction of arrival (DOA) of the sound source, that is, the estimation of the azimuth and elevation of the sound source relative to the reference position. Sound source localization plays an important role in applications such as speech recognition, speech enhancement, noise control, and room acoustic analysis.

[0004] The more commonly used methods in traditional sound source localization algorithms can be classified into the following three types: based on beamforming, based on high-resolution spectrum estimation, and based on time difference of arrival. Beamforming technology performs spatial filtering on the weighted elements of each array, and by controlling the pointing direction of the microphone array, traverses the spatial position, obtains the direction of maximum output power, and realizes sound source localization. It is not difficult to see that the sound source localization algorithm based on beamforming can locate multiple sound sources when there is more than one sound source. In the past few decades, conventional beamforming (CB) technology has proven its value in evaluating the spatial distribution of sound sources and their intensity, especially the ability to plot the energy of array output signals in different directions obtained by traversing the directions into images (beamforming diagrams, referred to as CB diagrams).

[0005] However, it cannot be ignored that there are difficulties in using conventional beamforming in the microphone array carried by drones: (1) The received signal of the microphone array carried by drones has an extremely low signal-to-noise ratio. From the energy point of view, the received signal signal-to-noise ratio is far less than -15dB, which has a serious impact on the accuracy of sound source positioning. At the same time, the visual representation of this CB map often shows incorrect source strength reconstruction; (2) Drone noise is not only high in energy, but also non-stationary. There are a large number of parts similar to interference sources on the CB map, and the location of the interference source changes over time; (3) The effective aperture and array element spacing of the microphone array that can be carried by drones are limited. The spatial distribution effect of conventional beamforming algorithms when estimating the spatial distribution of sound sources is limited by the effective aperture of the microphone array, especially for large wavelength signal sources. Summary of the invention

[0006] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method and device for realizing sound source positioning by using a microphone array carried by a drone are provided. The present invention aims to overcome the harsh acoustic array conditions on the drone (limited array aperture and small array element spacing) and the complex acoustic environment (extremely low signal-to-noise ratio and non-stationary characteristics of drone self-noise), and realize efficient and real-time sound source positioning of high-frequency sound sources by using a microphone array carried by the drone.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for realizing sound source localization by using a microphone array carried by an unmanned aerial vehicle comprises the following steps: S1. Obtain a multi-channel audio signal containing UAV flight noise measured by a microphone array carried by the UAV; S2, extracting a beamforming graph of a multi-channel audio signal using conventional beamforming techniques; S3. Input the beamforming image into the trained neural network model to obtain the predicted sound source position and sound source intensity.

[0008] Optionally, the UAV flight noise in step S1 includes part or all of the motor noise, rotor noise, naturally existing wind noise and aerodynamic noise in the working state of the UAV.

[0009] Optionally, step S2 includes: S2.1, pre-delay the multi-channel audio signal according to the following formula: , In the above formula, For the After the pre-delay of the microphone array element, The audio signal of sampling points, For the The microphone array elements are The audio signal at the moment, For time, For the The microphone array elements are Pre-delay in direction, , For the The scanning delay time of each microphone array element is is the angle between the position of the drone’s microphone array and the incident direction of the sound source, is the signal sampling period, , is the number of microphone array elements; S2.2, converting the pre-delayed audio signals of each microphone array element into digital sampling signals through A / D and then inputting them into a FIR filter bank optimized by norm constraint, wherein the filter length of the FIR filter bank is , No. Microphone array element The weight of a snapshot is , the function expression of the beam output time series obtained by the FIR filter group weighted summing all data is: , In the above formula, is the first time series of the beam output The audio signal of sampling points, For the After the pre-delay of the microphone array element, The audio signal of sampling points, For the Microphone array element The weight of a snapshot, , ; The function expression using norm constraint optimization is: , , In the above formula, is the impulse response of the FIR filter bank, ~ are the impulse responses of the 1st to Lth snapshots, For the Frequency points The error weighting coefficient is , is the filter passband frequency set The number of frequency points in For the Frequency points A vector function of For the Frequency points The expected response, For the The stopband frequency at the frequency point, , is the filter stopband frequency set The number of frequency points in is the side lobe value; S2.3, calculate the energy of the obtained beam output time series according to the following formula: , In the above formula, for The energy of direction, is the total number of sampling points, is the first time series of the beam output The audio signal of sampling points; S2.4, combining the drone altitude information, plotting the directional energy into a sound energy distribution diagram on a specified plane as a beamforming diagram of the obtained multi-channel audio signal.

[0010] Optionally, after the beamforming diagram is input into the trained neural network model in step S3, the neural network model obtains the predicted sound source position and sound source intensity, including: first, using a convolution layer with 64 convolution kernels of size 7×7 to extract features, and then sending them to the residual network unit after batch normalization processing of the batch normalization layer, activation of the ReLU activation function and maximum pooling processing of the pooling layer, wherein the residual network unit includes four cascade-connected residual layers, namely residual layer 1 to residual layer 4; the output of the residual network unit is sent to the fully connected output layer after average pooling processing of a pooling layer to obtain the predicted value of the sound source position. and the predicted value of the sound source intensity The output vector .

[0011] Optionally, each residual layer in the residual network unit processes the input features to obtain output features, including: first, extracting features from the input features using a convolutional layer, and then performing batch normalization processing through a batch normalization layer to achieve scaling and shifting, and then extracting features from the feature F obtained after scaling and shifting using a channel attention module, and then multiplying the feature with the original feature F obtained after scaling and shifting to obtain feature F', and then extracting features from feature F' using a spatial attention module, and then multiplying the feature with the original feature F' to obtain feature F", and then concatenating feature F" with the input features of the residual layer to obtain the output features of the residual layer.

[0012] Optionally, before step S3, it also includes setting a simulated sound source and collecting multi-channel audio signal samples containing drone flight noise, using the multi-channel audio signal samples containing drone flight noise obtained by actual measurement as the input of the neural network model, and the sound source position and sound source intensity of the simulated sound source as labels to construct a training data set, and using the training data set to train the neural network model, and using a preset loss function to update the model parameters of the neural network model to obtain a trained neural network model.

[0013] Optionally, when a preset loss function is used to update the model parameters of the neural network model, the function expression of the preset loss function is: , In the above formula, is the loss function, and are the label value and predicted value of the sound source position respectively, and are the label value and predicted value of the sound source intensity respectively.

[0014] In addition, the present invention also provides a system for realizing sound source positioning using a microphone array carried by an unmanned aerial vehicle, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the method for realizing sound source positioning using a microphone array carried by an unmanned aerial vehicle.

[0015] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the method of realizing sound source positioning using a microphone array carried by an unmanned aerial vehicle through a processor.

[0016] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the method of realizing sound source positioning by using a microphone array carried by an unmanned aerial vehicle through a processor.

[0017] Compared with the prior art, the present invention mainly has the following advantages: in order to realize the sound source positioning using a drone, it is necessary to consider the drone's own noise and the strong aerodynamic noise when the drone moves. These noises are picked up by the microphone array, which has a great impact on the accuracy of sound source positioning. The neural network proposed in the present invention extracts the characteristics of noise in the sound image by learning to reduce the influence of noise on sound source positioning; on the other hand, it overcomes the low resolution problem of conventional beamforming technology. The neural network is built based on the ResNet architecture and can locate the sound source in the extremely low signal-to-noise ratio signal picked up by the array. The sound image (beamforming image) obtained by traversing the spatial position by conventional beamforming is used as the network input, and the sound source position coordinates and sound source intensity are used as the output, so as to overcome the harsh sound array conditions (limited array aperture, small array element spacing) and complex acoustic environment (extremely low signal-to-noise ratio and non-stationary characteristics of drone self-noise) on the drone, and realize the drone-mounted microphone array for efficient and real-time sound source positioning of high-frequency sound sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0019] Figure 2 The figure is a schematic diagram of the arrangement of microphone array elements for signal collection in an embodiment of the present invention.

[0020] Figure 3 The flowchart of implementing a broadband time-domain beamformer in an embodiment of the present invention is shown in FIG.

[0021] Figure 4 Schematic diagram of the structure of the neural network model in an embodiment of the present invention.

[0022] Figure 5 Schematic diagram of the structure of the residual layer network structure in an embodiment of the present invention.

[0023] Figure 6 This is a diagram showing the model output effect obtained in an embodiment of the present invention.

[0024] Figure 7 The acoustic image and model output results are drawn when the drone is hovering 6.2m above the sound source in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] With the continuous improvement of the neural network theory system and the improvement of computing power, neural networks have achieved remarkable results in the field of image recognition, using neural networks to learn the mapping relationship between feature representation and input and output to describe nonlinear phenomena. Therefore, it is possible to bypass the complex mathematical and physical descriptions of specific problems. Taking the above points into consideration, the present invention aims to use a microphone array carried by a drone to use a neural network to bypass the complex acoustic environment on the drone to optimize the beamforming diagram to obtain a more accurate sound source localization result, thereby achieving high-precision, high-noise-resistant sound source localization based on conventional beamforming technology combined with neural networks. Due to the complex spatiotemporal characteristics of drone noise, neural networks bypassing the complex mathematical and physical descriptions of specific problems play an important role in reducing the impact of drone noise on positioning. In order to better understand the present technical solution, the method of the present invention is described in detail below in conjunction with the accompanying drawings through specific implementation methods.

[0026] like Figure 1 As shown, the method for realizing sound source localization by using a drone equipped with a microphone array in this embodiment includes the following steps: S1. Obtain a multi-channel audio signal containing UAV flight noise measured by a microphone array carried by the UAV; S2, extracting a beamforming map (CB map) of a multi-channel audio signal using conventional beamforming technology; S3. Input the beamforming image into the trained neural network model to obtain the predicted sound source position and sound source intensity.

[0027] It should be noted that the microphone array carried by the drone can be a microphone array carried by a single drone or a microphone array carried by multiple drones. In this embodiment, it is a microphone array carried by 32 drones, where the distribution of each microphone array element is as follows: Figure 2 The multi-channel audio signal containing the UAV flight noise obtained in step S1 of this embodiment includes the audio data of 32 microphone array elements. , , is the number of microphone array elements. In this embodiment, the number of microphone array elements is 32. The microphone array carried by the drone is used to obtain sound signal data, and the data contains the audio signal and noise of the desired sound source, including the noise of the motor operation when the drone is in working state, the wind noise generated by the rotor driving the surrounding air, and other noises. The drone flight noise in step S1 of this embodiment includes part or all of the motor noise, rotor noise, naturally existing wind noise, and aerodynamic noise when the drone is in working state. In addition, it is necessary to obtain the height information of the array that the drone can monitor at this time.

[0028] In order to obtain the beamforming graph (CB graph) required by the neural network model, step S2 of this embodiment adopts a broadband conventional beamforming algorithm. Specifically, step S2 of this embodiment includes: S2.1, pre-delay the multi-channel audio signal according to the following formula: , In the above formula, For the After the pre-delay of the microphone array element, The audio signal of sampling points, For the The microphone array elements are The audio signal at the moment, For time, For the The microphone array elements are Pre-delay in direction, , For the The scanning delay time of each microphone array element is is the angle between the position of the drone’s microphone array and the incident direction of the sound source, is the signal sampling period, , is the number of microphone array elements; S2.2, the pre-delayed audio signals of each microphone array element are converted into digital sampling signals through A / D and then input into the FIR filter bank optimized by norm constraint, such as Figure 3 As shown, ~ The distribution of microphone array elements from 1st to Mth is The audio signal at the moment ~ After pre-delay ~ , and then converted into a digital sampling signal through A / D ~ , the FIR filter bank can be input; in this embodiment, the filter length of the FIR filter bank is , No. Microphone array element The weight of a snapshot is , the function expression of the beam output time series obtained by the FIR filter group weighted summing all data is: , In the above formula, is the first time series of the beam output The audio signal of sampling points, For the After the pre-delay of the microphone array element, The audio signal of sampling points, For the Microphone array element The weight of a snapshot, , ; The function expression using norm constraint optimization is: , , In the above formula, is the impulse response of the FIR filter bank, ~ are the impulse responses of the 1st to Lth snapshots, For the Frequency points The error weighting coefficient is , is the filter passband frequency set The number of frequency points in For the Frequency points A vector function of For the Frequency points The expected response, For the The stopband frequency at the frequency point, , is the filter stopband frequency set The number of frequency points in is the side lobe value; S2.3, calculate the energy of the obtained beam output time series according to the following formula: , In the above formula, for The energy of direction, is the total number of sampling points, is the first time series of the beam output The audio signal of sampling points; S2.4, combined with the drone altitude information, the energy in the direction is plotted as a sound energy distribution diagram on a specified plane (generally the sound source plane) as the beamforming diagram of the obtained multi-channel audio signal. In this embodiment, the beamforming diagram size is 51*51. In this embodiment, the energy in the direction is plotted as a sound energy distribution diagram on the sound source plane. The coordinates of the sound source in the plane are .

[0029] The neural network model in step S3 is based on the residual neural network (ResNet) model as the basic network framework, and integrates the attention module to establish the network model; Figure 4 As shown, in step S3 of this embodiment, after the beamforming diagram is input into the trained neural network model, the neural network model obtains the predicted sound source position and sound source intensity, including: first, using a convolution layer with 64 convolution kernels of size 7×7 to extract features, and then sending them to the residual network unit after batch normalization processing of the batch normalization layer, activation of the ReLU activation function and maximum pooling processing of the pooling layer, wherein the residual network unit includes four cascade-connected residual layers, namely residual layer 1 to residual layer 4; the output of the residual network unit is sent to the fully connected output layer after average pooling processing of a pooling layer to obtain the predicted value of the sound source position and the predicted value of the sound source intensity The output vector . In this embodiment, the neural network model is built with the ResNet architecture, and normalization, pixel-by-pixel nonlinear transformation, and convolution with a certain number of filters are implemented in each unit. The channel attention mechanism and the spatial attention mechanism are then combined, and the unit output and input are finally combined using residual connections. Each residual layer contains a BatchNorm layer, i.e., a BN layer, which normalizes the unit input data by batches and scales and shifts the data; the ReLU function is applied to the normalized input data to perform pixel-by-pixel nonlinear transformation; followed by a weight layer (convolution layer), in which the data is convolved with a certain number of filters; an attention module is added after the convolution layer to compress the feature map in the spatial dimension and the channel dimension, respectively, to increase the quality of feature representation. After passing through four residual layers, the feature map is average pooled, and a three-dimensional vector output is obtained through a fully connected output layer. The vector includes two-dimensional coordinate information , and the sound source intensity information .

[0030] like Figure 5 As shown, each residual layer in the residual network unit described in this embodiment processes the input features to obtain the output features, including: first, the input features are extracted using a convolutional layer, and then the batch normalization is performed through a batch normalization layer to achieve scaling and shifting, and then the feature F obtained after scaling and shifting is extracted using a channel attention module, and then the feature is multiplied with the original feature F obtained after scaling and shifting to obtain feature F', and the feature F' is extracted using a spatial attention module and then multiplied with the original feature F' to obtain feature F", and then the feature F" is concatenated with the input feature of the residual layer to obtain the output feature of the residual layer.

[0031] In this embodiment, before step S3, it also includes setting a simulated sound source and collecting multi-channel audio signal samples containing drone flight noise, using the multi-channel audio signal samples containing drone flight noise obtained by actual measurement as the input of the neural network model, the sound source position and sound source intensity of the simulated sound source as labels to construct a training data set, and using the training data set to train the neural network model, and using a preset loss function to update the model parameters of the neural network model to obtain a trained neural network model. The data set used in the training phase is established by adding a simulated sound source to the drone self-noise data set actually measured. The input CB map size is 51×51, first through the convolution layer, extract features, and then further extract features through multiple units, and finally obtain the predicted output through the linear layer. The loss obtained by calculating the loss function is passed through the error back propagation algorithm to continuously optimize the network and obtain a positioning model for audio signals with a large amount of non-stationary drone noise. In this embodiment, the optimization of the neural network model follows supervised learning. In supervised learning, the model learns the approximate underlying mapping from a given sample pair in an iterative process called training. The sample pair consists of input data and the expected output of the model, which is called the target. Usually, a subset S of sample pairs from the training data set is evaluated once in each training step, which is called a batch, and the batch size is N. The difference between the estimated output and the desired target is measured by an objective function L called a loss function. The loss function here is the l_2 norm of the difference between the expected and actual position information and sound source intensity. Specifically, when the preset loss function is used to update the model parameters of the neural network model in this embodiment, the function expression of the preset loss function is: , In the above formula, is the loss function, and are the label value and predicted value of the sound source position respectively, and are the label value and predicted value of the sound source intensity, respectively. In training, this is the sum average of the individual errors between the model prediction and the target value in a batch of samples. The calculation is performed by the error back propagation algorithm, that is, the weights are adjusted by the optimization algorithm at each iteration step according to the calculated gradient. In addition, the judgment module, that is, the neural network with trained weights, is also used in this embodiment to locate the input CB map to obtain the positioning information and sound source intensity information. The prediction results of the training model are evaluated using: , , In the above formula, is the deviation of the sound source position, is the deviation of the sound source intensity, , , and are the label value and predicted value of the sound source position respectively, and are the label value and predicted value of the sound source intensity respectively. is the scanning spacing of conventional beamforming.

[0032] In order to verify the effect of the method of using a drone-mounted microphone array to realize sound source localization in this embodiment, the microphone array is first used in this embodiment to record the array receiving signal and the drone self-noise when the actual drone is running, respectively, when the sound source is playing and when the sound is not playing. In combination with the actual measurement of drone noise and the simulation of sound source, the localization effect diagram can be found in Figure 6 , where max is the maximum value position of the sound source intensity, and out is the predicted value of the sound source position output by the neural network model in this embodiment , label is the label value of the sound source position . See Figure 6 It can be seen that one of the single frequency f The 0=19kHz sound source is simulated at the coordinates of (-1.05, -0.14)m. The model output predicts the sound source position as a white dot. In this case, the distance between the predicted coordinates and the actual coordinates is 0.49 scanning grids, and the relative error between the model prediction and the actual sound source intensity is 0.18. It can be said that it has a good positioning effect and sound source intensity estimation effect.

[0033] In order to verify the effect of the method of realizing sound source localization by using a microphone array mounted on a drone in this embodiment on the signal received by the array when the sound source is played, please refer to Figure 7 , where max is the maximum value position of the sound source intensity, and out is the predicted value of the sound source position output by the neural network model in this embodiment , where the drone hovers 6.2m above the sound source during measurement, and the sound source position output by the model in the figure is shown as a white dot. It can be seen that the method of realizing sound source positioning using a drone-mounted microphone array in this embodiment can accurately locate the playback sound source.

[0034] In summary, in order to realize the sound source positioning using a drone, it is necessary to consider the drone's own noise and the strong aerodynamic noise when the drone moves. These noises are picked up by the microphone array, which has a great impact on the accuracy of sound source positioning. The neural network proposed in the method of realizing sound source positioning using a drone-mounted microphone array in this embodiment extracts the characteristics of noise in the sound image through learning, so as to reduce the influence of noise on sound source positioning; on the other hand, it overcomes the low resolution problem of conventional beamforming technology. The neural network model used in the method of realizing sound source positioning using a drone-mounted microphone array in this embodiment is a neural network model based on the ResNet architecture, which can realize the positioning of the sound source in the extremely low signal-to-noise ratio signal picked up by the array, and the sound image (beamforming image) obtained by traversing the spatial position by conventional beamforming is used as the network input, and the sound source position coordinates and sound source intensity are used as the output, so as to overcome the harsh sound array conditions (limited array aperture, small array element spacing) and complex acoustic environment (extremely low signal-to-noise ratio and non-stationary characteristics of drone self-noise) on the drone, and realize the efficient and real-time sound source positioning of the drone-mounted microphone array for high-frequency sound sources.

[0035] In addition, this embodiment also provides a system for realizing sound source positioning using a microphone array carried by an unmanned aerial vehicle, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the method for realizing sound source positioning using a microphone array carried by an unmanned aerial vehicle.

[0036] In addition, this embodiment also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the method of realizing sound source positioning using a microphone array carried by an unmanned aerial vehicle through a processor.

[0037] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the method of realizing sound source localization by using a microphone array carried by a drone through a processor.

[0038] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present application may be in the form of methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0039] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for realizing sound source localization using a microphone array carried by an unmanned aerial vehicle, characterized in that: The steps include: S1. Obtain a multi-channel audio signal containing UAV flight noise measured by a microphone array carried by the UAV; S2, extracting a beamforming graph of a multi-channel audio signal using conventional beamforming techniques; S3. Input the beamforming image into the trained neural network model to obtain the predicted sound source position and sound source intensity.

2. The method for realizing sound source localization by using a microphone array mounted on an unmanned aerial vehicle according to claim 1, characterized in that: The UAV flight noise in step S1 includes part or all of the motor noise, rotor noise, naturally existing wind noise and aerodynamic noise in the UAV working state.

3. The method for realizing sound source localization by using a microphone array mounted on an unmanned aerial vehicle according to claim 1, characterized in that: Step S2 includes: S2.1, pre-delay the multi-channel audio signal according to the following formula: ， In the above formula, For the After the pre-delay of the microphone array element, The audio signal of sampling points, For the The microphone array elements are The audio signal at the moment, For time, For the The microphone array elements are Pre-delay in direction, , For the The scanning delay time of each microphone array element is is the angle between the position of the drone’s microphone array and the incident direction of the sound source, is the signal sampling period, , is the number of microphone array elements; S2.2, converting the pre-delayed audio signals of each microphone array element into digital sampling signals through A / D and then inputting them into a FIR filter bank optimized by norm constraint, wherein the filter length of the FIR filter bank is , No. Microphone array element The weight of a snapshot is , the function expression of the beam output time series obtained by the FIR filter group weighted summing all data is: ， In the above formula, is the first time series of the beam output The audio signal of sampling points, For the After the pre-delay of the microphone array element, The audio signal of sampling points, For the Microphone array element The weight of a snapshot, , ; The function expression using norm constraint optimization is: ， ， In the above formula, is the impulse response of the FIR filter bank, ~ are the impulse responses of the 1st to Lth snapshots, For the Frequency points The error weighting coefficient is , is the filter passband frequency set The number of frequency points in For the Frequency points A vector function of For the Frequency points The expected response, For the The stopband frequency at the frequency point, , is the filter stopband frequency set The number of frequency points in is the side lobe value; S2.3, calculate the energy of the obtained beam output time series according to the following formula: ， In the above formula, for The energy of direction, is the total number of sampling points, is the first time series of the beam output The audio signal of sampling points; S2.4, combining the drone altitude information, plotting the directional energy into a sound energy distribution diagram on a specified plane as a beamforming diagram of the obtained multi-channel audio signal.

4. The method for realizing sound source localization by using a microphone array mounted on an unmanned aerial vehicle according to claim 1, characterized in that: After the beamforming diagram is input into the trained neural network model in step S3, the neural network model obtains the predicted sound source position and sound source intensity, including: first, using a convolution layer with 64 convolution kernels of size 7×7 to extract features, and then sending them to the residual network unit after batch normalization processing of the batch normalization layer, activation of the ReLU activation function and maximum pooling processing of the pooling layer, wherein the residual network unit includes four cascade-connected residual layers, namely residual layer 1 to residual layer 4; the output of the residual network unit is sent to the fully connected output layer after average pooling processing of a pooling layer to obtain the predicted value of the sound source position. and the predicted value of the sound source intensity The output vector .

5. The method for realizing sound source localization by using a drone-mounted microphone array according to claim 4, characterized in that: Each residual layer in the residual network unit processes the input features to obtain output features, including: first, extracting features from the input features using a convolutional layer, and then using batch normalization processing through a batch normalization layer to achieve scaling and shifting, then extracting features from the feature F obtained after scaling and shifting using a channel attention module, and then multiplying the features with the original feature F obtained after scaling and shifting to obtain feature F', extracting features from feature F' using a spatial attention module, and then multiplying the features with the original feature F' to obtain feature F", and then concatenating feature F" with the input features of the residual layer to obtain the output features of the residual layer.

6. The method for realizing sound source localization by using a microphone array mounted on an unmanned aerial vehicle according to claim 1, characterized in that: Before step S3, it also includes setting a simulated sound source and collecting multi-channel audio signal samples containing drone flight noise, using the multi-channel audio signal samples containing drone flight noise obtained by actual measurement as the input of the neural network model, and the sound source position and sound source intensity of the simulated sound source as labels to construct a training data set, and using the training data set to train the neural network model, and using a preset loss function to update the model parameters of the neural network model to obtain a trained neural network model.

7. The method for realizing sound source localization by using a microphone array mounted on an unmanned aerial vehicle according to claim 6, characterized in that: When the preset loss function is used to update the model parameters of the neural network model, the function expression of the preset loss function is: ， In the above formula, is the loss function, and are the label value and predicted value of the sound source position respectively, and are the label value and predicted value of the sound source intensity respectively.

8. A device for realizing sound source localization using a microphone array carried by an unmanned aerial vehicle, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the method for realizing sound source localization by using a microphone array carried by an unmanned aerial vehicle as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the method for realizing sound source localization by using a microphone array carried by an unmanned aerial vehicle as described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the method for realizing sound source localization by using a microphone array carried by an unmanned aerial vehicle as described in any one of claims 1 to 7 through a processor.

Images