Sound source localization method, unmanned aerial vehicle, storage medium and program product
By acquiring audio signals through a microphone array and calculating frequency spectrum similarity metrics, rotor noise interference is suppressed, thereby improving the sound source localization accuracy and spatial resolution of rotorcraft UAVs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-07
AI Technical Summary
The noise from rotary-wing drones results in lower accuracy in sound source localization.
Audio signals from multiple channels are acquired using a microphone array. The frequency spectrum similarity measure between the beamforming signal in each direction to be identified and the direction of the target rotor is calculated. The signals are then fused to obtain the target frequency spectrum similarity measure, which determines the direction of the sound source and suppresses noise interference from the target rotor.
It improves the sound source localization accuracy and spatial resolution of rotary-wing UAVs and reduces the impact of target rotor noise on positioning.
Smart Images
Figure CN121805949A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of positioning technology, and in particular to a sound source positioning method, a drone, a storage medium, and a program product. Background Technology
[0002] Currently, microphone array-based sound source localization technology combined with rotary-wing UAV equipment platforms has shown great potential in fields such as rescue, security, and industrial inspection. It can be used to locate the calls of people in earthquake ruins and field rescue, locate gunshots outdoors, and locate partial discharge of high-voltage cables in industrial inspection.
[0003] However, the noise from the rotorcraft itself results in low accuracy in locating the sound source. Summary of the Invention
[0004] In view of the above problems, this application provides a sound source localization method, a drone, a storage medium, and a program product to improve the sound source localization accuracy of rotary-wing drones. The specific solution is as follows:
[0005] The first aspect of this application provides a sound source localization method, applied to a rotary-wing unmanned aerial vehicle equipped with a microphone array, the method comprising:
[0006] Audio signals from multiple channels are acquired through the microphone array;
[0007] For each direction to be identified, the frequency spectrum similarity measure of the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the beamforming signal in the direction of at least one target rotor is calculated; the direction of the target rotor is the direction in which noise suppression is required.
[0008] The frequency spectrum similarity measures corresponding to the same direction to be identified are fused to obtain the target frequency spectrum similarity measures corresponding to each direction to be identified.
[0009] The direction to be identified corresponding to the target frequency spectrum similarity metric that represents the minimum similarity is determined as the direction of the sound source.
[0010] In one possible implementation, calculating the frequency spectrum similarity measure between the beamforming signal of the audio signals of the plurality of channels in the direction to be identified and the beamforming signal in the direction of the target rotor includes:
[0011] The audio signals of the multiple channels are converted into frequency domain signals of each channel at different frequency points;
[0012] For each frequency point, calculate the linear correlation between the frequency domain signals of each pair of channels at that frequency point;
[0013] Based on the linear correlation of the frequency domain signals of each pair of channels at this frequency point, the frequency spectrum similarity measure of the beamforming signal of the audio signals of the multiple channels at this frequency point in the direction to be identified and the beamforming signal in the direction of the target rotor is calculated.
[0014] In one possible implementation, calculating the frequency spectrum similarity measure between the beamforming signals of the multiple channels at that frequency point in the direction to be identified and the beamforming signals in the direction of the target rotor includes:
[0015] The weights of the beamforming signals of the multiple channels at the target direction at that frequency point are obtained according to the beamforming algorithm; the target direction includes the direction to be identified and the direction where the target rotor is located;
[0016] Based on the linear correlation of the frequency domain signals of each pair of channels at that frequency point, and the weight of the beamforming signal of the audio signals of the multiple channels in the target direction at that frequency point, the energy of the beamforming signal of the audio signals of the multiple channels in the target direction at that frequency point is calculated.
[0017] Calculate the distance between the energy of the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the energy in the direction of the target rotor at that frequency point, and use this distance as a frequency spectrum similarity measure between the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the beamforming signal in the direction of the target rotor at that frequency point.
[0018] In one possible implementation, the distance between the energy of the beamforming signal of the audio signals of the plurality of channels in the direction to be identified and the energy in the direction of the target rotor is the Itakura–Saito distance.
[0019] In one possible implementation, fusing the frequency spectrum similarity measures corresponding to the direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified includes:
[0020] The frequency spectrum similarity measures corresponding to the directions of each target rotor at the same frequency point are summed to obtain the frequency spectrum similarity measures corresponding to each frequency point.
[0021] The frequency spectrum similarity measures corresponding to each frequency point of the direction to be identified are summed to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified.
[0022] In one possible implementation, fusing the frequency spectrum similarity measures corresponding to the direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified includes:
[0023] The coherence factor corresponding to each frequency point is obtained based on the frequency domain signals of each channel at the same frequency point;
[0024] The frequency spectrum similarity measures corresponding to the directions of each target rotor at the same frequency point are summed to obtain the frequency spectrum similarity measures corresponding to each frequency point.
[0025] The frequency spectrum similarity measures corresponding to each frequency point corresponding to the direction to be identified are weighted and summed to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified; the weight of the frequency spectrum similarity measure corresponding to each frequency point is the coherence factor corresponding to that frequency point.
[0026] In one possible implementation, the coherence factor corresponding to each frequency point is obtained based on the frequency domain signals of each channel at the same frequency point, including:
[0027] The frequency domain signals of each channel at the same frequency point are summed to obtain the mixed frequency domain signal at each frequency point.
[0028] For each frequency point, the coherence factor corresponding to that frequency point is calculated based on the power density of the mixed frequency signal at that frequency point and the power density of the frequency domain signal of each channel. The coherence factor corresponding to that frequency point is negatively correlated with the power density of the frequency domain signal of each channel and positively correlated with the power density of the mixed frequency domain signal.
[0029] A second aspect of this application provides a computer program product including computer-readable instructions that, when executed on a rotary-wing unmanned aerial vehicle (UAV), cause the UAV to implement the sound source localization method described in the first aspect or any implementation thereof.
[0030] A third aspect of this application provides a rotary-wing unmanned aerial vehicle (UAV) including a microphone array, at least one processor, and a memory connected to the processor, wherein:
[0031] The memory is used to store computer programs;
[0032] The processor is used to execute the computer program so that the rotary-wing UAV can implement the sound source localization method of the first aspect or any implementation thereof.
[0033] A fourth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by a rotary-wing UAV, enable the UAV to implement the sound source localization method described in the first aspect or any implementation thereof.
[0034] By employing the above technical solution, the sound source localization method, UAV, storage medium, and program product provided in this application involve a rotary-wing UAV acquiring multiple channels of audio signals via a microphone array. For each direction to be identified, the beamforming signal of the multiple channels of audio signals in that direction is calculated, along with a frequency spectrum similarity measure between the beamforming signal in the direction of at least one target rotor. The direction of the target rotor is the direction requiring noise suppression. The frequency spectrum similarity measures corresponding to the same direction to be identified are fused to obtain a target frequency spectrum similarity measure for each direction to be identified. The direction to be identified corresponding to the target frequency spectrum similarity measure representing the minimum similarity is determined as the direction of the sound source. This application, after acquiring multiple channels of audio signals via a microphone array, compares the frequency spectrum similarity between the beamforming signal of the multiple channels of audio signals in each direction to be identified and the beamforming signal in the direction of each target rotor. Based on the comparison results, sound source localization is achieved, avoiding the influence of noise generated by the target rotor on sound source localization and improving the accuracy of sound source localization by the rotary-wing UAV. Attached Figure Description
[0035] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0036] Figure 1 A flowchart illustrating an implementation of the sound source localization method provided in this application;
[0037] Figure 2 A flowchart for implementing a frequency spectrum similarity measurement of the beamforming signal of multiple channels of audio signals in the q-th direction to be identified and the beamforming signal in the direction of the i-th target rotor, provided in this application;
[0038] Figure 3 A flowchart illustrating an implementation of this application for fusing the frequency spectrum similarity measures corresponding to the qth direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the qth direction to be identified;
[0039] Figure 4 A flowchart illustrating another implementation of this application for fusing the frequency spectrum similarity measures corresponding to the qth direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the qth direction to be identified;
[0040] Figure 5 A schematic diagram of the internal structure of a rotary-wing unmanned aerial vehicle provided in this application;
[0041] Figure 6 An example diagram of the appearance of the rotary-wing drone provided in this application. Detailed Implementation
[0042] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0043] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0044] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0045] Existing sound source localization algorithms are generally general-purpose algorithms. For example, they use beamforming algorithms to calculate the beamforming signal of the audio signals from multiple channels collected by a microphone array in each direction to be identified, calculate the power of the beamforming signal in each direction, and determine the direction of the sound source as the direction corresponding to the maximum power. This method has low computational cost, but the main lobe of the beamforming signal is wide and the side lobes are high, resulting in poor interference suppression capability and beam spatial resolution, leading to unsatisfactory localization results.
[0046] This application proposes a solution to improve the sound source localization accuracy of rotary-wing unmanned aerial vehicles (UAVs).
[0047] The sound source localization method of this application is applied to a rotary-wing UAV equipped with a microphone array. The rotary-wing UAV can be a single-rotor UAV or a multi-rotor UAV.
[0048] like Figure 1 The diagram shown is a flowchart of one implementation of the sound source localization method provided in this application, which may include:
[0049] Step S101: Acquire audio signals from multiple channels using a microphone array.
[0050] Each microphone in the microphone array is a separate acquisition channel; therefore, each microphone acquires one audio signal, which is a single channel of audio signal. The audio signals acquired by the microphones are time-domain signals.
[0051] Step S102: For each direction to be identified, calculate the frequency spectrum similarity measure between the beamforming signal of the audio signals of multiple channels in the direction to be identified and the beamforming signal in the direction of at least one target rotor.
[0052] The direction in which the target rotor is located is the direction in which noise suppression is required.
[0053] When a rotary-wing drone is performing positioning, it needs to compare beamforming signals from multiple directions (each direction being a direction to be identified) to determine the direction of the sound source.
[0054] In this application, the direction to be identified and the direction of the target rotor are both determined in the same coordinate system. For example, a coordinate system can be constructed with the center of the microphone array as the origin, and the directions to be identified and the direction of the target rotor can be determined in this coordinate system.
[0055] For the q-th (q=1, 2, 3, ..., Q; Q is the number of directions to be identified) direction to be identified and the i-th (i=1, 2, 3, ..., K; K is the number of target rotors) target rotor, this application calculates the frequency spectrum similarity measure between the beamforming signal of the audio signals of multiple channels in the q-th direction to be identified and the beamforming signal in the direction of the i-th target rotor. This frequency spectrum similarity measure characterizes the similarity in the frequency domain between the beamforming signal of the audio signals of multiple channels in the q-th direction to be identified and the beamforming signal in the direction of the i-th target rotor.
[0056] If there are K target rotors, then for the q-th direction to be identified, K frequency spectrum similarity measures are calculated.
[0057] If the rotary-wing drone has only one rotor, then the target rotor is that single rotor. If the rotary-wing drone has multiple rotors, then the target rotor may be one, two, or more; the specific number of target rotors can be determined based on the actual noise reduction requirements.
[0058] When a rotary-wing UAV has multiple rotors, the target rotor can be the rotor whose sound has a greater impact on the localization of the sound source. For example, it can be one or more rotors that are close to the microphone array. In this case, the intensity of the sound signal generated by the target rotor (referred to as noise signal for easy distinction) at the microphone array is greater than the intensity of the noise signal generated by the non-target rotor at the microphone array.
[0059] Step S103: Fuse the frequency spectrum similarity measures corresponding to the same direction to be identified to obtain the target frequency spectrum similarity measures corresponding to each direction to be identified.
[0060] The K frequency spectrum similarity measures corresponding to the q-th direction to be identified can be fused to obtain the target frequency spectrum similarity measure corresponding to the q-th direction to be identified.
[0061] If there is only one target rotor, then the frequency spectrum similarity measure corresponding to the qth direction to be identified is the target frequency spectrum similarity measure corresponding to the qth direction to be identified.
[0062] If there are Q directions to be identified, then Q target frequency spectrum similarity measures are obtained.
[0063] Step S104: Determine the direction to be identified corresponding to the target frequency spectrum similarity metric that represents the minimum similarity as the direction of the sound source.
[0064] When determining the direction of a sound source by comparing beamforming signals from multiple directions to be identified, this application does not directly compare the beamforming signals, but rather compares the similarity in the frequency domain between the beamforming signals from the direction to be identified and the beamforming signals from the direction where the target rotor is located, thereby suppressing the interference of noise generated by the target rotor on the sound source localization.
[0065] The sound source localization method provided in this application collects audio signals from multiple channels through a microphone array, and then compares the frequency spectrum similarity between the beamforming signals of the audio signals from multiple channels in each direction to be identified and the beamforming signals in the direction where each target rotor is located. Based on the comparison results, the sound source is located, avoiding the influence of noise generated by the target rotor on the sound source localization and improving the accuracy of sound source localization by the rotorcraft UAV.
[0066] In an optional embodiment, a flowchart illustrating an implementation of a frequency spectrum similarity measure between the beamforming signal of multiple channels' audio signals in the q-th direction to be identified and the beamforming signal in the direction of the i-th target rotor is shown below. Figure 2 As shown, it may include:
[0067] Step S201: Convert the audio signals of multiple channels into frequency domain signals of each channel at different frequency points.
[0068] The audio signal of each channel can be subjected to a Fourier transform (e.g., Fast Fourier Transform, FFT) to obtain the frequency domain signal of that channel at different frequency points. Assuming the microphone array has M elements, the audio signals from the M channels acquired by the M elements are denoted as follows: Perform FFT transformation on the audio signal of each channel to obtain multi-channel frequency domain signals at different frequency points: . This represents the frequency of the nth frequency point, where n = 0, 1, 2, ..., N-1.
[0069] Step S202: For the nth frequency point, calculate the linear correlation of the frequency domain signals of each pair of channels at the nth frequency point.
[0070] Optionally, for the nth frequency point, the covariance matrix of the M frequency domain signals at that nth frequency point can be calculated as the linear correlation of the M channels at that nth frequency point. This can be expressed by the formula:
[0071] (1)
[0072] in, Let be the covariance matrix of M frequency domain signals at the nth frequency point. Each element in this covariance matrix corresponds to the linear correlation between the frequency domain signals of two channels. The elements on the diagonal are the linear correlation between the frequency domain signals of the same channel, and the elements off-diagonal are the linear correlation between the frequency domain signals of two different channels.
[0073] Step S203: Based on the linear correlation of the frequency domain signals of each channel at the nth frequency point, calculate the frequency spectrum similarity measure between the beamforming signal of the audio signals of multiple channels at the nth frequency point in the qth direction to be identified and the beamforming signal in the direction of the ith target rotor.
[0074] In other words, for the q-th direction to be identified and the i-th target rotor, N spectral similarity measures are calculated.
[0075] Optionally, the frequency spectrum similarity measure between the beamforming signal of multiple channels at the nth frequency point in the qth direction to be identified and the beamforming signal in the direction of the i-th target rotor can be calculated using the following method:
[0076] The weights of the beamforming signals of multiple audio channels at the target direction at that frequency point are obtained based on the beamforming algorithm; the target direction includes the direction to be identified and the direction where the target rotor is located.
[0077] Beamforming algorithms may include, but are not limited to, any of the following: Delay and Sum (DAS) beamforming algorithm, Time Difference of Arrival (TDA) algorithm, etc.
[0078] Taking DAS as an example, the time-domain expression of the beamforming signal of the M-channel audio signal in the q-th direction to be identified is:
[0079] (2)
[0080] in, For an audio signal with M channels, in the q-th direction to be identified The time-domain representation of the beamforming signal; express The directional sound signal reaches the first The time delay of each array element relative to the reference point (i.e., the reference array element); when the m-th array element is the reference array element, ,therefore, The time-domain waveform signal received by the reference array element.
[0081] Performing a Fourier transform on equation (2) yields the frequency domain expression of the beamforming signal of the M-channel audio signals in the q-th direction to be identified:
[0082] (3)
[0083] (4)
[0084] (5)
[0085] in: for The weight of the beamforming signal at the nth frequency point in the direction; These are the coordinates of the array element relative to the origin. yes The direction vector of the directional beamforming signal, where c is the speed of sound. Let n be the frequency of the nth frequency point.
[0086] Based on the linear correlation of the frequency domain signals of each pair of channels at the nth frequency point, and the weight of the beamforming signal of the audio signals of multiple channels in the target direction at the nth frequency point, the energy of the beamforming signal of the audio signals of multiple channels in the target direction at the nth frequency point is calculated.
[0087] For the q-th direction to be identified, the energy of the beamforming signal of the audio signals of multiple channels at the n-th frequency point in the q-th direction to be identified can be calculated using the following formula. :
[0088] (6)
[0089] Similarly, corresponding to the direction of the i-th target rotor, the energy of the beamforming signal of the multiple channels of audio signal at the n-th frequency point in the direction of the i-th target rotor can be calculated using the following formula. :
[0090] (7)
[0091] Calculate the distance between the energy of the beamforming signal of the audio signals of multiple channels in the q-th identification direction and the energy in the direction of the i-th target rotor at the n-th frequency point. Use this distance as a frequency spectrum similarity measure between the beamforming signal of the audio signals of multiple channels in the q-th identification direction and the beamforming signal in the direction of the i-th target rotor at the n-th frequency point.
[0092] Optionally, at the nth frequency point, the distance between the energy of the beamforming signal of the audio signals of multiple channels in the qth direction to be identified and the energy in the direction of the ith target rotor can be either the KL divergence distance or the Itakura–Saito distance.
[0093] Taking the Itakura–Saito distance as an example, at the nth frequency point, the distance between the energy of the beamforming signal of the audio signals of multiple channels in the qth direction to be identified and the energy in the direction of the i-th target rotor. It can be:
[0094] (8)
[0095] In an optional embodiment, a flowchart illustrating one method of fusing the frequency spectrum similarity measures corresponding to the q-th direction to obtain the target frequency spectrum similarity measure corresponding to the q-th direction to be identified is shown below. Figure 3 As shown, it may include:
[0096] Step S301: Sum the frequency spectrum similarity measures of each target rotor at the same frequency point corresponding to the q-th direction to be identified, and obtain the frequency spectrum similarity measure corresponding to each frequency point.
[0097] Summing the frequency spectrum similarity metrics corresponding to the directions of each target rotor at the nth frequency point for the qth direction to be identified, we obtain the frequency spectrum similarity metric for the nth frequency point. This can be expressed by the formula:
[0098] (9)
[0099] Where K represents the number of target rotors.
[0100] Step S302: Sum the frequency spectrum similarity measures corresponding to each frequency point of the q-th direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the q-th direction to be identified.
[0101] Summing the frequency spectrum similarity metrics corresponding to the N frequency points for the q-th direction to be identified yields the target frequency spectrum similarity metric for the q-th direction to be identified. This can be expressed by the formula:
[0102] (10)
[0103] In an optional embodiment, another implementation flowchart of fusing the frequency spectrum similarity measures corresponding to the q-th direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the q-th direction to be identified is shown below. Figure 4 As shown, it may include:
[0104] Step S401: Obtain the coherence factor corresponding to each frequency point based on the frequency domain signals of each channel at the same frequency point.
[0105] The coherence factor corresponding to the nth frequency point represents the weight of the nth frequency point.
[0106] Optionally, the frequency domain signals of each channel at the same frequency point can be summed to obtain the mixed frequency domain signal at each frequency point;
[0107] For each frequency point, the coherence factor corresponding to that frequency point is calculated based on the power density of the mixed frequency signal at that frequency point and the power density of the frequency domain signal of each channel. The coherence factor corresponding to that frequency point is negatively correlated with the power density of the frequency domain signal of each channel and positively correlated with the power density of the mixed frequency domain signal.
[0108] Optionally, the coherence factor corresponding to the f-th frequency point can be calculated as follows: :
[0109] (11)
[0110] in, This represents the frequency domain signal of the m-th channel at the n-th frequency point. Let be the power density of the frequency domain signal of the m-th channel at the n-th frequency point; This is the mixed frequency domain signal at the nth frequency point; Let be the power density of the mixed frequency domain signal at the nth frequency point.
[0111] Step S402: Sum the frequency spectrum similarity measures of each target rotor at the same frequency point corresponding to the q-th direction to be identified, and obtain the frequency spectrum similarity measure corresponding to each frequency point.
[0112] For details on the implementation process, please refer to step S301, which will not be repeated here.
[0113] Step S403: Under the same direction to be identified, the frequency spectrum similarity measures corresponding to each frequency point are weighted and summed to obtain the target frequency spectrum similarity measure corresponding to each direction to be identified; the weight of the frequency spectrum similarity measure corresponding to each frequency point is the coherence factor corresponding to that frequency point.
[0114] This can be expressed as a formula:
[0115] (12)
[0116] in, This is the coherence factor corresponding to the nth frequency point.
[0117] By integrating coherence factor and frequency spectrum similarity metric into the sound source localization algorithm for rotorcraft UAVs in noisy scenarios, the accuracy of sound source localization for rotorcraft UAVs is further improved, while the spatial resolution of sound source localization is also enhanced. Combining these two data points (coherence factor and frequency spectrum similarity metric) can improve the overall localization performance of rotorcraft UAVs in scenarios with extremely low signal-to-noise ratios, particularly for weak signals and at long distances.
[0118] Corresponding to the method embodiments, this application also provides an electronic device, which is a rotary-wing drone, the rotary-wing drone including at least one rotor. (Reference) Figure 5 As shown, it illustrates an internal structure diagram suitable for implementing the embodiments of the present application of a rotary-wing unmanned aerial vehicle. Figure 5 The rotary-wing drone shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0119] like Figure 5 As shown, the rotary-wing drone may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. When the electronic device is powered on, the RAM 503 also stores various programs and data required for the operation of the electronic device. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 404. An input / output (I / O) interface 505 is also connected to the bus 504.
[0120] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, microphone arrays, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, memory cards, hard drives, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0121] like Figure 6 The image shown is an example of the appearance of a rotary-wing drone provided in an embodiment of this application. In this example, the rotary-wing drone has four rotors, namely rotor 1, rotor 2, rotor 3 and rotor 4. The microphone array is set near rotor 1 and rotor 2, so rotor 1 and rotor 2 can be used as target rotors.
[0122] This application also provides a computer program product including computer-readable instructions, which, when executed on a rotary-wing drone, enable the rotary-wing drone to implement any of the sound source localization methods provided in this application.
[0123] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by the rotary-wing UAV, the rotary-wing UAV can realize any of the sound source localization methods provided in this application.
[0124] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0126] In the above embodiments, the functionality can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product. Those skilled in the art can use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.
[0127] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0129] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A sound source localization method, applied to a rotary-wing unmanned aerial vehicle equipped with a microphone array, characterized in that, The method includes: Audio signals from multiple channels are acquired through the microphone array; For each direction to be identified, the frequency spectrum similarity measure of the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the beamforming signal in the direction of at least one target rotor is calculated; the direction of the target rotor is the direction in which noise suppression is required. The frequency spectrum similarity measures corresponding to the same direction to be identified are fused to obtain the target frequency spectrum similarity measures corresponding to each direction to be identified. The direction to be identified corresponding to the target frequency spectrum similarity metric that represents the minimum similarity is determined as the direction of the sound source.
2. The method according to claim 1, characterized in that, Calculating the frequency spectrum similarity measure between the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the beamforming signal in the direction of the target rotor includes: The audio signals of the multiple channels are converted into frequency domain signals of each channel at different frequency points; For each frequency point, calculate the linear correlation between the frequency domain signals of each pair of channels at that frequency point; Based on the linear correlation of the frequency domain signals of each pair of channels at this frequency point, the frequency spectrum similarity measure of the beamforming signal of the audio signals of the multiple channels at this frequency point in the direction to be identified and the beamforming signal in the direction of the target rotor is calculated.
3. The method according to claim 2, characterized in that, Calculating the frequency spectrum similarity measure between the beamforming signals of the multiple channels at this frequency point in the direction to be identified and the beamforming signals in the direction of the target rotor, including: The weights of the beamforming signals of the multiple channels at the target direction at that frequency point are obtained according to the beamforming algorithm; the target direction includes the direction to be identified and the direction where the target rotor is located; Based on the linear correlation of the frequency domain signals of each pair of channels at that frequency point, and the weight of the beamforming signal of the audio signals of the multiple channels in the target direction at that frequency point, the energy of the beamforming signal of the audio signals of the multiple channels in the target direction at that frequency point is calculated. Calculate the distance between the energy of the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the energy in the direction of the target rotor at that frequency point, and use this distance as a frequency spectrum similarity measure between the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the beamforming signal in the direction of the target rotor at that frequency point.
4. The method according to claim 3, characterized in that, The distance between the energy of the beamforming signal of the audio signals of the multiple channels in the direction to be identified and the energy in the direction of the target rotor is the Itakura–Saito distance.
5. The method according to claim 2, characterized in that, The step of fusing the frequency spectrum similarity measures corresponding to the direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified includes: The frequency spectrum similarity measures corresponding to the directions of each target rotor at the same frequency point are summed to obtain the frequency spectrum similarity measures corresponding to each frequency point. The frequency spectrum similarity measures corresponding to each frequency point of the direction to be identified are summed to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified.
6. The method according to claim 2, characterized in that, The step of fusing the frequency spectrum similarity measures corresponding to the direction to be identified to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified includes: The coherence factor corresponding to each frequency point is obtained based on the frequency domain signals of each channel at the same frequency point; The frequency spectrum similarity measures corresponding to the directions of each target rotor at the same frequency point are summed to obtain the frequency spectrum similarity measures corresponding to each frequency point. The frequency spectrum similarity measures corresponding to each frequency point corresponding to the direction to be identified are weighted and summed to obtain the target frequency spectrum similarity measure corresponding to the direction to be identified; the weight of the frequency spectrum similarity measure corresponding to each frequency point is the coherence factor corresponding to that frequency point.
7. The method according to claim 6, characterized in that, The coherence factor for each frequency point is obtained based on the frequency domain signals of each channel at the same frequency point, including: The frequency domain signals of each channel at the same frequency point are summed to obtain the mixed frequency domain signal at each frequency point. For each frequency point, the coherence factor corresponding to that frequency point is calculated based on the power density of the mixed frequency signal at that frequency point and the power density of the frequency domain signal of each channel. The coherence factor corresponding to that frequency point is negatively correlated with the power density of the frequency domain signal of each channel and positively correlated with the power density of the mixed frequency domain signal.
8. A rotary-wing unmanned aerial vehicle, characterized in that, The rotary-wing drone includes a microphone array, at least one rotor, at least one processor, and a memory connected to the processor; wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the rotary-wing UAV to implement the sound source localization method as described in any one of claims 1 to 7.
9. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on a rotary-wing drone, cause the rotary-wing drone to perform the sound source localization method as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by the rotary-wing UAV, enable the rotary-wing UAV to perform the sound source localization method as described in any one of claims 1 to 7.