An unmanned aerial vehicle target tracking method and an unmanned aerial vehicle target tracking system
By using a multi-dimensional microphone array and convolutional neural network to identify multiple sound categories in complex sound fields, and combining this with a spherical interpolation time delay estimation algorithm to achieve UAV target tracking, the problems of traditional visual blind spots and insufficient audio recognition are solved, thus improving the adaptability and accuracy of UAV target tracking.
Patent Information
- Application Number
- CN202310587056.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2043-05-23
AI Technical Summary
Traditional visual target tracking methods cannot track targets in visual blind spots and under the influence of external factors such as light, and existing audio signals have limited sound type recognition capabilities, making it difficult to cope with complex sound field scenarios.
Using a multi-dimensional microphone array and convolutional neural network, sound features are obtained from audio signals. A pre-trained model is used to identify multiple sound categories, and a spherical interpolation delay estimation algorithm is combined to locate the sound source and control the UAV to track the target.
It enables the recognition and target tracking of multiple sound types in complex sound field environments, reduces storage and processing difficulties, and improves the reliability and adaptability of UAV target tracking.
Smart Images

Figure CN116382350B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of unmanned aerial vehicle target tracking, and particularly relates to an unmanned aerial vehicle target tracking method, an unmanned aerial vehicle target tracking system, and a computer readable storage medium. BACKGROUND
[0002] In recent years, the unmanned aerial vehicle industry has developed rapidly, and unmanned aerial vehicle target tracking has gradually become one of the current hot research directions in the field of unmanned aerial vehicles. The target tracking of unmanned aerial vehicles can be achieved through various ways such as binding electronic tags or installing GPS trackers on the tracked targets, or through cameras to visually identify and track moving targets in videos.
[0003] At present, the traditional visual target tracking method has the problems that it cannot track the visual blind area, and it is easily affected by external factors such as light and field of view. These problems will lead to the occurrence of situations such as the inability of the unmanned aerial vehicle to identify the target or the loss of target tracking during the target identification process. Compared with the above visual target tracking based on image information, audio information can be free from the limitations of blind area, light and field of view, and has advantages in storage space and processing difficulty. Therefore, implementing unmanned aerial vehicle target tracking through audio signals is another feasible way. However, the current positioning and tracking of targets through audio signals are applied in the field of security monitoring, and the types of sounds identified are only two, which are not enough to cope with the complex sound field scene of sound categories.
[0004] In order to solve the above problems existing in the prior art, there is an urgent need in the field for an unmanned aerial vehicle target tracking technology that can identify multiple sound types in a sound field environment with complex sound categories and achieve target tracking of unmanned aerial vehicles based on sound signals. SUMMARY
[0005] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
[0006] In order to overcome the above-mentioned defects existing in the prior art, the present application provides an unmanned aerial vehicle target tracking method, an unmanned aerial vehicle target tracking system, and a computer readable storage medium, which can identify multiple sound types in a sound field environment with complex sound categories and achieve target tracking of unmanned aerial vehicles based on sound signals with lower storage and processing difficulty.
[0007] Specifically, the unmanned aerial vehicle target tracking method according to the first aspect of the present application comprises the following steps: obtaining a sound signal around the unmanned aerial vehicle and extracting a sound feature from the sound signal; determining, via a pre-trained sound recognition model, whether a specified tracking target exists around the unmanned aerial vehicle according to the sound feature; in response to a determination result that the tracking target exists, performing sound source localization on the sound signal to determine position information of the tracking target; and determining a flight instruction of the unmanned aerial vehicle according to the position information to control the unmanned aerial vehicle to track the tracking target.
[0008] Further, in some embodiments of the present application, the step of obtaining a sound signal around the unmanned aerial vehicle and extracting a sound feature from the sound signal comprises: performing frame processing and time-frequency domain conversion on the obtained sound signal to obtain a corresponding Mel spectrum representation; and determining the sound feature according to the Mel spectrum representation.
[0009] Further, in some embodiments of the present application, the step of performing frame processing and time-frequency domain conversion on the obtained sound signal to obtain a corresponding Mel spectrum representation comprises: performing endpoint detection on the sound signal subjected to the frame processing to extract valid audio data; and performing time-frequency domain conversion on the audio data to obtain the Mel spectrum representation.
[0010] Further, in some embodiments of the present application, the step of obtaining a sound signal around the unmanned aerial vehicle and extracting a sound feature from the sound signal comprises: analyzing the obtained sound signal to determine a sound field environment in which the unmanned aerial vehicle is located; and in response to the sound field environment being a preset target sound field environment, extracting a sound feature corresponding to the target sound field environment from the sound signal.
[0011] Further, in some embodiments of the present application, the step of determining, via a pre-trained sound recognition model, whether a specified tracking target exists around the unmanned aerial vehicle according to the sound feature comprises: calling a corresponding target sound recognition model to analyze the sound feature according to the target sound field environment to determine at least one sound category contained in the sound signal, wherein the target sound recognition model is learned via a sound signal training set of the target sound field environment; and comparing a specified sound category corresponding to the tracking target with the at least one sound category contained in the sound signal to determine whether the tracking target exists around the unmanned aerial vehicle.
[0012] Further, in some embodiments of the present application, before comparing the specified sound class corresponding to the tracking target with at least one sound class contained in the sound signal to determine whether the tracking target exists around the UAV, the UAV target tracking method further comprises the following steps: obtaining the specified information of the tracking target via the human-computer interaction interface; and determining the specified sound class corresponding to the tracking target from at least one candidate sound class involved in the target sound field environment according to the specified information.
[0013] Further, in some embodiments of the present application, the target sound field environment comprises a city sound field environment, which involves at least one candidate sound class of air conditioner running sound, car horn sound, children playing sound, dog barking sound, drilling sound, engine idling sound, siren sound, helicopter sound, and street music sound.
[0014] Further, in some embodiments of the present application, the step of obtaining the sound signal around the UAV comprises: obtaining the sound signal from different directions around the UAV via a multi-dimensional microphone array distributed in the sound field space.
[0015] Further, in some embodiments of the present application, the sound source positioning is implemented based on a spherical interpolation-based time delay estimation positioning algorithm, wherein the step of performing sound source positioning on the sound signal to determine the position information of the tracking target comprises: in the N+1 multi-dimensional microphone array, taking the position of the microphone M0 as a reference position, and defining the distance between the microphone M i and the microphone M0 as d i , and defining the distance between the sound source S and the microphone M0 as R i , to obtain:
[0016] d i =||m i -m||-||m||;
[0017] The distance from the microphone M i to the microphone M0 is defined as R i , and the distance from the sound source S to the microphone M0 is defined as R S , to obtain:
[0018]
[0019] The estimated value of the distance d i is represented as a function of error affected by environmental noise:
[0020]
[0021] to obtain a matrix form of N error functions in the N+1 dimensional microphone array:
[0022] ε = δ - 2R s d - 2Cs
[0023] wherein δ, d, M represent respectively:
[0024]
[0025] obtained when the N error functions reach a minimum value:
[0026]
[0027] and
[0028] solving the sound source position equation to determine an estimated position of the tracking target as:
[0029]
[0030] Further, in some embodiments of the present application, the step of determining flight instructions of the UAV according to the position information to control the UAV to track the tracking target comprises: taking the UAV as a controlled object, and controlling the UAV to fly to the estimated position of the tracking target via a PID controller.
[0031] Further, in some embodiments of the present application, the step of training the sound recognition model comprises: constructing a sound recognition model to be trained by using a convolutional neural network; defining at least one sound field environment, and obtaining sound signals of multiple sound sources in the sound field environment; performing feature extraction on the obtained sound signals to determine corresponding sound features; performing sound recognition on the sound features of each sound source in the sound field environment via the sound recognition model to be trained to determine corresponding sound category output values; and correcting learning parameters of the convolutional neural network according to each sound category output value and a corresponding true category label to train the sound recognition model.
[0032] Further, in some embodiments of the present application, the step of obtaining sound signals of multiple sound sources in the sound field environment comprises: randomly playing an audio of a candidate sound category via multiple loudspeakers at different positions at the same volume; and obtaining sound signals from different directions around the UAV via a multi-dimensional microphone array distributed in the sound field space.
[0033] Further, in some embodiments of the present application, the step of performing feature extraction on the acquired sound signal to determine a sound feature corresponding to the sound signal comprises: performing pre-processing of frame segmentation and / or windowing on the acquired sound signal; converting the pre-processed sound signal from time domain to frequency domain, and obtaining a spectrogram of each frame of the sound signal by using fast Fourier transform; and mapping the spectrogram of the sound signal to a Mel spectrogram by using a Mel filter bank, as the sound feature.
[0034] In addition, according to the second aspect of the present application, the above-mentioned UAV target tracking system comprises: a memory having computer instructions stored thereon; and a processor connected to the memory and configured to execute the computer instructions stored on the memory to implement the above-mentioned UAV target tracking method provided by the first aspect of the present application.
[0035] In addition, according to the third aspect of the present application, a computer readable storage medium having computer instructions stored thereon is also provided. When the computer instructions are executed by a processor, the above-mentioned UAV target tracking method provided by the first aspect of the present application is implemented. BRIEF DESCRIPTION OF DRAWINGS
[0036] The above features and advantages of the present application can be better understood by reading the following detailed description of embodiments of the present application in conjunction with the drawings, in which: In the drawings, components are not necessarily drawn to scale, and components of similar or identical related function or features can have the same or similar reference label.
[0037] Figure 1 A flow chart of a UAV target tracking method according to some embodiments of the present application is shown;
[0038] Figure 2 A flow chart of determining whether a specified tracking target exists around a UAV according to some embodiments of the present application is shown; and
[0039] Figure 3 A structural block diagram of a UAV target tracking system according to some embodiments of the present application is shown.
[0040] REFERENCE SIGNS:
[0041] S110-S140 steps;
[0042] S121-S122 steps;
[0043] 300 UAV target tracking system;
[0044] 310 memory;
[0045] 320 processor. DETAILED DESCRIPTION
[0046] The advantages and benefits of the present application will become apparent upon reading the following description in conjunction with the accompanying drawings. While the application will be described in conjunction with the preferred embodiments, it will be understood that they are given by way of example only and that the application is not intended to be limited by the preferred embodiments. On the contrary, it is intended to cover all alternatives, modifications and equivalents falling within the scope of the application. To provide for a complete disclosure without unnecessary restriction, the following description will contain specific details that can not be required. However, it should be understood that no unnecessary elements are included in order to not obscure the teachings of the present application.
[0047] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "linking" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium, or internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.
[0048] In addition, in the following description, "up", "down", "left", "right", "top", "bottom", "horizontal", "vertical" should be understood as the orientation shown in the paragraph and the related drawings. The relative terms are only used for the convenience of description, and they do not mean that the devices described should be manufactured or operated in a particular orientation, so they should not be understood as a limitation on the present application.
[0049] As described above, at present, the problems faced by the traditional visual target tracking mode are that the visual blind area cannot be tracked, and it is easy to be affected by external factors such as light and field of view, which will lead to the situation that the unmanned aerial vehicle cannot identify the target or the target tracking is lost during target identification. Compared with the above visual target tracking according to image information, audio information can not be limited by blind area, light and field of view, and has advantages in storage space and processing difficulty. Therefore, realizing unmanned aerial vehicle target tracking through audio signal is another feasible way. However, at present, the positioning and tracking of target through audio signal is applied in the field of security monitoring, and the type of sound identified is only two, which is not enough to deal with the sound field scene with complex sound categories.
[0050] To solve the above problems in the prior art, the present application provides a UAV target tracking method, a UAV target tracking system, and a computer readable storage medium, which can identify multiple sound types in a sound field environment with complex sound categories, and realize target tracking of a UAV based on sound signals with low storage and processing difficulty.
[0051] In some non-limiting embodiments, the UAV target tracking method provided by the first aspect of the present application can be implemented by the UAV target tracking system provided by the second aspect of the present application.
[0052] Please refer to Figure 1 , Figure 1 A flowchart of a UAV target tracking method according to some embodiments of the present application is shown.
[0053] As Figure 1 shown, in some embodiments of the present application, the UAV target tracking method mainly includes the following steps:
[0054] S110: Acquire sound signals around the UAV and extract sound features therefrom.
[0055] Specifically, in some embodiments, a plurality of loudspeakers can be placed at multiple positions in the sound field environment where the UAV is located, and a sound category of audio can be activated and randomly played through the loudspeakers as an unknown sound source in the sound field, wherein the spatial position of the unknown sound source can be represented as r s =(x s , y s , z s ). A multi-dimensional microphone array, such as a three-dimensional microphone array, can be distributed in the sound field space. Through the microphone array of such a topology, the embodiment can satisfy the reception of sound signals in different directions in the space, wherein the spatial position of the microphone can be represented as r p =(x p , y p , z p ), p∈{1, 2,..., P}, to acquire sound signals from multiple dimensions and different directions around the UAV.
[0056] Subsequently, the obtained sound signal is analyzed to determine the sound field environment in which the UAV is located. Specifically, it can be determined whether the current sound field environment in which the UAV is located is a preset target sound field environment. Optionally, in some embodiments, the target sound field environment can include a simulated city sound field environment, which can involve at least one candidate sound category of air conditioner running sound, car horn sound, children playing sound, dog barking sound, drilling sound, engine idling sound, siren sound, helicopter sound, and street music sound. Further, these candidate sound categories can be respectively assigned corresponding numbers (e.g., c1-c8), and played by multiple loudspeakers at different positions in the simulated sound field space. When it is determined that the sound field environment in which the UAV is located is the preset target sound field environment, the sound features of the corresponding target sound field environment are extracted from the obtained sound signal.
[0057] In some embodiments, all kinds of groups of sound signals in the target sound field environment can be obtained through the microphone array to construct an audio data set of the target sound field environment. All sound signals in the audio data set are subjected to frame processing and time-frequency domain conversion.
[0058] Specifically, the sound signal is first subjected to frame processing, and the overlap rate can be set to, for example, 50%. Then, the sound signal subjected to frame processing is subjected to endpoint detection to extract valid audio data. The audio data subjected to endpoint detection is subjected to time-frequency domain conversion, which can include filtering and standardization processing of the audio data. Then, the audio data subjected to filtering and standardization processing is analyzed to extract the Mel spectrogram features of the audio data. According to the obtained Mel spectrogram representation, the sound features are determined. In this embodiment, the Mel spectrogram representation is used as the features of the sound signal, which has a suitable size and is more consistent with the auditory characteristics of the human ear compared with sound features such as spectrograms.
[0059] S120: determining whether the specified tracking target exists around the UAV according to the sound features via the pre-trained sound recognition model.
[0060] Specifically, in the process of pre-training the sound model, a convolutional neural network can be selected to construct the sound recognition model to be trained. Further, a multi-scale attention convolutional neural network model is preferably trained to obtain the sound recognition model, wherein the multi-scale feature fusion structure can enrich the extracted feature information, and the presence of the attention module can improve the adaptability of the neural network model to different scale space information and channel information.
[0061] The at least one sound field environment is defined, and sound signals of multiple sound sources in the sound field environment are acquired. Optionally, audio of a candidate sound class can be randomly played at the same volume via multiple loudspeakers at different positions in the sound field environment, and sound signals from different directions around the UAV are acquired via a multi-dimensional microphone array distributed in the sound field space.
[0062] Subsequently, feature extraction is performed on the acquired sound signals to determine corresponding sound features. Specifically, in some embodiments, the acquired sound signals can be preprocessed by framing and / or windowing. Optionally, during the framing process, the frame shift is selected to be half of the frame length. The sound signals can be windowed using a Hamming window, the formula of which is as follows:
[0063]
[0064] Subsequently, feature extraction is performed, the preprocessed sound signals are converted from the time domain to the frequency domain, and the spectrum of each frame of sound signal is obtained using fast Fourier transform; the spectrum of the sound signals is mapped to a Mel spectrum using a Mel filter bank as the sound feature.
[0065] The sound features of the sound sources in the sound field environment are identified by the sound recognition model to be trained to determine corresponding sound class output values. Finally, the learning parameters of the convolutional neural network can be corrected according to the sound class output values and their corresponding true class labels to train the sound recognition model and obtain a trained sound recognition model.
[0066] Please refer to Figure 2 , Figure 2 A flowchart for determining whether a specified tracking target exists around a UAV according to some embodiments of the present application is shown, and the step S120 can further include the following steps S121-S122.
[0067] S121: According to the target sound field environment, a corresponding target sound recognition model is called to analyze the sound features to determine at least one sound class contained in the acquired sound signals.
[0068] Optionally, in some embodiments of the present application, the target sound recognition model can be learned from a sound signal training set of the target sound field environment, and the trained sound recognition model can finally recognize the eight candidate sound classes (for example, air conditioner running sound, car horn sound, children playing sound, dog barking sound, drilling sound, engine idling sound, siren sound, helicopter sound, street music sound) set above.
[0069] S122: Compare the specified sound category corresponding to the tracked target with at least one sound category contained in the sound signal to determine whether there is a tracked target around the drone.
[0070] In some preferred embodiments, user-specified information about the tracking target can be obtained via a human-computer interaction interface or other means. Based on this specified information, a specified sound category corresponding to the tracking target can be determined from at least one candidate sound category involved in the target sound field environment. Then, the specified sound category corresponding to the tracking target is compared with at least one sound category contained in the sound signal obtained in the target sound field environment to determine whether the tracking target exists around the drone.
[0071] For example, the features of audio data in the sound field environment can be input into a trained sound recognition model, and the recognition result can be output (which can correspond to the candidate sound category numbers c1 / c2 / ... / c8 mentioned above). The model then compares the data with the specified information of the target being tracked. If the information matches the recognition result, the current audio data belongs to the target that needs to be tracked; otherwise, the current audio data is not the target that needs to be tracked.
[0072] S130: In response to the determination that a tracking target exists, the sound signal is localized to determine the location information of the tracking target.
[0073] In some alternative embodiments, sound source localization can be achieved using a time delay estimation localization algorithm based on spherical interpolation. Specifically, in an N+1 element multidimensional microphone array, with vector m... i Microphone M i Spatial position, represented by vector m, with the location of microphone M0 selected as the reference position, and microphone M... i The distance between the microphone M0 and the microphone M0 is defined as:
[0074] d i =||m i -m||-||m||;
[0075] Microphone M i The distance to microphone M0 is defined as R. i The distance from the sound source S to the microphone M0 is defined as R. S, To obtain:
[0076]
[0077] Distance d i The estimated value The error function affected by environmental noise is expressed as:
[0078]
[0079] to obtain a matrix form of N error functions in an N+1 dimensional microphone array:
[0080] ε = δ - 2R s d - 2Cs
[0081] wherein δ, d, M represent respectively:
[0082]
[0083] obtained when the N error functions reach a minimum value:
[0084]
[0085] and
[0086] solving the above sound source position equation, so that the estimated position of the tracking target can be determined as:
[0087]
[0088] In the embodiment, the spherical interpolation-based time delay estimation positioning algorithm has the advantages of small amount of calculation and better real-time performance.
[0089] S140: Determine the flight instruction of the unmanned aerial vehicle according to the position information, so as to control the unmanned aerial vehicle to track the target.
[0090] In some embodiments, the unmanned aerial vehicle can be taken as a controlled object, the position information, i.e., coordinate information, of the tracking target is sent to the unmanned aerial vehicle, and the unmanned aerial vehicle is controlled to fly to the estimated position of the tracking target via a proportion integration differentiation (PID) controller. The information transmission in the process conforms to a Micro Air Vehicle Link (MAVLink) protocol.
[0091] Although the methods are illustrated and described as a series of acts, it will be appreciated that the methods are not limited by the order of acts as some acts can, in accordance with one or more embodiments, occur simultaneously or in different orders with other acts from that set of acts described herein or with acts described herein or with acts not described herein but understood by one of ordinary skill in the art.
[0092] So far, the unmanned aerial vehicle target tracking method provided by the first aspect of the application has been introduced, and the second aspect of the application further provides an unmanned aerial vehicle target tracking system. Please refer to Figure 3 , Figure 3A structural block diagram of a UAV target tracking system according to some embodiments of the present application is shown.
[0093] As shown in Figure 3 The UAV target tracking system 300 can be configured with a memory 310 and a processor 320. The memory 310 includes, but is not limited to, the computer readable storage medium provided by the third aspect of the present application, on which computer instructions are stored. The processor 320 is connected to the memory 310 and is configured to execute the computer instructions stored on the memory 310 to implement the UAV target tracking method provided by the first aspect of the present application.
[0094] In addition, those skilled in the art can understand that the embodiments of the UAV target tracking method described in the first aspect are only some non-limiting embodiments provided by the present application, which are intended to clearly show the main concept of the present application and provide some specific schemes for facilitating the public to implement, rather than to limit the overall working mode or overall function of the UAV target tracking system 300. Similarly, the UAV target tracking system 300 is also only a non-limiting embodiment provided by the present application, which does not limit the implementation subject of each step of the UAV target tracking method.
[0095] In summary, the present application provides a UAV target tracking method, a UAV target tracking system, and a computer readable storage medium, which can identify multiple sound types in a sound field environment with a relatively complex sound category, and realize target tracking of a UAV based on sound signals with relatively low storage and processing difficulty.
[0096] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the generic principles defined herein can be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the examples described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for tracking a target by a UAV, the method comprising: The method comprises the following steps: acquiring sound signals from different directions around the UAV via a multi-dimensional microphone array distributed in a sound field space, and extracting sound features from the sound signals; judging whether a specified tracking target exists around the UAV according to the sound features via a pre-trained sound recognition model; in response to a judgment result that the tracking target exists, performing sound source positioning on the sound signals to determine position information of the tracking target, wherein the sound source positioning is implemented based on a spherical interpolation time delay estimation positioning algorithm, and a sound source position equation involved in the sound source positioning process is: wherein, represents a sound source to a microphone as a reference position distance, , denotes a microphone to the microphone , a distance denotes a microphone an estimate of the distance between the microphone , i ranges from 1 to N, where N is the number of microphones in the multi-dimensional microphone array excluding the microphone at the reference position, ; and determining flight instructions of the UAV according to the position information to control the UAV to track the tracking target. 2.The UAV target tracking method of claim 1, wherein, The step of acquiring sound signals around the UAV and extracting sound features from the sound signals comprises: performing frame processing and time-frequency domain conversion on the acquired sound signals to obtain corresponding Mel spectrum representations; and determining the sound features according to the Mel spectrum representations. 3.The UAV target tracking method of claim 2, wherein, The step of performing frame processing and time-frequency domain conversion on the acquired sound signals to obtain corresponding Mel spectrum representations comprises: performing endpoint detection on the sound signals subjected to the frame processing to extract valid audio data; and performing time-frequency domain conversion on the audio data to obtain the Mel spectrum representations. 4.The UAV target tracking method of claim 1, wherein, The step of acquiring sound signals around the UAV and extracting sound features from the sound signals comprises: analyzing the acquired sound signals to determine a sound field environment in which the UAV is located; and in response to the sound field environment being a preset target sound field environment, extracting sound features corresponding to the target sound field environment from the sound signals. 5.The UAV target tracking method of claim 4, wherein, The step of judging whether a specified tracking target exists around the UAV according to the sound features via a pre-trained sound recognition model comprises: according to the target sound field environment, calling a corresponding target sound recognition model to analyze the sound features to determine at least one sound category contained in the sound signals, wherein the target sound recognition model is learned via a sound signal training set of the target sound field environment; and comparing a specified sound category corresponding to the tracking target with the at least one sound category contained in the sound signals to judge whether the tracking target exists around the UAV. 6.The UAV target tracking method of claim 5, wherein, Before comparing the specified sound category corresponding to the tracking target with the at least one sound category contained in the sound signals to judge whether the tracking target exists around the UAV, the UAV target tracking method further comprises the following steps: acquiring specified information of the tracking target of a user via a human-computer interaction interface; and determining the specified sound category corresponding to the tracking target from at least one candidate sound category involved in the target sound field environment according to the specified information. 7.The UAV target tracking method of claim 6, wherein, The target sound field environment comprises a city sound field environment, which involves at least one candidate sound category of air conditioner operation sound, car horn sound, children playing sound, dog barking sound, drilling sound, engine idling sound, siren sound, helicopter sound and street music sound. 8.The UAV target tracking method of claim 1, wherein, The step of sound source positioning on the sound signal to determine the position information of the tracking target comprises: exist In a multidimensional microphone array of elements, using vector microphone Spatial location, in vector form Indicates the sound source Spatial location, select microphone The location is used as a reference point, and the microphone is... With microphone The distance between them is defined as: Based on the microphone Distance to the microphone Distance to the microphone Distance to the microphone Distance to the microphone Distance to the microphone to obtain: The estimate of the distance of the estimate of the distance The error function affected by the environmental noise is represented as: to obtain a matrix form of the error functions in the multi-dimensional microphone array of the element wherein ; In the The sound source location equation is obtained when the error function reaches its minimum value; and solving the sound source position equation to determine the estimated position of the tracking target as: 9.The UAV target tracking method of claim 1, wherein, The step of determining the flight instruction of the UAV according to the position information to control the UAV to track the tracking target comprises: taking the UAV as a controlled object, and controlling the UAV to fly to the estimated position of the tracking target via a PID controller. 10.The UAV target tracking method of claim 1, wherein, The step of training the sound recognition model comprises: constructing a sound recognition model to be trained by using a convolutional neural network; defining at least one sound field environment, and obtaining sound signals of multiple sound sources in the sound field environment; extracting features of the obtained sound signals to determine corresponding sound features; performing sound recognition on the sound features of each sound source in the sound field environment via the sound recognition model to be trained to determine corresponding sound category output values; and correcting learning parameters of the convolutional neural network according to each sound category output value and a corresponding true category label to train the sound recognition model. 11.The UAV target tracking method of claim 10, wherein, The step of obtaining sound signals of multiple sound sources in the sound field environment comprises: randomly playing an audio of a candidate sound category via multiple loudspeakers at different positions at the same volume; and obtaining sound signals from different directions around the UAV via a multi-dimensional microphone array distributed in the sound field space. 12.The UAV target tracking method of claim 10, wherein, The step of extracting features of the obtained sound signals to determine corresponding sound features comprises: performing pre-processing such as framing and / or windowing on the obtained sound signals; converting the pre-processed sound signals from the time domain to the frequency domain, and obtaining a spectrum diagram of each frame of the sound signals by using fast Fourier transform; and mapping the spectrum diagram of the sound signals to a Mel spectrum diagram by using a Mel filter bank to serve as the sound features.
13. An unmanned aerial vehicle target tracking system, comprising: comprise: a memory having computer instructions stored thereon; and a processor connected to the memory and configured to execute the computer instructions stored on the memory to implement the UAV target tracking method according to any one of claims 1-12.
14. A computer readable medium having stored thereon computer instructions, wherein, The computer instructions are executed by the processor to implement the UAV target tracking method according to any one of claims 1-12.
Citation Information
Patent Citations
Unmanned plane target tracking method and device, unmanned plane and storage medium
CN107748860A
Multi-unmanned aerial vehicle multi-target cooperative tracking control method based on MAPPO algorithm
CN115509251A