Unmanned aerial vehicle target tracking system and method

Abstract

The invention provides an unmanned aerial vehicle target tracking system and method, and relates to the technical field of unmanned aerial vehicle target tracking, and the method comprises the steps: carrying out the processing of a real-time image captured by a visual imaging device, and obtaining the first position information of a target, when the visual confidence degree of the first position information is not smaller than a first threshold value, the first position information serves as target position information to be output, and when the visual confidence degree is smaller than the first threshold value, second position information of a target is obtained through processing according to sound signals, collected by a sound collection device, around the unmanned aerial vehicle; performing weighted fusion processing on the first position information and the second position information to obtain target position information; and controlling the unmanned aerial vehicle to track the target flight according to the target position information. The method has the beneficial effects that through dynamic complementation of visual tracking and sound source tracking, tracking loss caused by failure of a single mode is avoided, multiple scenes such as illumination abrupt change, shielding and noise can be effectively covered, and the tracking precision and robustness of the unmanned aerial vehicle in a complex scene are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) target tracking, and in particular to a UAV target tracking system and method. Background Art

[0002] With the rapid development of drone technology, its applications in military reconnaissance, security monitoring, emergency rescue and other fields are becoming increasingly widespread. As the core technology of drone intelligence, target tracking directly affects the efficiency of mission execution.

[0003] Traditional drone target tracking systems primarily rely on a single sensor, using image processing algorithms to achieve target positioning. Visible light cameras, as the most commonly used visual sensor, have imaging quality that is subject to complex optical physics. At night or in smoky environments, the quality of visible light imaging drops dramatically, leading to target loss. Specifically, at night, the signal intensity in the visible light band typically decays to less than 0.1% of that during the day, causing the signal-to-noise ratio (SNR) to drop sharply below 5dB. When encountering smoke interference, the Mie scattering effect can cause image contrast to drop by more than 70%, severely damaging the target's edge gradient features. Test data from the US DARPA shows that in sandstorm environments, the false detection rate of traditional visual tracking algorithms can be up to 15 times that of normal environments.

[0004] Furthermore, in dynamic occlusion scenarios, a single visual cue struggles to maintain stable tracking. When the target's occlusion exceeds 30%, the response confidence of tracking algorithms based on appearance models (such as KCF and MOSSE) plummets by over 60%. MIT research shows that in crowded scenes, monocular vision systems lose tracking once every five seconds on average, requiring manual intervention to recapture the target.

[0005] In summary, factors such as illumination changes, target occlusion, and background interference in complex environments seriously restrict the tracking robustness of traditional UAV target tracking systems. This not only restricts the combat effectiveness of the UAV system, but also affects the success rate of key tasks such as emergency rescue. Therefore, how to improve the tracking accuracy and robustness of UAVs in complex scenarios has become a fundamental problem that needs to be solved urgently. Summary of the Invention

[0006] In view of the problems existing in the prior art, the present invention provides a UAV target tracking system, comprising:

[0007] Multimodal sensors, including visual imaging devices and sound collection devices carried on UAVs;

[0008] A processor, integrated within the drone and connected to the multimodal sensor, includes:

[0009] The first processing module is configured to process the real-time image captured by the visual imaging device to obtain the first position information of the target, output the first position information as the target position information when the visual confidence of the first position information is not less than the first threshold, and generate an auxiliary signal when the visual confidence is less than the first threshold;

[0010] The second processing module is configured to obtain the sound signal around the UAV collected by the sound acquisition device according to the auxiliary signal, and process the sound signal to obtain the second position information of the target;

[0011] The position fusion module is respectively connected to the first processing module and the second processing module, and is configured to perform weighted fusion processing on the first position information and the second position information according to the auxiliary signal to obtain the target position information;

[0012] The flight control module is respectively connected to the first processing module and the position fusion module, and is configured to control the UAV to track the target flight according to the target position information.

[0013] Preferably, the visual acquisition device is an infrared thermal imaging dual-spectrum camera, which is configured to synchronously acquire visible light images and infrared images, and perform image registration on the visible light images and the infrared images to obtain dual-spectrum data as the real-time image.

[0014] Preferably, the sound acquisition device includes a plurality of directional microphones arranged circumferentially along the periphery of the UAV, which are configured to collect the sound signals of 360°.

[0015] Preferably, the first processing module includes:

[0016] A position acquisition unit, configured to acquire the position data of the region of interest where the target is located in the real-time image of the current frame as the first position information, and perform feature extraction on the region of interest to obtain a target embedding vector;

[0017] A position update unit, connected to the position acquisition unit, is configured to predict the region of interest where the target is located in the real-time image of the next frame according to the target embedding vector and using the particle filter algorithm, and then update the first position information according to the position data of the region of interest.

[0018] Preferably, the first processing module further includes:

[0019] A feature matching scoring unit, configured to select the maximum value of the particle weights corresponding to the real-time image of the current frame as the feature matching score of the real-time image of the current frame;

[0020] A motion continuity scoring unit for calculating the standard deviation of the displacement of the center point of the target in the current frame and a continuous multiple frames of the real-time image before the current frame as the motion continuity score of the real-time image of the current frame;

[0021] A size stability scoring unit for evaluating the change amount of the aspect ratio of the target in the real-time image of the current frame compared to the real-time image of the first frame as the size stability score;

[0022] A confidence calculation unit, respectively connected to the feature matching scoring unit, the motion continuity scoring unit, and the size stability scoring unit, for performing weighted summation on the feature matching score, the motion continuity score, and the size stability score to obtain the visual confidence of the real-time image of the current frame.

[0023] Preferably, the first processing module further includes:

[0024] A signal generation unit, connected to the confidence calculation unit, for generating the auxiliary signal when the visual confidence corresponding to the real-time images of a continuous preset number of frames is less than the first threshold.

[0025] Preferably, the second processing module includes:

[0026] A voiceprint matching unit for extracting features from the voice signal to obtain real-time voiceprint features, calculating the cosine similarity between the real-time voiceprint features and the actual voiceprint features of the target pre-sampled and stored, and outputting a determination signal indicating the existence of the target when the cosine similarity is greater than a preset similarity threshold;

[0027] A sound localization unit, connected to the voiceprint matching unit, for roughly localizing the voice signal according to the determination signal to generate a candidate sound source area, and then performing fine localization on the candidate sound source area to obtain the second position information of the target.

[0028] Preferably, the sound localization unit uses the generalized cross-correlation phase transform to calculate the time delay difference between microphone pairs for rough localization, and uses a regression network based on ResNet-18 to perform fine localization on the candidate sound source area.

[0029] Preferably, when the first position information and the second position information are weighted and fused, the first position information has a first preset weight, the second position information has a second preset weight, and the sum of the first preset weight and the second preset weight is 1;

[0030] And when the visual confidence corresponding to the real-time images in a continuous preset number of frames is less than the third threshold, which is less than the first threshold, configure the first preset weight to 0.

[0031] And when the Euclidean distance between the second position information and the first position information is greater than the maximum deviation distance, configure the first preset weight to 1.

[0032] The present invention also provides a method for tracking a target by a drone, which is applied to the above-mentioned drone target tracking system. The method for tracking a target by a drone includes:

[0033] Step S1, the drone target tracking system processes the real-time images captured by the visual imaging device carried on the drone to obtain the first position information of the target, and determines whether the visual confidence of the first position information is less than the first threshold:

[0034] If not, output the first position information as the target position information, and then turn to step S4;

[0035] If so, turn to step S2;

[0036] Step S2, the drone target tracking system acquires the sound signal around the drone collected by the sound acquisition device carried on the drone, and processes the sound signal to obtain the second position information of the target;

[0037] Step S3, the drone target tracking system performs weighted fusion processing on the first position information and the second position information to obtain the target position information;

[0038] Step S4, the drone target tracking system controls the drone to track the target flight according to the target position information.

[0039] The above technical solution has the following advantages or beneficial effects: By carrying multi-modal sensors on the drone, dynamic complementarity between visual tracking and sound source tracking is achieved. When the visual confidence is insufficient, the sound acquisition device is automatically activated for auxiliary positioning, avoiding tracking loss caused by the failure of a single modality, and can effectively cover multiple scenarios such as sudden changes in illumination, occlusion, and noise, significantly improving the tracking accuracy and robustness of the drone in complex scenarios, and having outstanding technological progressiveness and industrial application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic structural diagram of a drone target tracking system in a preferred embodiment of the present invention;

[0041] Figure 2In a preferred embodiment of the present invention, it is a schematic flowchart of a method for a drone to track a target. Detailed implementation manners

[0042] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this implementation manner, and as long as it conforms to the gist of the present invention, other implementation manners may also fall within the scope of the present invention.

[0043] In a preferred embodiment of the present invention, in view of the above problems existing in the prior art, a drone target tracking system is now provided, as Figure 1 shown, including:

[0044] A multimodal sensor 1, including a visual imaging device 11 and a sound collection device 12 mounted on the drone;

[0045] A processor 2, integrated inside the drone and connected to the multimodal sensor 1, including:

[0046] A first processing module 21, configured to process the real-time image captured by the visual imaging device 11 to obtain the first position information of the target, and output the first position information as the target position information when the visual confidence of the first position information is not less than the first threshold, and generate an auxiliary signal when the visual confidence is less than the first threshold;

[0047] A second processing module 22, configured to obtain the sound signal around the drone collected by the sound collection device 12 according to the auxiliary signal, and process the sound signal to obtain the second position information of the target;

[0048] A position fusion module 23, respectively connected to the first processing module 21 and the second processing module 22, configured to perform weighted fusion processing on the first position information and the second position information according to the auxiliary signal to obtain the target position information;

[0049] A flight control module 24, respectively connected to the first processing module 21 and the position fusion module 23, configured to control the drone to track the target flight according to the target position information.

[0050] Specifically, in this embodiment, the visual acquisition device 11 is an infrared thermal imaging dual-spectrum camera, which is used to synchronously acquire visible light images and infrared images, and perform image registration on the visible light images and infrared images to obtain dual-spectrum data as real-time images. Preferably, the infrared channel of the visual acquisition device 11 is preferentially enabled to capture the infrared image of the target under low light conditions. The target here can be a human body or a vehicle, which is not limited herein.

[0051] Further, to capture the first position information of the target and perform target tracking, it is first necessary to locate the region of interest in the real-time image. The region of interest locates the position of the target in the real-time image, and based on its position in the real-time image, coordinate transformation is performed to obtain the true physical position of the target, that is, the first position information. The coordinate transformation here is a prior art and is not the inventive point of the present invention, so it will not be elaborated here.

[0052] Based on this, in a preferred embodiment of the present invention, the first processing module 21 includes:

[0053] A position acquisition unit 211, configured to acquire the position data of the region of interest where the target is located in the real-time image of the current frame as the first position information, and perform feature extraction on the region of interest to obtain a target embedding vector;

[0054] A position update unit 212, connected to the position acquisition unit 211, configured to predict the region of interest where the target is located in the real-time image of the next frame according to the target embedding vector and using the particle filter algorithm, and then update the first position information according to the position data of the region of interest.

[0055] Specifically, in this embodiment, the region of interest in the real-time image of the next frame is predicted based on the region of interest in the real-time image of the current frame through the recursive principle. The basis of the recursive principle is that the region of interest initially acquired in the real-time image needs to be marked. The specific process is as follows:

[0056] 1. When the infrared thermal imaging dual-spectrum camera starts to synchronously collect visible light images and infrared images, it includes:

[0057] 1) Perform three-frame difference on consecutive frames of infrared images to generate a preliminary motion region mask. The specific calculation formula is as follows:

[0058] M diff (x,y) = |I t (x,y) - I t-1 (x,y)| ∩ |I t (x,y) - I t-2 (x,y)|

[0059] Among them, M diff (x,y) is the preliminary motion region mask, |I t (x,y) - I t-1 (x,y)| represents the absolute difference image between the t-th frame infrared image and the (t - 1)-th frame infrared image, and |I t (x,y) - I t-2 (x,y)| represents the absolute difference image between the t-th frame infrared image and the (t - 2)-th frame infrared image.

[0060] 2) Dilate the initially obtained motion area mask first and then erode it, followed by filling holes and smoothing the edges to generate a motion energy map;

[0061] 3) Extract connected components from the motion energy, filter out noise regions with too small an area or abnormal aspect ratio to obtain candidate regions;

[0062] 4) Combine the texture features (such as HOG descriptors) of the synchronously acquired visible light images to perform secondary verification on the candidate regions and exclude interference from static heat sources (such as heating pipes).

[0063] 5) Finally, select the region with the strongest energy and the highest texture matching degree as the initial target ROI, that is, the region of interest serving as the basis for recursion.

[0064] By introducing an infrared thermal imaging dual-spectrum camera, combining visible light texture and infrared heat source features, and jointly screening the initial target through a motion energy map and HOG descriptors, the detection robustness under complex lighting conditions is effectively improved.

[0065] After obtaining the above-mentioned region of interest, preferably use the particle filter algorithm to predict the region of interest where the target is located in the real-time image of the next frame, and then update the first position information according to the position data of the region of interest, that is, realize the update of the tracking position information of the target.

[0066] In this embodiment, visual tracking based on real-time images is the mainstay. However, due to the ever-changing tracking scenarios, visual occlusion environments, nighttime, or smoky environments may occur, resulting in a lower confidence level of visual tracking. At this time, it is necessary to enable sound source tracking as an auxiliary to avoid target loss. Therefore, during the process of visual tracking, it is necessary to monitor the visual confidence in real time. Based on this, in a preferred embodiment of the present invention, the first processing module 21 further includes:

[0067] A feature matching scoring unit 213, configured to select the maximum value of the particle weights corresponding to the real-time image of the current frame as the feature matching score of the real-time image of the current frame;

[0068] A motion continuity scoring unit 214, configured to calculate the displacement standard deviation of the center point of the target in the current frame and multiple consecutive previous frames of real-time images as the motion continuity score of the real-time image of the current frame;

[0069] A size stability scoring unit 215, configured to evaluate the change amount of the aspect ratio of the target in the real-time image of the current frame compared to the first frame of the real-time image as the size stability score;

[0070] The confidence calculation unit 216 is respectively connected to the feature matching score unit 213, the motion continuity score unit 214, and the size stability score unit 215, and is used to perform weighted summation on the feature matching score, the motion continuity score, and the size stability score to obtain the visual confidence of the real-time image of the current frame.

[0071] Specifically, in this embodiment, the calculation formula of the above motion continuity score is as follows:

[0072]

[0073] Among them, C motion is the motion continuity score, Δx t is the displacement of the target in the real-time image of the current frame relative to the target in the real-time image of the previous frame. This displacement is preferably a composite vector in multiple directions, including but not limited to the x, y, and z directions. For example, if the target moves 10 pixels horizontally and 5 pixels vertically between two adjacent frames, then

[0074] k is the preset length of the historical frame window. If the value is 5, the standard deviation of the past 5 frames is calculated uniquely.

[0075] std represents the standard deviation, and max_speed represents the preset maximum expected speed. If the tracking target is a pedestrian, assuming a frame rate of 30 fps and 1 pixel is approximately equal to 0.1 meter, then max_speed can be set to 10 - 30 pixels / frame, corresponding to the pedestrian's moving speed. The value of max_speed can be dynamically adjusted according to the type of the tracking target. For example, if the tracking target is a vehicle, max_speed can be set to 50 - 100 pixels / frame. If the tracking target is a flying object, then max_speed is set to a higher value, and so on. Details are not elaborated here.

[0076] The above motion continuity score is mainly used to statistically track the displacement fluctuation of the target within a short-time window and quantify the smoothness of the motion trajectory. If the target moves in a uniform straight line (the standard deviation approaches 0), then C motion ≈1; if the motion jitters violently (the standard deviation is close to max_speed), then C motion ≈0.

[0077] The calculation formula of the above size stability score is as follows:

[0078] C size = exp(-β * |AR t - AR init |

[0079] Among them, C size is the size stability score, AR tAR is the aspect ratio of the target in the real-time image of the current frame, preferably the ratio of the width to the height of the rectangular frame enclosing the target. If the rectangular frame is 100 pixels wide and 50 pixels high, then AR t has a value of 2.

[0080] AR init is the aspect ratio of the target in the real-time image of the initial frame and serves as a reference value. β is the attenuation coefficient, which is used to control the influence intensity of the aspect ratio difference on the confidence level. exp is the exponential function.

[0081] The above-mentioned size stability score is mainly used to quantify the stability of size changes. If the aspect ratio of the target is the same as the initial value, then C size has a value of 1. If the target rotates, deforms, or undergoes a sudden size change (such as the target being magnified due to the approach of a drone), the value of C size rapidly decreases.

[0082] The calculation formula for the above-mentioned visual confidence level is as follows:

[0083] C visual = λ1 * C feat + λ2 * C motion + λ3 * C size

[0084] where C visual is the visual confidence level, C feat is the feature matching score, λ1, λ2, and λ3 are weight coefficients, and λ1 > λ2 > λ3, indicating that the visual confidence level depends more on feature matching, followed by motion continuity, and lightly on size stability.

[0085] In a preferred embodiment of the present invention, the first processing module 21 further includes:

[0086] A signal generation unit 217, connected to the confidence calculation unit 216, is used to generate an auxiliary signal when the visual confidence levels corresponding to the real-time images of a continuous preset number of frames are all less than the first threshold.

[0087] Specifically, in this embodiment, when the visual confidence levels corresponding to the real-time images of a continuous preset number of frames are all less than the first threshold, it indicates that the confidence level of visual tracking remains low. At this time, the target may be occluded, or the environment may affect the acquisition of clear real-time images (such as sudden changes in light). If only relying on single visual tracking, it is very likely that the target will be lost. Based on this, the first processing module 21 generates an auxiliary signal to trigger sound source localization for assistance.

[0088] In a preferred embodiment of the present invention, the sound source localization depends on the sound collection device 12 carried on the drone. The sound collection device 12 includes a plurality of directional microphones arranged circumferentially around the drone for collecting 360° sound signals.

[0089] In a preferred embodiment of the present invention, the second processing module 22 includes:

[0090] A voiceprint matching unit 221, configured to extract features from the voice signal to obtain real-time voiceprint features, calculate the cosine similarity between the real-time voiceprint features and the actual voiceprint features of the target pre-sampled and stored, and output a determination signal indicating the presence of the target when the cosine similarity is greater than a preset similarity threshold;

[0091] A sound localization unit 222, connected to the voiceprint matching unit 221, configured to roughly localize the voice signal according to the determination signal to generate a candidate sound source area, and then perform fine localization on the candidate sound source area to obtain the second position information of the target.

[0092] Specifically, in this embodiment, after the sound acquisition device 12 acquires a voice signal, the second processing module 22 preferably performs a constant Q transform (CQT) on each frame of the voice signal to generate a high-resolution time-frequency map (frequency range: 80 Hz - 8 kHz), and then inputs the time-frequency map into the ECAPA-TDNN model to extract a 256-dimensional voiceprint vector, and calculates the cosine similarity S with the target voiceprint pre-sampled and stored in the database voice , if S voice is greater than the preset similarity threshold, it is determined that the target sound source exists. The above preset similarity threshold is preferably 75%.

[0093] After determining the existence of the target sound source, in a preferred embodiment of the present invention, the sound localization unit 222 uses the generalized cross-correlation phase transform to calculate the time delay difference between microphone pairs for rough localization, and uses a regression network based on ResNet-18 to perform fine localization on the candidate sound source area.

[0094] Specifically, in this embodiment, first, a candidate sound source area is obtained through rough localization, and then the three-dimensional coordinates of the target are output based on fine localization. After obtaining the three-dimensional coordinates, it is preferably further included to map the sound source coordinates to the visual coordinate system through a pre-calibrated external parameter matrix. The coordinate mapping process here is a prior art and is not an innovation point of the present invention, and the specific process will not be elaborated here.

[0095] In a preferred embodiment of the present invention, when the first position information and the second position information are weighted and fused, the first position information has a first preset weight, the second position information has a second preset weight, and the sum of the first preset weight and the second preset weight is 1;

[0096] And when the visual confidence degrees corresponding to real-time images of a continuous preset number of frames are all less than a third threshold, the first preset weight is configured to be 0, and the third threshold is less than the first threshold;

[0097] And when the Euclidean distance between the second position information and the first position information is greater than the maximum deviation distance, configure the first preset weight to 1.

[0098] Specifically, in this embodiment, when collecting real-time images and sound signals, it is preferred to add hardware and timestamps to ensure frame-level synchronization during position fusion, that is, to align visual and acoustic data in the feature space and ensure the consistency of the fusion result. When performing weighted fusion processing on the first position information and the second position information, it is preferred to dynamically calculate the above-mentioned first preset weight and second preset weight based on the following formula:

[0099] Among them,

[0100] In the above formula, α is the first preset weight, then the second preset weight is 1 - α, C visual is the visual confidence, S voice is the cosine similarity calculated during voiceprint matching, D max is the maximum deviation distance, and D is the Euclidean distance between the second position information and the first position information.

[0101] By dynamically calculating the above-mentioned first preset weight and second preset weight, the environmental adaptability of the drone during target tracking is effectively improved, so as to effectively cope with dynamic scene changes.

[0102] Furthermore, before calculating the above-mentioned first preset weight and second preset weight, it is preferred to first perform validity determination. The above-mentioned continuous preset number of frames is preferably 5 consecutive frames, that is, if the visual confidence corresponding to 5 consecutive frames of real-time images is less than the third threshold, then force α to be 0, completely rely on acoustic positioning, and start a target re-search for visual tracking. If the Euclidean distance between the second position information and the first position information is greater than the maximum deviation distance, then force α to be 1, disable acoustic data and record an abnormal event.

[0103] Even further, after performing weighted fusion processing on the first position information and the second position information, if it is detected that the visual confidence of the subsequent 5 consecutive frames of real-time images is not less than the first threshold, then the sound collection device can be turned off and switched to pure visual tracking.

[0104] The present invention also provides a drone target tracking method, which is applied to the above-mentioned drone target tracking system, as Figure 2 shown, the drone target tracking method includes:

[0105] Step S1, the drone target tracking system processes the real-time image captured by the visual imaging device carried on the drone to obtain the first position information of the target, and determines whether the visual confidence of the first position information is less than the first threshold:

[0106] If not, output the first position information as the target position information, and then go to step S4;

[0107] If so, go to step S2;

[0108] Step S2, the UAV target tracking system acquires the sound signal around the UAV collected by the sound collection device carried on the UAV, and processes the second position information of the target according to the sound signal;

[0109] Step S3, the UAV target tracking system performs weighted fusion processing on the first position information and the second position information to obtain the target position information;

[0110] Step S4, the UAV target tracking system controls the UAV to track the target flight according to the target position information.

[0111] The above are only the preferred embodiments of the present invention, and do not limit the implementation manners and protection scope of the present invention. For those skilled in the art, it should be realized that all the equivalent replacements and obvious changes made by using the content of this specification and the drawings should be included in the protection scope of the present invention.

Claims

1. A drone target tracking system, characterized in that, Including: A multi-modal sensor, including a visual imaging device and a sound acquisition device mounted on a drone; A processor, integrated inside the drone and connected to the multi-modal sensor, including: A first processing module, configured to process the real-time image captured by the visual imaging device to obtain the first position information of the target, and output the first position information as the target position information when the visual confidence of the first position information is not less than a first threshold, and generate an auxiliary signal when the visual confidence is less than the first threshold; A second processing module, configured to obtain the sound signal around the drone collected by the sound acquisition device according to the auxiliary signal, and process the sound signal to obtain the second position information of the target; A position fusion module, respectively connected to the first processing module and the second processing module, configured to perform weighted fusion processing on the first position information and the second position information according to the auxiliary signal to obtain the target position information; A flight control module, respectively connected to the first processing module and the position fusion module, configured to control the drone to track the target flight according to the target position information.

2. The drone target tracking system according to claim 1, characterized in that, The visual acquisition device is an infrared thermal imaging dual-spectrum camera, configured to synchronously acquire visible light images and infrared images, and perform image registration on the visible light images and the infrared images to obtain dual-spectrum data as the real-time image.

3. The drone target tracking system according to claim 1, characterized in that, The sound acquisition device includes a plurality of directional microphones arranged circumferentially along the periphery of the drone, configured to collect the sound signals of 360°.

4. The drone target tracking system according to claim 1, characterized in that, The first processing module includes: A position acquisition unit, configured to acquire the position data of the region of interest where the target is located in the current frame of the real-time image as the first position information; A position update unit, connected to the position acquisition unit, configured to predict the region of interest where the target is located in the next frame of the real-time image based on the region of interest where the target is located in the current frame of the real-time image by using a particle filter algorithm, and then update the first position information according to the position data of the region of interest.

5. The drone target tracking system according to claim 4, wherein, The first processing module further includes: A feature matching scoring unit, configured to select the maximum value of the particle weights corresponding to the current frame of the real-time image as the feature matching score of the current frame of the real-time image; A motion continuity scoring unit, configured to calculate the displacement standard deviation of the center point of the target in the current frame and a plurality of consecutive frames of the real-time image before the current frame as the motion continuity score of the current frame of the real-time image; A size stability scoring unit, configured to evaluate the change amount of the aspect ratio of the target in the current frame of the real-time image compared with the first frame of the real-time image as the size stability score; A confidence calculation unit, respectively connected to the feature matching scoring unit, the motion continuity scoring unit and the size stability scoring unit, configured to perform weighted summation on the feature matching score, the motion continuity score and the size stability score to obtain the visual confidence of the current frame of the real-time image.

6. The drone target tracking system according to claim 5, characterized in that, The first processing module further includes: A signal generation unit, connected to the confidence calculation unit, is configured to generate the auxiliary signal when the visual confidence levels corresponding to the real-time images in a continuous preset number of frames are all less than the first threshold.

7. The drone target tracking system according to claim 1, wherein The second processing module includes: A voiceprint matching unit, configured to extract features from the voice signal to obtain real-time voiceprint features, calculate the cosine similarity between the real-time voiceprint features and the actual voiceprint features of the target pre-sampled and stored, and output a determination signal indicating the presence of the target when the cosine similarity is greater than a preset similarity threshold; A sound localization unit, connected to the voiceprint matching unit, is configured to roughly localize the voice signal according to the determination signal to generate a candidate sound source area, and then precisely localize the candidate sound source area to obtain the second position information of the target.

8. The drone target tracking system according to claim 7, wherein, The sound localization unit uses the generalized cross-correlation phase transform to calculate the time delay difference between microphone pairs for rough localization, and uses a regression network based on ResNet-18 to precisely localize the candidate sound source area.

9. The drone target tracking system according to claim 7, wherein, When the first position information and the second position information are subjected to weighted fusion processing, the first position information has a first preset weight, the second position information has a second preset weight, and the sum of the first preset weight and the second preset weight is 1; And when the visual confidence levels corresponding to the real-time images in a continuous preset number of frames are all less than the third threshold, the first preset weight is configured to be 0, and the third threshold is less than the first threshold; And when the Euclidean distance between the second position information and the first position information is greater than the maximum deviation distance, the first preset weight is configured to be 1.

10. A method for unmanned aerial vehicle target tracking, characterized in that, Applied to the unmanned aerial vehicle target tracking system according to any one of claims 1-9, the unmanned aerial vehicle target tracking method includes: Step S1, the unmanned aerial vehicle target tracking system processes the real-time image captured by the visual imaging device carried on the unmanned aerial vehicle to obtain the first position information of the target, and determines whether the visual confidence level of the first position information is less than the first threshold: If not, the first position information is output as the target position information, and then it turns to step S4; If so, it turns to step S2; Step S2, the unmanned aerial vehicle target tracking system acquires the voice signal around the unmanned aerial vehicle collected by the voice acquisition device carried on the unmanned aerial vehicle, and processes the voice signal to obtain the second position information of the target; Step S3, the unmanned aerial vehicle target tracking system performs weighted fusion processing on the first position information and the second position information to obtain the target position information; Step S4, the unmanned aerial vehicle target tracking system controls the unmanned aerial vehicle to track the target flight according to the target position information.

Images