A multi-robot trajectory real-time tracking method based on multi-sensor fusion

Abstract

The application discloses a kind of multi-robot track real-time tracking method based on multi-sensing fusion, it is related to trajectory tracking technical field, for solving the problem of the task execution efficiency reduction of unmanned aerial vehicle group cooperative flight, by obtaining main unmanned aerial vehicle and secondary unmanned aerial vehicle group number, based on main unmanned aerial vehicle video analysis unmanned aerial vehicle number order and judge whether into depth tracking mechanism, in tracking period, the number of visible unmanned aerial vehicle is collected to calculate unmanned aerial vehicle group change rate, combined with the three-axis angular velocity obtained by main unmanned aerial vehicle inertial measurement unit generates attitude stability, comprehensive sampling deviation coefficient is obtained;According to sampling deviation coefficient, the number sequence of secondary unmanned aerial vehicle group is collected, and based on the number of occurrences, the alternative main unmanned aerial vehicle is screened, combined with the historical correction record and current position of alternative main unmanned aerial vehicle, the replacement benefit score is calculated, the unmanned aerial vehicle group is rechecked and handled, to ensure the task execution efficiency of unmanned aerial vehicle group cooperative flight.

Description

Technical Field

[0001] This invention relates to the field of trajectory tracking technology, and more specifically, to a method for real-time trajectory tracking of multiple unmanned aerial vehicles (UAVs) based on multi-sensor fusion. Background Technology

[0002] With the gradual opening of low-altitude airspace, multi-drone collaborative operations have been widely applied in scenarios such as urban low-altitude logistics, emergency search and rescue, power line inspection, and urban security patrols. In these applications, multiple drones typically need to coordinate to perform tasks in formation flight, convoy following, or area coverage, and at least one drone needs to transmit real-time visual information back to the ground control platform for global situational awareness and task scheduling.

[0003] The existing technology has the following shortcomings:

[0004] Currently, most existing technologies employ fixed master UAVs or multi-UAV trajectory tracking and cooperative control methods based on static rules. However, these methods do not fully consider the impact of differences in the sampling timing of heterogeneous sensors and changes in the attitude stability of the master UAV in complex environments. They also lack dynamic evaluation and adaptive adjustment mechanisms for changes in the UAV swarm numbering order, fluctuations in the number of visible UAVs, and the effectiveness of the master UAV. As a result, the trajectory fusion results of multi-UAVs are prone to deviations during long-term flights or in complex urban environments, affecting the cooperative control and safety of the UAV swarm and reducing the mission execution efficiency of the UAV swarm cooperative flight. Therefore, this paper proposes a real-time trajectory tracking method for multi-UAVs based on multi-sensor fusion.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a real-time tracking method for multi-UAV trajectories based on multi-sensor fusion. This method utilizes a joint evaluation mechanism based on the visible change rate of the UAV swarm and the attitude stability of the master UAV, and combines a multi-sensor collaborative tracking strategy with dynamic switching of the master UAV and historical calibration constraints to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for real-time tracking of multi-UAV trajectories based on multi-sensor fusion, comprising the following steps:

[0008] Step S1: Obtain the number of the main drone in the current drone swarm and the number of each drone in the secondary drone swarm. Analyze the numbering order of the current drone swarm based on the real-time video stream transmitted back by the main drone and determine whether to enter the deep tracking mechanism.

[0009] Step S2: Set the tracking period, collect the number of visible UAVs during the tracking period, calculate the UAV swarm change rate, obtain the three-axis angular velocity of the main UAV through the inertial measurement unit of the main UAV to generate attitude stability, and obtain the sampling deviation coefficient by combining the UAV swarm change rate.

[0010] Step S3: Determine whether to collect the drone number sequence in the visual sensor of the secondary drone swarm based on the sampling deviation coefficient, and filter the candidate main drones based on the frequency of occurrence of each drone number in the drone number sequence of the main drone and the secondary drone swarm.

[0011] Step S4: Access the historical calibration records of the candidate master drone to obtain the number of historical calibrations. Calculate the replacement distance difference based on the current coordinates of the candidate master drone and the master drone. Combine the replacement distance difference with the number of historical calibrations to obtain the replacement benefit score. Re-verify the current drone group according to the replacement benefit score.

[0012] In a preferred embodiment, in step S1, the task identifier of the current UAV task is obtained through the local storage unit and matched with the task information database to obtain the main UAV number in the current UAV group and the UAV numbers in the secondary UAV group.

[0013] The real-time video stream of the current drone swarm is acquired by the visual sensor on the main drone and divided into continuous video frame images, which are then integrated into a video frame sequence.

[0014] That is, to perform QR code decoding on the QR code on the body of each drone in each video frame image to obtain the device ID of each drone, and to obtain the order of device IDs of each drone in each video frame image according to the order of QR code decoding operations.

[0015] Match the device ID of each drone with the number of each drone in the sub-drone group to obtain the drone number sequence corresponding to each video frame image.

[0016] Count the number of times each drone number appears in the entire video frame sequence, and use the drone number order that appears most frequently as the current drone group number order.

[0017] In a preferred embodiment, in step S1, the task identifier of the current UAV mission is matched with the task scheduling database to obtain the target number order of the current UAV swarm.

[0018] Compare the current drone swarm's serial number sequence with the target's serial number sequence, and count the number of drones with abnormal sequences;

[0019] Divide the number of drones with abnormal order by the total number of drones in the current drone swarm to obtain the abnormal order value;

[0020] The outlier is compared with a preset outlier threshold for ordering.

[0021] If the sequence anomaly value is greater than or equal to the preset sequence anomaly threshold, the deep tracking mechanism will be activated.

[0022] If the out-of-order value is less than the preset out-of-order threshold, it will be determined that the deep tracking mechanism will not be entered.

[0023] In a preferred embodiment, in step S2, after entering the depth tracking mechanism, a tracking period is set, and the real-time video stream of the drone swarm during the tracking period is obtained through the visual sensor carried by the main drone, and divided into continuous video frame images of the tracking period.

[0024] UAV target detection is performed on video frame images for each tracking period to obtain the number of UAVs in the video frame images for each tracking period;

[0025] If the number of drones in the video frame images of each tracking period is different from the number of drones in the drone swarm, then the video frame images of that tracking period are determined to be abnormal video frame images.

[0026] If the number of drones in the video frame image of each tracking period is the same as the number of drones in the drone swarm, then the video frame image of that tracking period is determined to be a normal video frame image.

[0027] The number of abnormal video frames is counted, and the number of abnormal video frames is divided by the total number of video frames in each tracking period to obtain the change rate of the drone swarm.

[0028] In a preferred embodiment, in step S2, the three-axis angular velocity of the main UAV is obtained through the inertial measurement unit of the main UAV;

[0029] The angular velocity modulus of the main UAV is obtained by processing the three-axis angular velocity vector using the Euclidean norm calculation method of the three-axis angular velocity vector.

[0030] The attitude stability is obtained by dividing the angular velocity module of the main UAV by the preset standard angular velocity module.

[0031] Attitude stability is a technical indicator used to quantify the dynamic stability of a drone's attitude during flight. The lower the attitude stability, the more stable the drone's attitude; the higher the attitude stability, the more obvious the attitude jitter or deviation.

[0032] The rate of change and attitude stability of the UAV swarm are standardized to obtain the change factor and stability factor;

[0033] By using the Bayesian confidence assessment method, a joint analysis of the changing factors and the stable factors is performed to obtain the sampling bias coefficient.

[0034] In a preferred embodiment, in step S3, if the sampling deviation coefficient is greater than or equal to a preset sampling deviation threshold, the sequence of drone numbers present in the visual sensors of the drone swarm is collected.

[0035] If the sampling deviation coefficient is greater than the preset sampling deviation threshold, the drone number sequence present in the visual sensor of the drone swarm will not be collected.

[0036] The system acquires real-time video streams from each drone in the swarm using the visual sensors mounted on each drone.

[0037] In a preferred embodiment, the real-time video stream processing method in step S1 is used to obtain the drone number appearing in the real-time video stream of each drone in the sub-drone swarm, and integrate them into a drone number sequence.

[0038] Remove the master drone number from the drone number sequence;

[0039] Count the number of times each drone number appears in the drone number sequence;

[0040] The drone that appears most frequently will be selected as the primary drone.

[0041] In a preferred embodiment, in step S4, the historical calibration records of the candidate master drone are accessed to obtain the historical calibration count of the candidate master drone;

[0042] The current coordinates of the candidate main drone and the main drone are obtained by the global satellite navigation system receiver module carried by the drone;

[0043] Based on the three-dimensional Euclidean distance calculation method, the current coordinates of the candidate main UAV and the main UAV are processed to obtain the replacement distance difference;

[0044] The historical calibration count and replacement distance difference of the candidate master UAV are standardized to obtain the calibration factor and distance factor.

[0045] The replacement benefit score is obtained by comprehensively analyzing the calibration factor and distance factor through the TOPSIS multi-attribute decision evaluation algorithm.

[0046] In a preferred embodiment, in step S4, if the replacement benefit score is less than or equal to a preset replacement benefit score threshold, it is determined that the current drone swarm should be re-verified.

[0047] If the replacement benefit score is greater than the preset replacement benefit score threshold, then the candidate master drone is selected to replace the master drone; the re-verification process refers to the re-screening and verification operation of the candidate master drones in the current drone fleet.

[0048] The technical effects and advantages of this invention are as follows:

[0049] This invention obtains the IDs of the main UAV and each UAV in the secondary UAV swarm. It analyzes the ID sequence through video transmission from the main UAV to determine whether to enter a deep tracking mechanism, sets a tracking period, collects the number of visible UAVs to calculate the UAV swarm's change rate, obtains the three-axis angular velocity from the main UAV's inertial measurement unit to generate attitude stability, and calculates a sampling deviation coefficient based on the overall change rate. It then determines whether to collect the visual sensor ID sequence of the secondary UAV swarm based on the sampling deviation coefficient, filters candidate main UAVs based on the frequency of ID occurrence, accesses the historical calibration records of candidate main UAVs, calculates the replacement distance difference based on the current position, generates a replacement benefit score, and re-verifies the UAV swarm. This allows for real-time monitoring of the UAV swarm's ID sequence and state deviation, accurate quantification of the group's change rate and attitude stability, rapid screening of candidate main UAVs, and optimized replacement decisions, ensuring the stability of the UAV swarm's coordinated flight and mission execution efficiency. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the implementation of a real-time tracking method for multi-UAV trajectories based on multi-sensor fusion according to the present invention.

[0051] Figure 2 This is a schematic diagram illustrating the steps of a real-time tracking method for multiple unmanned aerial vehicle trajectories based on multi-sensor fusion according to the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] This invention obtains the IDs of the main UAV and each UAV in the secondary UAV swarm. It analyzes the ID sequence through video transmission from the main UAV to determine whether to enter a deep tracking mechanism, sets a tracking period, collects the number of visible UAVs to calculate the UAV swarm's change rate, obtains the three-axis angular velocity from the main UAV's inertial measurement unit to generate attitude stability, and calculates a sampling deviation coefficient based on the overall change rate. It then determines whether to collect the visual sensor ID sequence of the secondary UAV swarm based on the sampling deviation coefficient, filters candidate main UAVs based on the frequency of ID occurrence, accesses the historical calibration records of candidate main UAVs, calculates the replacement distance difference based on the current position, generates a replacement benefit score, and re-verifies the UAV swarm. This allows for real-time monitoring of the UAV swarm's ID sequence and state deviation, accurate quantification of the swarm's change rate and attitude stability, rapid filtering of candidate main UAVs, and optimization of replacement decisions.

[0054] Example 1: A method for real-time tracking of multiple UAV trajectories based on multi-sensor fusion, such as... Figures 1 to 2 As shown, it includes the following steps:

[0055] Step S1: Obtain the number of the main drone in the current drone swarm and the number of each drone in the secondary drone swarm. Analyze the numbering order of the current drone swarm based on the real-time video stream transmitted back by the main drone and determine whether to enter the deep tracking mechanism.

[0056] Step S2: Set the tracking period, collect the number of visible UAVs during the tracking period, calculate the UAV swarm change rate, obtain the three-axis angular velocity of the main UAV through the inertial measurement unit of the main UAV to generate attitude stability, and obtain the sampling deviation coefficient by combining the UAV swarm change rate.

[0057] Step S3: Determine whether to collect the drone number sequence in the visual sensor of the secondary drone swarm based on the sampling deviation coefficient, and filter the candidate main drones based on the frequency of occurrence of each drone number in the drone number sequence of the main drone and the secondary drone swarm.

[0058] Step S4: Access the historical calibration records of the candidate master drone to obtain the number of historical calibrations. Calculate the replacement distance difference based on the current coordinates of the candidate master drone and the master drone. Combine the replacement distance difference with the number of historical calibrations to obtain the replacement benefit score. Re-verify the current drone group according to the replacement benefit score.

[0059] The specific implementation is as follows:

[0060] In step S1, during the collaborative flight mission of the UAV swarm, in order to ensure the stable formation and sequential execution of the swarm, it is necessary to monitor the numbering order of the UAVs and their deviation from the preset target in real time. If individual UAVs deviate from the predetermined position or the order is abnormal during the formation mission, it will affect the overall collaborative efficiency and mission safety of the swarm. At the beginning of the mission, the composition of the main UAV and the secondary UAV swarm is obtained, and real-time video information is collected to provide basic data for the judgment of the order abnormality and subsequent monitoring.

[0061] Obtain the mission identifier of the current drone mission from the local storage unit;

[0062] Match the task identifier of the current drone mission with the task information database to obtain the main drone number in the current drone swarm and the drone numbers in the secondary drone swarm.

[0063] The real-time video stream of the current drone swarm is obtained through the visual sensor carried by the main drone;

[0064] The real-time video stream of the current drone swarm is divided into continuous video frame images and integrated into a video frame sequence.

[0065] By processing each video frame image in the video frame sequence, that is, by decoding the QR code on each drone body in each video frame image, the device ID of each drone can be obtained.

[0066] The device ID order of each drone in each video frame image is obtained according to the QR code decoding operation sequence;

[0067] Match the device ID of each drone with the number of each drone in the sub-drone group to obtain the drone number sequence corresponding to each video frame image.

[0068] Iterate through the drone number sequence corresponding to each video frame image in the video frame sequence, and count the number of times each drone number sequence appears in the entire video frame sequence;

[0069] The order of the drone numbers that appear most frequently will be used as the numbering order of the current drone swarm;

[0070] It should be noted that the local storage unit refers to the data storage module deployed in the UAV swarm control terminal, used to obtain the task identifier of the current UAV mission; the task identifier refers to the identification data generated during the UAV mission creation or scheduling phase, used to uniquely identify a UAV swarm collaborative flight mission; the task information database refers to the data storage unit that stores basic information related to the UAV swarm mission, used to obtain the main UAV number and the UAV numbers of each UAV in the current UAV swarm; the visual sensor refers to the image acquisition device deployed on the UAV, used to acquire the real-time video stream of the current UAV swarm; the QR code on each UAV is a two-dimensional barcode identifier formed by encoding the UAV's unique device ID, number, or other identification information through optical graphic encoding; the QR code decoding operation refers to the process of parsing the device ID encoded in the QR code after acquiring the image of the QR code on the UAV through the visual sensor, and then using computer vision algorithms to parse the device ID in the QR code; the device ID refers to the coded information used to uniquely identify each UAV.

[0071] Match the task identifier of the current UAV mission with the task scheduling database to obtain the target number sequence of the current UAV swarm;

[0072] Compare the current drone swarm's serial number sequence with the target's serial number sequence, and count the number of drones with abnormal sequences;

[0073] Divide the number of drones with abnormal order by the total number of drones in the current drone swarm to obtain the abnormal order value;

[0074] The outlier is compared with a preset outlier threshold for ordering.

[0075] If the sequence anomaly value is greater than or equal to the preset sequence anomaly threshold, the deep tracking mechanism will be activated.

[0076] If the out-of-order value is less than the preset out-of-order threshold, it will be determined that the deep tracking mechanism will not be entered.

[0077] It should be explained that the task scheduling database refers to the data storage unit that stores and manages the task scheduling information of the UAV swarm, and is used to obtain the target number sequence of the current UAV swarm; the preset sequence anomaly threshold can be set according to the UAV swarm size, the accuracy requirements of cooperative flight, and the reliability of visual recognition; the deep tracking mechanism refers to the operation process of the system to perform high-precision, fine-grained continuous monitoring and number sequence tracking of the UAV swarm when the sequence anomaly value reaches the preset threshold during the execution of cooperative flight tasks.

[0078] By acquiring real-time video streams of drone swarms and analyzing the drone number sequence, the current formation status and sequence of the drone swarm can be accurately identified, providing a reliable data foundation for determining sequence anomalies and improving the accuracy and stability of drone swarm coordinated flight.

[0079] In step S2, during the collaborative flight mission of the UAV swarm, in order to ensure stable formation and flight efficiency, the dynamic changes of the UAV swarm are monitored in real time. If the number or position of the UAVs deviates during the mission, it will affect the overall collaborative effect. By setting a tracking period and collecting dynamic information of the UAV swarm, basic data is provided for anomaly detection and swarm stability assessment.

[0080] After entering the deep tracking mechanism, a tracking period is set, and the real-time video stream of the drone swarm during the tracking period is obtained through the visual sensor carried by the main drone, and divided into continuous video frame images of the tracking period.

[0081] UAV target detection is performed on video frame images for each tracking period to obtain the number of UAVs in the video frame images for each tracking period;

[0082] The number of drones in each tracking time period video frame image is compared with the number of drones in the sub-drone swarm to determine:

[0083] If the number of drones in the video frame images of each tracking period is different from the number of drones in the drone swarm, then the video frame images of that tracking period are determined to be abnormal video frame images.

[0084] If the number of drones in the video frame image of each tracking period is the same as the number of drones in the drone swarm, then the video frame image of that tracking period is determined to be a normal video frame image.

[0085] Count the number of abnormal video frame images;

[0086] The rate of change of the drone swarm is obtained by dividing the number of abnormal video frames by the total number of video frames in each tracking period.

[0087] It should be noted that the tracking period can be set according to the frequency of the UAV swarm's coordinated flight, the UAV's propulsion speed, and the video acquisition frame rate requirements; UAV target detection refers to processing the video frame images of the tracking period acquired by the visual sensor through computer vision algorithms to obtain the number of UAVs in the video frame images of each tracking period.

[0088] The three-axis angular velocity of the main UAV is obtained through the inertial measurement unit of the main UAV;

[0089] The angular velocity modulus of the main UAV is obtained by processing the three-axis angular velocity vector using the Euclidean norm calculation method of the three-axis angular velocity vector.

[0090] The attitude stability is obtained by dividing the angular velocity module of the main UAV by the preset standard angular velocity module.

[0091] Attitude stability is a technical indicator used to quantify the dynamic stability of a drone's attitude during flight. The lower the attitude stability, the more stable the drone's attitude; the higher the attitude stability, the more obvious the attitude jitter or deviation.

[0092] The rate of change and attitude stability of the UAV swarm are standardized to obtain the change factor and stability factor;

[0093] By using the Bayesian confidence assessment method, a joint analysis of the changing factors and the stable factors is performed to obtain the sampling bias coefficient.

[0094] The sampling deviation coefficient reflects the consistency between the actual flight state and the expected state of the UAV swarm within the preset tracking period, as well as its relative deviation level within the overall UAV swarm. The larger the sampling deviation coefficient, the more the current rate of change of the UAV swarm deviates from the expected attitude stability of the main UAV, and the higher the level of its group state deviation within the UAV swarm. The smaller the sampling deviation coefficient, the closer the current rate of change and attitude stability of the UAV swarm are to the expected values, and the higher the group state stability and the lower the deviation level.

[0095] It should be explained that the inertial measurement unit (IMU) refers to a multi-degree-of-freedom inertial sensor deployed on the UAV to obtain the three-axis angular velocities of the main UAV; the Euclidean norm calculation method is a mathematical method for calculating the magnitude of a three-dimensional vector to obtain the angular velocity magnitude of the main UAV; the preset standard angular velocity magnitude can be set according to the characteristics of the UAV model, flight stability requirements, and the sampling accuracy of the IMU; the standardization processing methods include, but are not limited to, standard linear transformation based on interval scaling, Z-Score standardization based on statistics, or normalization based on nonlinear mapping functions. The application methods of standardization processing will not be elaborated here; the Bayesian confidence assessment method is a calculation method based on Bayesian statistical theory, which evaluates the confidence of UAV swarm change factors and attitude stability factors through joint analysis of prior probabilities and observation data, thereby quantifying the sampling bias of the UAV swarm.

[0096] By setting a tracking period and acquiring dynamic information of the drone swarm, and combining the analysis of changes in the number of drones and the attitude stability of the main drone, the rate of change and sampling deviation coefficient of the drone swarm can be quantified in real time, thereby realizing continuous monitoring and anomaly detection of the drone swarm's flight status and improving the safety and reliability of group mission execution.

[0097] In step S3, in the collaborative flight mission of the UAV swarm, in order to ensure the stability of the swarm, the deviation of the UAV swarm is judged and individuals suitable as candidate master UAVs are selected.

[0098] The sampling deviation coefficient is compared with the preset sampling deviation threshold for determination.

[0099] If the sampling deviation coefficient is greater than or equal to the preset sampling deviation threshold, the drone number sequence present in the visual sensor of the drone swarm will be collected.

[0100] If the sampling deviation coefficient is greater than the preset sampling deviation threshold, the drone number sequence present in the visual sensor of the drone swarm will not be collected.

[0101] Real-time video streams of each drone in the swarm are obtained using the visual sensors carried by each drone in the swarm.

[0102] The real-time video stream processing method in step S1 is used to obtain the drone numbers appearing in the real-time video streams of each drone in the drone swarm and integrate them into a drone number sequence.

[0103] Remove the master drone number from the drone number sequence;

[0104] Count the number of times each drone number appears in the drone number sequence;

[0105] The drone that appears most frequently will be selected as the primary drone.

[0106] It should be explained that the preset sampling deviation threshold can be set according to the size of the UAV swarm, the accuracy requirements of cooperative flight, and the reliability of visual recognition and attitude measurement.

[0107] By comparing the sampling deviation coefficient with a preset threshold and screening candidate master drones based on video analysis using visual sensors, the system can quickly identify the most suitable replacement drone when abnormal deviations occur in the drone swarm, providing an accurate basis for master drone replacement and improving the adaptability and continuity of drone swarm collaborative flight.

[0108] In step S4, during the collaborative flight mission of the UAV swarm, in order to ensure the reliability of the main UAV and the stability of the UAV swarm, a comprehensive evaluation of the candidate main UAVs is conducted. During the flight of the UAV swarm, some UAVs may have increased replacement risks due to position deviation or insufficient historical experience, thereby affecting the group's collaborative efficiency and mission execution effect. During the mission execution, the historical calibration records and current positions of the candidate main UAVs are first obtained, and their relative adaptability with the main UAV is evaluated in order to select the optimal candidate main UAV as the main UAV replacement candidate.

[0109] Access the historical calibration records of the candidate primary drone to obtain the historical calibration count of the candidate primary drone;

[0110] The current coordinates of the candidate main drone and the main drone are obtained by the global satellite navigation system receiver module carried by the drone;

[0111] Based on the three-dimensional Euclidean distance calculation method, the current coordinates of the candidate main UAV and the main UAV are processed to obtain the replacement distance difference;

[0112] The historical calibration count and replacement distance difference of the candidate master UAV are standardized to obtain the calibration factor and distance factor.

[0113] The replacement benefit score is obtained by comprehensively analyzing the calibration factor and distance factor through the TOPSIS multi-attribute decision evaluation algorithm.

[0114] The replacement benefit score reflects the overall adaptability of the candidate main UAV as a replacement for the main UAV and its relative advantages and disadvantages in the UAV swarm. The higher the replacement benefit score, the more extensive the candidate main UAV's historical calibration experience and the closer its position is to the main UAV, and its adaptability to replace the main UAV is among the best in the UAV swarm. The lower the replacement benefit score, the less extensive the candidate main UAV's historical calibration experience or the further its position is from the main UAV, and its replacement adaptability is lower, with obvious relative disadvantages.

[0115] The replacement benefit score is compared with a preset replacement benefit score threshold for evaluation.

[0116] If the replacement benefit score is less than or equal to the preset replacement benefit score threshold, it is determined that the current drone swarm should be re-verified.

[0117] If the replacement benefit score is greater than the preset replacement benefit score threshold, then the candidate main drone is selected to replace the main drone.

[0118] It needs to be explained that the historical calibration record of the candidate master UAV refers to the accumulated number of calibrations and related calibration data sets of the candidate master UAV during past mission execution or verification in UAV swarm collaborative flight missions, thus obtaining the historical calibration count of the candidate master UAV; the global satellite navigation system receiving module refers to the positioning sensor device deployed on the UAV, used to obtain the current coordinates of the candidate master UAV and the master UAV; the three-dimensional Euclidean distance calculation method is a mathematical method that calculates the three-dimensional coordinates of two points in space to obtain the straight-line distance between the two points, used to obtain the replacement distance difference; the TOPSIS multi-attribute decision evaluation algorithm is a decision-making method that, in the presence of multiple evaluation indicators, comprehensively evaluates the merits of each scheme by calculating the distance between each scheme and the ideal optimal scheme and the ideal worst scheme, and ranks them, used to obtain the replacement benefit score; the preset replacement benefit score threshold can be set according to the UAV swarm size, mission collaboration accuracy requirements, and the statistical characteristics of historical calibration data; the re-verification processing refers to the re-screening and verification operation of the candidate master UAV in the current UAV swarm when the replacement benefit score of the candidate master UAV is less than or equal to the preset replacement benefit score threshold, to ensure that the selected master UAV can meet the reliability and stability requirements of the collaborative flight mission.

[0119] By accessing the historical calibration records of the candidate master drone, obtaining its current position, and performing three-dimensional Euclidean distance calculation and multi-attribute decision analysis, the replacement adaptability of the candidate master drone is comprehensively evaluated, and the optimal candidate master drone is selected to replace the current master drone, thus ensuring the reliability and stability of the drone swarm collaborative flight mission.

[0120] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0121] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0122] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0123] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0124] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for real-time tracking of multiple UAV trajectories based on multi-sensor fusion, characterized in that: Includes the following steps: Step S1: Obtain the number of the main drone in the current drone swarm and the number of each drone in the secondary drone swarm. Analyze the numbering order of the current drone swarm based on the real-time video stream transmitted back by the main drone and determine whether to enter the deep tracking mechanism. In step S2, after entering the depth tracking mechanism, a tracking period is set, and the real-time video stream of the drone swarm during the tracking period is obtained through the visual sensor carried by the main drone, and divided into continuous video frame images of the tracking period. UAV target detection is performed on video frame images for each tracking period to obtain the number of UAVs in the video frame images for each tracking period; If the number of drones in the video frame images of each tracking period is different from the number of drones in the drone swarm, then the video frame images of that tracking period are determined to be abnormal video frame images. If the number of drones in the video frame image of each tracking period is the same as the number of drones in the drone swarm, then the video frame image of that tracking period is determined to be a normal video frame image. The number of abnormal video frames is counted, and the number of abnormal video frames is divided by the total number of video frames in each tracking period to obtain the change rate of the drone swarm. The attitude stability is generated by obtaining the three-axis angular velocity of the main UAV through the inertial measurement unit of the main UAV, and the sampling deviation coefficient is obtained by combining the change rate of the UAV swarm. In step S3, if the sampling deviation coefficient is greater than or equal to the preset sampling deviation threshold, it is determined that the drone number sequence present in the visual sensor of the drone swarm will be collected. If the sampling deviation coefficient is less than the preset sampling deviation threshold, it is determined that the drone number sequence existing in the visual sensor of this drone swarm will not be collected. Real-time video streams of each drone in the swarm are obtained using the visual sensors carried by each drone in the swarm. The real-time video stream processing method in step S1 is used to obtain the drone numbers appearing in the real-time video streams of each drone in the drone swarm and integrate them into a drone number sequence. Remove the master drone number from the drone number sequence; Count the number of times each drone number appears in the drone number sequence; The drone that appears most frequently will be selected as the primary drone. In step S4, access the historical calibration records of the candidate master drone to obtain the historical calibration count of the candidate master drone; The current coordinates of the candidate main drone and the main drone are obtained by the global satellite navigation system receiver module carried by the drone; Based on the three-dimensional Euclidean distance calculation method, the current coordinates of the candidate main UAV and the main UAV are processed to obtain the replacement distance difference; The historical calibration count and replacement distance difference of the candidate master UAV are standardized to obtain the calibration factor and distance factor. The TOPSIS multi-attribute decision evaluation algorithm is used to comprehensively analyze the calibration factor and distance factor to obtain the replacement benefit score. The current drone swarm is then re-verified according to the replacement benefit score.

2. The method for real-time tracking of multiple UAV trajectories based on multi-sensor fusion according to claim 1, characterized in that: In step S1, the task identifier of the current UAV task is obtained through the local storage unit and matched with the task information database to obtain the number of the main UAV in the current UAV group and the number of each UAV in the secondary UAV group. The real-time video stream of the current drone swarm is acquired by the visual sensor on the main drone and divided into continuous video frame images, which are then integrated into a video frame sequence. That is, to perform QR code decoding on the QR code on the body of each drone in each video frame image to obtain the device ID of each drone, and to obtain the order of device IDs of each drone in each video frame image according to the order of QR code decoding operations. Match the device ID of each drone with the number of each drone in the sub-drone group to obtain the drone number sequence corresponding to each video frame image. Count the number of times each drone number appears in the entire video frame sequence, and use the drone number order that appears most frequently as the current drone group number order.

3. The method for real-time tracking of multiple UAV trajectories based on multi-sensor fusion according to claim 2, characterized in that: In step S1, the task identifier of the current UAV mission is matched with the task scheduling database to obtain the target number order of the current UAV swarm; Compare the current drone swarm's serial number sequence with the target's serial number sequence, and count the number of drones with abnormal sequences; Divide the number of drones with abnormal order by the total number of drones in the current drone swarm to obtain the abnormal order value; The outlier is compared with a preset outlier threshold for ordering. If the sequence anomaly value is greater than or equal to the preset sequence anomaly threshold, the deep tracking mechanism will be activated. If the out-of-order value is less than the preset out-of-order threshold, it will be determined that the deep tracking mechanism will not be entered.

4. The method for real-time tracking of multiple UAV trajectories based on multi-sensor fusion according to claim 1, characterized in that: In step S2, the three-axis angular velocity of the main UAV is obtained through the inertial measurement unit of the main UAV; The angular velocity modulus of the main UAV is obtained by processing the three-axis angular velocity vector using the Euclidean norm calculation method of the three-axis angular velocity vector. The attitude stability is obtained by dividing the angular velocity module of the main UAV by the preset standard angular velocity module. Attitude stability is a technical indicator used to quantify the dynamic stability of a drone's attitude during flight. The lower the attitude stability, the more stable the drone's attitude. The greater the attitude stability, the more obvious the attitude jitter or deviation; The rate of change and attitude stability of the UAV swarm are standardized to obtain the change factor and stability factor; By using the Bayesian confidence assessment method, a joint analysis of the changing factors and the stable factors is performed to obtain the sampling bias coefficient.

5. The method for real-time tracking of multiple UAV trajectories based on multi-sensor fusion according to claim 1, characterized in that: In step S4, if the replacement benefit score is less than or equal to the preset replacement benefit score threshold, it is determined that the current drone swarm should be re-verified. If the replacement benefit score is greater than the preset replacement benefit score threshold, then the candidate master drone is selected to replace the master drone; the re-verification process refers to the re-screening and verification operation of the candidate master drones in the current drone fleet.

Images