A method and system for dynamic target relocation of a UAV

Abstract

The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical field.The application discloses a kind of unmanned aerial vehicle dynamic target relocation method and system, it is related to unmanned aerial vehicle target tracking technical filed.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target tracking technology, and specifically to a method and system for dynamic target relocation of UAVs. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are characterized by flexible viewing angles, wide coverage, and convenient deployment, and are increasingly being used in scenarios such as urban traffic monitoring, road inspection, and dynamic vehicle tracking. Existing UAV target tracking methods typically utilize onboard cameras to acquire road images, continuously determine the position of target vehicles through target detection, visual tracking, or multi-target association algorithms, and control the UAV to adjust its flight attitude or tracking waypoints based on changes in the target vehicle's position in the image. When the target vehicle is continuously visible or only briefly obscured, these methods can maintain relatively continuous target tracking.

[0003] After a target vehicle leaves the drone's field of view, existing technologies typically predict the target vehicle's trajectory based on its position, speed, and direction of motion before disappearing, and then generate the drone's search trajectory accordingly. For example, prior art CN118519446A discloses a drone tracking and control method after a vehicle target leaves the field of view. This method predicts the target vehicle's coordinate trajectory based on changes in the target vehicle's position in the world coordinate system before disappearing, determines the drone's search speed and direction based on changes in the target vehicle's position in the image frame coordinate system before disappearing, and performs a weighted fusion of the predicted and search coordinate trajectories to determine the drone's desired search trajectory.

[0004] Furthermore, the Vision-Language-Action (VLA) strategy model can fuse multimodal information such as images, task commands, and device status, and output navigation or control actions, providing a new technical approach for UAV autonomous tracking and search. However, existing VLA strategy models mainly focus on command following and current observation decisions when the target is visible, and do not adequately consider continuous search after the target has completely left the UAV's field of vision, confirmation of the same target, and safe resumption of following.

[0005] In urban road environments, target vehicles may disappear from the UAV's field of view for extended periods due to obstructions from buildings, road turns, intersections, trees, or other vehicles. Extrapolating solely from the vehicle's trajectory before disappearance fails to adequately reflect road connectivity, vehicle travel directions, and the limitations imposed by road branches on the target's movement range, easily leading to unreasonable search areas and duplicate searches. Furthermore, when multiple visually similar vehicles exist within the search area, directly updating target cues with new detection results can easily cause target confusion and contaminate the tracking model. If tracking is resumed only based on a single visual feature or predicted location after the target reappears, similar vehicles can easily be mistakenly identified as the original target. Moreover, existing methods typically lack a closed-loop connection between search decision-making, candidate vehicle verification, waypoint safety checks, and execution result feedback, making it difficult to balance the accuracy of target relocation, search efficiency, and flight safety. Summary of the Invention

[0006] The technical problem this invention aims to solve is that existing UAV target tracking methods mainly rely on historical movement trajectories for searching after a target vehicle leaves the field of view, failing to effectively combine credible target clues, road topology, and road observation status to determine the search area. Furthermore, these methods are easily interfered with by similar vehicles after the target reappears, and lack closed-loop feedback between target search, identity verification, action generation, and safe execution. This invention provides a UAV dynamic target relocation method and system to improve the reliability of target vehicle relocation and follow-up in complex road environments.

[0007] This invention is achieved through the following technical solution:

[0008] A method for dynamic target relocalization of unmanned aerial vehicles (UAVs) includes:

[0009] Acquire mission instructions, current UAV image, UAV status, historical target clues, local road context, and candidate verification results;

[0010] The drone determines whether the target vehicle is in a verified visible state based on the current image of the drone. If the target vehicle is not in a verified visible state and the target loss time exceeds a preset loss tolerance threshold, a dynamic target relocation process is triggered.

[0011] In response to triggering the dynamic target relocation process, the last verified target clue is retained, and the location uncertainty range of the target vehicle is updated according to the target loss time, wherein unauthenticated candidate vehicles are not used to replace the last verified target clue;

[0012] Candidate road segments are determined based on the last verified target clue and the local road context. The search confidence of each candidate road segment is determined based on the motion consistency between the candidate road segment and the target vehicle, road drivability constraints, recent observation state weights, and unvisited road branch rewards. Candidate road areas that meet the safe reachability conditions of the UAV are determined based on the search confidence.

[0013] The drone is controlled to search the candidate road area and to verify the identity of candidate vehicles detected during the search. The identity verification includes determining the identity score of the candidate vehicle based on the visibility quality, attribute consistency, motion consistency, spatial consistency and appearance re-identification similarity of the candidate vehicle, and selecting the corresponding identity confirmation conditions according to the number of candidate vehicles to confirm whether the candidate vehicle is the target vehicle.

[0014] The task instruction, the current image of the UAV, the UAV status, the historical target clues, the local road context, and the candidate verification results are input into the visual language action strategy model to obtain a structured relocation decision including action phase, spatial target, and policy confidence, and a waypoint to be executed by the UAV is generated based on the spatial target.

[0015] The system performs safety constraint checks on the proposed waypoints of the UAV. If the safety constraints are met, the UAV is controlled to execute the corresponding action; if the safety constraints are not met, the proposed waypoint is rejected and either replanning or maintaining the search state.

[0016] Based on the road observation results, candidate vehicle authentication results, and action execution results, update the last verified target clue, the local road context, the candidate verification results, and the input of the visual language action strategy model for the next round.

[0017] Furthermore, a verified visibility flag is used to indicate whether the target vehicle is in a verified visible state, and the target loss condition is represented as:

[0018]

[0019] in, This indicates the verified visibility flag at the current moment. This indicates that the target vehicle is not visible or its identity is ambiguous. Indicates the current moment. Indicates the time of the last verified observation. This represents the preset loss tolerance threshold;

[0020] The last verified target clue is represented as follows:

[0021]

[0022]

[0023] in, Indicates the target category, This indicates that the target appearance description has been verified for the last time. This indicates that the target location has been verified for the last time. Indicates target status information. This indicates that the target speed has been verified for the last time. This indicates that the target course has been verified for the last time. Indicates the radius of positional uncertainty at the time of the last verified observation;

[0024] When the target vehicle is in a verified visible state, the last verified target clue is updated based on the current observation results; when the target vehicle is not in a verified visible state, the last verified target clue remains unchanged.

[0025] Furthermore, updating the uncertainty range of the target vehicle's location based on the target loss time includes:

[0026] The radius of the uncertain position at the current moment is calculated using the following formula:

[0027]

[0028] in, The radius represents the current position at an uncertain time. Indicates the growth factor of the uncertain radius;

[0029] Based on the current position of the UAV, the last verified target position, the last verified target heading, the last verified target speed, the current position uncertainty radius, and the local road topology, a set of candidate road segments is determined.

[0030] Furthermore, the search confidence of the candidate road segment is determined according to the following formula:

[0031] ;

[0032] in, Indicates candidate road segments Search confidence, Represents the normalization coefficient. This indicates that the road segment is related to the last verified target clue. Consistency of the extrapolated direction of motion This indicates road connectivity and vehicle mobility constraints. Indicates the weight of recently observed states. This indicates the reward for a candidate road segment that is not a visited road branch;

[0033] For the candidate road segment set The search confidence of each candidate road segment satisfies:

[0034] ;

[0035] Specifically, the motion consistency is determined based on the deviation between the road direction of the candidate road segment and the last verified target heading, as well as the distance from the extrapolated target position to the candidate road segment; when the candidate road segment has been fully observed recently and the target vehicle has not been found, the corresponding recent observation state weight is reduced and then restored over time; the search priority of unvisited road branches is increased through the reward item.

[0036] Furthermore, regarding the first detected during the search process... For each candidate vehicle, the candidate vehicle's identity score is determined according to the following formula:

[0037] ;

[0038] in, Indicates the first Candidate vehicle identity score for each candidate vehicle. This represents the visible mass of the candidate vehicle. This indicates the consistency of category and appearance attributes between the candidate vehicle and the target vehicle. This indicates the motion consistency between the candidate vehicle and the last verified target clue. This indicates the spatial consistency between the road area where the candidate vehicle is located and the candidate road area. This represents the appearance re-identification similarity between the candidate vehicle and the last verified target appearance description. , , , and Let represent the corresponding non-negative weights, and satisfy: .

[0039] Furthermore, based on the number of candidate vehicles, appropriate identity verification conditions are selected to confirm whether a candidate vehicle is the target vehicle, including:

[0040] When there are two or more candidate vehicles that meet the detection conditions, the candidate vehicles are sorted from high to low according to their candidate vehicle identity scores, and the optimal candidate vehicle is confirmed as the target vehicle when the following conditions are met simultaneously:

[0041] ;

[0042] in, This represents the candidate vehicle identity score of the optimal candidate vehicle. This represents the candidate vehicle identity score of the second-best candidate vehicle. Indicates the identity verification threshold. This represents the threshold for the identity scoring interval between candidate vehicles;

[0043] When there is only one candidate vehicle that meets the detection conditions in the current candidate road area, the identity scoring interval condition between candidate vehicles does not apply. Only it is determined whether the identity score of the candidate vehicle is not lower than the identity confirmation threshold. When the corresponding confirmation condition is not met, the last verified target clue is not updated, and the UAV is controlled to continue searching, adjust the observation waypoint, or re-detect candidate vehicles in the current image of the UAV acquired in the subsequent period.

[0044] Furthermore, the input representation of the visual language action strategy model is as follows:

[0045] ;

[0046] in, This indicates the task instruction. This indicates the current image of the drone. Indicates the status of the drone. This indicates historical target clues up to the last verified observation time. This represents the local road context. This indicates the candidate verification result;

[0047] The drone's status is represented as follows:

[0048] ;

[0049] in, , , and These represent the position, speed, heading, and altitude of the UAV, respectively.

[0050] When no candidate vehicle is detected in the current candidate road area, the candidate verification result is an empty set or an unconfirmed status mark; when a candidate vehicle is detected, the candidate verification result includes the identity score and identity confirmation result of the candidate vehicle; when two or more candidate vehicles are detected, the candidate verification result also includes the score interval information between the best candidate vehicle and the second best candidate vehicle.

[0051] Furthermore, the structured relocation decision is represented as:

[0052] ;

[0053] in, The action phase refers to the action phase, which includes at least the following steps: search, inspection, confirmation, follow, and return. The spatial target is indicated by road area markers, candidate vehicle markers, confirmed target vehicle markers, or UAV waypoints. Indicates the confidence level of the stated strategy;

[0054] When the spatial target is a road area identifier, an observation waypoint is generated based on the location of the road area; when the spatial target is a candidate vehicle identifier and the action phase is inspection or confirmation, a corresponding waypoint is generated based on the location of the candidate vehicle; when the spatial target is a confirmed target vehicle identifier and the action phase is follow, a follow waypoint is generated based on the location of the confirmed target vehicle; when the spatial target is a UAV waypoint, the UAV waypoint is determined as the UAV's to-execute waypoint.

[0055] Furthermore, a safety constraint check is performed on the waypoint to be executed by the UAV, including:

[0056] Based on the flight altitude, obstacle distance, waypoint accessibility, and local collision risk of the waypoint to be executed by the UAV, determine whether the waypoint to be executed by the UAV meets the safety constraints.

[0057] When the waypoint to be executed by the UAV does not meet the safety constraints, the safety check failure result is fed back to the visual language action strategy model, and the waypoint to be executed by the UAV is regenerated or the search state is maintained.

[0058] When a candidate vehicle is confirmed as the target vehicle, the confirmed candidate vehicle is used as a new verified target, the last verified target clue is updated and following is resumed; the results of the most recent complete observation that no verified target vehicle was found, the candidate vehicle failed to be verified, and the UAV waypoint failed the security constraint check are used to update the local road context, the candidate verification results, and the input of the visual language action strategy model for the next round.

[0059] The present invention also provides a UAV dynamic target relocation system for implementing the aforementioned UAV dynamic target relocation method, comprising:

[0060] The multi-source input module is used to acquire mission instructions, current UAV images, UAV status, historical target clues, local road context, and candidate verification results;

[0061] The target loss judgment module is used to determine whether the target vehicle is in a verified visible state based on the current image of the UAV, and output the target loss status when the target vehicle is not in a verified visible state and the target loss time exceeds a preset loss tolerance threshold.

[0062] The historical target clue management module is used to save the last verified target clue, update the uncertain range of the target vehicle's location based on the target loss time, and prohibit unauthenticated candidate vehicles from replacing the last verified target clue.

[0063] The road constraint search module is used to determine candidate road segments based on the last verified target clue and the local road context, determine the search confidence of each candidate road segment based on target motion consistency, road drivability constraints, recent observation state weights and unvisited road branch rewards, and determine candidate road regions based on the search confidence.

[0064] The candidate vehicle verification module is used to determine the identity score of candidate vehicles based on their visible quality, attribute consistency, motion consistency, spatial consistency, and appearance re-identification similarity. When there are two or more candidate vehicles that meet the detection conditions, the module confirms whether the optimal candidate vehicle is the target vehicle based on its candidate vehicle identity score and the score difference between the optimal and second-best candidate vehicles. When there is only one candidate vehicle that meets the detection conditions, the module confirms whether the candidate vehicle is the target vehicle based solely on its identity score and the identity confirmation threshold.

[0065] A visual language action structured decision module is used to output a structured relocation decision including action phase, spatial target, and policy confidence based on the task instruction, the current image of the UAV, the UAV state, the historical target cues, the local road context, and the candidate verification results; and to generate a UAV waypoint to be executed based on the spatial target; and

[0066] The safety execution and feedback module is used to perform safety constraint checks on the waypoints to be executed by the UAV, control the UAV to execute corresponding actions when the waypoints to be executed by the UAV meet the safety constraints, replan or maintain the search state when the waypoints to be executed by the UAV do not meet the safety constraints, and update the last verified target clue, the local road context, the candidate verification results, and the input of the visual language action structured decision module for the next round based on the road observation results, candidate vehicle authentication results, and action execution results.

[0067] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned UAV dynamic target relocation method.

[0068] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned UAV dynamic target relocation method.

[0069] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0070] This invention saves the last verified target clue when the target vehicle is confirmed to be visible, and prohibits the use of the corresponding candidate vehicle to replace the last verified target clue when the target vehicle is not visible or the candidate vehicle has not yet passed the identity verification. This avoids false detection results and similar vehicles contaminating the original target identity information during the occlusion period. At the same time, it updates the uncertain range of the target vehicle's location according to the target loss time, providing a reliable historical status basis for the determination of the subsequent search area.

[0071] This invention determines candidate road segments based on the last verified target clue and local road context, and determines the search confidence by comprehensively considering target motion consistency, road drivability constraints, recent observation state weights, and rewards for unvisited road branches. This ensures that the target search range matches the road connectivity, possible vehicle driving directions, and existing UAV observations, reducing the search for areas that do not conform to vehicle motion patterns and the repeated search of already observed road areas.

[0072] This invention comprehensively considers the visible quality, attribute consistency, motion consistency, spatial consistency, and appearance re-identification similarity of candidate vehicles to determine the identity score of the candidate vehicle. At the same time, it uses the identity confirmation threshold and the identity score interval threshold between candidate vehicles to confirm the identity. Only when the best candidate vehicle has sufficient credibility and sufficient distinguishability from other candidate vehicles will the following be resumed, thereby reducing the risk of erroneous relocation and erroneous resumption of following caused by similar vehicles.

[0073] This invention inputs task instructions, current UAV image, UAV status, historical target clues, local road context, and candidate verification results into a visual language action strategy model, outputting a structured relocation decision including action phase, spatial target, and policy confidence, and performing safety constraint checks on the generated UAV waypoints to be executed; further, it updates the next round of model input based on road observation results, candidate vehicle identity verification results, and action execution results, thereby forming a dynamic target relocation closed loop that connects target search, candidate confirmation, action generation, safe execution, and feedback updates. Attached Figure Description

[0074] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0075] Figure 1 This is a flowchart of the UAV dynamic target relocation method of the present invention;

[0076] Figure 2 This is a schematic diagram of the hierarchical reasoning framework of the dynamic target relocation method of the present invention;

[0077] Figure 3 This is a schematic diagram of the visual language action structured decision-making and safety feedback closed loop of the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0079] Example 1

[0080] In this embodiment, "tracking" means continuously acquiring the target's position and motion state based on continuous images and updating target cues; "following" means planning UAV waypoints based on the confirmed target's position and motion state and controlling the UAV to perform follow-flying. The search, candidate vehicle screening, and identity verification after target loss are part of the relocation process; following the target vehicle will only resume after successful identity verification.

[0081] This embodiment provides a dynamic target relocation method for unmanned aerial vehicles (UAVs) applied to urban road traffic inspection scenarios. The UAV executing the method is equipped with an onboard visible light camera, a positioning and inertial navigation unit, a flight controller, and a wireless communication unit. The UAV's onboard computing equipment or a ground control device communicatively connected to the UAV is connected to the onboard visible light camera, the positioning and inertial navigation unit, the road information source, and the flight controller, respectively, to acquire the UAV's current image, UAV status, and road information, and to perform target vehicle detection, target loss judgment, road constraint search, candidate vehicle verification, structured relocation decision-making, and flight action control based on the acquired information.

[0082] The road information source can be a high-precision map, a road topology map, or road information obtained through lane line detection and road semantic segmentation. The road information includes at least road topology, road branches, road connectivity, lane travel direction, and vehicle-passable areas.

[0083] At the start of the mission, the upper-level mission system issues mission commands to the UAV targeting the vehicle. The UAV continuously observes and follows the target vehicle based on road images acquired by its onboard visible light camera, historical observation information of the target vehicle, and its own flight status. When the target vehicle temporarily leaves the current image due to obstructions from buildings, overpasses, trees, other vehicles, road curves, intersections, or limitations in the UAV's field of view, the UAV enters a dynamic target relocation process.

[0084] like Figure 1 As shown, the UAV dynamic target relocation method in this embodiment includes acquiring multi-source input, determining the target loss state and maintaining the last verified target clue, constructing a road constraint candidate search region, verifying the candidate vehicle identity and outputting a visual language action structured decision, and performing safety constraint checks, executing actions, and providing feedback updates. Specifically, it includes the following steps.

[0085] S101: Obtain mission instructions, current UAV image, UAV status, historical target clues, local road context, and candidate verification results;

[0086] The drone acquires its current image using an onboard visible light camera. and obtain task instructions. Drone status Historical target clues Local road context and candidate validation results , is represented as:

[0087] ;

[0088] At the start of each round of execution, the candidate verification results The candidate vehicles obtained in the previous search process are used for verification; when the method is executed for the first time, the results will be... Initialize to an empty set or an unconfirmed status flag. After completing S105, update the candidate vehicle verification results obtained in this round to the new flag. And serve as the input to S106.

[0089] The task instructions describe the tracking tasks to be performed by the target vehicle and the drone. The drone's status is indicated as follows:

[0090] ;

[0091] in, , , and These represent the position, speed, heading, and altitude of the UAV, respectively.

[0092] The historical target clues Including the time up to the last verified observation The resulting historical observation information of the target vehicle. The local road context. This includes road topology, road branches, road connectivity, lane direction of travel, traversable areas, road areas where no recently identified target vehicles have been found, unvisited road branches, and road areas currently accessible to drones.

[0093] When no candidate vehicle has been detected in the current search area, the candidate verification result The status is marked as empty or unconfirmed; when a candidate vehicle is detected, the candidate verification result... The candidate vehicle's identity score and identity verification result are included. When two or more candidate vehicles are detected, the candidate verification result also includes the score interval information between the best candidate vehicle and the second-best candidate vehicle.

[0094] like Figure 2 As shown, the aforementioned task instructions, current UAV image, UAV status, historical target cues, local road context, and candidate verification results constitute the multi-source inputs for the dynamic target relocalization process. These multi-source inputs sequentially pass through target loss judgment and cue preservation, road constraint search area estimation, candidate vehicle verification, and structured decision processing, and the action execution results are fed back to the next round of multi-source inputs.

[0095] S102 determines whether the target vehicle is in a verified visible state based on the current image of the UAV, and triggers the dynamic target relocation process when the target loss condition is met.

[0096] This embodiment uses a verified visibility flag. This indicates whether the target vehicle is currently in a verified visible state. When =1, it indicates that a target vehicle exists in the current drone image, and the identity of the target vehicle is reliable; when When =0, it indicates that the target vehicle is not visible in the current drone image, or that there is an identity ambiguity between the vehicle detected in the current image and the target vehicle.

[0097] Specifically, vehicle target detection is performed on the current UAV image to obtain candidate vehicle detection boxes, vehicle categories, and detection confidence scores. For candidate vehicles with detection confidence scores not lower than a preset detection threshold, the identity score of the candidate vehicle is determined according to the identity verification method described in S105, and the candidate vehicles are sorted from high to low according to their identity scores.

[0098]

[0099] When there are two or more candidate vehicles that meet the detection criteria, the candidate vehicle with the highest identity score is selected as the optimal candidate vehicle, and its detection confidence is denoted as . Record their identity score as The identity score of the second-best candidate vehicle is recorded as The verified visibility flag is determined according to the following formula:

[0100]

[0101] When only one candidate vehicle meets the detection criteria, the identity score of that candidate vehicle is recorded as follows: No interval condition is set for identity scoring between candidate vehicles; if If it is not lower than the identity verification threshold, then =1, otherwise =0. The aforementioned Indicates the detection threshold, the Indicates the identity verification threshold, the This represents the threshold for the identity scoring interval between candidate vehicles. As a non-limiting implementation, the detection threshold can be set to 0.5, the identity verification threshold can be set to 0.72, and the identity scoring interval threshold between candidate vehicles can be set to 0.06; wherein, the detection threshold is determined based on the target detection accuracy on the validation set, and the identity verification threshold and the identity scoring interval threshold between candidate vehicles are determined based on the candidate vehicle identity verification results.

[0102] when When =1, the system continues to follow the target vehicle and updates the target vehicle's historical target cues based on current observations. When =0, but at the current moment Compared with the last verified observation time The time difference between them did not exceed the preset loss tolerance threshold. At this time, the system maintains the current tracking state, or performs partial inspections through preset flight control strategies.

[0103] The system triggers a dynamic target relocation process when the following target loss conditions are met:

[0104]

[0105] in, The parameters are preset based on the drone image acquisition frequency, the target vehicle's speed, and the allowable short-term obstruction duration.

[0106] S103, retain the last verified target clue and update the uncertain range of the target vehicle's location based on the time the target was lost.

[0107] The system creates and updates the last verified target clue when the target vehicle is in a verified visible state. The last verified target clue is represented as:

[0108]

[0109] The last verified target status information is represented as follows:

[0110]

[0111] in, Indicates the target category, This indicates that the target appearance description has been verified for the last time. This indicates that the target location has been verified for the last time. Indicates target status information. This indicates that the target speed has been verified for the last time. This indicates that the target course has been verified for the last time. This represents the radius of the positional uncertainty at the last verified observation time.

[0112] The target appearance description may include at least one of the target vehicle's color, model, texture, and vehicle re-identification features. When the target vehicle is in a verified visible state, the system updates the last verified target clue based on the current observation results; when the target vehicle is not visible or the current candidate vehicle has not yet passed authentication, the system does not replace the last verified target clue with the corresponding candidate vehicle, so as to avoid similar vehicles or erroneous detection results contaminating the target vehicle's identity information.

[0113] After the target vehicle is lost, the system calculates the current position uncertainty radius based on the position uncertainty radius at the last verified observation time, the target loss time, and the uncertainty radius growth factor, using the following formula:

[0114] ;

[0115] in, The radius represents the current position at an uncertain time. Indicates the growth factor of the uncertain radius;

[0116] In one implementation, both the current time and the last verified observation time are in seconds, the position uncertainty radius is in meters, and the uncertainty radius growth factor is in meters per second. The position uncertainty radius at the last verified observation time is determined based on the target's world coordinate positioning error.

[0117]

[0118] in, This represents the standard deviation of the target's world coordinate positioning error at the last verified observation time. This represents the preset minimum uncertainty radius. This represents the confidence coefficient (confidence factor) indicating the location uncertainty. This represents the radius of positional uncertainty at the time of the last verified observation. The growth factor α of the uncertainty radius is determined based on the prediction error in historical vehicle trajectories or the verification set.

[0119]

[0120] in, This indicates the true location of the target in the verification data. This indicates the predicted position obtained by extrapolating from the last verified target speed and heading. This represents the 95th percentile of the verification error. As a non-limiting example, the initial position uncertainty radius can be taken as 2 meters, and the uncertainty radius growth factor can be taken as 1.5 meters / second. Then, the position uncertainty radius of the target vehicle after 5 seconds of loss is 9.5 meters.

[0121] In real-world drone applications, the center pixel of the bottom edge of the candidate vehicle detection bounding box is taken, and based on the intrinsic parameters of the onboard camera, the extrinsic parameters of the camera relative to the drone body, and the drone's positioning and attitude information, the pixel is back-projected into a spatial ray:

[0122]

[0123]

[0124]

[0125] in, Let K represent the homogeneous pixel coordinates of the center point of the bottom edge of the i-th candidate vehicle detection box in the image, and let K represent the camera intrinsic parameter matrix. This represents the ray direction from the camera center to the i-th candidate vehicle in the camera coordinate system. This represents the rotation matrix from the camera coordinate system to the world coordinate system. This represents the direction of the ray in the world coordinate system. This indicates the position of the camera center in the world coordinate system. This represents the scale parameter of the intersection point between the ray corresponding to the current candidate vehicle and the road surface. Indicates the local height of the road surface. and They represent and The component in the vertical direction, Represents a point on a ray. This represents the position of the i-th candidate vehicle in the world coordinate system.

[0126] The location of the camera center in the world coordinate system, the local road height given by the road plane or digital elevation model, and the rotation relationship between the camera coordinate system and the world coordinate system are used together to determine the intersection point of the candidate vehicle's line of sight and the road surface. After obtaining the candidate vehicle's position in the world coordinate system, it is matched with the centerline of the passable lane in the road map to obtain the corresponding road segment number and road coordinates. If the simulation environment can directly provide the candidate vehicle's world coordinates, the candidate vehicle position output by the simulation environment can be directly used as its world coordinates.

[0127] When the vertical component of the ray in the world coordinate system is zero, the ray is parallel to the road surface and the intersection point cannot be determined by the above formula; when the scale parameter calculated by the above formula is not greater than zero, the intersection point is located behind the camera or not in the direction of the selected ray, and the back projection result of the candidate vehicle should be discarded.

[0128] S104. Based on the last verified target clue and the local road context, candidate road segments are determined, and the search confidence of each candidate road segment is determined.

[0129] The system constructs a set of candidate road segments based on the UAV's current position, the last verified target position, the last verified target speed, the last verified target heading, the current position uncertainty radius, and the local road topology. The candidate road segments in the candidate road segment set are road segments that the target vehicle may pass through after being lost, determined based on road connectivity and the direction of vehicle travel.

[0130] like Figure 2 As shown, in the road constraint search area estimation stage, the system extrapolates the direction of motion of the target vehicle based on the last verified target position, target speed and target heading. On the other hand, it constrains the extrapolation results based on road topology, road connectivity and vehicle travel directions, and selects candidate road segments accordingly.

[0131] For the candidate road segment set any candidate road segment The search confidence of the candidate road segment is calculated using the following formula:

[0132] ;

[0133] in, Indicates candidate road segments Search confidence, Represents the normalization coefficient. This indicates that the road segment is related to the last verified target clue. Consistency of the extrapolated direction of motion This indicates road connectivity and vehicle mobility constraints. Indicates the weight of recently observed states. This indicates the reward for a candidate road segment that is not a visited road branch;

[0134] In one implementation, the predicted free motion position of the target vehicle is first calculated based on the last verified target position and velocity:

[0135]

[0136] This implementation uses uniform heading extrapolation, and the predicted target heading is the last verified target heading. The minimum angle between the candidate road segment direction and the predicted target heading is recorded as the road direction deviation, and the angle is normalized to [0,π].

[0137] For candidate road segment ρ, let Let ρ represent the shortest distance from the predicted location to the candidate road segment ρ. This indicates the angular deviation between the candidate road segment direction and the last verified target heading. The heading error scale parameter is used to determine the motion consistency according to the following formula:

[0138]

[0139] Road drivability constraints are assigned values ​​according to the following formula:

[0140]

[0141] If a candidate road segment was last fully observed and no authenticated target vehicle was found, the weight of the recent observation state is reduced based on the observation time, and this weight is restored over time.

[0142]

[0143] Where γ∈(0,1) represents the suppression strength of target vehicles that have not been recently detected through authentication. This indicates the moment when no authenticated target vehicle was found in the most recent complete observation of candidate road segment ρ. >0 represents the recovery time constant for recent observation state weights. When there are no historical undetected, authenticated target vehicle observation records for the candidate road segment, let... =1; When there is no historical observation record of a target vehicle that has not been verified, the reward is calculated according to the above formula. The reward for not visiting road branches is assigned according to the following formula:

[0144]

[0145] As a non-limiting example, the inhibition strength γ can be 0.2, and the recovery time constant... 10 seconds can be taken, weight of visited road branches. A value of 0.8 is acceptable. For a non-empty set of candidate road segments where the sum of the unnormalized weights of all candidate road segments is greater than zero, the normalization coefficient is the reciprocal of the sum of the unnormalized weights of all candidate road segments:

[0146]

[0147] For the set of non-empty candidate road segments The search confidence of each candidate road segment satisfies:

[0148] ;

[0149] When the candidate road segment set When the normalization coefficient is empty or the sum of the unnormalized weights of all candidate road segments is zero, the system will not execute the current search waypoint and will replan the candidate road segments or keep the target lost to avoid the normalization coefficient being divided by zero.

[0150] The generation of the candidate road segment set includes two steps: generating a set of possible roads for vehicles and filtering the safe reachability of drones. The system first matches the last verified target location map to the starting road segment in the road topology map; the road topology map represents the road connection relationships and the directions that vehicles can travel.

[0151] The road topology is denoted as G=(V,E), where V represents the set of road nodes and E represents the road connection relationships and vehicle travel directions; This represents the set of road segments that the target vehicle might reach during the period when the target is lost. This indicates that the drone can safely reach the candidate road segment ρ.

[0152]

[0153]

[0154]

[0155]

[0156]

[0157] Starting from the initial road segment, the system expands the road segment along the legal vehicle travel direction in the road topology map using breadth-first search or the shortest path expansion method based on the cumulative road length. Road segments whose cumulative road distance falls within the above possible travel distance range, are connected to the initial road segment, and allow vehicle passage are added to the vehicle's possible road set. At the same time, roads that are in the wrong direction, prohibited roads, roads that are not connected to the initial road, and roads that obviously exceed the vehicle's possible travel distance are deleted.

[0158] The set of possible roads represents the road segments that the target vehicle might reach during the period when the target is lost, while the set of candidate road segments represents the road segments filtered by the drone's safe reachability criteria. The drone's safe reachability is determined based on the drone's current location, remaining range, flight altitude, no-fly zones, obstacle distance, and collision safety distance.

[0159] in, This represents the upper limit of the maximum longitudinal acceleration of the target vehicle on the road. This represents the aforementioned positional uncertainty amplification factor. As a non-limiting example, A speed of 2 m / s² can be taken. Option 2 is acceptable.

[0160] Specifically, the motion consistency is determined based on the deviation between the road direction of the candidate road segment and the last verified target heading, as well as the distance from the extrapolated target position to the candidate road segment; when the candidate road segment has been fully observed recently and the target vehicle has not been found, the corresponding recent observation state weight is reduced and then restored over time; the search priority of unvisited road branches is increased through the reward item.

[0161] Specifically, the motion consistency can be determined based on the deviation between the road direction of the candidate road segment and the last verified target heading, as well as the distance from the extrapolated target position to the candidate road segment; the road connectivity and vehicle drivability constraints can be determined based on whether the candidate road segment is connected to the road segment where the last verified target position is located, whether the travel direction of the candidate road segment is consistent with the possible movement direction of the target vehicle, and whether the candidate road segment allows vehicle travel.

[0162] When a candidate road segment has been fully observed by a drone recently and no authenticated target vehicle has been detected, its recent search priority is reduced by the recent observation state weight, and its search priority is restored over time without permanently excluding the candidate road segment. For road branches that have not yet been visited by a drone, their search priority is increased by the aforementioned reward item.

[0163] The system sorts candidate road segments from high to low search confidence and, in combination with the drone's flight altitude, remaining range, no-fly zone, and collision safety distance, selects candidate road segments that meet the drone's safe accessibility conditions to form a candidate road area.

[0164] S105 controls the drone to search for candidate road areas and authenticate the detected candidate vehicles.

[0165] At the start of S105, if a previous round of structured relocation decision exists, search waypoints are generated based on that decision. If a previous round of structured relocation decision is not present during the first execution, search waypoints are generated according to a preset search strategy. The road observations and candidate verification results obtained in S105 are used in S106 to generate the current round of structured relocation decision, and are then used as input for the next round after feedback updates.

[0166] The UAV determines the execution order of search waypoints based on the search confidence of candidate road segments and the current UAV status, and observes the candidate road areas sequentially. When a candidate road area is fully observed and no authenticated target vehicle is found, the system records the corresponding road observation results and lowers the priority of that candidate road area in recent search processes.

[0167] For the first detected during the search process For each candidate vehicle, its visible quality, attribute consistency, motion consistency, spatial consistency, and appearance re-identification similarity are determined, and the candidate vehicle's identity score is determined according to the following formula:

[0168] ;

[0169] in, Indicates the first Candidate vehicle identity score for each candidate vehicle. This represents the visible mass of the candidate vehicle. This indicates the consistency of category and appearance attributes between the candidate vehicle and the target vehicle. This indicates the motion consistency between the candidate vehicle and the last verified target clue. This indicates the spatial consistency between the road area where the candidate vehicle is located and the candidate road area. This represents the appearance re-identification similarity between the candidate vehicle and the last verified target appearance description. , , , and Let represent the corresponding non-negative weights, and satisfy: .

[0170] Among them, candidate vehicle features can be one or more of the following: color, model, texture, motion trajectory, and vehicle re-identification features.

[0171] Based on the number of candidate vehicles, select the corresponding identity verification criteria. When there are two or more candidate vehicles that meet the detection criteria, sort the candidate vehicles from high to low according to their identity scores. When the optimal candidate vehicle meets the following conditions simultaneously, confirm that the optimal candidate vehicle as the target vehicle:

[0172] ;

[0173] in, This represents the candidate vehicle identity score of the optimal candidate vehicle. This represents the candidate vehicle identity score of the second-best candidate vehicle. Indicates the identity verification threshold. This represents the threshold for the identity scoring interval between candidate vehicles;

[0174] When there is only one candidate vehicle that meets the detection conditions in the current candidate road area, the identity scoring interval condition between candidate vehicles does not apply. Only it is determined whether the identity score of the candidate vehicle is not lower than the identity confirmation threshold. When the corresponding confirmation condition is not met, the last verified target clue is not updated, and the drone is controlled to continue searching, adjust the observation waypoint, or re-detect candidate vehicles in the current drone image acquired in the subsequent period.

[0175] In one implementation, each authentication component is normalized to [0,1]. The apparent quality is determined based on the detection confidence and the detection box area quality.

[0176]

[0177]

[0178] in, This represents the detection confidence level of candidate vehicle i. Indicates the area mass of the detection frame. This represents the area of ​​the detection box for candidate vehicle i. This represents the minimum detection box area that meets the recognition requirements. Represents the area of ​​the reference detection box under clear observation conditions. The clip function restricts the result to [0,1] and satisfies... > . ∈[0,1] represents the fusion coefficient between detection confidence and detection box area quality. Attribute consistency is determined based on target category consistency and color similarity:

[0179]

[0180] Among them, category consistency uses a consistency indicator function. , ∈[0,1] represents the fusion coefficient between category consistency and color similarity. This represents the color histogram similarity between the candidate vehicle and the last verified target vehicle, normalized to [0,1]. Motion consistency is determined by the following formula:

[0181]

[0182] in, The positions of the aforementioned candidate vehicles in the world coordinate system will be used for representation. Using the predicted target location obtained from S104, This indicates the angular deviation between the candidate vehicle's heading and the predicted target heading. and The aforementioned uncertainty range and heading error scale parameters are used.

[0183] In the above motion consistency calculation, the current position uncertainty radius is used as the scale parameter of the position error, and the heading error scale parameter is used to normalize the heading deviation.

[0184] Spatial consistency is determined based on the distance from candidate vehicles to the current candidate road area:

[0185]

[0186] in, Indicates candidate vehicles The shortest road distance to the current candidate road area. This represents the spatial distance scale parameter.

[0187] Appearance re-identification similarity is determined based on the cosine similarity between the features of the last verified target image and the features of the candidate vehicle image:

[0188]

[0189] in, and Let i represent the normalized image features of candidate vehicle i and the last verified target vehicle, respectively. This represents the cosine similarity between the two. Linearly map it to [0,1].

[0190] Each weight is determined offline using the validation dataset and fixed before evaluation. Specifically, under the constraint that each weight is non-negative and the sum of the weights is 1, grid search or numerical optimization is performed using the candidate vehicle identification accuracy, F1 score, or false confirmation rate as optimization metrics. As a non-restrictive example, the five weights can be set to 0.10, 0.15, 0.20, 0.15, and 0.40, respectively; the identification threshold can be set to 0.72; and the threshold for the identification scoring interval between candidate vehicles can be set to 0.06.

[0191] S106, the multi-source input described in S101 Input the Vision-Language-Action (VLA) policy model to obtain a structured relocation decision including action phase, spatial target and policy confidence, and generate UAV waypoints to be executed based on the spatial target.

[0192] like Figure 3 As shown, before feeding the multi-source input into the visual language action strategy model, the current image of the UAV is input into the visual encoding module of the multimodal base model to generate a visual token; the task instructions, UAV status, historical target clues, local road context and candidate verification results are serialized into structured text according to preset fields, and converted into text tokens through a text segmenter; the visual token and text token are jointly input into the multimodal language model for fusion reasoning.

[0193] The visual language action strategy model can use the Qwen2.5-VL-7B series model as the multimodal base model, and adapt the attention projection layer to the task using the Low-Rank Adaptation (LoRA) method. The frozen base model weights (W) after task adaptation are expressed as follows:

[0194]

[0195]

[0196] in, The weights represent the adaptation weights used during inference, and W represents the frozen base model weights. and Let each represent a trainable low-rank matrix. Indicates the rank of the low-rank fit. This represents the scaling factor. Thus, while keeping most of the parameters of the base model frozen, the visual language action strategy model is adapted to the search, candidate validation, and relocation decision-making tasks after the target vehicle is lost.

[0197] In one implementation, the specific base model uses Qwen2.5-VL-7B-Instruct, with low-rank adaptation applied to the attention projection layer. During training, the main parameters of the base model are frozen, and the low-rank matrix is ​​trained. LoRA dropout is set to 0.05. Training data is generated from simulation logs and rule-validated labeled samples. A single training sample includes the current UAV image, mission instructions, UAV status, target loss status, historical target cues, candidate road regions, candidate vehicle labels, and candidate validation results. The supervision label is a structured relocation decision sequence including action phase, spatial target type, and spatial target identifier. Policy confidence is not used as an independent supervision field but is calculated during the inference phase based on the output token probabilities of the model for the action phase and spatial target fields.

[0198] The structured relocation decision sequence is trained using autoregressive cross-entropy loss. As a non-limiting embodiment, the training samples include 1160 training samples and 117 validation samples, with 216 training steps and an optimal validation loss of 0.1869.

[0199] The visual language action strategy model is not limited to the Qwen2.5-VL-7B series model. Other multimodal models that can integrate visual information, task instructions and device status and output action decisions can also be used. The model adaptation method is not limited to low-rank adaptation. It can also adopt cue learning, adapter or other parameter fine-tuning methods.

[0200] The structured relocalization decision output by the visual language action strategy model is represented as follows:

[0201] ;

[0202] in, The action phase at the t-th decision moment includes at least the following actions: search, inspection, confirmation, follow, and return. The spatial target corresponding to the action phase is indicated, and the spatial target includes road area markers, candidate vehicle markers, confirmed target vehicle markers, or UAV waypoints. Indicates the confidence level of the stated strategy;

[0203] When the spatial target is a road area identifier, an observation waypoint is generated based on the location of the road area; when the spatial target is a candidate vehicle identifier and the action phase is inspection or confirmation, a corresponding waypoint is generated based on the location of the candidate vehicle; when the spatial target is a confirmed target vehicle identifier and the action phase is follow, a follow waypoint is generated based on the location of the confirmed target vehicle; when the spatial target is a UAV waypoint, the UAV waypoint is determined as the UAV's to-be-executed waypoint. Thus, the reasoning results of the visual language action strategy model are converted into waypoints or action commands that the flight controller can parse.

[0204] The visual language action policy model generates structured text fields corresponding to action stages and spatial targets through autoregression using the original output head of the language model. Policy confidence is calculated based on the probability of the actual output tokens corresponding to the action stages and spatial targets. Spatial targets are categorized by action stage in the following form:

[0205]

[0206] In this embodiment, the action phase, spatial target, and policy confidence are all generated as structured text fields from the original output header of the language model. The policy confidence is calculated by the system based on the probability of the actual output token corresponding to the structured text field. The system parses the structured text according to the field name and preset delimiter, and converts the parsing result into road area, candidate vehicle, confirmed target vehicle, or UAV waypoint.

[0207] Among them, when For inspection or confirmation, the vehicle identifier represents a candidate vehicle identifier; when When following, the vehicle identifier indicates that the target vehicle identifier has been confirmed; when When returning, This indicates the UAV's direct waypoint, which includes its position coordinates and heading angle. Road area identifiers are used to indicate the road area to be searched.

[0208] To clarify the confidence level of the strategy The actual generation method will be the action phase. With space targets The field values ​​are concatenated in a preset order to obtain the output token sequence:

[0209]

[0210] in, This represents the length of the token sequence. Let the model select candidate tokens at position j. The output Logit is After calibration using the temperature parameter T>0 determined by the validation set, the probability of the actual output token is:

[0211]

[0212] in, The model vocabulary is represented. Policy confidence is calculated using the following formula:

[0213]

[0214] The ∈(0,1] represents the normalized sequence confidence of the combination of the current action stage and the spatial target. The larger the value, the higher the model's support for the structured relocalization decision. The token sequence does not contain... This is to avoid circular definitions of policy confidence. The system parses according to field name and preset delimiter. and And calculate according to the above method This leads to a structured relocation decision; subsequently, the spatial target is converted into road areas, candidate vehicles, confirmed target vehicles, or UAV waypoints.

[0215] The aforementioned strategy confidence is calculated based on the output probability of the token corresponding to the action stage and the spatial target according to the visual language action strategy model. It is used as a confidence field in the structured relocation decision to characterize the credibility of the combination of the current action stage and the spatial target.

[0216] When the spatial target is a road area identifier, query the road map to obtain the road segment centerline or area centroid, and generate an observation waypoint:

[0217]

[0218] in, This indicates the waypoints generated based on the road area. and This indicates the coordinates of the center of the road segment or the centroid of the road area on the horizontal plane of the world coordinate system. Indicates the flight altitude of the drone when performing a search mission. This indicates the heading angle of the UAV at the observation waypoint. The observation waypoint is used to enable the UAV to reach above or near the road area and obtain effective observation of the target vehicle.

[0219] The search flight altitude and follow flight altitude mentioned above represent absolute coordinate values ​​in the vertical direction of the world coordinate system, and do not represent relative altitudes relative to the road surface.

[0220] When the action phase is inspection or confirmation and the spatial target is a candidate vehicle identifier, or when the action phase is following and the spatial target is a confirmed target vehicle identifier, corresponding waypoints are generated based on the world coordinates and heading of the corresponding vehicle:

[0221]

[0222] in, and This represents the position coordinates of the corresponding vehicle in the horizontal plane of the world coordinate system. This represents the heading angle of the corresponding vehicle in the world coordinate system. This indicates the distance between the drone and the corresponding vehicle at the rear. , and These represent the position coordinates of the corresponding waypoints of the UAV. This indicates the flight altitude of the UAV when performing candidate vehicle observation or following tasks; in this embodiment, inspection, confirmation, and following all use the same target vehicle observation altitude. The coordinate calculations above use the x-axis and y-axis of the world coordinate system as the horizontal coordinate axes, and... The calculation starts from the positive x-axis direction; where the distance in the corresponding vehicle's driving direction is subtracted to set the UAV waypoint behind the corresponding vehicle.

[0223] The waypoints followed in this implementation are ( , , This indicates that the heading angle is not an independent output of the waypoint, but is determined by the flight controller based on the relative direction between the UAV's current position and the target vehicle's position.

[0224] When the action phase is return, if a search waypoint saved from the previous round exists, it is used as the waypoint to be executed by the UAV; if no search waypoint saved from the previous round exists, the return point specified in the mission command is used as the waypoint to be executed by the UAV. If no return point is specified, a preset return point is invoked. Thus, the visual language action strategy model is responsible for high-level relocation decisions, and the deterministic waypoint generation module is responsible for converting road area markers, candidate vehicle markers, or confirmed target vehicle markers into waypoints to be executed by the UAV according to the action phase. After the safety constraints are checked as described in S107, the waypoints are then handed over to the flight controller for execution.

[0225] S107 performs a safety constraint check on the waypoint to be executed by the UAV and executes the corresponding action based on the check results.

[0226] like Figure 3As shown, the safety execution and feedback module first converts the spatial target in the structured relocation decision into a waypoint to be executed by the UAV, and then checks the waypoint's accessibility, flight altitude, obstacle distance, and local collision risk.

[0227] When the UAV's intended waypoint meets the safety constraints, the safety execution and feedback module sends the corresponding waypoint or action command to the flight controller, controlling the UAV to perform search, inspection, confirmation, follow, or return actions. When the UAV's intended waypoint does not meet the safety constraints, the safety execution and feedback module refuses to execute the UAV's intended waypoint and controls the system to replan the UAV's intended waypoint or maintain the current search state.

[0228] S108 updates the system based on road observation results, candidate vehicle authentication results, and action execution results.

[0229] When a candidate vehicle is authenticated and confirmed as the target vehicle, the system treats the candidate vehicle as a new verified target, updates the last verified target cues, and controls the drone to resume following the target vehicle.

[0230] When no authenticated target vehicle is found in the most recent complete observation of the candidate road area, the system updates the observation status and search priority of the corresponding road segment in the local road context; when the candidate vehicle fails to pass authentication, the system updates the candidate verification result, but does not update the last verified target clue; when the UAV waypoint fails the safety constraint check, the system records the safety check failure result and uses the result for the generation of the next round of UAV waypoints.

[0231] The updated historical target cues, local road context, candidate validation results, and new observations and action execution results obtained by the UAV together constitute the input to the next round of the visual-language action strategy model. For example... Figure 2 and Figure 3 As shown, through the above feedback updates, a closed loop is formed, which includes multi-source input, target loss judgment, road constraint search, candidate vehicle verification, structured relocation decision, safety action execution, and state update, until the target vehicle is reconfirmed and following resumes, or a return action is executed according to the task instructions.

[0232] Example 2

[0233] A UAV dynamic target relocation system, used to implement the UAV dynamic target relocation method of Embodiment 1, includes:

[0234] The multi-source input module is used to acquire mission instructions, current UAV images, UAV status, historical target clues, local road context, and candidate verification results;

[0235] The target loss judgment module is used to determine whether the target vehicle is in a verified visible state based on the current image of the UAV, and output the target loss status when the target vehicle is not in a verified visible state and the target loss time exceeds a preset loss tolerance threshold.

[0236] The historical target clue management module is used to save the last verified target clue, update the uncertain range of the target vehicle's location based on the target loss time, and prohibit unauthenticated candidate vehicles from replacing the last verified target clue.

[0237] The road constraint search module is used to determine candidate road segments based on the last verified target clue and the local road context, determine the search confidence of each candidate road segment based on target motion consistency, road drivability constraints, recent observation state weights and unvisited road branch rewards, and determine candidate road regions based on the search confidence.

[0238] The candidate vehicle verification module is used to determine the identity score of candidate vehicles based on their visible quality, attribute consistency, motion consistency, spatial consistency, and appearance re-identification similarity. When there are two or more candidate vehicles that meet the detection conditions, the module confirms whether the optimal candidate vehicle is the target vehicle based on its candidate vehicle identity score and the score difference between the optimal and second-best candidate vehicles. When there is only one candidate vehicle that meets the detection conditions, the module confirms whether the candidate vehicle is the target vehicle based solely on its identity score and the identity confirmation threshold.

[0239] A visual language action structured decision module is used to output a structured relocation decision including action phase, spatial target, and policy confidence based on the task instruction, the current image of the UAV, the UAV state, the historical target cues, the local road context, and the candidate verification results; and to generate a UAV waypoint to be executed based on the spatial target; and

[0240] The safety execution and feedback module is used to perform safety constraint checks on the waypoints to be executed by the UAV, control the UAV to execute corresponding actions when the waypoints to be executed by the UAV meet the safety constraints, replan or maintain the search state when the waypoints to be executed by the UAV do not meet the safety constraints, and update the last verified target clue, the local road context, the candidate verification results, and the input of the visual language action structured decision module for the next round based on the road observation results, candidate vehicle authentication results, and action execution results.

[0241] Example 3

[0242] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a dynamic target relocation method for a UAV as described in Embodiment 1.

[0243] Example 4

[0244] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a dynamic target relocation method for unmanned aerial vehicles as described in Example 1.

[0245] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0246] This invention is described with reference to flowchart illustrations and / or block diagrams of methods (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0247] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0248] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0249] Those skilled in the art will understand that all or part of the steps in the above method can be implemented by a program instructing related hardware. The program or the program described therein can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: at this time, the corresponding method steps are introduced. The storage medium can be ROM / RAM, magnetic disk, optical disk, etc.

[0250] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dynamic target relocation of an unmanned aerial vehicle (UAV), characterized in that, include: Acquire mission instructions, current UAV image, UAV status, historical target clues, local road context, and candidate verification results; The drone determines whether the target vehicle is in a verified visible state based on the current image of the drone. If the target vehicle is not in a verified visible state and the target loss time exceeds a preset loss tolerance threshold, a dynamic target relocation process is triggered. In response to triggering the dynamic target relocation process, the last verified target clue is maintained, and the uncertain range of the target vehicle's location is updated according to the target loss time, wherein unauthenticated candidate vehicles are not used to replace the last verified target clue; Candidate road segments are determined based on the last verified target clue and the local road context. The search confidence of each candidate road segment is determined based on the motion consistency between the candidate road segment and the target vehicle, the road drivability constraints of the candidate road segment, the weight of recent observation state, and the reward for unvisited road branches. Candidate road areas that meet the safe reachability conditions of the UAV are determined based on the search confidence. The drone is controlled to search the candidate road area and to verify the identity of candidate vehicles detected during the search. The identity verification includes determining the identity score of the candidate vehicle based on the visibility quality, attribute consistency, motion consistency, spatial consistency and appearance re-identification similarity of the candidate vehicle, and selecting the corresponding identity confirmation conditions according to the number of candidate vehicles to confirm whether the candidate vehicle is the target vehicle. The task instruction, the current image of the UAV, the UAV status, the historical target clues, the local road context, and the candidate verification results are input into the visual language action strategy model to obtain a structured relocation decision including action phase, spatial target, and policy confidence, and a waypoint to be executed by the UAV is generated based on the spatial target. The system performs a safety constraint check on the waypoints to be executed by the UAV. If the waypoints meet the safety constraints, the system controls the UAV to execute the corresponding actions. If the waypoints do not meet the safety constraints, the system refuses to execute the waypoints and either re-plans or maintains the search state. Based on the road observation results, candidate vehicle authentication results, and action execution results, update the last verified target clue, the local road context, the candidate verification results, and the input of the visual language action strategy model for the next round.

2. The UAV dynamic target relocation method according to claim 1, characterized in that, The verified visibility flag is used to indicate whether the target vehicle is in a verified visible state, and the target loss condition is represented as follows: in, This indicates the verified visibility flag at the current moment. This indicates that the target vehicle is not visible or its identity is ambiguous. Indicates the current moment. Indicates the time of the last verified observation. This represents the preset loss tolerance threshold; The last verified target clue is represented as follows: in, Indicates the target category. This indicates the last verified target appearance description. This indicates that the target location has been verified for the last time. Indicates target status information. This indicates that the target speed has been verified for the last time. This indicates that the target course has been verified for the last time. Indicates the radius of positional uncertainty at the time of the last verified observation; When the target vehicle is in a verified visible state, the last verified target clue is updated based on the current observation results; when the target vehicle is not in a verified visible state, the last verified target clue remains unchanged.

3. The UAV dynamic target relocation method according to claim 2, characterized in that, Update the uncertain range of the target vehicle's location based on the target loss time, including: The radius of the uncertain position at the current moment is calculated using the following formula: in, The radius represents the current position at an uncertain time. Indicates the growth factor of the uncertain radius; Based on the current position of the UAV, the last verified target position, the last verified target heading, the last verified target speed, the current position uncertainty radius, and the local road topology, a set of candidate road segments is determined.

4. The UAV dynamic target relocation method according to claim 3, characterized in that, The search confidence of the candidate road segment is determined according to the following formula: ； in, Indicates candidate road segments Search confidence, Represents the normalization coefficient. This indicates that the road segment is related to the last verified target clue. Consistency of the extrapolated direction of motion This indicates road connectivity and vehicle mobility constraints. Indicates the weight of recently observed states. This indicates the reward for a candidate road segment that is not a visited road branch; For the set of non-empty candidate road segments The search confidence of each candidate road segment satisfies: ； Specifically, the motion consistency is determined based on the deviation between the road direction of the candidate road segment and the last verified target heading, as well as the distance from the extrapolated target position to the candidate road segment; when the candidate road segment has been fully observed recently and the target vehicle has not been found, the corresponding recent observation state weight is reduced and then restored over time; the search priority of unvisited road branches is increased through the reward item.

5. The UAV dynamic target relocation method according to claim 4, characterized in that, For the first detected during the search process For each candidate vehicle, the candidate vehicle's identity score is determined according to the following formula: ； in, Indicates the first Candidate vehicle identity score for each candidate vehicle. This represents the visible mass of the candidate vehicle. This indicates the consistency of category and appearance attributes between the candidate vehicle and the target vehicle. This indicates the motion consistency between the candidate vehicle and the last verified target clue. This indicates the spatial consistency between the road area where the candidate vehicle is located and the candidate road area. This represents the appearance re-identification similarity between the candidate vehicle and the last verified target appearance description. , , , and Let represent the corresponding non-negative weights, and satisfy: .

6. The UAV dynamic target relocation method according to claim 5, characterized in that, Based on the candidate vehicle identity score of the optimal candidate vehicle, confirm whether a candidate vehicle is the target vehicle, including: when there are two or more candidate vehicles that meet the detection conditions, sort the candidate vehicles from high to low according to their candidate vehicle identity scores, and confirm the optimal candidate vehicle as the target vehicle when the following conditions are met simultaneously: ； in, This represents the candidate vehicle identity score of the optimal candidate vehicle. This represents the candidate vehicle identity score of the second-best candidate vehicle. Indicates the identity verification threshold. This represents the threshold for the identity scoring interval between candidate vehicles; When there is only one candidate vehicle that meets the detection conditions in the current candidate road area, the identity scoring interval condition between candidate vehicles does not apply. Only it is determined whether the identity score of the candidate vehicle is not lower than the identity confirmation threshold. When the corresponding confirmation condition is not met, the last verified target clue is not updated, and the UAV is controlled to continue searching, adjust the observation waypoint, or re-detect candidate vehicles in the current image of the UAV acquired in the subsequent period.

7. The UAV dynamic target relocation method according to claim 6, characterized in that, The input representation of the visual language action strategy model is as follows: ； in, This indicates the task instruction. This indicates the current image of the drone. Indicates the status of the drone. This indicates historical target clues up to the last verified observation time. This represents the local road context. This indicates the candidate verification result; The drone's status is represented as follows: ； in, , , and These represent the position, speed, heading, and altitude of the UAV, respectively. When no candidate vehicle is detected in the current candidate road area, the candidate verification result is an empty set or an unconfirmed status mark; when a candidate vehicle is detected, the candidate verification result includes the identity score and identity confirmation result of the candidate vehicle; when two or more candidate vehicles are detected, the candidate verification result also includes the score interval information between the best candidate vehicle and the second best candidate vehicle.

8. The UAV dynamic target relocation method according to claim 7, characterized in that, The structured relocation decision is represented as follows: ； in, The action phase refers to the action phase, which includes at least the following steps: search, inspection, confirmation, follow, and return. The spatial target is indicated by road area markers, candidate vehicle markers, confirmed target vehicle markers, or UAV waypoints. Indicates the confidence level of the stated strategy; When the spatial target is a road area identifier, an observation waypoint is generated based on the location of the road area; when the spatial target is a candidate vehicle identifier and the action phase is inspection or confirmation, a corresponding waypoint is generated based on the location of the candidate vehicle; when the spatial target is a confirmed target vehicle identifier and the action phase is follow, a follow waypoint is generated based on the location of the confirmed target vehicle; when the spatial target is a UAV waypoint, the UAV waypoint is determined as the UAV's to-execute waypoint.

9. The UAV dynamic target relocation method according to claim 8, characterized in that, The safety constraint check for the waypoint to be executed by the UAV includes: Based on the flight altitude, obstacle distance, waypoint accessibility, and local collision risk of the waypoint to be executed by the UAV, determine whether the waypoint to be executed by the UAV meets the safety constraints. When the waypoint to be executed by the UAV does not meet the safety constraints, the safety check failure result is fed back to the visual language action strategy model, and the waypoint to be executed by the UAV is regenerated or the search state is maintained. When a candidate vehicle is confirmed as the target vehicle, the confirmed candidate vehicle is used as a new verified target, the last verified target clue is updated and following is resumed; the results of the most recent complete observation that no verified target vehicle was found, the candidate vehicle failed to be verified, and the UAV waypoint failed the security constraint check are used to update the local road context, the candidate verification results, and the input of the visual language action strategy model for the next round.

10. A dynamic target relocation system for unmanned aerial vehicles (UAVs), characterized in that, include: The multi-source input module is used to acquire mission instructions, current UAV images, UAV status, historical target clues, local road context, and candidate verification results; The target loss judgment module is used to determine whether the target vehicle is in a verified visible state based on the current image of the UAV, and output the target loss status when the target vehicle is not in a verified visible state and the target loss time exceeds a preset loss tolerance threshold. The historical target clue management module is used to save the last verified target clue, update the uncertain range of the target vehicle's location based on the target loss time, and prohibit unauthenticated candidate vehicles from replacing the last verified target clue; The road constraint search module is used to determine candidate road segments based on the last verified target clue and the local road context, determine the search confidence of each candidate road segment based on target motion consistency, road drivability constraints, recent observation state weights and unvisited road branch rewards, and determine candidate road regions based on the search confidence. The candidate vehicle verification module is used to determine the identity score of the candidate vehicle based on the visible quality, attribute consistency, motion consistency, spatial consistency and appearance re-identification similarity of the candidate vehicle; when there are two or more candidate vehicles that meet the detection conditions, it confirms whether the optimal candidate vehicle is the target vehicle based on the candidate vehicle identity score of the optimal candidate vehicle and the score difference between the optimal candidate vehicle and the second-best candidate vehicle. When there is only one candidate vehicle that meets the detection conditions, the candidate vehicle is confirmed as the target vehicle based solely on its identity score and identity confirmation threshold. The visual language action structured decision module is used to output a structured relocation decision including action stage, spatial target and policy confidence based on the task instruction, the current image of the UAV, the state of the UAV, the historical target clues, the local road context and the candidate verification results, and generate the UAV waypoint to be executed based on the spatial target; The safety execution and feedback module is used to perform safety constraint checks on the waypoints to be executed by the UAV, control the UAV to execute corresponding actions when the waypoints to be executed by the UAV meet the safety constraints, replan or maintain the search state when the waypoints to be executed by the UAV do not meet the safety constraints, and update the last verified target clue, the local road context, the candidate verification results, and the input of the visual language action structured decision module for the next round based on the road observation results, candidate vehicle authentication results, and action execution results.

Images