Control method and system for tracking a turning vehicle by a vehicle-mounted unmanned aerial vehicle

Abstract

The application discloses a kind of vehicle-mounted unmanned aerial vehicle tracking turning vehicle control method and system, by fusing the state estimation value of target vehicle of visual detection information and V2X communication information of target vehicle;Predict the trajectory of target vehicle in prediction time domain based on the state estimation value of target vehicle, obtain target prediction position sequence;With camera field of view constraint as constraint condition, minimizing unmanned aerial vehicle motion cost and field of view center error as objective function to build optimization model;The optimization model is solved, and the unmanned aerial vehicle expected pose sequence that target prediction position sequence in prediction time domain is located in the safe region of camera field of view is obtained;Control unmanned aerial vehicle to fly according to the unmanned aerial vehicle expected pose sequence.The application can reduce tracking time sequence lag by fusing information to predict trajectory, build optimization model and solve, ensure that target vehicle is always located in the safe region of camera field of view during turning process.

Description

Technical Field

[0001] This invention belongs to the field of vehicle-mounted unmanned aerial vehicle (UAV) control technology, specifically relating to a control method and system for a vehicle-mounted UAV to track turning vehicles. Background Technology

[0002] Vehicle-mounted drone tracking systems typically consist of a drone, a vehicle-mounted mobile base station, and the target vehicle. Existing technologies mostly employ visual tracking based on image sensors combined with GPS / BeiDou positioning tracking. However, when the tracked vehicle makes a sharp turn, two major technical challenges arise: First, tracking has a certain lag. The drone follows the target based on its current or past position, and the generation and execution of its control commands have inherent delays. When the target's trajectory curvature changes abruptly, such as in some turning situations, the drone, due to inertia, cannot match the turning action in time, causing the target to move out of the drone's camera's field of view. Second, traditional position tracking (such as maintaining relative distance) does not use the camera's field of view as a hard constraint for active control planning. Once the target's relative position exceeds the field of view, even if the physical distance is very close, the vehicle will not be displayed in the image.

[0003] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the aforementioned background technology and provide a control method, system, and computer program product for tracking turning vehicles using a vehicle-mounted unmanned aerial vehicle (UAV), which can reduce tracking time lag and ensure that the target vehicle remains within the safe area of ​​the camera's field of view during the turning process.

[0005] The technical solution adopted in this invention is: a control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle, comprising the following steps: The target vehicle's state estimate is obtained by fusing visual detection information and V2X communication information. Based on the state estimate of the target vehicle, a hybrid prediction model is used to predict the trajectory of the target vehicle in the prediction time domain, and the target predicted position sequence is obtained. An optimization model is constructed with camera field of view constraints as the constraints and minimizing the UAV motion cost and field of view center error as the objective function. Solving the optimization model yields the UAV's desired pose sequence that satisfies the condition that the predicted target position sequence is within the camera's safe field of view in the prediction time domain. Control the drone to fly according to the desired pose sequence of the drone.

[0006] Furthermore, the coordinate system includes a fixed world coordinate system W, a UAV body coordinate system B with its origin at the UAV's centroid, and a camera coordinate system C fixed to the UAV. The optical axis of the camera coordinate system C forms a fixed angle with the X-axis of the UAV body coordinate system B.

[0007] Furthermore, the fusion of the visual detection information and V2X communication information of the target vehicle to obtain the state estimate of the target vehicle includes: The state vector of the target vehicle is recursively estimated by the extended Kalman filter algorithm, and the optimal state estimate is output. The prediction step of the extended Kalman filter algorithm uses a constant rotation rate and velocity model to predict the prediction state vector at the current moment. The update step of the extended Kalman filter algorithm uses the V2X communication information as a direct observation and the visual detection information as a nonlinear observation to correct the predicted state vector and obtain the optimal state estimate at the current time.

[0008] Furthermore, the visual detection information includes the azimuth and pitch angles of the target vehicle relative to the UAV, calculated from images from the UAV's onboard camera; the V2X communication information includes the target vehicle's own GNSS position, speed, and heading angle broadcast by the vehicle itself.

[0009] Furthermore, the target vehicle's trajectory in the prediction time domain is predicted using a hybrid prediction model based on the target vehicle's state estimate, resulting in a target predicted position sequence, including: The first predicted position sequence of the target vehicle is predicted using a linear prediction model. ; The second predicted position sequence of the target vehicle is predicted using a circular arc prediction model. ; Confidence weights are determined based on the heading angle in the state estimate of the target vehicle. Based on the confidence weight, the first predicted position sequence Second predicted position sequence The target predicted location sequence is obtained by performing weighted fusion.

[0010] Furthermore, the target predicted location sequence is determined using the following formula: ; ; Among them, P pred (k+1|t) represents the predicted target position of the target vehicle at the time corresponding to preview point k+1; w is the confidence weight. The first predicted position of the target vehicle at the time corresponding to preview point k+1; Let σ be the second predicted position of the target vehicle at the time corresponding to preview point k+1; σ is the adjustment parameter. t represents the rate of change of heading angle in the state estimate; t represents the current time; and k represents the prediction step index.

[0011] Furthermore, the constraint conditions are as follows: For any preview point k in the prediction time domain, the target vehicle is in the desired pose X of the UAV relative to the UAV. d The azimuth angle α(k|t) and elevation angle β(k|t) satisfy the following inequality: ; ; Among them, [p dx (k|t), p dy (k|t), p dz (k|t)] T These represent the x, y, and z components of the vector representing the target vehicle's position relative to the UAV's desired position at preview point k in the world coordinate system; ψ d (k|t) represents the expected yaw angle of the UAV at the corresponding moment of the preview point; θ h and θ v η represents the horizontal and vertical field of view of the camera, respectively. h and η v These are the safety factors for the horizontal and vertical fields of view, respectively.

[0012] Furthermore, the objective function is: ; Where, p d (k preview |t) represents the desired position of the UAV at the corresponding time at the baseline preview point; ψ d (k preview |t) represents the expected yaw angle of the UAV at the corresponding time at the baseline preview point; α(k preview |t), β(k) preview |t) represents the azimuth and pitch angles of the target vehicle relative to the desired pose of the UAV at the corresponding time at the reference preview point; a represents the acceleration of the UAV; λ1, λ2, and λ3 are the weighting coefficients of the azimuth, pitch, and acceleration, respectively.

[0013] Furthermore, solving the optimization model to obtain the UAV's desired pose sequence that satisfies the condition that the target predicted position sequence is within the camera's safe field of view in the prediction time domain includes: Set a baseline preview point within the prediction time domain; For the baseline preview point k previewThe camera field of view constraint is applied in several steps before and after, and the UAV at the reference preview point k is obtained through constrained nonlinear optimization. preview The optimal expected pose; Based on the benchmark preview point k preview The optimal desired pose is obtained, and a complete UAV desired pose sequence is generated by interpolation.

[0014] A control system for implementing the control method described above for tracking turning vehicles by an onboard drone includes: The state perception module is used to fuse the visual detection information and V2X communication information of the target vehicle to obtain the state estimate of the target vehicle; The trajectory prediction module is used to predict the trajectory of the target vehicle in the prediction time domain based on the state estimate of the target vehicle and a hybrid prediction model, so as to obtain the target predicted position sequence. The model building module is used to build an optimization model with camera field of view constraints as the constraints and minimizing the UAV motion cost and field of view center error as the objective functions. The trajectory planning module is used to solve the optimization model to obtain the UAV's expected pose sequence that satisfies the condition that the target predicted position sequence is within the camera's field of view in the prediction time domain. The flight control module is used to control the UAV to fly according to the desired pose sequence of the UAV.

[0015] A computer program product includes a computer program that, when run, implements the method described above for tracking turning vehicles by a vehicle-mounted drone.

[0016] The beneficial effects of this invention are as follows: This invention reduces tracking time lag by establishing a coordinate system, fusing information to predict trajectories, constructing and solving an optimization model, and ensuring that the target vehicle remains within the safe area of ​​the camera's field of view during turning, thereby improving the system's real-time performance and robustness. Attached Figure Description

[0017] Figure 1 This is a flowchart of the control method of the present invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments to facilitate a clear understanding of the present invention, but these descriptions do not constitute a limitation on the present invention.

[0019] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0020] Furthermore, references to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in still other embodiments" appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0021] like Figure 1 As shown, this application proposes a control method for tracking a turning vehicle using a vehicle-mounted unmanned aerial vehicle (UAV), comprising the following steps: Establish a coordinate system; In the aforementioned coordinate system, the state estimate of the target vehicle is obtained by fusing the visual detection information and V2X communication information of the target vehicle through extended Kalman filtering. Based on the state estimate of the target vehicle, a hybrid prediction model is used to predict the trajectory of the target vehicle in the prediction time domain, and the target predicted position sequence is obtained. An optimization model is constructed with camera field of view constraints as the constraints and minimizing the UAV motion cost and field of view center error as the objective function. Solving the optimization model yields the UAV's desired pose sequence that satisfies the condition that the predicted target position sequence is within the camera's safe field of view in the prediction time domain. Control the drone to fly according to the desired pose sequence of the drone.

[0022] For ease of understanding, the following explains some key terms in this embodiment: A coordinate system refers to a reference frame used to describe the spatial position and attitude of a drone, target vehicle, and its environment. It can include one or more interrelated coordinate systems, such as the world coordinate system, the drone's body coordinate system, and the camera coordinate system, used to uniformly represent data from different sensors and motion states.

[0023] Extended Kalman Filter (EPF) is a nonlinear filtering algorithm used to estimate the state of nonlinear systems. This algorithm linearizes the nonlinear function, approximating the nonlinear problem as a linear one, thereby achieving a recursive estimate of the target vehicle's state by fusing data from multiple sensors. In the prediction phase, the filter uses the system's dynamic model to predict the state and state covariance at the next time step. In the update phase, the filter uses new measurements to correct the predicted state and covariance, thus obtaining the optimal state estimate for the current time step. This recursive estimation characteristic allows the filter to process continuous sensor data streams in real time and continuously optimize its understanding of the target vehicle's state.

[0024] Visual inspection information refers to the observation data of the target vehicle relative to the drone, obtained by processing image data acquired through the onboard camera of the drone and solving it using image processing and computer vision algorithms. This data may include information such as the target vehicle's azimuth and pitch angles.

[0025] V2X communication refers to communication technologies that enable vehicles to interact with the outside world, including various modes such as vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), vehicle-to-pedestrian (V2P), and vehicle-to-network (V2N). In this embodiment, it specifically refers to the target vehicle broadcasting its own GNSS position, speed, and heading angle via wireless communication.

[0026] The state estimate refers to the real-time estimation of the motion state of a target vehicle using a data fusion algorithm. This estimate integrates information from multiple sensors and has high accuracy and robustness.

[0027] Hybrid prediction models refer to vehicle trajectory prediction models that combine multiple prediction mechanisms. They can use different kinematic or dynamic models to predict the trajectory based on the current motion state and historical data of the target vehicle, in order to adapt to the vehicle's motion characteristics under different operating conditions, such as straight-line driving and turning.

[0028] The prediction time domain refers to a period of time extending into the future from the current moment during trajectory prediction. Within this time domain, the prediction model will generate a series of discrete target location points at future moments, forming a sequence of predicted target locations.

[0029] The target predicted position sequence refers to the set of expected positions of the target vehicle at a series of discrete moments in the future, predicted by a hybrid prediction model within the prediction time domain. This sequence reflects the trajectory of the target vehicle over a period of time.

[0030] Camera field-of-view constraint refers to the limitation of the field of view that the drone's onboard camera can effectively capture the target vehicle. This constraint ensures that the target vehicle remains within the camera's effective shooting area throughout the drone's flight, preventing the target from moving out of the frame.

[0031] The motion cost of a drone refers to the energy consumption, changes in control input, or attitude changes incurred by the drone when performing a tracking task. In optimization models, minimizing the motion cost helps achieve smooth and efficient drone flight.

[0032] Field of view center error refers to the deviation between the target vehicle's position in the camera's field of view and the camera's field of view center. Minimizing this error helps keep the target vehicle centered in the frame, improving image quality.

[0033] An optimization model is a mathematical model that includes an objective function, constraints, and decision variables. By solving this model, an optimal set of decision variables can be found that minimizes or maximizes the objective function while satisfying all constraints.

[0034] The desired pose sequence of a UAV refers to the set of ideal positions and attitudes of the UAV at a series of discrete future moments, obtained by solving an optimization model within the prediction time domain. The UAV will perform flight control based on this sequence.

[0035] This application accurately estimates the state of the target vehicle by fusing multi-source information and predicts its future trajectory based on a hybrid prediction model. It also constructs an optimization model for trajectory planning by combining camera field-of-view constraints and motion costs, ensuring that the UAV can match the turning action of the target vehicle in a timely manner, preventing the target from moving out of the camera's field of view. Furthermore, it actively uses the camera's field of view as a mandatory condition for control planning, ensuring that the target vehicle remains within the camera's view.

[0036] The following details the implementation steps of the control method of this application: S1. Establish a coordinate system, including: World coordinate system W: Fixed to the ground, Northeast-Ultra-Upper (ENU) coordinate system. This coordinate system is a global, fixed reference frame whose origin and axes remain unchanged throughout the mission. It is typically used to describe the absolute position and attitude of the UAV and target vehicle. Data from all other local coordinate systems (such as the UAV body coordinate system B and the camera coordinate system C) are ultimately transformed to or from the world coordinate system W to ensure the consistency of global spatial information.

[0037] UAV body coordinate system B: The origin is at the UAV's center of mass, X B Forward, Y B To the right, Z B Downward. This coordinate system is a local coordinate system that moves with the UAV and is mainly used to describe the UAV's own kinematics and dynamics, as well as the mounting position and attitude of the onboard sensors relative to the UAV body.

[0038] Camera coordinate system C: Assuming it is fixed to the UAV, the optical axis is parallel to the X-axis. B The axes coincide (or a known fixed angle can be processed through coordinate transformation). This coordinate system is a local coordinate system fixed to the UAV, used to describe the viewpoint and direction when the camera captures images. By explicitly defining a fixed angle between the optical axis of the camera coordinate system C and the X-axis of the UAV body coordinate system B, the transformation relationship between camera attitude and UAV attitude is simplified. This allows target information (such as azimuth and pitch angles) calculated from camera images to be accurately mapped to the UAV body coordinate system B, and then transformed to the world coordinate system W for unified processing.

[0039] Target vehicle: Simplified to a centroid point, its state vector is defined as its position, velocity, and heading angle in the world coordinate system: =[p x ,p y ,v,ψ] T , represents the state vector of the target vehicle in the world coordinate system. Where p x p is the x-coordinate of the target vehicle in the world coordinate system (usually its eastward position); y y is the target vehicle's y-coordinate (northward position) in the world coordinate system; v is the target vehicle's velocity (scalar, assumed to be the velocity in the forward direction); ψ is the target vehicle's heading angle (usually the angle of counterclockwise rotation from the north direction to the vehicle's forward direction, or the angle of counterclockwise rotation from the east direction to the vehicle's forward direction, depending on the coordinate system definition; here it is assumed to be the angle of counterclockwise rotation from the east direction, which is consistent with the conventional ENU coordinate system).

[0040] This application clearly defines the form of each coordinate system, providing a clear, unified and stable spatial reference framework for the control method of vehicle-mounted UAV tracking turning vehicles. This is crucial for subsequent extended Kalman filter fusion, target trajectory prediction and solving the UAV's desired pose sequence, ensuring the accuracy and robustness of the entire control method in spatial positioning and motion planning, and improving the accuracy and stability of UAV tracking turning vehicles.

[0041] S2, Multi-source target state perception and fusion.

[0042] The extended Kalman filter (EKF) algorithm is used to fuse information from two asynchronous sources: Visual detection information: From the images acquired by the UAV's onboard camera, the pixel coordinates (u,v) of the target vehicle in the image are obtained through detection algorithms such as YOLO. Combined with the UAV's own pose, camera intrinsic parameters and altitude, the azimuth angle α and pitch angle β of the target vehicle relative to the UAV are calculated.

[0043] V2X communication information: Through vehicle-to-everything (V2X) technology, directly receive the target vehicle's broadcast high-precision GNSS position. The information includes velocity v and heading angle ψ. This information is unaffected by visual obstruction and is highly reliable.

[0044] This application integrates information from two different sources with different characteristics, effectively combining the real-time and relative nature of visual information with the precision and absoluteness of V2X information. This improves the accuracy and robustness of target vehicle state estimation, thereby providing more reliable basic data for subsequent trajectory prediction and UAV trajectory planning, ensuring that the UAV can track turning vehicles more stably and accurately.

[0045] The fusion process is as follows: the state vector of the target vehicle is recursively estimated using the extended Kalman filter algorithm, and the optimal state estimate is output; the state vector of the algorithm is... The prediction step uses a constant rev and velocity (CTRV) model to predict the predicted state vector at the current moment; the update step uses the V2X communication information as a direct observation and the visual detection information as a nonlinear observation to correct the predicted state vector and obtain the optimal state estimate at the current moment. .

[0046] This application improves the accuracy and robustness of target state estimation when a vehicle-mounted UAV tracks a turning vehicle. It employs a constant rev rate and velocity model, enabling the extended Kalman filter to more accurately simulate the dynamic behavior of the turning vehicle during the prediction phase. Simultaneously, it fuses V2X communication information as direct observations with visual detection information as nonlinear observations, fully utilizing the complementary characteristics of the two sensors. V2X information provides high-precision absolute state data, while visual information provides crucial relative angle information. Through the optimized fusion mechanism of the extended Kalman filter, sensor noise is effectively suppressed, improving the accuracy of state estimation. This accurate and real-time state estimation provides a solid foundation for subsequent trajectory prediction and UAV control, ensuring that the UAV can track turning vehicles more stably and accurately, thereby improving the overall performance and reliability of the tracking system.

[0047] S3. Target trajectory prediction: A hybrid prediction model is used to predict the trajectory of the target vehicle in the prediction time domain to adapt to different modes of vehicle straight driving and turning.

[0048] The first predicted position sequence of the target vehicle is predicted using a linear prediction model. ; The second predicted position sequence of the target vehicle is predicted using a circular arc prediction model. .

[0049] Linear prediction model (Model A): Applicable to straight lines or gentle curves, it has high prediction accuracy when the target vehicle is traveling in a straight line or a gentle curve. It assumes that the speed and heading angle are constant in the short term.

[0050] ; ; Where k∈(0,Np), and Np is the total number of prediction steps.

[0051] (p x (k+1|t), p y (k+1|t)) represents the predicted target position of the target vehicle at the time corresponding to preview point k+1; ( ) is the estimated position of the target vehicle at the current time (time t).

[0052] This is an estimate of the target vehicle's speed at the current moment.

[0053] This is the estimated heading angle of the target vehicle at the current moment.

[0054] ΔT: The time interval for predicting the step size (i.e., the time from the current moment to the next step).

[0055] Circular arc prediction model (Model B): Applicable to obvious turns, providing more accurate predictions when the target vehicle is turning. It assumes a constant turning radius R over a short period.

[0056] Turning radius can be determined by the rate of change of heading angle. Velocity calculation: R = v / .

[0057] ; .

[0058] Adaptive fusion: A confidence weight w∈[0,1] is designed to measure whether the target vehicle is more inclined to travel in a straight line or turn at the current moment. The confidence weight is determined based on the heading angle in the target vehicle's state estimate. The first predicted position sequence is then fused based on the confidence weight. Second predicted position sequence The predicted target location sequence is obtained by performing weighted fusion. The specific formula is as follows: ; ; Among them, P pred (k+1|t) represents the predicted target position of the target vehicle at the time corresponding to preview point k+1 (i.e., time t+(k+1)×ΔT); w is the confidence weight. The first predicted position of the target vehicle at the time corresponding to preview point k+1; Let w be the second predicted position of the target vehicle at the time corresponding to preview point k+1; σ is an adjustment parameter used to control the sensitivity of the weight to the rate of change of heading angle. The larger σ is, the faster the weight w increases for small changes in heading angle. t is the rate of change of heading angle in the state estimate; t is the current time; k is the prediction step index, k∈(1,Np), and Np is the total number of prediction steps.

[0059] When | | Approaching 0 (straight ahead), w→0, model A is dominant; when | | Increase (turn), w→1, with model B as the primary model. The final target position sequence in the prediction time domain is obtained: {P pred (1|t), ..., P pred (N p |t)}.

[0060] This application combines a linear prediction model and a circular prediction model, dynamically adjusting their weights based on the target vehicle's heading angle change rate, to generate more accurate and robust target vehicle trajectory predictions. When the vehicle is traveling in a straight line, the linear model plays a dominant role; when the vehicle is turning, the circular model takes over. Furthermore, in the transition region between straight and turning, the two models are weighted and fused to achieve a smooth transition. This adaptive hybrid prediction method improves the prediction accuracy for complex motion patterns (especially turning vehicles), providing more reliable input for subsequent UAV trajectory planning. This ensures that the UAV can track the target vehicle more stably and accurately, continuously keeping it within the camera's field of view, thus improving the overall performance and reliability of the tracking and control system.

[0061] S4. Desired UAV Pose Generation: The goal is to find a sequence of future UAV poses such that all predicted target vehicle positions fall within the safe area of ​​the camera's field of view (e.g., the rectangular box in the center of the image). This is achieved by constructing an optimization model with camera field of view constraints as the constraints and minimizing the UAV motion cost and the field of view center error as the objective function. The desired UAV pose is then obtained by solving the optimization model.

[0062] Define camera field of view constraints: Set the camera field of view angle (horizontal θ) as the constraint. h , vertical θ v This is mapped to a cone-shaped region centered on the optical axis in the world coordinate system. For any preview point k at any given time, the desired pose X of the UAV is... d (k|t)=[p d (k|t),ψ d (k|t)] T Target predicted location P pred (k|t) must satisfy the following inequality to ensure that at the time corresponding to preview point k, the azimuth angle α(k|t) (horizontal angle) and pitch angle β(k|t) (vertical angle) of the target relative to the expected pose of the UAV are both within the camera's field of view (with a certain safety margin): ; ; Among them, [p dx (k|t), p dy (k|t), p dz (k|t)] TThese represent the x, y, and z components of the vector representing the target vehicle's position relative to the UAV's desired position at preview point k in the world coordinate system; p d (k|t) represents the expected position of the UAV at the preview point k at the corresponding time; ψ d (k|t) represents the expected yaw angle of the UAV at the corresponding moment of the preview point; θ h and θ v η represents the horizontal and vertical field of view of the camera, respectively. h and η v These are the horizontal and vertical field-of-view safety factors, used to define a more stringent safety zone, such as 80% of the field of view, to allow for control margins.

[0063] The prediction time domain is the period of time that the UAV looks ahead during trajectory planning. This time domain is discretized into a series of time points (corresponding moments), each corresponding to a preview point k. At these preview points, the system predicts the position of the target vehicle and the desired pose of the UAV. Ensuring that the target vehicle can be stably tracked by the camera throughout the entire prediction time domain is crucial for achieving robust tracking.

[0064] This application restricts the azimuth and pitch angles of the target vehicle relative to the desired pose of the UAV to within the safe area of ​​the camera's field of view. This allows the optimization model to accurately assess whether the desired pose of the UAV ensures that the target vehicle remains within the camera's field of view and provides sufficient safety margin. Therefore, when the UAV performs trajectory planning, it can generate a more stable and robust sequence of desired poses, avoiding the problem of the target vehicle deviating from the camera's field of view or getting too close to the edge of the field of view during complex movements such as turning. This improves the continuity and reliability of the vehicle-mounted UAV in tracking turning vehicles.

[0065] The objective function is defined as minimizing the UAV motion cost and the field-of-view center error: ; Where, p d (k preview |t) represents the desired position of the UAV at the corresponding time at the baseline preview point; ψ d (k preview |t) represents the expected yaw angle of the UAV at the corresponding time at the baseline preview point; α(k preview |t), β(k) preview |t) represents the azimuth and pitch angles of the target vehicle relative to the desired pose of the UAV at the corresponding time at the reference preview point; a represents the acceleration of the UAV; λ1, λ2, and λ3 are the weighting coefficients of the azimuth, pitch, and acceleration, respectively.

[0066] This application optimizes the desired pose of the UAV by minimizing a comprehensive cost J. It quantifies and integrates multiple key performance indicators in the UAV tracking process, namely the field of view center error (reflected by azimuth and pitch angles) and the UAV motion stability (reflected by acceleration), into a single mathematical expression. By solving this objective function, the optimal desired pose of the UAV under the constraints can be obtained, thereby achieving stable and accurate tracking of the target vehicle.

[0067] Optimize the model solution: Set a baseline preview point k in the prediction time domain [1, Np]. preview ∈[1,Np] (e.g., N p / 2), apply the camera field of view constraint to the reference preview point and several steps before and after it, and obtain the optimal desired pose of the UAV at the reference preview point through constrained nonlinear optimization.

[0068] First, optimize the drone in k preview The expected position and yaw angle at the corresponding time moment ensure that the predicted target position at that moment lies in the image, while simultaneously satisfying the field-of-view constraints for the forward and subsequent steps. Also, for k=k... preview -m,...,k preview The +m step applies the aforementioned field-of-view constraint inequality. This is a constrained nonlinear optimization problem that can be solved quickly online using sequential quadratic programming (SQP).

[0069] This application sets a baseline preview point k. preview This approach decomposes complex global optimization problems into a combination of local optimization and global interpolation, effectively reducing computational complexity. A baseline preview point k was determined. preview Then, the focus of the optimization solution will be placed on the baseline preview point k. preview Within a small segment of the prediction time domain, and its vicinity, strict camera field-of-view constraints are applied to ensure the target vehicle remains within the safe field of view of the UAV's onboard camera. This allows the UAV to obtain the prediction at the reference preview point k. preview The optimal desired position and attitude (such as yaw angle, pitch angle, etc.) are determined at the location. This local fine-grained optimization strategy can reduce the amount of computation while ensuring tracking accuracy at critical moments.

[0070] After obtaining the optimal expected pose of the baseline preview point, the entire expected trajectory of the UAV {X} is generated by interpolation from the current time to the corresponding time of the baseline preview point, and then to the corresponding time of the predicted endpoint. d (0), ..., X d (N p Interpolation methods can include linear interpolation, cubic spline interpolation, or polynomial interpolation, etc.

[0071] This application sets a baseline preview point k within the prediction time domain. preview The constrained nonlinear optimization solution is then focused on the baseline preview point k. preview This, along with several preceding and following steps, significantly reduces the computational complexity of the optimization problem, thereby improving the efficiency and real-time performance of generating the desired UAV pose sequence. Based on this, and using the baseline preview point k... preview The optimal expected pose is obtained, and a complete sequence of expected UAV poses is generated through interpolation. This not only ensures the smoothness and continuity of the UAV trajectory, avoiding the computational burden and trajectory instability caused by frequent full-time-domain optimization, but also ensures that the target vehicle remains within the camera's safe field of view at critical prediction moments, effectively improving the stability and reliability of vehicle-mounted UAVs tracking turning vehicles. This strategy of combining step-by-step solution with interpolation enables the system to quickly respond and generate high-quality UAV tracking trajectories in dynamically changing tracking scenarios, thereby achieving continuous and stable tracking of the target vehicle.

[0072] S5. Call the flight control to make the drone fly along the desired position. Through this method, the drone can adjust its own trajectory in advance when the target vehicle turns, and always keep the target within the camera's field of view, thereby solving the problem of losing track.

[0073] In one embodiment, this application proposes a control system for implementing the above-described control method for tracking turning vehicles using a vehicle-mounted unmanned aerial vehicle. The control system includes a state perception module, a trajectory prediction module, a model building module, a trajectory planning module, and a flight control module.

[0074] The state perception module is used to obtain the state estimate of the target vehicle by fusing visual detection information and V2X communication information from the target vehicle through extended Kalman filtering. By fusing multi-source information, the state perception module improves the accuracy and robustness of state estimation, providing reliable input for trajectory prediction.

[0075] The trajectory prediction module uses a hybrid prediction model based on the target vehicle's state estimate to predict the target vehicle's trajectory in the prediction time domain, resulting in a predicted target position sequence. This prediction mechanism can adapt to both straight-line driving and turning conditions of the target vehicle, improving the accuracy of trajectory prediction.

[0076] The model building module is used to construct an optimization model with camera field-of-view constraints as the constraint condition and minimizing the UAV motion cost and field-of-view center error as the objective function. By treating the camera field-of-view constraint as a hard condition, this optimization model ensures that the target vehicle is always within the effective shooting area.

[0077] The trajectory planning module is used to solve the optimization model to obtain the UAV's expected pose sequence that satisfies the condition that the predicted position sequence of the target is within the safe area of ​​the camera's field of view in the prediction time domain.

[0078] The flight control module controls the UAV to fly according to its desired pose sequence. The module receives the desired pose sequence, generates control commands, and adjusts the UAV's motor speeds and attitude to enable the UAV to track the target vehicle in real time and keep it within the camera's field of view. Through closed-loop control, the system can dynamically respond to changes in the target vehicle's motion, ensuring the continuity and stability of the tracking process.

[0079] The control system of this application enables the UAV to respond to the turning action of the target vehicle in a timely manner. By actively planning the camera field of view constraint as a hard condition, it ensures that the target vehicle is always within the camera frame, which significantly improves the performance and reliability of the tracking system.

[0080] It should be noted that the information interaction and execution process between the above modules / units are based on the same concept as the method embodiments of this application. For details on their specific implementation process, functions, and technical effects, please refer to the method embodiments section, and they will not be repeated here. The descriptions of each embodiment have their own emphasis; parts not detailed or recorded in a certain embodiment can be referred to the relevant descriptions of other embodiments.

[0081] It is understood that those skilled in the art will clearly recognize that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0082] This application provides a computer program product, including a computer program, which, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0083] In applications, computer programs include computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable media can include at least: any entity or device capable of carrying computer program code to an electronic device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical discs. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0084] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0085] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. Content not described in detail in this specification belongs to the prior art known to those skilled in the art.

Claims

1. A control method for tracking turning vehicles using a vehicle-mounted unmanned aerial vehicle (UAV), characterized in that: The target vehicle's state estimate is obtained by fusing visual detection information and V2X communication information. Based on the state estimate of the target vehicle, a hybrid prediction model is used to predict the trajectory of the target vehicle in the prediction time domain, and the target predicted position sequence is obtained. An optimization model is constructed with camera field of view constraints as the constraints and minimizing the UAV motion cost and field of view center error as the objective function. Solving the optimization model yields the UAV's desired pose sequence that satisfies the condition that the predicted target position sequence is within the camera's safe field of view in the prediction time domain. Control the drone to fly according to the desired pose sequence of the drone.

2. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that, The fusion of visual detection information and V2X communication information of the target vehicle to obtain the state estimate of the target vehicle includes: The state vector of the target vehicle is recursively estimated by the extended Kalman filter algorithm, and the optimal state estimate is output. The prediction step of the extended Kalman filter algorithm uses a constant rotation rate and velocity model to predict the prediction state vector at the current moment. The update step of the extended Kalman filter algorithm uses the V2X communication information as a direct observation and the visual detection information as a nonlinear observation to correct the predicted state vector and obtain the optimal state estimate at the current time.

3. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that: The visual detection information includes the azimuth and pitch angles of the target vehicle relative to the UAV, calculated from images from the UAV's onboard camera; the V2X communication information includes the target vehicle's broadcast GNSS position, speed, and heading angle.

4. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that: The target vehicle's trajectory in the prediction time domain is predicted using a hybrid prediction model based on the target vehicle's state estimate, resulting in a target predicted position sequence, including: The first predicted position sequence of the target vehicle is predicted using a linear prediction model. ; The second predicted position sequence of the target vehicle is predicted using a circular arc prediction model. ; Confidence weights are determined based on the heading angle in the state estimate of the target vehicle. Based on the confidence weight, the first predicted position sequence Second predicted position sequence The target predicted location sequence is obtained by performing weighted fusion.

5. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 4, characterized in that: The target predicted location sequence is determined using the following formula: ； ； Among them, P pred (k+1|t) represents the predicted target position of the target vehicle at the time corresponding to preview point k+1; w is the confidence weight. The first predicted position of the target vehicle at the time corresponding to preview point k+1; Let σ be the second predicted position of the target vehicle at the time corresponding to preview point k+1; σ is the adjustment parameter. t represents the rate of change of heading angle in the state estimate; t represents the current time; and k represents the prediction step index.

6. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that: The constraints are as follows: For any preview point k in the prediction time domain, the target vehicle is in the desired pose X of the UAV relative to the UAV. d The azimuth angle α(k|t) and elevation angle β(k|t) satisfy the following inequality: ； ； Among them, [p dx (k|t), p dy (k|t), p dz (k|t)] T These represent the x, y, and z components of the vector representing the target vehicle's position relative to the UAV's desired position at preview point k in the world coordinate system; ψ d (k|t) represents the expected yaw angle of the UAV at the corresponding moment of the preview point; θ h and θ v η represents the horizontal and vertical field of view of the camera, respectively. h and η v These are the safety factors for the horizontal and vertical fields of view, respectively.

7. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that: The objective function is: ； Where, p d (k preview |t) represents the desired position of the UAV at the corresponding time at the baseline preview point; ψ d (k preview |t) represents the expected yaw angle of the UAV at the corresponding time at the baseline preview point; α(k preview |t), β(k) preview |t) represents the azimuth and pitch angles of the target vehicle relative to the desired pose of the UAV at the corresponding time at the reference preview point; a represents the acceleration of the UAV; λ1, λ2, and λ3 are the weighting coefficients of the azimuth, pitch, and acceleration, respectively.

8. The control method for tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that: Solving the optimization model yields the UAV's desired pose sequence that satisfies the condition that the predicted target position sequence is within the camera's safe field of view in the prediction time domain, including: Set a baseline preview point within the prediction time domain; The camera field of view constraint is applied to the reference preview point and several steps before and after it. The optimal desired pose of the UAV at the reference preview point is obtained by solving the constraint-bound nonlinear optimization. Based on the optimal desired pose of the reference preview point, a complete UAV desired pose sequence is generated by interpolation.

9. A control system for implementing the control method of tracking turning vehicles by a vehicle-mounted unmanned aerial vehicle as described in claim 1, characterized in that, include: The state perception module is used to fuse the visual detection information and V2X communication information of the target vehicle to obtain the state estimate of the target vehicle; The trajectory prediction module is used to predict the trajectory of the target vehicle in the prediction time domain based on the state estimate of the target vehicle and a hybrid prediction model, so as to obtain the target predicted position sequence. The model building module is used to build an optimization model with camera field of view constraints as the constraints and minimizing the UAV motion cost and field of view center error as the objective functions. The trajectory planning module is used to solve the optimization model to obtain the UAV's expected pose sequence that satisfies the condition that the target predicted position sequence is within the camera's field of view in the prediction time domain. The flight control module is used to control the UAV to fly according to the desired pose sequence of the UAV.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is run, it implements the method for tracking turning vehicles by a vehicle-mounted drone as described in any one of claims 1-8.

Images