Intelligent camera scheduling method and system based on UAV trajectory prediction

Abstract

The present invention provides an intelligent camera scheduling method and system based on unmanned aerial vehicle (UAV) trajectory prediction, which relates to the field of UAV monitoring technology. The method comprises generating a predicted trajectory set including a spatiotemporal probability distribution based on collected real-time UAV flight data in combination with a motion probability coupling model; generating a dynamic coverage model in combination with the predicted trajectory set according to collected camera parameters and environmental modeling data; dynamically generating target tracking priority weights based on the spatiotemporal probability distribution of the predicted trajectory set and the spatial overlap of a geographic fence area preset in the environmental modeling data; generating an optimal camera scheduling scheme through a multi-objective optimization method based on the spatial constraints and target tracking priority weights of the dynamic coverage model; converting the optimal camera scheduling scheme into camera control instructions, and performing closed-loop control based on the real-time deviation of the dynamic coverage model. The present invention realizes intelligent collaborative scheduling and target tracking of a UAV monitoring system.

Description

Technical Field

[0001] The present invention relates to the field of drone monitoring technology, and in particular to an intelligent camera scheduling method and system based on drone trajectory prediction. Background Art

[0002] With the rapid development of drone technology, drones are increasingly being used in logistics, security inspections, disaster monitoring, and other fields. However, the dynamic nature of drone flight paths, the complexity of the environment, and the real-time requirements for multi-target tracking pose significant challenges to traditional fixed surveillance systems. Traditional surveillance systems typically rely on pre-set camera layouts and fixed monitoring areas, making them difficult to adapt to the rapid changes in drone flight paths. This results in increased blind spots and inefficient target tracking.

[0003] Therefore, it is necessary to provide an intelligent camera scheduling method and system based on drone trajectory prediction to solve the above technical problems. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an intelligent camera scheduling method and system based on drone trajectory prediction, which realizes intelligent collaborative monitoring and target tracking of multi-camera systems by integrating drone flight trajectory prediction, three-dimensional dynamic field of view modeling, multi-objective optimization scheduling and closed-loop control.

[0005] The present invention provides an intelligent camera scheduling method based on drone trajectory prediction, comprising the following steps:

[0006] Based on the collected real-time UAV flight data and combined with the motion probability coupling model, a set of predicted trajectories containing spatiotemporal probability distribution is generated;

[0007] Calculating the three-dimensional dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and performing occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model;

[0008] dynamically generating a target tracking priority weight based on the spatial overlap of the spatiotemporal probability distribution of the predicted trajectory set and the geo-fenced area preset in the environmental modeling data;

[0009] Based on the spatial constraints of the dynamic coverage model and the target tracking priority weights, an optimal camera scheduling solution is generated through a multi-objective optimization method;

[0010] The optimal camera scheduling solution is converted into a camera control instruction, closed-loop control is performed based on the real-time deviation of the dynamic coverage model, and the camera control instruction is executed.

[0011] Preferably, the optimization objectives of the multi-objective optimization method include:

[0012] The track coverage rate within the geo-fenced area exceeds a preset threshold, using the target tracking priority weight as a coefficient;

[0013] The camera posture adjustment time is less than the preset ratio of the prediction time window;

[0014] The number of activated cameras does not exceed a preset ratio of the total available number.

[0015] Preferably, the generation process of the predicted trajectory set specifically includes:

[0016] Obtain real-time flight data of the drone, input the real-time flight data into the pre-trained spatiotemporal attention RNN model, and output the basic motion trajectory of the drone within the predicted time window;

[0017] Based on the kinematic constraint model of the UAV, Monte Carlo sampling is performed on the basic motion trajectory to generate multiple candidate trajectories;

[0018] Calculate the dynamic survival probability of each candidate trajectory based on the covariance matrix of historical flight data statistics and the obstacle distribution in the environmental modeling data;

[0019] The probability density of motion parameters of candidate trajectories is fused through Gaussian mixture model to generate a probability distribution matrix of spatiotemporal association;

[0020] Based on the dynamic survival probability, the probability distribution matrix is ​​dynamically pruned and normalized, and a predicted trajectory set including a timestamp sequence, a three-dimensional coordinate confidence interval, and a trajectory weight is output.

[0021] Preferably, the generation process of the dynamic coverage model specifically includes:

[0022] Construct the viewing cone based on the focal length, field of view angle and pan / tilt adjustable range parameters in the collected camera parameters;

[0023] Mapping the three-dimensional coordinate confidence intervals of the predicted trajectory set to the viewing cone of each camera, and calculating the visibility probability of the drone trajectory in each frame;

[0024] Based on the 3D obstacle model in the environment modeling data, a ray casting algorithm is used to detect the view occlusion area and dynamically clip the view cone;

[0025] A dynamic coverage model is generated based on the dynamically cropped viewing cone and the visibility probability of the predicted trajectory, where the dynamic coverage model includes the effective monitoring area of ​​each camera within the predicted time window and the corresponding coverage confidence.

[0026] Preferably, the dynamic generation process of the target tracking priority weight includes:

[0027] Divide the geo-fenced area into multiple sub-areas with preset security levels and assign a static safety factor to each sub-area;

[0028] Calculate the spatiotemporal integral of the probability density of drone trajectory points in each sub-area to generate an initial threat score;

[0029] The initial threat score is multiplied by the corresponding static safety factor to obtain a dynamic tracking priority vector.

[0030] Preferably, the specific process of generating the optimal camera scheduling solution by the multi-objective optimization method includes:

[0031] Based on the spatial constraints of the dynamic coverage model, a set of coverage areas for each camera within the prediction time window is constructed;

[0032] Using the target tracking priority weight as an optimization coefficient, establishing a trajectory coverage optimization target for the geo-fenced area;

[0033] Using the preset threshold, preset proportion and preset ratio as optimization boundary conditions;

[0034] A constrained optimization algorithm is used to solve the problem and output the optimal camera scheduling solution that meets all optimization objectives.

[0035] Preferably, the closed-loop control specifically includes:

[0036] Converting the camera activation instruction in the optimal camera scheduling solution into a pan / tilt control protocol instruction, wherein the pan / tilt control protocol instruction at least includes an azimuth angle and a pitch angle;

[0037] Based on the real-time coverage deviation calculated by the dynamic coverage model, the PTZ control protocol instructions are adjusted by a PID controller;

[0038] The adjusted PTZ control protocol instructions are sent to the corresponding camera PTZ actuator to complete target tracking.

[0039] The present invention also provides an intelligent camera scheduling system based on drone trajectory prediction, which is used to execute the intelligent camera scheduling method based on drone trajectory prediction. The system includes:

[0040] The trajectory prediction module is used to generate a predicted trajectory set containing spatiotemporal probability distribution based on the collected real-time UAV flight data and combined with the motion probability coupling model;

[0041] A model generation module is used to calculate the three-dimensional dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and perform occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model;

[0042] a dynamic weight generation module for dynamically generating a target tracking priority weight based on the spatiotemporal probability distribution of the predicted trajectory set and the spatial overlap of the geo-fenced area preset in the environmental modeling data;

[0043] A multi-objective optimization scheduling module is used to generate an optimal camera scheduling solution through a multi-objective optimization method based on the spatial constraints of the dynamic coverage model and the target tracking priority weights;

[0044] A closed-loop control module is used to convert the optimal camera scheduling solution into camera control instructions, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instructions.

[0045] Compared with related technologies, the intelligent camera scheduling method and system based on drone trajectory prediction provided by the present invention has the following beneficial effects:

[0046] The present invention first generates multiple candidate trajectories based on the spatiotemporal attention RNN model and Monte Carlo sampling, and combines the dynamic survival probability with the Gaussian mixture model to output the spatiotemporal probability distribution matrix, thereby improving the robustness of trajectory prediction. Subsequently, the field of view cone is dynamically clipped through a ray casting algorithm to generate a dynamic coverage model that includes the effective monitoring area and coverage confidence, ensuring that there are no blind spots in the field of view. Finally, in the multi-objective optimization scheduling, the target tracking priority weight is used as the optimization coefficient, and combined with constraints such as trajectory coverage, posture adjustment time, and the number of activated cameras, a constrained optimization algorithm is used to solve the optimal camera scheduling solution.

[0047] The present invention realizes the intelligent allocation and efficient utilization of camera resources, significantly improves the target tracking coverage and response speed of the UAV monitoring system in complex environments, reduces the camera posture adjustment time and resource redundancy, effectively reduces the monitoring blind spots, and enhances the stability and real-time performance of multi-target collaborative monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of an intelligent camera scheduling method based on drone trajectory prediction provided by the present invention;

[0049] Figure 2 This is a module structure diagram of an intelligent camera scheduling system based on drone trajectory prediction provided by the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all of the structures. Furthermore, the embodiments of the present invention and the features of the embodiments may be combined with one another unless there is a conflict.

[0051] It should also be noted that, for ease of description, only portions relevant to the present invention are shown in the accompanying drawings, rather than all of the contents. Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the various operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operations are completed, but may also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0052] Example 1

[0053] The present invention provides an intelligent camera scheduling method based on UAV trajectory prediction. Figure 1 As shown, the following steps are included:

[0054] S1: Based on the collected real-time UAV flight data and combined with the motion probability coupling model, a predicted trajectory set containing spatiotemporal probability distribution is generated.

[0055] Specifically, the process of generating the predicted trajectory set in step S1 includes:

[0056] S11: Acquire real-time flight data of the UAV, input the real-time flight data into a pre-trained spatiotemporal attention RNN model, and output the basic motion trajectory of the UAV within the predicted time window.

[0057] In this embodiment, this step first uses sensors to collect the drone's six-degree-of-freedom state parameters in real time, including 3D position, velocity vector, heading, pitch, and roll angles. After Z-score normalization, this data is fed into a pretrained spatiotemporal attention recurrent neural network (RNN) model. This model uses a bidirectional long short-term memory (LSTM) network as the encoding layer to extract features from historical time series data. An attention mechanism dynamically assigns importance weights to different time steps, focusing on the drone's recent motion trends. Based on the attention-weighted contextual information, the decoding layer predicts the drone's basic motion trajectory frame by frame within a prediction window (set to actual requirements, 5 seconds in this embodiment) with a temporal resolution of 0.1 seconds. The model is trained using a dataset of tens of thousands of historical trajectories encompassing various flight modes (such as constant speed, acceleration, and hovering). The Adam optimizer adjusts parameters to minimize the error between the predicted and actual trajectories.

[0058] S12: Based on the kinematic constraint model of the UAV, Monte Carlo sampling is performed on the basic motion trajectory to generate multiple candidate trajectories.

[0059] In this example, the physical kinematic constraints of the drone are introduced based on the basic trajectory: a maximum flight speed of 15 m / s, a maximum acceleration of 5 m / s, and a heading angle change rate of no more than 30 degrees / s. Gaussian noise perturbations are added to each predicted point using the Monte Carlo method to simulate sensor noise and flight uncertainty, generating multiple candidate trajectories.

[0060] Each candidate trajectory also undergoes kinematic validation: if the velocity or heading angle variation between adjacent trajectory points exceeds physical limits, the trajectory is marked as "infeasible" and resampled. For feasible trajectories, high-probability paths that conform to historical flight patterns are further screened to ensure that the candidate trajectory set covers a variety of maneuvers, including sharp turns, climbs, and dives.

[0061] S13: Calculate the dynamic survival probability of each candidate trajectory based on the covariance matrix of historical flight data statistics and the obstacle distribution in the environment modeling data.

[0062] In this embodiment, the survival probability of each candidate trajectory is quantified by combining the environmental characteristics of historical flight data statistics and the real-time obstacle distribution.

[0063] First, a position covariance matrix is ​​constructed based on historical trajectories to describe the typical range of motion of the drone in the same environment. Second, the candidate trajectories are subjected to frame-by-frame collision detection using the obstacle bounding boxes (AABBs) in the environment modeling data. If a trajectory point falls within the obstacle bounding box, the number of collisions is accumulated. The survival probability is composed of two components: the degree of match between the trajectory point and the historical distribution (calculated using probability density) and a collision penalty term (the probability decreases with the number of collisions). Finally, these two components are combined using exponential weighting (preset weights) to output a dynamic survival probability for each trajectory.

[0064] S14: The motion parameter probability densities of the candidate trajectories are fused through the Gaussian mixture model to generate a probability distribution matrix of spatiotemporal associations.

[0065] In this embodiment, the motion parameters of candidate trajectories (such as velocity, acceleration, and rate of change of heading angle) are input into a Gaussian mixture model (GMM) to model their joint probability distribution. The model is trained using the expectation-maximization (EM) algorithm to automatically determine the weights and covariance parameters of the five Gaussian components. Each component corresponds to a typical motion pattern (such as uniform straight-line motion or accelerated turn), and the plausibility of the trajectory is quantified using probability density. Ultimately, a spatiotemporal probability distribution matrix is ​​generated, clarifying the probability of the drone appearing at different future times and spatial locations, providing data support for trajectory screening.

[0066] S15: Dynamically prune and normalize the probability distribution matrix based on the dynamic survival probability, and output a predicted trajectory set including a timestamp sequence, a three-dimensional coordinate confidence interval, and a trajectory weight.

[0067] In this example, this step first eliminates trajectories with a survival probability below 1% or multiple collisions with obstacles, retaining a set of physically feasible and safe candidates. Subsequently, the top 20% of high-probability trajectories are retained by probability density ranking to ensure the reliability of the prediction results.

[0068] The retained trajectories are normalized and a weight is calculated for each trajectory, which reflects its overall confidence (the product of the survival probability and the probability density of the motion parameters). The final output is a structured set of predicted trajectories, including a timestamp sequence, 3D coordinate confidence intervals, and normalized weights.

[0069] S2: Calculating the three-dimensional dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and performing occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model.

[0070] Specifically, the process of generating the dynamic coverage model in step S2 includes:

[0071] S21: Construct a viewing cone according to the focal length, field of view angle, and pan / tilt adjustable range parameters of the collected camera parameters.

[0072] In this embodiment, a three-dimensional viewing cone is constructed based on the camera's optical and mechanical parameters. Optical parameters include focal length (12mm), horizontal field of view (60°), and vertical field of view (45°). Mechanical parameters cover the pan / tilt's horizontal rotation range (±180°) and pitch adjustment range (-30° to +90°). With the camera's optical center as the vertex, the viewing angle parameters are combined to generate a quadrangular pyramidal viewing space. The horizontal direction extends left and right according to the field of view angle, and the vertical direction extends up and down along the pitch axis. The effective monitoring distance is set between 5 and 100 meters. The viewing cone's spatial orientation is dynamically adjusted by binding the pan / tilt's attitude data (azimuth and pitch) in real time, outputting a four-dimensional viewing cone model (three-dimensional space and time axis) consisting of a vertex coordinate sequence and a direction matrix.

[0073] S22: Mapping the three-dimensional coordinate confidence interval of the predicted trajectory set to the viewing cone of each camera, and calculating the visibility probability of the drone trajectory in each frame.

[0074] In this embodiment, the three-dimensional coordinate confidence intervals of the predicted trajectory set are converted to the camera's local coordinate system, and the projection of the global coordinates onto the imaging plane is achieved through a homogeneous coordinate transformation matrix. The projected area of ​​each trajectory point on the camera's imaging plane is calculated. If the projection is completely within the viewing cone, it is marked as "fully visible." If it partially exceeds the viewing cone boundary, the visibility probability is calculated based on the coverage area ratio (for example, if it covers 60% of the imaging area, the probability is 0.6). The visibility probability of the trajectory points in each frame is time-integrated (time window 5 seconds, step size 0.1 seconds) to generate a spatiotemporal visibility probability heat map, with the output dimension being a matrix of time (50 frames) × space (3D grid resolution 0.5 meters).

[0075] S23: Based on the 3D obstacle model in the environment modeling data, a ray casting algorithm is used to detect the view occlusion area and dynamically clip the view cone.

[0076] In this embodiment, a ray casting algorithm is used to detect field of view occlusion based on the obstacle triangular mesh model in the environment modeling data. Dense detection rays are emitted from the optical center of the camera to the boundary of the field of view cone (one ray every 1° in the horizontal direction and one ray every 0.5° in the vertical direction). The Bresenham algorithm is used to traverse the ray path and perform collision detection with the obstacle mesh. If the ray intersects with the obstacle within the effective distance, the direction is marked as an occlusion area. The vertices of the field of view cone boundary are dynamically cropped according to the occlusion result, and the occlusion edge is smoothed by Gaussian filtering to avoid mutations. Octree space partitioning is used to optimize collision detection, and the single-frame processing time is controlled within 10 milliseconds. The cropped field of view cone vertex sequence and the occlusion mark matrix are output.

[0077] S24: Generate a dynamic coverage model based on the dynamically cropped viewing cone and the visibility probability of the predicted trajectory, wherein the dynamic coverage model includes the effective monitoring area of ​​each camera within the prediction time window and the corresponding coverage confidence.

[0078] In this embodiment, a logical AND operation is performed on the cropped view cone and the spatiotemporal visibility probability matrix to extract valid monitoring areas that simultaneously meet the following conditions: they are within the cropped view cone and have a visibility probability greater than or equal to 0.5. Coverage confidence consists of two components: a static component calculated based on camera parameters, and a dynamic component adjusted based on the real-time visibility probability and occlusion ratio (a higher occlusion ratio results in a more pronounced confidence degradation).

[0079] The confidence level of historical frames is exponentially decayed (with a decay coefficient of 0.9) to balance real-time performance and stability. The final output is a dynamic coverage model, structured as a four-dimensional tensor. Each grid cell records whether it belongs to a valid area and a confidence value (0.0 to 1.0). The temporal coverage window is 5 seconds, and the spatial resolution is 0.5 meters. The model file includes timestamps, spatial extents, confidence parameters, and version metadata. Generation takes less than 50 milliseconds for a single camera, and reliability is ensured by verifying trajectory overlap (greater than or equal to 90%).

[0080] S3: Dynamically generate a target tracking priority weight based on the spatiotemporal probability distribution of the predicted trajectory set and the spatial overlap of the geo-fenced area preset in the environment modeling data.

[0081] Specifically, the dynamic generation process of the target tracking priority weight in step S3 includes:

[0082] S31: Divide the geo-fenced area into a plurality of sub-areas with preset security levels, and assign a static safety factor to each sub-area.

[0083] In this embodiment, based on a pre-set security policy, the geo-fenced area is divided into three sub-areas based on security level: a core area, a buffer area, and a warning area. The core area is a high-priority monitoring area (such as a no-fly zone or the perimeter of a critical facility) and is divided using a high-precision 5m x 5m grid. The buffer area is a transitional area (such as the edge of a flight path) and is divided using a medium-precision 10m x 10m grid. The warning area is a low-priority boundary area and is divided using a coarse-grained 20m x 20m grid.

[0084] Each sub-area is assigned a static safety factor: the core area receives the highest factor (1.0), the buffer zone receives a medium factor (0.6), and the warning zone receives the lowest factor (0.3). Grid boundaries are aligned with the geographic coordinate system to ensure seamless coverage, and configuration files support dynamic adjustment of safety factors to accommodate different scenarios (such as security upgrades for temporary events).

[0085] S32: Calculate the spatiotemporal integral value of the probability density of the drone trajectory points in each sub-area to generate an initial threat score.

[0086] In this embodiment, the spatiotemporal probability density integration of the drone trajectory points in each sub-area is performed to quantify the threat level. The specific process is as follows:

[0087] Spatial integration: Accumulate the probability density values ​​of all trajectory points within the sub-region grid. For example, if a core area grid contains three trajectory points with probability densities of 0.8, 0.7, and 0.5, the spatial integration result is 2.0.

[0088] Time integration: Within the prediction time window (5 seconds), the probability density of each time slice is accumulated at 0.1 second intervals. For example, the total probability density of a grid cell over 50 time slices is 10.5.

[0089] Threat Score Normalization: Map the spatiotemporal integration results to a range of 0 to 1. Set a maximum integration threshold (e.g., 20.0), truncating any values ​​above that threshold to 1.0, and values ​​below the minimum threshold (e.g., 0.1) to 0.0. For example, an integration value of 10.0 corresponds to a normalized threat score of 0.5.

[0090] The output is a list of initial threat scores for each sub-region (e.g. core grid A: 0.8, buffer grid B: 0.4).

[0091] S33: Multiply the initial threat score by the corresponding static safety factor to obtain a dynamic tracking priority vector.

[0092] In this embodiment, the initial threat score is combined with the static safety factor to generate a dynamic priority weight:

[0093] Weight calculation: Multiply the sub-region's threat score by its corresponding safety factor. For example, a core region's threat score of 0.8 × a safety factor of 1.0 = a dynamic weight of 0.8, and a buffer region's threat score of 0.4 × a safety factor of 0.6 = a dynamic weight of 0.24.

[0094] The dynamic weights are logarithmically transformed to amplify differences in high-threat areas. For example, a weight of 0.8 is amplified to 2.08, and a weight of 0.24 is amplified to 1.31.

[0095] Normalization is performed to ensure that the sum is 1.0, generating a priority vector (e.g., core area grid A: 0.25, buffer area grid B: 0.10).

[0096] The threat score is recalculated every 1 second and the priority vector is updated.

[0097] A sliding window mechanism is used to smooth weight fluctuations: the current weight accounts for 70%, and the historical 3-second data accounts for 30%, to avoid sudden changes in priority.

[0098] S4: Based on the spatial constraints of the dynamic coverage model and the target tracking priority weights, an optimal camera scheduling solution is generated through a multi-objective optimization method.

[0099] Specifically, the optimization objectives of the multi-objective optimization method in step S4 include:

[0100] The track coverage rate within the geo-fenced area exceeds a preset threshold, using the target tracking priority weight as a coefficient;

[0101] The camera posture adjustment time is less than the preset ratio of the prediction time window;

[0102] The number of activated cameras does not exceed a preset ratio of the total available number.

[0103] Specifically, the specific process of generating the optimal camera scheduling solution using the multi-objective optimization method in step S4 includes:

[0104] S41: Based on the spatial constraints of the dynamic coverage model, a set of coverable areas of each camera within the prediction time window is constructed.

[0105] In this embodiment, the effective monitoring area information of each camera in the next 5 seconds is extracted from the dynamic coverage model. Each area contains the precise three-dimensional spatial boundary coordinates and the corresponding coverage confidence value, where the confidence threshold is set to no less than 0.5. These areas are encoded at time intervals of 0.1 seconds to form spatiotemporal grid data. Each grid cell records the camera number, real-time coverage confidence, and current activation status in detail. For example, camera A can monitor a spatial range of 10 meters × 10 meters × 10 meters at 1 second with a confidence level of 0.7. Finally, a three-dimensional constraint matrix is ​​generated, clearly marking the specific areas that each camera can cover at different times.

[0106] S42: Using the target tracking priority weight as an optimization coefficient, establishing a trajectory coverage optimization target for the geo-fenced area.

[0107] In this embodiment, a multi-objective optimization function is constructed based on the priority weights of geo-fenced areas. First, a weight coefficient is assigned to each sub-area based on its security level: the core high-security area is assigned a weight of 0.8, the buffer zone is assigned a weight of 0.6, and the early warning zone is assigned a weight of 0.3. The weighted coverage of each sub-area is then calculated, and the contributions of all sub-areas are summed to obtain the total coverage. The optimization goal is to maximize this total coverage while ensuring that the preset minimum threshold of 85% is met. The system dynamically adjusts the weight assignments of each area in real time based on changes in threat levels, ensuring that high-priority areas are always monitored.

[0108] S43: Using the preset threshold, preset proportion and preset ratio as optimization boundary conditions.

[0109] In this embodiment, three key constraints are set for the optimization problem: first, the total coverage of the entire geo-fenced area must exceed 85%; second, the total time for camera pan / tilt adjustments cannot exceed 1 second (i.e., 20% of the 5-second prediction window); and third, the number of cameras activated at any one time must not exceed 30% of the total system capacity (e.g., out of 10 cameras, a maximum of 3 can be activated simultaneously). Furthermore, a spatial resource conflict avoidance mechanism is implemented to ensure that only one camera is responsible for the same monitoring area at any one time. These constraints are updated in real time and stored in a dedicated configuration file for the optimization algorithm to use.

[0110] S44: Use a constrained optimization algorithm to solve and output the optimal camera scheduling solution that meets all optimization objectives.

[0111] In this embodiment, a linear weighted optimization algorithm is used to solve the problem, transforming multiple optimization objectives into a single objective function. During algorithm execution, an initial solution is randomly generated. Then, through iterative calculations, the camera activation combination and PTZ parameters are gradually optimized. After each iteration, all constraints are rigorously checked for compliance, and solutions that violate constraints are penalized. The final optimal solution output includes the precise activation time period for each camera (e.g., camera A activates between 1.0 and 3.0 seconds) and the corresponding PTZ control parameters (30 degrees in azimuth and 15 degrees in pitch). The system verifies solution performance in real time to ensure coverage meets requirements and that adjustment time is manageable. Furthermore, through technologies such as parallel computing and sparse matrix storage, the entire optimization process is completed within 100 milliseconds. In the event of an anomaly, the system automatically activates the historical optimal solution as a backup.

[0112] S5: Convert the optimal camera scheduling solution into a camera control instruction, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instruction.

[0113] Specifically, the closed-loop control of step S5 includes:

[0114] S51: Convert the camera activation instruction in the optimal camera scheduling solution into a pan / tilt control protocol instruction, where the pan / tilt control protocol instruction at least includes an azimuth angle and a pitch angle.

[0115] In this embodiment, the camera activation instructions in the optimal camera scheduling plan are converted into protocol instructions executable by the PTZ. First, the scheduling plan is parsed to extract the camera activation time period (e.g., 1.0 to 3.0 seconds) and the three-dimensional coordinates of the target monitoring location. The target location is converted into PTZ control parameters through geometric projection calculations: the azimuth angle is calculated based on the horizontal projection of the target point, and the pitch angle is determined based on the ratio of height to horizontal distance. The instructions are encapsulated using the Pelco-D protocol and contain the camera ID, azimuth angle, pitch angle, motion speed, and a millisecond-accurate timestamp. A CRC checksum is added to ensure command integrity, and the PTZ response status is verified by pre-issuing test commands.

[0116] S52: Based on the real-time coverage deviation calculated by the dynamic coverage model, the PTZ control protocol instruction is adjusted by a PID controller.

[0117] In this embodiment, the gimbal parameters are dynamically adjusted using a PID controller based on real-time monitoring data from a dynamic coverage model. Real-time image recognition is used to determine the drone's actual position and calculate the deviation from the predicted position (e.g., a horizontal deviation of 0.3 meters and an altitude deviation of 0.2 meters). If the deviation exceeds a threshold (0.5 meters), the PID control process is triggered: the proportional coefficient quickly responds to position deviations, the integral coefficient eliminates accumulated errors, and the differential coefficient suppresses oscillations. Adjustments to the azimuth and pitch angles are calculated based on the deviations and added to the original control instructions to generate an updated target angle. The adjustment process limits the maximum rotational speed (20° / second) and acceleration (50° / second²) to prevent gimbal overload.

[0118] S53: Send the adjusted PTZ control protocol instruction to the corresponding camera PTZ actuator to complete target tracking.

[0119] In this embodiment, control instructions are sent via the RS-485 bus at a rate of 115200bps. After receiving the instructions, the PTZ servo motor drives the mechanical structure to move. Closed-loop control is used to read the actual angle of the encoder feedback in real time to verify the execution deviation (≤0.5°). If the instruction is lost or times out (no response within 200ms), the retransmission mechanism is triggered (up to 3 times). After execution, the camera image is used to confirm whether the drone is in the center of the field of view, and the dynamic coverage confidence level is monitored (≥0.7). In abnormal situations (such as PTZ failure or network interruption), the backup camera is automatically switched or the historical instruction cache is enabled. The entire process delay is controlled within 150 milliseconds to ensure real-time tracking and ultimately achieve accurate closed-loop tracking of the drone trajectory.

[0120] Example 2

[0121] The present invention also provides an intelligent camera scheduling system based on drone trajectory prediction, which is used to execute the intelligent camera scheduling method based on drone trajectory prediction. Figure 2 As shown, the system includes:

[0122] The trajectory prediction module 100 is used to generate a predicted trajectory set containing spatiotemporal probability distribution based on the collected real-time flight data of the UAV and in combination with the motion probability coupling model.

[0123] The model generation module 200 is used to calculate the 3D dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and perform occlusion correction on the 3D dynamic field of view to generate a dynamic coverage model.

[0124] The weight dynamic generation module 300 is used to dynamically generate target tracking priority weights based on the spatiotemporal probability distribution of the predicted trajectory set and the spatial overlap of the geo-fenced area preset in the environmental modeling data.

[0125] The multi-objective optimization scheduling module 400 is used to generate an optimal camera scheduling solution through a multi-objective optimization method based on the spatial constraints of the dynamic coverage model and the target tracking priority weights.

[0126] The closed-loop control module 500 is configured to convert the optimal camera scheduling solution into camera control instructions, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instructions.

[0127] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, magnetic disk storage, or magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0129] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

Claims

1. An intelligent camera scheduling method based on drone trajectory prediction, characterized in that: The following steps are involved: Based on the collected real-time UAV flight data and combined with the motion probability coupling model, a predicted trajectory set containing spatiotemporal probability distribution is generated. The generation process of the predicted trajectory set specifically includes: Obtain real-time flight data of the drone, input the real-time flight data into the pre-trained spatiotemporal attention RNN model, and output the basic motion trajectory of the drone within the predicted time window; Based on the kinematic constraint model of the UAV, Monte Carlo sampling is performed on the basic motion trajectory to generate multiple candidate trajectories; Calculate the dynamic survival probability of each candidate trajectory based on the covariance matrix of historical flight data statistics and the obstacle distribution in the collected environmental modeling data; The probability density of motion parameters of candidate trajectories is fused through Gaussian mixture model to generate a probability distribution matrix of spatiotemporal association; Dynamically pruning and normalizing the probability distribution matrix based on the dynamic survival probability, and outputting a predicted trajectory set including a timestamp sequence, a three-dimensional coordinate confidence interval, and a trajectory weight; Calculating the three-dimensional dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and performing occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model; dynamically generating a target tracking priority weight based on the spatial overlap of the spatiotemporal probability distribution of the predicted trajectory set and the geo-fenced area preset in the environmental modeling data; Based on the spatial constraints and target tracking priority weights of the dynamic coverage model, an optimal camera scheduling solution is generated through a multi-objective optimization method; The optimal camera scheduling solution is converted into a camera control instruction, closed-loop control is performed based on the real-time deviation of the dynamic coverage model, and the camera control instruction is executed.

2. The intelligent camera scheduling method based on drone trajectory prediction according to claim 1 is characterized in that: The optimization objectives of the multi-objective optimization method include: The track coverage rate within the geo-fenced area exceeds a preset threshold, using the target tracking priority weight as a coefficient; The camera posture adjustment time is less than the preset ratio of the prediction time window; The number of activated cameras does not exceed a preset ratio of the total available number.

3. The intelligent camera scheduling method based on drone trajectory prediction according to claim 2 is characterized in that: The generation process of the dynamic coverage model specifically includes: Construct the viewing cone based on the focal length, field of view angle and pan / tilt adjustable range parameters in the collected camera parameters; Mapping the three-dimensional coordinate confidence intervals of the predicted trajectory set to the viewing cone of each camera, and calculating the visibility probability of the drone trajectory in each frame; Based on the 3D obstacle model in the environment modeling data, a ray casting algorithm is used to detect the view occlusion area and dynamically clip the view cone; A dynamic coverage model is generated based on the dynamically cropped viewing cone and the visibility probability of the predicted trajectory, where the dynamic coverage model includes the effective monitoring area of ​​each camera within the predicted time window and the corresponding coverage confidence.

4. The intelligent camera scheduling method based on drone trajectory prediction according to claim 3 is characterized in that: The dynamic generation process of the target tracking priority weight includes: Divide the geo-fenced area into multiple sub-areas with preset security levels and assign a static safety factor to each sub-area; Calculate the spatiotemporal integral of the probability density of drone trajectory points in each sub-area to generate an initial threat score; The initial threat score is multiplied by the corresponding static safety factor to obtain a dynamic tracking priority vector.

5. The intelligent camera scheduling method based on drone trajectory prediction according to claim 4 is characterized in that: The specific process of generating the optimal camera scheduling solution by the multi-objective optimization method includes: Based on the spatial constraints of the dynamic coverage model, a set of coverage areas for each camera within the prediction time window is constructed; Using the target tracking priority weight as an optimization coefficient, establishing a trajectory coverage optimization target for the geo-fenced area; Using the preset threshold, preset proportion and preset ratio as optimization boundary conditions; A constrained optimization algorithm is used to solve the problem and output the optimal camera scheduling solution that meets all optimization objectives.

6. The intelligent camera scheduling method based on drone trajectory prediction according to claim 5 is characterized in that: The closed-loop control specifically includes: Converting the camera activation instruction in the optimal camera scheduling solution into a pan / tilt control protocol instruction, wherein the pan / tilt control protocol instruction at least includes an azimuth angle and a pitch angle; Based on the real-time coverage deviation calculated by the dynamic coverage model, the PTZ control protocol instructions are adjusted by a PID controller; The adjusted PTZ control protocol instructions are sent to the corresponding camera PTZ actuator to complete target tracking.

7. An intelligent camera scheduling system based on drone trajectory prediction, used to execute the intelligent camera scheduling method based on drone trajectory prediction according to any one of claims 1 to 6, characterized in that: The system comprises: The trajectory prediction module is used to generate a predicted trajectory set containing spatiotemporal probability distribution based on the collected real-time UAV flight data and combined with the motion probability coupling model; In the trajectory prediction module, the process of generating the predicted trajectory set specifically includes: Obtain real-time flight data of the drone, input the real-time flight data into the pre-trained spatiotemporal attention RNN model, and output the basic motion trajectory of the drone within the predicted time window; Based on the kinematic constraint model of the UAV, Monte Carlo sampling is performed on the basic motion trajectory to generate multiple candidate trajectories; Calculate the dynamic survival probability of each candidate trajectory based on the covariance matrix of historical flight data statistics and the obstacle distribution in the collected environmental modeling data; The probability density of motion parameters of candidate trajectories is fused through Gaussian mixture model to generate a probability distribution matrix of spatiotemporal association; Dynamically pruning and normalizing the probability distribution matrix based on the dynamic survival probability, and outputting a predicted trajectory set including a timestamp sequence, a three-dimensional coordinate confidence interval, and a trajectory weight; A model generation module is used to calculate the three-dimensional dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and perform occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model; a dynamic weight generation module for dynamically generating a target tracking priority weight based on the spatiotemporal probability distribution of the predicted trajectory set and the spatial overlap of the geo-fenced area preset in the environmental modeling data; A multi-objective optimization scheduling module is used to generate an optimal camera scheduling solution through a multi-objective optimization method based on the spatial constraints of the dynamic coverage model and the target tracking priority weights; A closed-loop control module is used to convert the optimal camera scheduling solution into camera control instructions, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instructions.

Images