Intelligent camera scheduling algorithm and system based on unmanned aerial vehicle trajectory prediction

Abstract

The invention provides an intelligent camera scheduling algorithm and system based on unmanned aerial vehicle trajectory prediction, and relates to the technical field of unmanned aerial vehicle monitoring, and the algorithm comprises the steps: generating a prediction trajectory set containing space-time probability distribution based on the collected real-time flight data of an unmanned aerial vehicle in combination with a motion probability coupling model; according to the collected camera parameters and environment modeling data, generating a dynamic coverage model in combination with the prediction trajectory set; dynamically generating a target tracking priority weight based on the space-time probability distribution of the prediction trajectory set and the space coincidence degree of a preset geo-fencing region in the environment modeling data; based on the spatial constraint of the dynamic coverage model and the target tracking priority weight, generating an optimal camera scheduling scheme through a multi-target optimization method; and converting the optimal camera scheduling scheme into a camera control instruction, and performing closed-loop control based on the real-time deviation of the dynamic coverage model, thereby realizing intelligent cooperative scheduling and target tracking of the unmanned aerial vehicle monitoring system.

Description

Technical Field

[0001] The present invention relates to the technical field of UAV monitoring, and particularly to an intelligent camera scheduling algorithm and system based on UAV trajectory prediction. Background Art

[0002] With the rapid development of UAV technology, UAVs are increasingly widely used in fields such as logistics distribution, security patrol, and disaster monitoring. However, the dynamic nature of UAV flight trajectories, environmental complexity, and real-time requirements for multi-target tracking pose severe challenges to traditional fixed monitoring systems. Traditional monitoring systems usually rely on pre-set camera layouts and fixed monitoring areas, and it is difficult to adapt to the rapid changes in UAV flight trajectories, resulting in an increase in monitoring blind spots and low target tracking efficiency.

[0003] Therefore, it is necessary to provide an intelligent camera scheduling algorithm and system based on UAV trajectory prediction to solve the above technical problems. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an intelligent camera scheduling algorithm and system based on UAV trajectory prediction, which realizes intelligent collaborative monitoring and target tracking of a multi-camera system by integrating UAV flight trajectory prediction, three-dimensional dynamic field-of-view modeling, multi-target optimal scheduling, and closed-loop control. The present invention provides an intelligent camera scheduling algorithm based on UAV trajectory prediction, including the following steps: Based on the collected real-time flight data of the UAV and combined with a motion probability coupling model, generate a prediction trajectory set including spatio-temporal probability distribution; According to the collected camera parameters and environmental modeling data, calculate the three-dimensional dynamic field-of-view of each camera in combination with the prediction trajectory set, and correct the occlusion of the three-dimensional dynamic field-of-view to generate a dynamic coverage model; Based on the spatio-temporal probability distribution of the prediction trajectory set and the spatial overlap degree of the preset geofence area in the environmental modeling data, dynamically generate target tracking priority weights; Based on the spatial constraints of the dynamic coverage model and the target tracking priority weights, generate an optimal camera scheduling scheme through a multi-target optimization method; Convert the optimal camera scheduling scheme into a camera control instruction, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instruction.

[0005] Preferably, the optimization objectives of the multi-target optimization method include: Taking the target tracking priority weight as a coefficient, the trajectory coverage rate within the geofence area exceeds a preset threshold; The camera attitude adjustment time is less than a preset percentage of the prediction time window; The number of activated cameras does not exceed a preset ratio of the total available number.

[0006] Preferably, the generation process of the predicted trajectory set specifically includes: Obtain the real-time flight data of the drone, and input the real-time flight data into a pre-trained spatio-temporal attention RNN model to output the basic motion trajectory of the drone within the prediction time window; Based on the kinematic constraint model of the drone, perform Monte Carlo sampling on the basic motion trajectory to generate multiple candidate trajectories; According to the covariance matrix statistically calculated from historical flight data and the obstacle distribution in the environmental modeling data, calculate the dynamic survival probability of each candidate trajectory; Generate a spatio-temporal correlation probability distribution matrix by fusing the probability density of the motion parameters of the candidate trajectories through a Gaussian mixture model; Based on the dynamic survival probability, perform dynamic pruning and normalization processing on the probability distribution matrix, and output a predicted trajectory set including a time stamp sequence, a three-dimensional coordinate confidence interval, and a trajectory weight.

[0007] Preferably, the generation process of the dynamic coverage model specifically includes: Construct a field of view cone according to the focal length, field of view angle, and pan-tilt adjustable range parameters in the collected camera parameters; Map the three-dimensional coordinate confidence interval of the predicted trajectory set to the field of view cones of each camera, and calculate the visible probability of the drone trajectory in each frame of the image; Based on the three-dimensional model of the obstacle in the environmental modeling data, use the ray casting algorithm to detect the field of view occlusion area and perform dynamic clipping on the field of view cone; Generate a dynamic coverage model according to the dynamically clipped field of view cone and the visible probability of the predicted trajectory, where the dynamic coverage model includes the effective monitoring area of each camera within the prediction time window and the corresponding coverage confidence level.

[0008] Preferably, the dynamic generation process of the target tracking priority weight includes: Divide the geofence area into multiple sub-areas with preset safety levels, and assign a static safety factor to each sub-area; Calculate the spatio-temporal integral value of the probability density of the drone trajectory points in each sub-area to generate an initial threat score; Multiply the initial threat score by the corresponding static safety factor to obtain a dynamic tracking priority vector.

[0009] Preferably, the specific process of the multi-objective optimization method for generating the optimal camera scheduling scheme includes: Based on the spatial constraints of the dynamic coverage model, construct a set of coverable areas of each camera within the prediction time window; Taking the target tracking priority weight as an optimization coefficient, an optimization objective for trajectory coverage of the geofence area is established; Taking the preset threshold, preset ratio, and preset proportion as optimization boundary conditions; An optimization algorithm with constraints is used for solving, and an optimal camera scheduling scheme that meets all optimization objectives is output.

[0010] Preferably, the closed-loop control specifically includes: Converting the camera activation instruction in the optimal camera scheduling scheme into a pan-tilt control protocol instruction, and the pan-tilt control protocol instruction includes at least an azimuth angle and a pitch angle; Based on the real-time coverage deviation calculated by the dynamic coverage model, adjusting the pan-tilt control protocol instruction through a PID controller; Sending the adjusted pan-tilt control protocol instruction to the corresponding camera pan-tilt actuator to complete target tracking.

[0011] The present invention also provides an intelligent camera scheduling system based on UAV trajectory prediction for executing the intelligent camera scheduling algorithm based on UAV trajectory prediction, and the system includes: A trajectory prediction module for generating a prediction trajectory set including spatio-temporal probability distribution based on the collected real-time flight data of the UAV and in combination with a motion probability coupling model; A model generation module for calculating the three-dimensional dynamic field of view of each camera according to the collected camera parameters and environmental modeling data, and performing occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model; A weight dynamic generation module for dynamically generating a target tracking priority weight based on the spatio-temporal probability distribution of the prediction trajectory set and the spatial coincidence degree of the preset geofence area in the environmental modeling data; A multi-objective optimization scheduling module for generating an optimal camera scheduling scheme through a multi-objective optimization method based on the spatial constraint of the dynamic coverage model and the target tracking priority weight; A closed-loop control module for converting the optimal camera scheduling scheme into a camera control instruction, performing closed-loop control based on the real-time deviation of the dynamic coverage model, and executing the camera control instruction.

[0012] Compared with the related technology, the intelligent camera scheduling algorithm and system based on UAV trajectory prediction provided by the present invention have the following beneficial effects: The present invention first generates multiple candidate trajectories based on a spatio-temporal attention RNN model and Monte Carlo sampling, and combines the dynamic survival probability and Gaussian mixture model to output a spatio-temporal correlation probability distribution matrix, thereby enhancing the robustness of trajectory prediction. Subsequently, the frustum of view is dynamically cropped through a ray casting algorithm to generate a dynamic coverage model including an effective monitoring area and coverage confidence, ensuring no occluded blind areas in the field of view. Finally, in multi-objective optimization scheduling, with the target tracking priority weight as the optimization coefficient, combined with constraint conditions such as trajectory coverage rate, attitude adjustment time, and the number of activated cameras, a constrained optimization algorithm is used to solve the optimal camera scheduling scheme.

[0013] The present invention realizes the intelligent allocation and efficient utilization of camera resources, significantly improves the target tracking coverage rate and response speed of the UAV monitoring system in complex environments, reduces the camera attitude adjustment time and resource redundancy, effectively reduces monitoring blind areas, and enhances the stability and real-time performance of multi-objective collaborative monitoring. Brief Description of the Drawings

[0014] Figure 1 is a flowchart of an intelligent camera scheduling algorithm based on UAV trajectory prediction provided by the present invention; Figure 2 is a module structure diagram of an intelligent camera scheduling system based on UAV trajectory prediction provided by the present invention. Detailed Embodiments

[0015] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention rather than all structures are shown in the drawings. Furthermore, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0016] It should also be noted that for the sake of description, only parts related to the present invention rather than all content are shown in the drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the process can be terminated, but there can also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0017] Embodiment 1 The present invention provides an intelligent camera scheduling algorithm based on UAV trajectory prediction, referring toFigure 1 As shown, it includes the following steps: S1: Based on the collected real-time flight data of the drone and combined with the motion probability coupling model, generate a prediction trajectory set containing spatio-temporal probability distributions.

[0018] Specifically, the generation process of the prediction trajectory set in step S1 specifically includes: S11: Obtain the real-time flight data of the drone, and input the real-time flight data into a pre-trained spatio-temporal attention RNN model to output the basic motion trajectory of the drone within the prediction time window.

[0019] In this embodiment, this step first collects the six-degree-of-freedom state parameters of the drone in real time through sensors, including three-dimensional position, velocity vector, heading angle, pitch angle, and roll angle. After these data are normalized by Z-score, they are input into a pre-trained spatio-temporal attention recurrent neural network (RNN) model. This model uses a bidirectional long short-term memory (LSTM) network as the encoding layer to extract the features of historical time-series data; through the attention mechanism, dynamically allocate the importance weights of different time steps, and focus on the recent motion trend of the drone. The decoding layer is based on the attention-weighted context information to predict the basic motion trajectory of the drone within the prediction time window (set according to actual needs, set to 5 seconds in this embodiment) frame by frame, and the time resolution is 0.1 second. When the model is trained, a dataset of tens of thousands of historical trajectories containing various flight modes (such as uniform motion, acceleration, hovering) is used, and the parameters are adjusted through the Adam optimizer to ensure that the error between the predicted trajectory and the actual trajectory is minimized.

[0020] S12: Based on the kinematic constraint model of the drone, perform Monte Carlo sampling on the basic motion trajectory to generate multiple candidate trajectories.

[0021] In this embodiment, on the basis of the basic trajectory, introduce the physical kinematic constraints of the drone: the maximum flight speed is 15 m / s, the maximum acceleration is 5 m / s, and the change rate of the heading angle does not exceed 30 degrees / second. Add Gaussian noise perturbation to each prediction point through the Monte Carlo method to simulate sensor noise and flight uncertainty, and generate multiple candidate trajectories.

[0022] In addition, each candidate trajectory also needs to pass through kinematic verification: if the change in speed or heading angle between adjacent trajectory points exceeds the physical limit, it is marked as "infeasible" and re-sampled. For feasible trajectories, further screen out the high-probability paths that conform to the historical flight rules to ensure that the candidate trajectory set covers various maneuvering behaviors such as sharp turns, climbs, and dives.

[0023] S13: Calculate the dynamic survival probability of each candidate trajectory according to the covariance matrix statistically calculated from the historical flight data and the obstacle distribution in the environmental modeling data.

[0024] In this embodiment, the survival probability of each candidate trajectory is quantified by combining the environmental characteristics of historical flight data statistics and the real-time obstacle distribution.

[0025] First, a position covariance matrix is constructed based on historical trajectories to describe the typical range of motion of the drone in the same environment. Second, the candidate trajectories are subjected to frame-by-frame collision detection using the obstacle bounding boxes (AABBs) in the environment modeling data. If a trajectory point falls within the obstacle bounding box, the number of collisions is accumulated. The survival probability is composed of two components: the degree of match between the trajectory point and the historical distribution (calculated using probability density) and a collision penalty term (the probability decreases with the number of collisions). Finally, these two components are combined using exponential weighting (preset weights) to output a dynamic survival probability for each trajectory.

[0026] S14: The motion parameter probability densities of the candidate trajectories are fused through the Gaussian mixture model to generate a probability distribution matrix of spatiotemporal associations.

[0027] In this embodiment, the motion parameters of candidate trajectories (such as velocity, acceleration, and rate of change of heading angle) are input into a Gaussian mixture model (GMM) to model their joint probability distribution. The model is trained using the expectation-maximization (EM) algorithm to automatically determine the weights and covariance parameters of the five Gaussian components. Each component corresponds to a typical motion pattern (such as uniform straight-line motion or accelerated turn), and the plausibility of the trajectory is quantified using probability density. Ultimately, a spatiotemporal probability distribution matrix is generated, clarifying the probability of the drone appearing at different future times and spatial locations, providing data support for trajectory screening.

[0028] S15: Dynamically prune and normalize the probability distribution matrix based on the dynamic survival probability, and output a predicted trajectory set including a timestamp sequence, a three-dimensional coordinate confidence interval, and a trajectory weight.

[0029] In this example, this step first eliminates trajectories with a survival probability below 1% or multiple collisions with obstacles, retaining a set of physically feasible and safe candidates. Subsequently, the top 20% of high-probability trajectories are retained by probability density ranking to ensure the reliability of the prediction results.

[0030] The retained trajectories are normalized and a weight is calculated for each trajectory, which reflects its overall confidence (the product of the survival probability and the probability density of the motion parameters). The final output is a structured set of predicted trajectories, including a timestamp sequence, 3D coordinate confidence intervals, and normalized weights.

[0031] S2: Calculating the three-dimensional dynamic field of view of each camera based on the collected camera parameters and environment modeling data in combination with the predicted trajectory set, and performing occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model.

[0032] Specifically, the generation process of the dynamic coverage model in step S2 specifically includes: S21: Construct a field-of-view cone according to the focal length, field-of-view angle, and pan-tilt adjustable range parameters in the collected camera parameters.

[0033] In this embodiment, a three-dimensional field-of-view cone is constructed based on the optical and mechanical parameters of the camera. The optical parameters include the focal length (12 mm), horizontal field-of-view angle (60°), and vertical field-of-view angle (45°), and the mechanical parameters cover the horizontal rotation range (±180°) and pitch angle adjustment range (-30° to +90°) of the pan-tilt. Taking the optical center of the camera as the vertex, a quadrangular pyramid-shaped field-of-view space is generated in combination with the field-of-view angle parameters, which expands left and right in the horizontal direction according to the field-of-view angle and extends up and down along the pitch axis in the vertical direction. The effective monitoring distance is set to 5 meters to 100 meters. The pan-tilt attitude data (azimuth angle and pitch angle) is bound in real time, and the spatial orientation of the field-of-view cone is dynamically adjusted, and a four-dimensional field-of-view cone model (three-dimensional space and time axis) including the vertex coordinate sequence and direction matrix is output.

[0034] S22: Map the three-dimensional coordinate confidence intervals of the predicted trajectory set to the field-of-view cones of each camera, and calculate the visibility probability of the UAV trajectory in each frame of the picture.

[0035] In this embodiment, the three-dimensional coordinate confidence intervals of the predicted trajectory set are converted to the local coordinate system of the camera, and the projection from the global coordinates to the imaging plane is realized through the homogeneous coordinate transformation matrix. Calculate the projection area of each trajectory point on the camera imaging plane. If the projection is completely located within the field-of-view cone, it is marked as "fully visible"; if part of it exceeds the field-of-view boundary, the visibility probability is calculated according to the coverage area ratio (for example, if 60% of the imaging area is covered, the probability is 0.6). The time integral of the visibility probability of the trajectory points in each frame of the picture is performed (time window 5 seconds, step size 0.1 second), and a spatio-temporal visibility probability heat map is generated, and a matrix with the dimension of time (50 frames) × space (three-dimensional grid resolution 0.5 meter) is output.

[0036] S23: Based on the three-dimensional model of the obstacle in the environmental modeling data, use the ray casting algorithm to detect the field-of-view occlusion area and dynamically clip the field-of-view cone.

[0037] In this embodiment, based on the obstacle triangular mesh model in the environmental modeling data, the ray casting algorithm is used to detect field-of-view occlusion. Dense detection rays are emitted from the camera optical center to the boundary of the field-of-view cone (one ray every 1° in the horizontal direction and one ray every 0.5° in the vertical direction). The Bresenham algorithm is used to traverse the ray path and perform collision detection with the obstacle grid. If the ray intersects with an obstacle within the effective distance, the direction is marked as an occluded area. The vertices of the field-of-view cone boundary are dynamically cropped according to the occlusion result, and Gaussian filtering is performed on the occlusion edge to avoid sudden changes. Octree space partitioning is used to optimize collision detection, and the single-frame processing time is controlled within 10 milliseconds. The cropped field-of-view cone vertex sequence and occlusion marking matrix are output.

[0038] S24: Generate a dynamic coverage model based on the visible probability of the dynamically cropped field-of-view cone and the predicted trajectory, where the dynamic coverage model includes the effective monitoring areas of each camera within the predicted time window and the corresponding coverage confidence levels.

[0039] In this embodiment, a logical "AND" operation is performed on the cropped field-of-view cone and the spatio-temporal visible probability matrix to extract the effective monitoring areas that simultaneously meet the following conditions: located within the cropped field-of-view and the visible probability is greater than or equal to 0.5. The coverage confidence level consists of two parts, static and dynamic: the static part is calculated based on the camera parameters, and the dynamic part is adjusted according to the real-time visible probability and occlusion ratio (the higher the occlusion ratio, the more significant the confidence decay).

[0040] The confidence level of historical frames is exponentially decayed (decay coefficient 0.9) to balance real-time performance and stability. Finally, a dynamic coverage model is output, with a data structure of a four-dimensional tensor. Each grid cell records whether it belongs to the effective area and the confidence value (from 0.0 to 1.0). The time covers a 5-second window, and the spatial resolution is 0.5 meters. The model file contains a timestamp, spatial range, confidence parameters, and version metadata. The generation time for a single camera is less than 50 milliseconds, and the reliability is ensured through verification of the trajectory coincidence rate (greater than or equal to 90%).

[0041] S3: Dynamically generate target tracking priority weights based on the spatio-temporal probability distribution of the predicted trajectory set and the spatial coincidence degree of the preset geofence area in the environmental modeling data.

[0042] Specifically, the dynamic generation process of the target tracking priority weights in step S3 includes: S31: Divide the geofence area into multiple sub-areas with preset security levels, and assign a static security coefficient to each sub-area.

[0043] In this embodiment, according to the preset security policy, the geofence area is divided into three types of sub-areas, namely the core area, the buffer area, and the warning area, according to the security level. The core area is a high-priority monitoring area (such as a no-fly zone or around important facilities), and is divided into high-precision grids of 5 meters × 5 meters; the buffer area is a transition area (such as the edge of a flight path), and is divided into medium-precision grids of 10 meters × 10 meters; the warning area is a low-priority boundary area, and is divided into coarse-grained grids of 20 meters × 20 meters.

[0044] Each sub-area is assigned a static security coefficient: the core area is given the highest coefficient (1.0), the buffer area is a medium coefficient (0.6), and the warning area is the lowest coefficient (0.3). The grid boundaries are aligned with the geographic coordinate system to ensure seamless coverage, and the security coefficient can be dynamically adjusted through a configuration file to adapt to different scenario requirements (such as the upgrade of temporary event security).

[0045] S32: Calculate the spatio-temporal integral value of the probability density of the UAV trajectory points in each sub-area to generate an initial threat score.

[0046] In this embodiment, the spatio-temporal probability density integration of the UAV trajectory points in each sub-area is performed to quantify the threat level. The specific process is as follows: Spatial integration: Accumulate the probability density values of all trajectory points within the sub-area grid. For example, a certain core area grid contains three trajectory points, and the probability densities are 0.8, 0.7, and 0.5 respectively, then the spatial integration result is 2.0.

[0047] Temporal integration: Within the prediction time window (5 seconds), accumulate the probability density of each time slice at intervals of 0.1 second. For example, the total probability density of a certain grid within 50 time slices is 10.5.

[0048] Threat score normalization: Map the spatio-temporal integration result to the range of 0 to 1. Set the maximum integration threshold (such as 20.0), and the part exceeding it is truncated to 1.0, and the part lower than the minimum threshold (such as 0.1) is set to 0.0. For example, the integration value of 10.0 corresponds to the normalized threat score of 0.5.

[0049] The output result is a list of initial threat scores for each sub-area (such as core area grid A: 0.8, buffer area grid B: 0.4).

[0050] S33: Multiply the initial threat score by the corresponding static security coefficient to obtain a dynamic tracking priority vector.

[0051] In this embodiment, the initial threat score is fused with the static security coefficient to generate a dynamic priority weight: Weight calculation: Multiply the threat score of the sub-region by the corresponding safety factor. For example, threat score of the core area 0.8 × safety factor 1.0 = dynamic weight 0.8, threat score of the buffer area 0.4 × safety factor 0.6 = dynamic weight 0.24.

[0052] Perform a logarithmic transformation on the dynamic weights to amplify the differences in high-threat regions. For example, after enhancement, weight 0.8 becomes 2.08, and 0.24 becomes 1.31.

[0053] Normalization ensures that the sum is 1.0, generating a priority vector (e.g., grid A in the core area: 0.25, grid B in the buffer area: 0.10).

[0054] Recalculate the threat score every 1 second and update the priority vector.

[0055] Adopt a sliding window mechanism to smooth the weight fluctuations: the current weight accounts for 70%, and the data of the past 3 seconds accounts for 30%, to avoid sudden changes in priorities.

[0056] S4: Based on the spatial constraints of the dynamic coverage model and the target tracking priority weights, generate an optimal camera scheduling scheme through a multi-objective optimization method.

[0057] Specifically, the optimization objectives of the multi-objective optimization method in step S4 include: With the target tracking priority weight as a coefficient, the trajectory coverage rate within the geofence area exceeds a preset threshold; The camera attitude adjustment time is less than a preset proportion of the prediction time window; The number of activated cameras does not exceed a preset proportion of the total available number.

[0058] Specifically, the specific process of the multi-objective optimization method in step S4 to generate an optimal camera scheduling scheme includes: S41: Based on the spatial constraints of the dynamic coverage model, construct a set of coverable areas of each camera within the prediction time window.

[0059] In this embodiment, extract the effective monitoring area information of each camera within the next 5 seconds from the dynamic coverage model. Each area contains precise three-dimensional spatial boundary coordinates and corresponding coverage confidence values, where the confidence threshold is set to not less than 0.5. Encode these areas at time intervals of 0.1 second to form spatio-temporal grid data. Each grid cell details the camera number, real-time coverage confidence, and current activation status. For example, camera A can monitor a space range of 10 m × 10 m × 10 m at the 1-second moment, with a confidence of 0.7. Finally, generate a three-dimensional constraint matrix clearly marking the specific areas that each camera can cover at different times.

[0060] S42: Establish an optimization objective for trajectory coverage of the geofence area with the target tracking priority weight as the optimization coefficient.

[0061] In this embodiment, a multi-objective optimization function is constructed based on the priority weights of the geofence area. First, different weight coefficients are assigned according to the security levels of each sub-area. The core high-security area is given a weight of 0.8, the buffer area is given 0.6, and the warning area is set to 0.3. Then, the weighted coverage rate of each sub-area is calculated, and the contribution values of all sub-areas are added together to obtain the total coverage rate. The optimization objective is to maximize this total coverage rate while ensuring that the preset minimum threshold requirement of 85% is met. The system will dynamically adjust the weight allocation of each area in real time according to the change of threat level to ensure that high-priority areas are always under key monitoring.

[0062] S43: Use the preset threshold, preset proportion, and preset ratio as optimization boundary conditions.

[0063] In this embodiment, three key constraint conditions are set for the optimization problem: First, the total coverage rate of the entire geofence area must exceed 85%; second, the total time for the camera pan-tilt to adjust its posture cannot exceed 1 second (i.e., 20% of the 5-second prediction window); third, the number of cameras activated at the same time does not exceed 30% of the total system quantity (for example, at most 3 out of 10 cameras can be activated simultaneously). In addition, a spatial resource conflict avoidance mechanism is set to ensure that only one camera is responsible for the same monitoring area at the same time. These constraint conditions will be updated in real time and stored in a dedicated configuration file for the optimization algorithm to call.

[0064] S44: Solve using a constrained optimization algorithm and output the optimal camera scheduling scheme that meets all optimization objectives.

[0065] In this embodiment, a linear weighted optimization algorithm is used for solving, converting multiple optimization objectives into a single objective function. During the execution of the algorithm, an initial solution is randomly generated first, and then the camera activation combination and pan-tilt parameters are gradually optimized through iterative calculations. After each iteration, it is strictly checked whether all constraint conditions are met, and penalties are given to the solutions that violate the constraints. The final output optimal scheme includes the exact activation time period of each camera (such as camera A is activated from 1.0 to 3.0 seconds) and the corresponding pan-tilt control parameters (azimuth angle 30 degrees, elevation angle 15 degrees). The system will verify the performance of the scheme in real time to ensure that the coverage rate meets the standard and the adjustment time is controllable. At the same time, through technologies such as parallel computing and sparse matrix storage, the entire optimization process can be completed within 100 milliseconds. When an abnormal situation occurs, the system will automatically enable the historical optimal scheme as an emergency backup.

[0066] S5: Convert the optimal camera scheduling scheme into camera control instructions, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instructions.

[0067] Specifically, the closed-loop control in step S5 specifically includes: S51: Convert the camera activation instruction in the optimal camera scheduling scheme into a pan-tilt control protocol instruction, and the pan-tilt control protocol instruction at least includes an azimuth angle and a pitch angle.

[0068] In this embodiment, convert the camera activation instruction in the optimal camera scheduling scheme into a protocol instruction executable by the pan-tilt head. First, parse the scheduling scheme, extract the activation time period of the camera (such as 1.0 to 3.0 seconds) and the three-dimensional coordinates of the target monitoring position. Convert the target position into pan-tilt control parameters through geometric projection calculation: the azimuth angle is calculated based on the horizontal projection of the target point, and the pitch angle is determined according to the ratio of the height to the horizontal distance. Package the instruction using the Pelco-D protocol, including the camera ID, azimuth angle, pitch angle, movement speed, and a timestamp accurate to milliseconds. Add a CRC check code to ensure the integrity of the instruction, and verify the pan-tilt head response status through pre-sending test instructions.

[0069] S52: Adjust the pan-tilt control protocol instruction through a PID controller based on the real-time coverage deviation calculated by the dynamic coverage model.

[0070] In this embodiment, based on the real-time monitoring data of the dynamic coverage model, dynamically adjust the pan-tilt head parameters through a PID controller. Obtain the actual position of the UAV through real-time image recognition, and calculate the deviation from the predicted position (such as a horizontal deviation of 0.3 meters and a height deviation of 0.2 meters). If the deviation exceeds the threshold (0.5 meters), trigger the PID control process: the proportional coefficient quickly responds to the position deviation, the integral coefficient eliminates the cumulative error, and the derivative coefficient suppresses the oscillation. Calculate the adjustment amounts of the azimuth angle and the pitch angle according to the deviation, and superimpose them on the original control instruction to generate an updated target angle. Limit the maximum rotation speed (20° / second) and acceleration (50° / second²) during the adjustment process to avoid overloading the pan-tilt head.

[0071] S53: Send the adjusted pan-tilt control protocol instruction to the corresponding camera pan-tilt head actuator to complete the target tracking.

[0072] In this embodiment, control instructions are sent at a rate of 115,200 bps through the RS-485 bus. After receiving the instructions, the pan-tilt servo motor drives the mechanical structure to move. The actual angle fed back by the encoder is read in real time using closed-loop control to verify the execution deviation (≤0.5°). If an instruction is lost or times out (no response within 200 ms), a retransmission mechanism is triggered (up to 3 times). After execution, it is confirmed through the camera image whether the drone is located in the center area of the field of view, and at the same time, the dynamic coverage confidence level is monitored (≥0.7). In case of an abnormality (such as a pan-tilt failure or network interruption), the backup camera is automatically switched or the historical instruction cache is enabled. The full process delay is controlled within 150 milliseconds to ensure real-time tracking, and finally, precise closed-loop tracking of the drone trajectory is achieved.

[0073] Embodiment 2 The present invention also provides an intelligent camera scheduling system based on drone trajectory prediction for executing the intelligent camera scheduling algorithm based on drone trajectory prediction, referring to Figure 2 As shown, the system includes: A trajectory prediction module 100 for generating a prediction trajectory set containing spatio-temporal probability distributions based on the collected real-time flight data of the drone and in combination with a motion probability coupling model.

[0074] A model generation module 200 for calculating the three-dimensional dynamic field of view of each camera according to the collected camera parameters and environmental modeling data, and correcting the occlusion of the three-dimensional dynamic field of view to generate a dynamic coverage model in combination with the prediction trajectory set.

[0075] A weight dynamic generation module 300 for dynamically generating target tracking priority weights based on the spatio-temporal probability distribution of the prediction trajectory set and the spatial overlap degree of the preset geofence area in the environmental modeling data.

[0076] A multi-objective optimization scheduling module 400 for generating an optimal camera scheduling scheme through a multi-objective optimization method based on the spatial constraints of the dynamic coverage model and the target tracking priority weights.

[0077] A closed-loop control module 500 for converting the optimal camera scheduling scheme into camera control instructions, performing closed-loop control based on the real-time deviation of the dynamic coverage model, and executing the camera control instructions.

[0078] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0079] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disc memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0080] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of another identical element in the process, method, commodity or device comprising the element.

Claims

1. An intelligent camera scheduling algorithm based on UAV trajectory prediction, characterized in that, It includes the following steps: Based on the collected real-time flight data of the drone and in combination with the motion probability coupling model, generate a prediction trajectory set containing spatio-temporal probability distributions; According to the collected camera parameters and environmental modeling data, calculate the three-dimensional dynamic field of view of each camera in combination with the prediction trajectory set, and perform occlusion correction on the three-dimensional dynamic field of view to generate a dynamic coverage model; Based on the spatio-temporal probability distribution of the prediction trajectory set and the spatial coincidence degree with the preset geofence area in the environmental modeling data, dynamically generate target tracking priority weights; Based on the spatial constraints of the dynamic coverage model and the target tracking priority weights, generate an optimal camera scheduling scheme through a multi-objective optimization method; Convert the optimal camera scheduling scheme into a camera control instruction, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instruction.

2. The intelligent camera scheduling algorithm based on UAV trajectory prediction according to claim 1, wherein The optimization objectives of the multi-objective optimization method include: Taking the target tracking priority weight as a coefficient, the trajectory coverage rate within the geofence area exceeds a preset threshold; The camera attitude adjustment time is less than a preset proportion of the prediction time window; The number of activated cameras does not exceed a preset proportion of the total available number.

3. An intelligent camera scheduling algorithm based on UAV trajectory prediction according to claim 2, characterized in that The generation process of the prediction trajectory set specifically includes: Obtain the real-time flight data of the drone, and input the real-time flight data into a pre-trained spatio-temporal attention RNN model to output the basic motion trajectory of the drone within the prediction time window; Based on the kinematic constraint model of the drone, perform Monte Carlo sampling on the basic motion trajectory to generate multiple candidate trajectories; According to the covariance matrix statistically calculated from historical flight data and the obstacle distribution in the environmental modeling data, calculate the dynamic survival probability of each candidate trajectory; Fuse the probability density of the motion parameters of the candidate trajectories through a Gaussian mixture model to generate a spatio-temporally correlated probability distribution matrix; Based on the dynamic survival probability, perform dynamic pruning and normalization processing on the probability distribution matrix, and output a prediction trajectory set containing a time stamp sequence, a three-dimensional coordinate confidence interval, and trajectory weights.

4. An intelligent camera scheduling algorithm based on UAV trajectory prediction according to claim 3, characterized in that, The generation process of the dynamic coverage model specifically includes: According to the focal length, field of view angle, and pan-tilt adjustable range parameters in the collected camera parameters, construct a field of view cone; Map the three-dimensional coordinate confidence interval of the prediction trajectory set to the field of view cones of each camera, and calculate the visible probability of the drone trajectory in each frame of the image; Based on the three-dimensional model of the obstacle in the environmental modeling data, use the ray casting algorithm to detect the field of view occlusion area and perform dynamic cropping on the field of view cone; According to the dynamically cropped field of view cone and the visible probability of the prediction trajectory, generate a dynamic coverage model, where the dynamic coverage model includes the effective monitoring area of each camera within the prediction time window and the corresponding coverage confidence level.

5. An intelligent camera scheduling algorithm based on UAV trajectory prediction according to claim 4, characterized in that, The dynamic generation process of the target tracking priority weight includes: Divide the geofence area into multiple sub-areas with preset safety levels, and assign static safety coefficients to each sub-area; Calculate the spatio-temporal integral value of the probability density of the drone trajectory points in each sub-area to generate an initial threat score; Multiply the initial threat score by the corresponding static safety coefficient to obtain a dynamic tracking priority vector.

6. The intelligent camera scheduling algorithm based on UAV trajectory prediction according to claim 5, wherein, The specific process of the multi-objective optimization method for generating the optimal camera scheduling scheme includes: Based on the spatial constraints of the dynamic coverage model, construct the set of coverable areas of each camera within the prediction time window; Taking the target tracking priority weight as the optimization coefficient, establish the trajectory coverage optimization objective for the geofence area; Use the preset threshold, preset ratio, and preset proportion as the optimization boundary conditions; Solve using a constrained optimization algorithm and output the optimal camera scheduling scheme that meets all optimization objectives.

7. An intelligent camera scheduling algorithm based on UAV trajectory prediction according to claim 6, characterized in that, The closed-loop control specifically includes: Convert the camera activation instruction in the optimal camera scheduling scheme into a pan-tilt control protocol instruction, and the pan-tilt control protocol instruction includes at least the azimuth angle and the elevation angle; Based on the real-time coverage deviation calculated by the dynamic coverage model, adjust the pan-tilt control protocol instruction through a PID controller; Send the adjusted pan-tilt control protocol instruction to the corresponding camera pan-tilt actuator to complete target tracking.

8. An intelligent camera scheduling system based on UAV trajectory prediction, which is used to execute an intelligent camera scheduling algorithm based on UAV trajectory prediction as described in any one of claims 1 to 7, characterized in that, The system includes: A trajectory prediction module, which is used to generate a prediction trajectory set containing spatio-temporal probability distributions based on the collected real-time flight data of the unmanned aerial vehicle and in combination with the motion probability coupling model; A model generation module, which is used to calculate the three-dimensional dynamic field of view of each camera according to the collected camera parameters and environmental modeling data, correct the occlusion of the three-dimensional dynamic field of view, and generate a dynamic coverage model; A weight dynamic generation module, which is used to dynamically generate the target tracking priority weight based on the spatio-temporal probability distribution of the prediction trajectory set and the spatial coincidence degree of the preset geofence area in the environmental modeling data; A multi-objective optimization scheduling module, which is used to generate the optimal camera scheduling scheme through a multi-objective optimization method based on the spatial constraints of the dynamic coverage model and the target tracking priority weight; A closed-loop control module, which is used to convert the optimal camera scheduling scheme into a camera control instruction, perform closed-loop control based on the real-time deviation of the dynamic coverage model, and execute the camera control instruction.

Images