Urban low-altitude unmanned aerial vehicle team scheduling method and system based on multi-modal data

Abstract

The present application relates to the field of flight control technology, and more particularly to a kind of urban low-altitude unmanned aerial vehicle team scheduling method and system based on multi-modal data.The method comprises the following steps: obtaining the multi-modal data of unmanned aerial vehicle in target area, including city three-dimensional map and meteorological data;Determine the navigable area of city three-dimensional map;Flight safety area is divided based on meteorological data in navigable area;Flight conflict path is predicted based on flight safety area;First flight path set is generated using flight conflict path, and first flight path set is sent to unmanned aerial vehicle, and unmanned aerial vehicle scheduling is carried out, to generate second flight path set;Flight error is determined according to first flight path set and second flight path set;Heading compensation value is determined based on flight error;In response to heading compensation value, flight control instruction is generated.The present application improves the safety and path scheduling success rate of urban low-altitude unmanned aerial vehicle cluster flight based on flight control technology.

Description

Technical Field

[0001] This invention relates to the field of flight control technology, and in particular to a method and system for scheduling urban low-altitude unmanned aerial vehicle (UAV) fleets based on multimodal data. Background Technology

[0002] Currently, multimodal data is increasingly used in the scheduling of urban low-altitude unmanned aerial vehicle (UAV) fleets, especially in path planning by combining urban 3D maps with real-time meteorological data. However, traditional scheduling methods still suffer from problems such as insufficient information integration and untimely scheduling response in practical applications. Existing technologies for UAV fleet path scheduling, when integrating urban spatial structure and dynamic meteorological factors, mainly have the following shortcomings: Traditional methods largely rely on static map data for navigable area identification, failing to fully consider dynamic changes in building distribution, road width limitations, and flight buffer zones, among other urban details. They lack accurate modeling and real-time updates for complex airspace in high-density urban areas, resulting in a lack of flexibility and safety boundaries in path planning. In flight conflict prediction and mission scheduling, existing methods typically rely solely on rule-driven or single-parameter priority ranking, ignoring the synergistic trade-offs between multiple factors such as mission urgency, remaining battery power, and flight risk. This makes it difficult to perform refined scheduling and priority reconstruction of conflict paths, easily leading to path overlap and resource waste. Summary of the Invention

[0003] Therefore, it is necessary for the present invention to provide a method for scheduling urban low-altitude unmanned aerial vehicle (UAV) fleets based on multimodal data, in order to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, a method for scheduling urban low-altitude unmanned aerial vehicle (UAV) fleets based on multimodal data includes the following steps:

[0005] Step S1: Acquire multimodal data of UAVs within the target area, including urban 3D maps and meteorological data; determine the navigable area of ​​the urban 3D map; delineate the flight safety zone of the navigable area based on the meteorological data;

[0006] Step S2: Predict flight conflict paths based on flight safety zones; generate a first flight path set using the flight conflict paths, send the first flight path set to the UAV, and perform UAV scheduling to generate a second flight path set;

[0007] Step S3: Determine the flight error based on the first flight path set and the second flight path set; determine the heading compensation value based on the flight error; generate flight control commands in response to the heading compensation value;

[0008] Step S4: Execute flight control commands and monitor the UAV status; if the UAV status shows a communication link interruption, record it as an abnormal flight status and trigger an automatic return-to-home command to obtain the first corrected flight trajectory.

[0009] This specification also provides a multimodal data-based urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling system for executing the multimodal data-based urban low-altitude UAV fleet scheduling method described above. The multimodal data-based urban low-altitude UAV fleet scheduling system includes:

[0010] The flight safety zone identification module is used to acquire multimodal data of UAVs within the target area, including urban 3D maps and meteorological data; determine the navigable area of ​​the urban 3D map; and delineate the flight safety zone of the navigable area based on the meteorological data.

[0011] The UAV scheduling module is used to predict flight conflict paths based on flight safety areas; generate a first flight path set using the flight conflict paths, send the first flight path set to the UAVs, and perform UAV scheduling to generate a second flight path set;

[0012] The heading compensation value module is used to determine the flight error based on the first flight path set and the second flight path set; determine the heading compensation value based on the flight error; and generate flight control commands in response to the heading compensation value.

[0013] The flight trajectory correction module is used to execute flight control commands and monitor the status of the UAV. If the communication link of the UAV is interrupted, it is recorded as an abnormal flight status and an automatic return command is triggered to obtain the first corrected flight trajectory.

[0014] The beneficial effects of this invention are as follows:

[0015] On the one hand, by integrating urban 3D map data with real-time meteorological data, it is possible to dynamically identify navigable areas in urban airspace. By combining building boundary points and road centerline points, it is possible to accurately construct flight buffer zones and restricted areas. Furthermore, based on wind speed and visibility parameters, it is possible to screen safe take-off points, which significantly improves the environmental adaptability and safety of path planning. This avoids the problems of coarse airspace division and blind selection of take-off points in traditional methods, and effectively enhances the feasibility of take-off and landing of UAVs in complex urban environments and the success rate of missions.

[0016] On the other hand, by constructing a three-dimensional grid space in the flight safety zone and detecting the spatial intersection between flight paths in real time within a short prediction window, it is possible to identify flight conflict paths with high accuracy. Furthermore, by combining the urgency of the flight mission with the remaining battery power to generate a priority ranking table, it can dynamically generate the optimal flight path set based on the obstacle avoidance path. This effectively solves the path conflict and resource scheduling problems in the high-density operation of multiple UAVs, and improves the rationality, flexibility and real-time performance of UAV scheduling.

[0017] On the other hand, by calculating flight error through path matching and combining spatial offset vector with theoretical heading value, a heading deviation model is constructed. Furthermore, the heading compensation value is calculated based on filtering algorithm and angle mean, which can correct flight control commands in real time, ensuring that the UAV runs along the optimal path. This significantly improves the accuracy and stability of path tracking, solves the path drift problem caused by disturbance factors in traditional methods, and enhances the robustness of the scheduling system. Attached Figure Description

[0018] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0019] Figure 1 This is a flowchart illustrating the steps of a method for scheduling urban low-altitude unmanned aerial vehicle (UAV) fleets based on multimodal data according to the present invention.

[0020] Figure 2 This is a detailed flowchart of step S2 in the present invention;

[0021] Figure 3 This is a schematic diagram illustrating the determination of velocity disturbance compensation values ​​in this invention;

[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] The technical method of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0024] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0025] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a method for scheduling urban low-altitude unmanned aerial vehicle (UAV) fleets based on multimodal data, the method comprising the following steps:

[0027] Step S1: Acquire multimodal data of UAVs within the target area, including urban 3D maps and meteorological data; determine the navigable area of ​​the urban 3D map; delineate the flight safety zone of the navigable area based on the meteorological data;

[0028] In one embodiment, a 3D urban map, with a resolution of 5 cm and a coverage radius of 10 km, is obtained using LiDAR (Light Detection and Ranging) and drone aerial photography. Wind speeds range from 2 to 6 m / s, and temperature variations are within ±5°C. The 3D urban map identifies navigable areas containing buildings, roads, bridges, and open spaces. Then, based on meteorological data, areas with wind speeds greater than 5 m / s are marked as high-risk areas, requiring flight avoidance. Safe flight zones are determined based on meteorological data, taking into account wind speed and temperature variations, to ensure stable drone flight.

[0029] In another embodiment, assuming the target area is a metropolitan area, the 3D map is obtained through LiDAR scanning, covering an area of ​​12 kilometers. 2 Meteorological data was obtained from ground-based weather stations, with wind speeds ranging from 3 to 8 m / s and temperatures from 15°C to 22°C. Areas within this region with wind speeds exceeding 6 m / s were designated as no-fly zones. The safe flight zone was narrowed to 5 kilometers. 2This ensures that drones can fly within the optimal area, avoiding areas heavily affected by weather conditions.

[0030] Step S2: Predict flight conflict paths based on flight safety zones; generate a first flight path set using the flight conflict paths, send the first flight path set to the UAV, and perform UAV scheduling to generate a second flight path set;

[0031] In one embodiment, potential flight conflict paths are predicted using obstacle data (such as buildings, bridges, etc.) within the flight safety area. A cost function is used for calculation.

[0032]

[0033] Where L(P) is the total length of path P, E(P) is the obstacle avoidance cost of path P, W(P) is the weather factor cost of path P, α is the weight of path length (set to 0.6), β is the weight of obstacle avoidance (set to 0.3), and γ is the weight of weather factors (set to 0.1). The system calculates a first set of flight paths, where each path is 10-15 kilometers long. The UAV receives the first set of flight paths based on real-time data and dynamically adjusts its path during flight. After optimization by the scheduling system, a second set of flight paths is generated, with path lengths adjusted to 12-18 kilometers depending on the flight environment, weather conditions, and flight mission.

[0034] In another embodiment, it is assumed that the safe flight zone within the target area is 8 kilometers. 2 Five flight paths were predicted using a path planning algorithm, with lengths of 7 km, 9 km, 10 km, 12 km, and 14 km, respectively. Based on real-time meteorological data (wind speed of 4 m / s and temperature of 19°C), the generation of the second flight path set took into account avoiding areas impacted by airflow, and the final selected flight path length was 10 km to ensure flight stability.

[0035] Step S3: Determine the flight error based on the first flight path set and the second flight path set; determine the heading compensation value based on the flight error; generate flight control commands in response to the heading compensation value;

[0036] In one embodiment, based on the flight trajectory data of the first and second flight path sets, error calculation is performed using a real-time positioning system (GPS and inertial measurement unit) to determine that the error range for each path is ±3 meters. If the error exceeds 5 meters, the system calculates a compensation value using a heading compensation algorithm, with the heading compensation value being ±2°. Based on the heading compensation value, the flight control system generates corresponding corrective flight control commands to adjust the flight direction, ensuring that the UAV accurately flies along the target path.

[0037] In another embodiment, assuming the error of the first flight path set is ±4 meters and the error of the second flight path set is ±6 meters, the flight system calculates a compensation value of ±3°. Based on this compensation value, the flight control system adjusts the UAV's heading and ensures, through commands, that the UAV's flight deviation does not exceed 10 meters, thereby guaranteeing flight accuracy.

[0038] Step S4: Execute flight control commands and monitor the UAV status; if the UAV status shows a communication link interruption, record it as an abnormal flight status and trigger an automatic return-to-home command to obtain the first corrected flight trajectory.

[0039] In one embodiment, flight control commands are sent to the UAV via a ground control station. These commands include information such as flight speed, heading, and altitude. The UAV monitors its own flight status in real time (e.g., battery level, GPS signal strength, communication link strength). If the UAV's communication link strength drops below 50%, it is considered an abnormal flight state. The system automatically triggers a return-to-home command and calculates a first corrected flight path based on the distance between the current location and the takeoff point (assumed to be 6 kilometers). The optimized flight path ensures a safe return in the shortest possible time.

[0040] In another embodiment, assuming the drone is in the target area with 20% battery remaining and the communication link is lost for more than 30 seconds, the system immediately determines this as an abnormal state and calculates the shortest return path (5 kilometers) from the takeoff point. Based on the corrected flight trajectory generated by the return-to-home algorithm, the system ensures that the drone can return to the takeoff point within 5 minutes, avoid obstacles, and complete the automatic return.

[0041] Preferably, the method further includes:

[0042] Obtain meteorological disturbance parameters and communication interference parameters of the airspace where the UAV is currently located;

[0043] Determine the attitude disturbance compensation value based on meteorological disturbance parameters;

[0044] Determine the speed disturbance compensation value based on communication interference parameters;

[0045] Based on the attitude disturbance compensation value and the velocity disturbance compensation value, the flight control command is compensated and corrected to obtain the corrected control command;

[0046] The first corrected flight path is corrected a second time using the correction control command to generate the second corrected flight path.

[0047] In one embodiment, the drone's sensors are used to acquire meteorological parameters (such as wind speed, temperature, humidity, etc.) and interference data of the communication link (such as signal strength, signal-to-noise ratio, etc.) in the airspace. Assume a wind speed of 10 m / s, a temperature of 25°C, a signal strength of -80 dBm, and a signal-to-noise ratio of 15 dB.

[0048] In another embodiment, it is assumed that the wind speed in the meteorological disturbance parameters is 10 m / s, and the angle between the wind direction and the UAV's heading is 45°. The system calculates the attitude disturbance compensation value using the relationship between wind speed, wind direction, and UAV attitude. According to the meteorological disturbance model, the calculation formula is: ;in, This is the proportionality coefficient of the effect of wind speed on attitude (assumed to be 0.2). Let be the angle between the wind direction and the UAV's heading, with a wind speed of 10 m / s. The calculated attitude disturbance compensation value is ±2°.

[0049] In another embodiment, the communication interference parameters are assumed to be a signal strength of -80 dBm and a signal-to-noise ratio of 15 dB. Based on the relationship between communication quality and speed control, the system calculates the speed disturbance compensation value using the following formula: ; Let be the coefficient representing the influence of communication signals on flight speed (assumed to be 0.1). Calculations show that the speed disturbance compensation value is ±0.5 m / s.

[0050] In another embodiment, the flight control system first corrects the current flight control command based on the acquired attitude disturbance compensation value ±2° and velocity disturbance compensation value ±0.5 m / s. Assuming the initial flight command is a heading angle of 45° and a flight speed of 10 m / s, the system adds the attitude compensation value and the velocity compensation value to the original control command respectively, resulting in the corrected flight command: Corrected control command = (heading ±2°, velocity ±0.5 m / s).

[0051] In another embodiment, the flight control system applies the revised control commands to the UAV to adjust its flight trajectory. Based on the revised control commands, the system performs a second correction on the first corrected flight trajectory to ensure that the UAV's flight path meets the requirements of the new weather and communication environment. The final generated second corrected flight trajectory deviates from the expected path by ±1 meter, ensuring the accuracy and stability of the flight.

[0052] Preferably, determining the attitude disturbance compensation value based on meteorological disturbance parameters includes:

[0053] Wind speed disturbance values ​​and airflow gradient change rates are extracted based on meteorological disturbance parameters;

[0054] By using wind speed disturbance values, the disturbance angle of the UAV in the pitch direction can be obtained;

[0055] Based on the rate of change of the airflow gradient, the disturbance angle of the UAV in the roll direction is obtained;

[0056] Calculate the pitch attitude compensation value based on the disturbance angle and the preset pitch stability angle;

[0057] Calculate the roll attitude compensation value based on the disturbance angle and the preset roll stability angle;

[0058] Based on the pitch attitude compensation value and the roll attitude compensation value, the attitude disturbance compensation value is determined.

[0059] In one embodiment, the pitch disturbance angle The calculation formula is: ,in This represents the wind speed disturbance value. The pitch response coefficient is expressed in degrees per meter per second (° / (m / s)). Obtained from UAV wind tunnel tests and flight data calibration, the typical value range is 2°~5° / (m / s).

[0060] It should be noted that, taking a wind speed disturbance value of 1.0 m / s as an example, if Taking 3° / (m / s), the corresponding pitch disturbance angle is 3°. This calculation process is implemented in the flight control system using a digital signal processor with an update frequency of 10Hz. After inputting the wind speed disturbance value, the pitch disturbance angle signal is output in real time for subsequent compensation value calculation. The angle value range is limited to ±15°; any value exceeding this range is truncated by a limiting function to avoid abnormal flight control commands.

[0061] In another embodiment, the roll disturbance angle The calculation formula is: ,in This represents the rate of change of the airflow gradient obtained in step one, in m / s / m. The roll angle response coefficient is expressed in degrees per (m / s / m) (° / (m / s / m)). Also obtained through flight test data, the typical value range is 10°~20° / (m / s / m). For example, if the airflow gradient change rate is 0.5 m / s / m, then... Taking 15° / (m / s / m), the roll disturbance angle is 7.5°. The roll disturbance angle is also calculated in real time by the flight controller processor, with an update frequency of 10Hz and a value limited to ±20°.

[0062] In another embodiment, pitch attitude compensation value The calculation is the difference between the disturbance angle and the stable angle, i.e. In practice, that is... = The compensation value serves as the input to the flight control attitude controller, used to adjust the UAV's pitch angle to counteract disturbances. Before being transmitted to the attitude adjustment module, the compensation value is smoothed by a finite response filter. The filter coefficients are set according to the system's dynamic characteristics, with a typical bandwidth of 1Hz to avoid rapid fluctuations that could cause control oscillations. The range of the compensation value is controlled by a soft-limiting function, with a threshold set at ±15°. Values ​​exceeding this range are truncated using a step function to ensure the stability of flight control commands.

[0063] In another embodiment, the roll attitude compensation value The calculation method is as follows ,Right now = The compensation value serves as the control input for the flight control roll channel, guiding the adjustment of the airframe roll angle. The compensation value is smoothed using a finite response filter similar to that used in the pitch channel, with consistent filtering parameters and a limit range of ±20°. To prevent attitude adjustment overshoot, a rate limiter is integrated into the flight control module input, limiting the rate of change of angular velocity to within 5° / s.

[0064] In another embodiment, the final attitude perturbation compensation value is defined as a two-dimensional vector. This vector serves as the unified input to the attitude adjustment module of the flight control system. The combination process employs a linear weighting method, with weights of [weights to be filled in]. and The weight values ​​are determined by the system design requirements and flight tests.

[0065] It should be noted that the weighted calculation formula is as follows: The compensation vector is updated in real time, with an update frequency maintained above 10Hz to meet the dynamic response requirements of UAV flight. After the attitude disturbance compensation value is input into the flight control algorithm, it drives the actuators to adjust the UAV's control surfaces or power distribution to achieve attitude stabilization control.

[0066] Preferably, determining the speed disturbance compensation value based on communication signal interference parameters includes:

[0067] Convert communication signal interference parameters into a signal-to-noise ratio sequence;

[0068] Obtain flight speed data from the flight log;

[0069] The signal-to-noise ratio sequence and flight speed data are time-stamped to obtain the perturbation sequence;

[0070] Velocity components along the X, Y, and Z coordinate axes are extracted using a perturbation sequence;

[0071] Calculate the three-dimensional velocity vector based on the velocity components;

[0072] Calculate the average difference based on the three-dimensional velocity vector;

[0073] The speed disturbance compensation value is determined based on the average difference.

[0074] In one embodiment, the signal-to-noise ratio (SNR) is defined as the ratio of signal power to noise power, measured in decibels (dB). At the device end, the acquired signal strength and noise level data are input into a digital signal processing unit, and the formula is used... Calculate the signal-to-noise ratio (SNR) value at the corresponding time point. The sampling frequency is set to 50Hz. During the conversion process, the sampling window for both signal and noise is set to 20 milliseconds to balance the real-time performance and stability of the data. The calculated SNR data is stored in timestamp order to form a continuous SNR sequence for use in subsequent steps.

[0075] In another embodiment, due to the difference between the communication signal sampling frequency (50Hz) and the flight speed sampling frequency (100Hz), the two data sequences need to be aligned based on timestamps. The timestamps of both sets of data are unified to Coordinated Universal Time (UTC) to eliminate any clock skew. Then, using the signal-to-noise ratio (SNR) sequence as the reference time point, the flight speed data is resampled using nearest-neighbor interpolation, reducing the speed data sampling rate to 50Hz to ensure consistent time series length. During synchronization, data segments with timestamp differences exceeding 10 milliseconds are discarded to avoid data deviations caused by time synchronization errors. After synchronization is complete, the SNR values ​​and the corresponding three-dimensional velocity vectors at each time point are combined to form a perturbation sequence with the same length as the SNR sequence.

[0076] In another embodiment, the magnitude of the three-dimensional velocity vector at the corresponding time point is calculated using the three-axis velocity components, and the calculation formula is as follows: ,in , , These are the zero-mean values ​​respectively. Axial velocity component, in meters per second (m / s).

[0077] In another embodiment, for a sequence of three-dimensional velocity vector magnitudes within a certain time window (e.g., the most recent 10 seconds), the difference between adjacent time points is calculated, i.e. The arithmetic mean of all differences is taken to obtain the average velocity disturbance difference within the time window. The unit is meters per second (m / s).

[0078] It should be noted that the average difference reflects the magnitude of velocity disturbance changes and is a key parameter for measuring aircraft motion instability. During the calculation, a sliding window mechanism is used with a window step size of 1 second, updating the calculation results in real time. Outlier differences (exceeding the threshold of 5 m / s) are removed to ensure the robustness of the average difference calculation.

[0079] In another embodiment, the average velocity disturbance difference is mapped to a velocity disturbance compensation value. The calculation formula is: ,in This is the proportionality coefficient, with a value ranging from 0.1 to 0.5. The compensation value is determined through ground test flights and flight data calibration based on the dynamic characteristics and safety requirements of the aircraft control system. The value is limited to 0 to 2.0 m / s and is used as the adjustment input for the speed control channel of the flight control system. This compensation value is fed back to the flight control system in real time, enabling flight attitude and speed adjustment based on communication interference and speed changes.

[0080] Preferably, step S1 includes the following steps:

[0081] Step S11: Acquire multimodal data of UAVs within the target area, including 3D city maps and meteorological data;

[0082] Step S12: Based on the building boundary points and road centerline points in the city 3D map, filter out 5 or more consecutive map tile data;

[0083] Step S13: When the spatial distance between the building boundary points and the road centerline points in the continuous map tiles is maintained above the preset safe navigation width and the height difference is below the set threshold, it is determined to be a navigable area. The path trajectory formed by connecting the road centerline points in this area is extended to both sides to form a flight buffer zone, and the remaining area is used as a no-entry buffer zone.

[0084] Step S14: Extract the takeoff position points of the flight buffer zone; based on meteorological data, obtain the wind speed and visibility of the takeoff position points; when the wind speed is less than the set wind speed threshold and the visibility is higher than the set visibility threshold, it is judged as a safe takeoff point, and a set of safe takeoff points is obtained; sort the set of safe takeoff points by the shortest distance, and connect adjacent points in sequence according to the relative position of the points to form a continuous boundary line; connect the end of the boundary line to form the flight safety zone.

[0085] In one embodiment, the building boundary is extracted from the point cloud data using the RANSAC algorithm to extract the outer contour polygon; the road centerline is extracted based on the center point coordinates in the GIS road vector map, in the format of Polyline, with an interval accuracy of 1m.

[0086] It should be noted that, in the Python environment, the Shapely library is used to perform spatial overlay analysis on the building boundaries and centerlines of each slice to determine whether the slice contains a road centerline point and to assess whether that centerline point is more than 30m outside the building boundary. If five consecutive slices (consecutively numbered and with continuous centerlines) meet this condition, they are considered to constitute a continuous analyzable segment, which is then numbered, recorded, and used as the basis for subsequent determination of navigation areas.

[0087] In another embodiment, the minimum Euclidean distance between the centerline point and the nearest building boundary point within each continuous slice is calculated using the following formula: ;

[0088] It should be noted that the minimum safe navigation width threshold of 15 meters is used as the criterion. If the distance from all centerline points in five consecutive slices to the building boundary is greater than 15 meters, then the navigation width requirement is initially met.

[0089] It should be noted that the average altitude of the centerline point (calculated from the Z-value in the point cloud) is compared with the heights of buildings within a 50-meter radius. If the maximum altitude difference is less than a set threshold of 10 meters, the area is considered to have high-altitude navigation stability. After meeting the above two conditions, the centerline sequence is extracted, and a buffer operation is used to extend the centerline by 5 meters on each side to generate a flight buffer zone (Polygon). All areas outside the Polygon are defined as no-entry buffer zones and marked as no-fly zones.

[0090] In another embodiment, candidate takeoff points are sampled at equidistant intervals of 50 meters from the boundary of the flight buffer zone to obtain their latitude and longitude coordinates. Based on the wind speed and visibility values ​​recorded in the meteorological data, spatial matching is performed with these coordinates (using the KD tree nearest neighbor matching method, with spatial error controlled within 1 meter) to extract the corresponding meteorological sampling point data.

[0091] It should be noted that a wind speed threshold of 7 m / s and a visibility threshold of 3000 meters are set. Candidate points with wind speeds less than 7 m / s and visibility greater than 3000 meters are marked as safe takeoff points. After extracting the set of all takeoff points that meet the conditions, the Dijkstra algorithm is used to calculate the shortest straight-line distance between each point on the GIS map, and the points are sorted according to the shortest path. The sorted points are then connected sequentially using Polylines to generate continuous boundary lines. Using a line closure algorithm, the boundary lines are connected end to end to form a closed polygon, ultimately forming the local flight safety zone.

[0092] Preferably, step S2 includes the following steps:

[0093] Step S21: Convert the flight safety zone into a three-dimensional grid space. Within a 15-second prediction time window, detect the spatial intersection of all UAV trajectory points and the flight safety zone of other aircraft at 1-second intervals. If the trajectory point appears three or more times in the flight safety zone of other aircraft, it is determined to be a flight conflict path.

[0094] Step S22: Extract flight missions with conflicting flight paths; calculate the mission urgency of the flight missions;

[0095] Step S23: Extract the remaining battery power of all UAVs along the flight path; generate a task priority ranking table using the remaining battery power and task urgency; detect obstacles on the flight conflict path and obtain obstacle data; remove obstacle paths on the flight conflict path based on the obstacle data to determine the obstacle avoidance path;

[0096] Step S24: Generate the first flight path set based on the obstacle avoidance path and the task priority sorting table;

[0097] Step S25: Send the first flight path set to the UAV and perform UAV scheduling to generate the second flight path set.

[0098] In one embodiment, a voxel grid method is used to transform the entire flight airspace into a unified three-dimensional grid spatial coordinate system with a 3D grid granularity of 50m × 50m × 20m. Based on the current position information (x, y, z) and flight speed (m / s) and flight direction (angle) of each UAV, the spatial coordinates of 15 predicted trajectory points are calculated sequentially in 1-second increments within a 15-second prediction time window.

[0099] It should be noted that this prediction is based on the formula for calculating uniform linear motion: ,in As the initial coordinates, For speed, For time, The direction vector is a unit vector. The predicted trajectory points of all UAVs are stored frame by frame. At each time point, the predicted trajectory point number of a UAV is compared with the 3D grid number corresponding to the flight safety area of ​​other UAVs. If the predicted trajectory point number of a certain UAV appears in the safety grid number of other UAVs three times or more, the current trajectory is determined to be a flight conflict path, and its corresponding conflict start and end grid number and corresponding time point are recorded.

[0100] In another embodiment, task urgency is calculated based on the remaining task time and the task path length, with a standard urgency time window set at 300 seconds. When the remaining task time is less than this standard time window, an urgency coefficient is calculated. The remaining distance of the path is calculated by summing the lengths of each path segment using GPS coordinates, in meters. The final task urgency is then calculated in seconds per meter.

[0101] In another embodiment, the power data is updated by the onboard BMS (Battery Management System) at a frequency of 10Hz and uploaded to the ground dispatch center. The remaining battery power of all path-related drones is aggregated and associated with the task urgency list generated in step S22 to construct a priority ranking table. The ranking principle is: tasks with higher urgency and larger remaining battery power have higher priority. On the flight conflict path, based on the city's 3D building model and road structure data, the ray projection method is used to detect the intersection area between the path and obstacles. Starting from each path point, rays are extended 10 meters, 30 meters, and 50 meters in its flight direction to determine whether they intersect with building facades or elevated road models. If they intersect, they are marked as obstacle paths. The obstacle paths are reconstructed, and a new path to avoid obstacles is calculated between the start and end points to obtain the obstacle avoidance path. The obstacle avoidance path requires the minimum detour distance and the area traversed does not overlap with the identified conflict paths.

[0102] In another embodiment, the first flight path set is a set of UAV flight paths generated based on obstacle avoidance paths and task priority ranking results, and its scope is limited to the intersection of the navigable area and the flight buffer area in the urban low-altitude three-dimensional grid space.

[0103] It should be noted that the path set is generated in order of task priority, constructing paths for each UAV in a 3D flight buffer zone. This buffer zone is the flyable space predicted 15 seconds ahead of each UAV's current trajectory. Within this space, 3D nodes are sampled in 2-meter increments, and directed edges are established between nodes. The edge weights comprehensively consider the actual Euclidean distance between nodes and the impact of wind speed disturbances in the corresponding direction (wind speed disturbance values ​​are analyzed in real-time based on regional meteorological data). During the path planning phase, Dijkstra's algorithm is used to find the path with the minimum total cost from the current UAV position to the target point, while avoiding nodes that spatially intersect with the predicted trajectories of other UAVs.

[0104] In another embodiment, the system first obtains the obstacle avoidance path and task priority ranking table, and constructs a three-dimensional node mesh in 2-meter units within the expected flight buffer area of ​​the UAV (a spatial range predicted forward for 15 seconds based on the current state). Directed edges are established between nodes, and the edge weights comprehensively consider the flight distance and wind speed disturbance direction. Dijkstra's algorithm is used to plan the shortest cost path from the starting point to the target point for each UAV, and nodes that spatially overlap with the predicted trajectories of other UAVs are preferentially eliminated during path screening to reduce the risk of conflict. This path set is then sent to the UAV via the MQTT communication protocol. The scheduling system sends back the flight status every 5 seconds to verify the trajectory. If an offset of more than 5 meters is found, the path is determined to be invalid, and the system returns to step S21 to start the path reconstruction process.

[0105] Preferably, step S3 includes the following steps:

[0106] Step S31: Perform path matching based on the first flight path set and the second flight path set to determine the flight error path;

[0107] Step S32: Calculate the horizontal and vertical errors based on the flight error path; sum the squares of the horizontal and vertical errors to determine the flight error;

[0108] Step S33: Determine the heading compensation value based on the flight error;

[0109] Step S34: In response to the heading compensation value, generate flight control commands.

[0110] In one embodiment, the first path of each UAV and its second path adopted after scheduling reconstruction are extracted. The set of start point, end point, and intermediate path node numbers in the path array are read respectively, and the node coordinates are compared with a tolerance threshold of 0.5 meters. If there is a point on either path where the spatial position deviation exceeds the threshold, the point is recorded as a flight error path node. The construction of the error path is based on three cases: first, the path node numbers are completely inconsistent; second, the numbers are consistent but the three-dimensional coordinate deviation exceeds the threshold; and third, the path lengths are unequal, causing the end point to shift.

[0111] In another embodiment, let the first path node be P1(x1,y1,z1) and the second path node be P2(x2,y2,z2), then the formula for calculating the horizontal error is: The formula for calculating vertical error is: The Eh and Ev values ​​for all nodes are recorded separately, forming two error arrays. The error aggregation method involves summing the squares of the horizontal and vertical errors for each pair of nodes, and the final flight error E is calculated using the following formula: .

[0112] In another embodiment, the start and end vectors of the error path are represented as two direction vectors. and These correspond to the three-dimensional headings of the expected path and the actual path, respectively. The direction vector is calculated as follows: Let the starting point of the path be (x1, y1, z1) and the ending point be (x2, y2, z2), then the direction vector is... The direction deviation angle is calculated using the vector angle formula. , where “⋅” represents the vector dot product and “||||” represents the vector magnitude. The yaw angle error is derived from α, in degrees. To improve accuracy, all angle calculation results are rounded to two decimal places.

[0113] In another embodiment, a real-time flight control command sequence is generated based on the heading compensation value θ and the duration T. θ is converted into a yaw angle control command, with the command frequency set to 10Hz, meaning one control message is sent every 100ms. In the command parameters, attitude control maintains a pitch angle of 0° and a roll angle of 0°, with only the yaw angle being dynamically updated. Each control command is accompanied by a unique timestamp and a UAV identifier to ensure correct command timing. The transmission mechanism employs QoS1 level protection in the MQTT channel to ensure command delivery. The controller executes each received flight control command and transmits the feedback status back to the scheduling system in real time. The entire control cycle equals the value T, and control parameters are linearly distributed piecewise across the θ value to achieve smooth compensation.

[0114] It should be noted that if θ exceeds 30°, in order to prevent the drone from yawing violently and causing instability, the strategy is to divide θ into 3 stages, with each stage lasting for T / 3, and apply compensation of 10°, 10°, and θ-20° respectively.

[0115] Preferably, step S33 includes the following steps:

[0116] Step S331: Calculate the spatial offset vector based on flight error;

[0117] Step S332: Calculate the heading deviation angle based on the spatial offset vector and the preset theoretical heading value;

[0118] Step S333: Use median filtering to remove abnormal deviations in the heading deviation angle to obtain the standard heading deviation angle;

[0119] Step S334: Calculate the mean heading angle of the standard heading deviation angle;

[0120] Step S335: Determine the heading compensation value based on the average heading angle.

[0121] In one embodiment, a one-to-one correspondence is performed between the first set of flight paths (theoretical paths) and the second set of flight paths (actual flight paths) at the same timestamp. Each corresponding point contains its three-dimensional coordinate information (x, y, z). For each matching point at any given time, the Cartesian coordinate difference method is used to calculate the offset values ​​between the actual coordinates and the theoretical coordinates in the x, y, and z directions, respectively, using the following formula: , The three-dimensional offset values ​​are used to construct a spatial offset vector. For each time step in each flight path, the above operation is performed to generate a sequence of spatial offset vectors of the same length as the number of time steps.

[0122] In another embodiment, the theoretical heading value is determined based on the coordinate difference of the first flight path set between two consecutive time steps, i.e., assuming the theoretical position at time t is... At time t+1, Then the theoretical heading vector is The actual heading vector is based on the spatial offset vector. and The direction of the line connecting the two vectors is determined. Calculate the angle θ between these two vectors using the dot product formula: Then, θ is calculated using the arccosine function (arccos), in radians, which can be converted to an angle. This angle is the heading deviation angle. To ensure the accuracy of the heading angle, quaternion rotation or Euler angle transformation methods are used to normalize all angle values ​​in three-dimensional space to eliminate the influence of coordinate system rotation errors.

[0123] In another embodiment, the heading deviation angle values ​​are compiled into a time series and processed using a sliding window median filter with a window size of 5. The median filtering steps are as follows: Using the current position as the center, select four heading angle values ​​before and after the current position, plus the current value, for a total of five values. Calculate the median value after sorting these values, and use it as the standard heading deviation angle for the current time step. Using the median instead of the average value is to eliminate the influence of abrupt changes on the final deviation calculation.

[0124] In another embodiment, the mean of the standard heading deviation angle sequence after median filtering is calculated. The arithmetic mean method is used, and the formula is: Where θ_i is the standard heading deviation angle at the i-th time step, and N is the total length of the angle sequence. This mean is the core angle value used for the final heading compensation calculation. To avoid zero-point looping errors (for example, the mean of 359° and 1° should be 0° instead of 180°), a unit vector projection method is used, that is, all angles are first converted into two-dimensional vector coordinates, and then the mean is calculated to inversely deduce the angle.

[0125] It should be noted that the specific method is as follows: , then calculate Obtain the angle.

[0126] In another embodiment, the compensation value is compared with the UAV's current heading angle. If the deviation exceeds a set threshold (e.g., ±2°), it is marked as requiring heading adjustment. The threshold value is set based on the measured UAV flight error tolerance range, with ±2° as the standard minimum perceptible correction unit. The compensation value is output as an integer angle and fed into the flight control module to adjust the UAV's control surface offset or rotor differential speed strategy to achieve heading correction.

[0127] Preferably, step S4 includes the following steps:

[0128] Step S41: Execute flight control commands and monitor the drone's status;

[0129] Step S42: When the frequency of continuous loss of communication data packets of the drone status exceeds the fault tolerance threshold, it is determined that the communication link is interrupted and recorded as an abnormal flight status;

[0130] Step S43: Receive abnormal flight status and trigger automatic return command to obtain the first corrected flight trajectory.

[0131] In one embodiment, during flight, the monitoring module performs status monitoring through continuous communication data feedback. Specifically, status data packets are sent to the dispatch center at 100ms intervals, and include fields such as GPS location information, flight speed, attitude angles (pitch, yaw, roll), remaining battery power, and network signal strength. The monitoring server parses the data packets uploaded by each UAV in parallel using a multi-threaded approach, caches them using high-speed message queues such as Redis or Kafka, and archives them by flight number.

[0132] In another embodiment, the communication connection status of each drone is maintained by a dedicated daemon thread, which checks the number of valid data packets within the most recent N seconds (N is 3 seconds in this embodiment) every second. If no valid status data packets are received for 3 consecutive seconds, and there is no obvious interference signal on the wireless link during this period (RSSI greater than -85dBm), then the communication link is determined to be interrupted.

[0133] It should be noted that the fault tolerance threshold is preset to the loss of 30 consecutive data packets (i.e., 3 consecutive seconds), with a configuration period of 100ms. Upon reaching this threshold, the drone's flight status field is immediately updated to "abnormal" in the database, and information such as the GPS location, timestamp, flight altitude, and flight direction of the interruption is recorded and written to the flight abnormality log table. This log is used for subsequent return-to-home logic triggering and supports real-time highlighting of alerts on the interface layer.

[0134] In another embodiment, the return path starts from the last valid GPS coordinate point recorded by the current UAV and ends at the takeoff point (recorded in the flight mission initialization configuration). The return path planning uses a three-dimensional point set linear interpolation method, maintaining a constant altitude (e.g., 50 meters) along the current altitude. If airspace obstacles exist (obstacle mesh models marked on the city's 3D map are higher than the current flight altitude), the vertical ascent altitude is adjusted to the top of the obstacle plus a safety margin (set to 10 meters), and the path segment is re-interpolated. The return command includes the return path point set, flight speed (default 5 m / s), and target point spacing threshold (set to 2 meters to determine if a path point has been reached). This information is encapsulated in an emergency control command format and broadcast to the terminal corresponding to the current UAV ID via the LTE network. Upon receiving the return control command, the UAV immediately interrupts the current mission command, loads the new return path, and autonomously executes the first corrected flight trajectory.

[0135] This specification also provides a multimodal data-based urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling system for executing the multimodal data-based urban low-altitude UAV fleet scheduling method described above. The multimodal data-based urban low-altitude UAV fleet scheduling system includes:

[0136] The flight safety zone identification module is used to acquire multimodal data of UAVs within the target area, including urban 3D maps and meteorological data; determine the navigable area of ​​the urban 3D map; and delineate the flight safety zone of the navigable area based on the meteorological data.

[0137] The UAV scheduling module is used to predict flight conflict paths based on flight safety areas; generate a first flight path set using the flight conflict paths, send the first flight path set to the UAVs, and perform UAV scheduling to generate a second flight path set;

[0138] The heading compensation value module is used to determine the flight error based on the first flight path set and the second flight path set; determine the heading compensation value based on the flight error; and generate flight control commands in response to the heading compensation value.

[0139] The flight trajectory correction module is used to execute flight control commands and monitor the status of the UAV. If the communication link of the UAV is interrupted, it is recorded as an abnormal flight status and an automatic return command is triggered to obtain the first corrected flight trajectory.

[0140] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0141] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for scheduling urban low-altitude unmanned aerial vehicle (UAV) fleets based on multimodal data, applied to UAVs, characterized in that, Includes the following steps: Step S1: Acquire multimodal data of UAVs within the target area, including 3D city maps and meteorological data; Determine navigable areas on a 3D map of the city; delineate flight safety zones within navigable areas based on meteorological data; Step S2: Predict flight conflict paths based on flight safety zones; generate a first flight path set using the flight conflict paths, send the first flight path set to the UAV, and perform UAV scheduling to generate a second flight path set; wherein, step S2 includes the following steps: Step S21: Convert the flight safety zone into a three-dimensional grid space. Within a 15-second prediction time window, detect the spatial intersection of all UAV trajectory points and the flight safety zone of other aircraft at 1-second intervals. If the trajectory point appears three or more times in the flight safety zone of other aircraft, it is determined to be a flight conflict path. Step S22: Extract flight missions with conflicting flight paths; calculate the mission urgency of the flight missions; Step S23: Extract the remaining battery power of all UAVs along the flight path; generate a task priority ranking table using the remaining battery power and task urgency; detect obstacles on the flight conflict path and obtain obstacle data; remove obstacle paths on the flight conflict path based on the obstacle data to determine the obstacle avoidance path; Step S24: Generate the first flight path set based on the obstacle avoidance path and the task priority sorting table; Step S25: Send the first flight path set to the UAV and perform UAV scheduling to generate the second flight path set; Step S3: Determine the flight error based on the first flight path set and the second flight path set; determine the heading compensation value based on the flight error; generate flight control commands in response to the heading compensation value; Step S4: Execute flight control commands and monitor the UAV status; if the UAV status shows a communication link interruption, record it as an abnormal flight status and trigger an automatic return-to-home command to obtain the first corrected flight trajectory.

2. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 1, characterized in that, The method further includes: Obtain meteorological disturbance parameters and communication interference parameters of the airspace where the UAV is currently located; Determine the attitude disturbance compensation value based on meteorological disturbance parameters; Determine the speed disturbance compensation value based on communication interference parameters; Based on the attitude disturbance compensation value and the velocity disturbance compensation value, the flight control command is compensated and corrected to obtain the corrected control command; The first corrected flight path is corrected a second time using the correction control command to generate the second corrected flight path.

3. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 2, characterized in that, The attitude disturbance compensation value is determined based on meteorological disturbance parameters, including: Wind speed disturbance values ​​and airflow gradient change rates are extracted based on meteorological disturbance parameters; By using wind speed disturbance values, the disturbance angle of the UAV in the pitch direction can be obtained; Based on the rate of change of the airflow gradient, the disturbance angle of the UAV in the roll direction is obtained; Calculate the pitch attitude compensation value based on the disturbance angle and the preset pitch stability angle; Calculate the roll attitude compensation value based on the disturbance angle and the preset roll stability angle; Based on the pitch attitude compensation value and the roll attitude compensation value, the attitude disturbance compensation value is determined.

4. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 2, characterized in that, The speed disturbance compensation value is determined based on communication signal interference parameters, including: Convert communication signal interference parameters into a signal-to-noise ratio sequence; Obtain flight speed data from the flight log; The signal-to-noise ratio sequence and flight speed data are time-stamped to obtain the perturbation sequence; Velocity components along the X, Y, and Z coordinate axes are extracted using a perturbation sequence; Calculate the three-dimensional velocity vector based on the velocity components; Calculate the average difference based on the three-dimensional velocity vector; The speed disturbance compensation value is determined based on the average difference.

5. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Acquire multimodal data of UAVs within the target area, including 3D city maps and meteorological data; Step S12: Based on the building boundary points and road centerline points in the city 3D map, filter out 5 or more consecutive map tile data; Step S13: When the spatial distance between the building boundary points and the road centerline points in the continuous map tiles is maintained above the preset safe navigation width and the height difference is below the set threshold, it is determined to be a navigable area. The path trajectory formed by connecting the road centerline points in this area is extended to both sides to form a flight buffer zone, and the remaining area is used as a no-entry buffer zone. Step S14: Extract the takeoff position points of the flight buffer zone; based on meteorological data, obtain the wind speed and visibility of the takeoff position points; when the wind speed is less than the set wind speed threshold and the visibility is higher than the set visibility threshold, it is judged as a safe takeoff point, and a set of safe takeoff points is obtained; sort the set of safe takeoff points by the shortest distance, and connect adjacent points in sequence according to the relative position of the points to form a continuous boundary line; connect the end of the boundary line to form the flight safety zone.

6. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Perform path matching based on the first flight path set and the second flight path set to determine the flight error path; Step S32: Calculate the horizontal and vertical errors based on the flight error path; sum the squares of the horizontal and vertical errors to determine the flight error; Step S33: Determine the heading compensation value based on the flight error; Step S34: In response to the heading compensation value, generate flight control commands.

7. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 6, characterized in that, Step S33 includes the following steps: Step S331: Calculate the spatial offset vector based on flight error; Step S332: Calculate the heading deviation angle based on the spatial offset vector and the preset theoretical heading value; Step S333: Use median filtering to remove abnormal deviations in the heading deviation angle to obtain the standard heading deviation angle; Step S334: Calculate the mean heading angle of the standard heading deviation angle; Step S335: Determine the heading compensation value based on the average heading angle.

8. The urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Execute flight control commands and monitor the drone's status; Step S42: When the frequency of continuous loss of communication data packets of the drone status exceeds the fault tolerance threshold, it is determined that the communication link is interrupted and recorded as an abnormal flight status; Step S43: Receive abnormal flight status and trigger automatic return command to obtain the first corrected flight trajectory.

9. A city low-altitude unmanned aerial vehicle (UAV) fleet scheduling system based on multimodal data, characterized in that, For executing the urban low-altitude unmanned aerial vehicle (UAV) fleet scheduling method based on multimodal data as described in claim 1, the urban low-altitude UAV fleet scheduling system based on multimodal data includes: The flight safety zone identification module is used to acquire multimodal data of UAVs within the target area, including urban 3D maps and meteorological data; determine the navigable area of ​​the urban 3D map; and delineate the flight safety zone of the navigable area based on the meteorological data. The UAV scheduling module is used to predict flight conflict paths based on flight safety areas; generate a first flight path set using the flight conflict paths, send the first flight path set to the UAVs, and perform UAV scheduling to generate a second flight path set; The heading compensation value module is used to determine the flight error based on the first flight path set and the second flight path set; determine the heading compensation value based on the flight error; and generate flight control commands in response to the heading compensation value. The flight trajectory correction module is used to execute flight control commands and monitor the status of the UAV. If the communication link of the UAV is interrupted, it is recorded as an abnormal flight status and an automatic return command is triggered to obtain the first corrected flight trajectory.

Images