An unmanned aerial vehicle intelligent management and control and command platform

Abstract

The application discloses an unmanned aerial vehicle intelligent management and control and command platform, which comprises a multi-modal artificial intelligence (AI) fusion intelligent identification and event real-time triggering and disposal system and an environment and task self-adaptive intelligent flight path planning system; data intercommunication and module linkage are achieved between the two systems, and a full-process intelligent closed-loop management and control system from multi-source sensing, intelligent identification, event triggering, flight path planning, dynamic adjustment to feedback optimization is jointly constructed. The application belongs to the technical field of unmanned aerial vehicle low-altitude management and control, and the structure can realize multi-modal fusion high-precision intelligent identification, algorithm dynamic scheduling, full-process automatic closed loop of identification, event and disposal, environment, task and equipment self-adaptive dynamic flight path planning, multi-machine cooperative obstacle avoidance and energy consumption optimization.

Description

Technical Field

[0001] This invention relates to the field of low-altitude control technology for unmanned aerial vehicles (UAVs), and in particular to an intelligent control system for UAVs. Background Technology

[0002] In the practical application of regional-level UAV low-altitude integrated service platforms, two major challenges are faced: Firstly, in terms of problem identification and incident handling, existing platforms mostly rely on single-modal data for identification, which is easily affected by environmental interference and has limited accuracy. Artificial intelligence AI algorithms are fixed and rigid, and incident generation and responsibility matching are highly dependent on manual intervention, resulting in slow response and low efficiency. Secondly, in terms of route planning, static routes are generally pre-set manually, which cannot be dynamically adjusted according to real-time environment such as weather, airspace and mission requirements. Multi-aircraft coordination is prone to conflicts, and there is a lack of energy consumption optimization and active protection mechanisms for flight anomalies, which seriously affects operational safety and efficiency.

[0003] Therefore, there is an urgent need to develop an intelligent control and command platform for unmanned aerial vehicles (UAVs) to solve the problems in existing technologies. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent management and command platform for unmanned aerial vehicles (UAVs) that can achieve high-precision intelligent identification through multimodal fusion, dynamic algorithm scheduling, and a fully automated closed loop for identification, events, and handling. It also features dynamic route planning that adapts to the environment, tasks, and equipment, multi-UAV collaborative obstacle avoidance, and energy consumption optimization. Furthermore, it has a simple structure and is easy to use, thereby solving the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A drone intelligent control and command platform, comprising: A multimodal artificial intelligence (AI) fusion system for intelligent recognition and real-time event triggering and handling, and an intelligent route planning system that adapts to the environment and tasks; The two systems share data and link modules to jointly build a full-process intelligent closed-loop management and control system, from multi-source perception, intelligent identification, event triggering, route planning, dynamic adjustment to feedback optimization; The multimodal artificial intelligence (AI) fusion intelligent recognition and real-time event triggering and handling system includes a multimodal perception fusion module, an AI algorithm dynamic scheduling module, a real-time and post-processing AI recognition data complementarity module, an intelligent event real-time triggering module, and a problem priority intelligent classification module, which are connected in sequence to form a data closed loop. The intelligent route planning system that adapts to the environment and tasks includes a multi-source environmental perception fusion module that links with environmental data in real time, a task-route parameter intelligent matching module, an environment-adaptive dynamic route adjustment module, a multi-aircraft collaborative route planning and obstacle avoidance module, an equipment status-route energy consumption optimization module, and a waypoint anomaly real-time early warning and self-correction module. The platform supports switching between intelligent and manual modes, and the two systems are seamlessly integrated with the original basic modules of the regional-level UAV low-altitude integrated service platform without the need to reconstruct the original system.

[0006] By adopting the above technical solutions and through the modular connection and data closed loop of the two major systems, integrated intelligent operation and continuous optimization are achieved, from data collection, intelligent identification, automatic event generation and hierarchical response, to dynamic route planning, real-time adjustment, multi-aircraft collaboration and anomaly self-correction.

[0007] As a further aspect of the present invention: the multimodal perception fusion module uses spatiotemporal correlation data fusion technology to align and remove redundancy between the UAV's multi-source perception data and the platform's geographic airspace data, and uses a hybrid CNN and Transformer model to simultaneously extract local spatial features and global spatiotemporal correlation features to construct a unified multimodal fusion feature library; The multi-source sensing data includes UAV visual data, laser ranging data, and environmental sensing data; the platform's geographic airspace data includes operational areas, restricted flight areas, and geographic grid data.

[0008] By adopting the above technical solution, a unified and standardized multimodal fusion feature library was constructed by performing spatiotemporal alignment and deep feature fusion on multi-source heterogeneous data. This provides a reliable and comprehensive information source for all subsequent artificial intelligence (AI) recognition tasks, and significantly improves the quality of input data for recognition algorithms.

[0009] As a further aspect of the present invention: the AI ​​algorithm dynamic scheduling module adopts a rule engine and reinforcement learning fusion architecture to build a scheduling model, and trains it using historical operation data of the platform to achieve dynamic algorithm scheduling based on task type and flight scenario; The scheduling model supports hot-plugging of algorithms, visual configuration, and online incremental training. Its reinforcement learning reward function is based on a weighted calculation of recognition accuracy and running time.

[0010] By adopting the above technical solutions and using a scheduling model that integrates rules and learning, the appropriate algorithm is dynamically matched according to real-time tasks and scenarios, and flexible configuration and continuous learning are supported, ensuring the efficiency, adaptability and evolvability of the recognition system.

[0011] As a further aspect of the present invention: the real-time-post-processor AI recognition data complementary module realizes bidirectional interaction between real-time frame-sampling data and post-processor high-definition data through a low-latency data transmission protocol, with the transmission latency controlled within 100ms; The module establishes a feature library sharing mechanism, enabling real-time recognition and post-recognition to share the same multimodal feature library, and to complement each other's data in terms of false alarm filtering and recognition accuracy.

[0012] By adopting the above technical solutions and constructing a low-latency bidirectional data channel and feature sharing mechanism, real-time rapid identification and post-processing fine identification can be mutually verified and supplemented, effectively filtering false alarms and significantly improving the reliability of the final identification results.

[0013] As a further aspect of the present invention: the preset rules of the intelligent event real-time triggering module include a dynamic relational database of problem type, handling subject, and triggering conditions. The event information includes at least the problem type, shooting location, latitude and longitude, and associated media data, and is automatically bound to the platform grid responsible subject to realize the integration of event generation and responsibility traceability.

[0014] By adopting the above technical solution, the identification results are automatically associated with the handling entity and bound to complete event information through a preset rule base, thus constructing an automated link for problem discovery, event generation, and responsibility assignment, which greatly improves the initiation efficiency and traceability of event handling.

[0015] As a further aspect of the present invention: the problem priority intelligent classification module uses a fuzzy comprehensive evaluation model to quantitatively evaluate events, taking problem type, scope of impact, and occurrence area as evaluation indicators, and pushes them in a graded manner based on priority scores; The results of the handling are fed back to each artificial intelligence (AI) module to enable model self-learning and parameter optimization.

[0016] By adopting the above technical solutions, events are prioritized through a quantitative evaluation model to ensure priority response to emergencies; at the same time, the handling results are fed back to the model to drive the entire identification and handling system to continuously learn and evolve, forming a virtuous cycle of becoming smarter the more it is used.

[0017] As a further aspect of the present invention, the workflow of the environment- and task-adaptive intelligent route planning system is as follows: T1: The multi-source environmental perception fusion module collects UAV environmental sensor data, equipment status data, platform real-time airspace data and geographic grid data, and builds a dynamic environment, airspace and geographic fusion database and updates it in real time. T2: The task-route parameter intelligent matching module, based on the fused database, task type, and UAV performance data, outputs the optimal basic parameters of the route with one click through the parameter matching model, and supports manual fine-tuning; T3: The environment-adaptive dynamic flight path adjustment module, based on the optimal flight path parameters, the fusion database, and the UAV's real-time location data, uses a hybrid path planning algorithm and an adjustment threshold model to achieve real-time dynamic flight path adjustment. T4: When multiple drones work together, the multi-drone collaborative route planning and obstacle avoidance module realizes conflict-free route planning and dynamic obstacle avoidance during flight based on dynamic route data and the status data of each drone. T5: Equipment Status - Route Energy Consumption Optimization Module. Based on route data and UAV equipment status data, it achieves optimal energy consumption planning through energy consumption model and dynamic programming algorithm, sets alternate landing points and marks breakpoints for resuming flight. T6: The waypoint anomaly real-time warning and self-correction module is based on energy-optimal route data and fusion database. It verifies waypoint parameters in real time, realizes automatic correction and warning of anomalies, and triggers hovering and waits for manual intervention when it cannot be corrected.

[0018] By adopting the above technical solutions, the process starts with multi-source environmental perception and sequentially completes intelligent parameter matching, dynamic route adjustment, multi-aircraft collaborative obstacle avoidance, energy consumption optimization and anomaly protection, realizing automated and intelligent management of the entire lifecycle of the route from generation to execution to safety monitoring.

[0019] As a further aspect of the present invention: the environment adaptive dynamic route adjustment module adopts a hybrid path planning algorithm that combines the A* algorithm with the artificial potential field method, and has a route adjustment threshold model that triggers route adjustments based on wind speed, rainfall, and airspace status to avoid meaningless frequent adjustments; The adjusted flight path data is synchronized to the platform's flight control center and the drone terminal in real time.

[0020] By adopting the above technical solutions, fast and flexible local obstacle avoidance and replanning are achieved through a hybrid path planning algorithm. At the same time, a threshold model is used to avoid overreacting to environmental disturbances, ensuring timely, effective and smooth route adjustments. Real-time synchronization ensures the consistency of control commands.

[0021] As a further aspect of the present invention: the multi-aircraft collaborative route planning and obstacle avoidance module adopts a distributed collaborative algorithm to achieve multi-aircraft load balancing and conflict-free route planning, and adopts obstacle avoidance technology that combines ultrasonic and visual methods to monitor the relative positions of multiple aircraft in real time and trigger a dynamic obstacle avoidance mechanism when the distance is ≤50m. The route planning results are synchronized in real time to each drone terminal and the panoramic command module of the platform.

[0022] By adopting the above technical solutions, the system achieves a reasonable allocation of tasks and resources through distributed collaborative algorithms and utilizes fusion perception technology to achieve real-time dynamic obstacle avoidance during flight, thereby maximizing the overall operational efficiency of the multi-aircraft system and fundamentally avoiding the risk of collisions.

[0023] As a further aspect of the present invention: the device status-route energy consumption optimization module constructs an UAV energy consumption model, quantifies the relationship between flight altitude, speed, wind speed and battery consumption, and uses a dynamic programming algorithm to achieve optimal route energy consumption planning, automatically marks breakpoints and resume flight points and connects with the platform's breakpoint resume flight function. The waypoint anomaly real-time warning and self-correction module is equipped with a waypoint anomaly real-time verification model, which can automatically correct waypoint altitude, speed and yaw angle anomalies, and trigger hovering and warning when it cannot be corrected.

[0024] By adopting the above technical solutions, the flight time and distance of a single operation are extended through refined energy consumption models and optimization algorithms, and the fault tolerance of the mission is improved by combining the breakpoint resume flight mechanism; at the same time, the real-time verification and self-correction of waypoint parameters provide a final automated protection for flight safety.

[0025] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs an intelligent identification and real-time event triggering and handling system that integrates multimodal artificial intelligence (AI), achieving a fully automated closed loop from multi-source data fusion perception, dynamic scheduling of AI algorithms, to automatic event generation and hierarchical handling. This significantly improves the accuracy of problem identification, the speed of event response, and the level of intelligence in handling processes in complex scenarios. This invention constructs an intelligent flight path planning system that adapts to the environment and tasks, enabling intelligent matching of flight path parameters, dynamic adjustment based on real-time environment, conflict-free multi-aircraft collaborative operation, and global optimization of energy consumption and safety. This comprehensively ensures the flight autonomy, safety, and mission execution efficiency of UAVs in various tasks and environments.

[0026] Other features and advantages of the present invention will be disclosed in detail in the following detailed description and accompanying drawings. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the steps of a multimodal AI fusion intelligent recognition and real-time event triggering and handling system for an intelligent drone control and command platform according to an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] In this embodiment of the invention, an intelligent control and command platform for unmanned aerial vehicles (UAVs) is described below. Figure 1 As shown, the system comprises two core intelligent systems that work collaboratively to achieve intelligent management and control of drones throughout the entire process, from mission planning and flight execution to incident handling. These two core systems are: a multimodal artificial intelligence (AI) fusion-based intelligent recognition and real-time event triggering and handling system, and an environment- and mission-adaptive intelligent flight path planning system. These two systems can operate independently or collaborate through data interfaces and module linkages to complete complex drone management tasks.

[0030] I. Platform Overall Architecture and Collaboration Mechanism The platform's overall architecture adopts a layered design, from bottom to top: a multi-source perception layer, an intelligent processing layer, an execution control layer, and a feedback optimization layer. The multimodal AI-integrated intelligent recognition and real-time event triggering and handling system is primarily deployed in the intelligent processing layer, responsible for data processing and decision generation. The environment- and task-adaptive intelligent flight path planning system spans both the intelligent processing and execution control layers, responsible for flight path generation and dynamic control. These two systems seamlessly connect through standard data interfaces defined within the platform, and achieve data interoperability and functional linkage with the existing basic modules of the regional-level UAV low-altitude integrated service platform, such as the system management module, panoramic command module, flight control center module, and event handling module, without requiring reconstruction of the original system.

[0031] The platform supports both intelligent and manual operation modes, allowing users to switch flexibly according to task requirements and scenario complexity. In intelligent mode, the platform automatically performs the entire process of identification, event generation, and route planning; in manual mode, operators can intervene or adjust any step through a visual interface, ensuring the system's flexibility and reliability.

[0032] II. Intelligent Recognition and Real-time Event Triggering and Handling System Based on Multimodal Artificial Intelligence (AI) This system aims to address the problems of low recognition accuracy, reliance on manual handling, and slow response of traditional drone platforms in complex scenarios. The various modules within the system are connected sequentially to form a closed data loop encompassing data acquisition, feature fusion, algorithm scheduling, complementary recognition, event triggering, and priority allocation. The specific modules and processes are as follows: (a) Multimodal sensing fusion module This module provides a high-quality, standardized data foundation for the entire identification and handling system. Its core function is to perform spatiotemporal alignment, fusion processing, and deep feature extraction on various heterogeneous data from UAVs and platforms.

[0033] The input data includes two main categories: 1. Multi-source perception data from UAVs: This includes visual data collected by visible light cameras, infrared thermal imagers, and other equipment carried by the UAV; laser ranging and point cloud data collected by lidar or laser ranging modules; and environmental sensing data collected by airborne meteorological sensors, such as real-time wind speed, precipitation, light intensity, and atmospheric pressure.

[0034] 2. Platform geospatial data: including the boundary coordinates of pre-defined operating areas, restricted flight zones and no-fly zones, as well as geographic grid data based on Geographic Information System (GIS), with each grid associated with pre-defined responsible entity information.

[0035] The data processing process consists of two main steps: First, spatiotemporal correlation data fusion technology is used to process the input data. This technology aligns all multi-source heterogeneous data to the same spatiotemporal reference through a unified timestamp synchronization mechanism and spatial coordinate transformation algorithm, effectively eliminating data redundancy and deviation caused by differences in sensor sampling frequency, accuracy, and location, forming a spatiotemporally consistent original dataset.

[0036] Secondly, a hybrid neural network model combining CNN and Transformer is employed to extract cross-modal features from the aforementioned dataset. Specifically, a CNN branch is used to extract local spatial features of the target, such as edges, textures, and shapes, from visual images and point cloud projection maps. Simultaneously, a Transformer encoder branch is used, leveraging its self-attention mechanism to model global spatiotemporal relationships between different modalities, such as between images and laser point clouds, and within data sequences, including the interaction between the target's motion trajectory and the background environment. Finally, the local features extracted by CNN and the global features extracted by Transformer are concatenated and fused to generate a unified multimodal fusion feature library.

[0037] The output is the standardized, high-dimensional multimodal fusion feature library described above. This feature library serves as a public data source, providing real-time synchronization to the subsequent AI algorithm dynamic scheduling module and the real-time-post AI recognition data complementarity module, and supports dynamic updates based on newly incoming data.

[0038] (II) Artificial Intelligence (AI) Algorithm Dynamic Scheduling Module This module is responsible for intelligently selecting and scheduling the most suitable artificial intelligence (AI) recognition algorithm based on the current specific task and flight scenario, in order to improve the overall efficiency and accuracy of the recognition system.

[0039] Input data includes: a multimodal fusion feature library from the multimodal perception fusion module; the type of job task specified by the user in the platform task creation interface, such as straw burning inspection, road surface defect detection, or building facade oblique photography; and flight scene data transmitted back by the UAV in real time, such as current lighting conditions, weather conditions, and flight area type.

[0040] The core processing mechanism relies on a scheduling model built using a fusion architecture of rule engine and reinforcement learning.

[0041] The rule engine is pre-configured with a basic task, scenario, and algorithm mapping matrix. For example, a rule can be defined as: when the task type is straw burning inspection and the scenario is nighttime, the smoke detection algorithm based on infrared thermal imaging should be called first.

[0042] The reinforcement learning component is responsible for more refined dynamic optimization based on the rules. This reinforcement learning model is trained using historical task data. The training process involves collecting task data accumulated over the past six months from the platform, including task type, scenario conditions, the actual algorithm used, and the accuracy and execution time of the algorithm, totaling approximately 400,000 valid records. This data is divided into training, validation, and test sets in a 7:2:1 ratio. Approximately 280,000 records are used for model training, 80,000 records for hyperparameter tuning in the validation set, and 40,000 records for final performance evaluation in the test set. During model training, the number of iterations is set to 500 rounds, and the learning rate is 0.03 to control the training speed. The reward function is designed as a weighted sum of the improvement in recognition accuracy and the reduction in algorithm execution time, thereby guiding the model to learn efficient and accurate scheduling strategies.

[0043] Module Features: The trained scheduling model can dynamically output the optimal combination of AI algorithms and their operating parameters based on real-time input task and scenario information. This module supports dynamic loading and unloading of algorithms, i.e., hot-swapping, facilitating algorithm library updates and maintenance. It also provides a visual configuration interface, allowing administrators to adjust rules and model parameters. Furthermore, the scheduling model has online incremental training capabilities, continuously optimizing itself using newly generated job data to adapt to ever-changing task requirements.

[0044] The output is the optimal combination of artificial intelligence (AI) algorithms and their configuration parameters determined for the current task and flight scenario. This result is directly pushed to the real-time-post AI recognition data complementation module.

[0045] (III) Real-time-post AI recognition data complementary module This module aims to resolve the contradiction between the fast real-time identification speed and limited accuracy in UAV inspections and the high accuracy but lagging post-recognition, achieving complementary advantages through two-way data interaction.

[0046] The input data includes: the optimal algorithm combination output by the AI ​​algorithm dynamic scheduling module, the multimodal fusion feature library output by the multimodal perception fusion module, real-time recognition data consisting of frame-stripped images or video streams transmitted back by the UAV in real time during flight, and post-recognition data consisting of the original high-definition images or videos uploaded to the server after the mission is completed.

[0047] The implementation of key technologies involves two aspects: 1. Low-latency bidirectional data transmission: A dedicated low-latency data transmission protocol was designed and implemented to ensure high-speed bidirectional interaction between the frame-sampling data in the real-time recognition process and the high-definition data in the subsequent recognition. The end-to-end latency of the entire interaction process is strictly controlled within 100 milliseconds.

[0048] 2. Feature Library Sharing and Complementary Verification Mechanism: A shared feature library update and access mechanism has been constructed. During the initial analysis in the real-time recognition process, it can not only utilize the current multimodal feature library but also access more refined high-definition image features generated and added to the library in subsequent recognition stages. This allows for rapid verification of its initial judgment and effective filtering of false alarms. Conversely, during subsequent depth recognition, the algorithm can utilize scene context information recorded during real-time recognition (such as the impact of lighting angle and wind speed on image clarity) to assist in the interpretation of high-definition images, thereby supplementing details and improving recognition accuracy.

[0049] The output is a precise and effective structured problem identification result after complementary verification. This result not only includes the identified problem type (such as fire point, road surface crack), but also accurately associates the latitude and longitude of the shooting location where the problem occurred, as well as multimedia evidence such as problem images or video clips, providing complete information input for subsequent event generation.

[0050] (iv) Intelligent event real-time triggering module This module achieves a leap in automation from problem identification to generating manageable events, reducing human intervention and improving response speed.

[0051] The input data consists of valid problem identification results output by the real-time-post-processing AI recognition data complementary module, as well as pre-set grid management data in the platform (i.e., the relationship between geographic grids and responsible entities).

[0052] The core processing logic is based on a dynamically configurable rule base. This rule base is managed through a visual rule engine, which defines the relationships between problem type, handling entity, and triggering conditions. For example, a rule can be defined as: when the problem type is identified as illegal dumping of garbage and the location is in grid 03 of Street A, an event is automatically triggered, and the handling entity is bound to the Environmental Sanitation Management Office of Street A.

[0053] Once the module receives a valid identification result, it automatically matches it to the rule base, extracts information such as the issue type, latitude and longitude, and media data from the result, and locates the specific platform geographic grid based on the latitude and longitude. Subsequently, the module calls the event generation interface to automatically create a standard event ticket, which contains all the above information and automatically binds the responsible entity corresponding to the grid according to the rules. The entire process requires no manual input or assignment.

[0054] The output is a complete initial event information package with the responsible party. This package is sent to the problem priority intelligent classification module and simultaneously written into the platform's original event database, achieving seamless integration.

[0055] (v) Intelligent problem priority classification module This module intelligently categorizes automatically generated events, ensuring that limited processing resources are prioritized for high-risk, high-urgency events.

[0056] The input data consists of the initial event information output by the intelligent event real-time triggering module, as well as the risk level assessment standards preset by the platform according to management requirements.

[0057] Priority quantification assessment is achieved through a fuzzy comprehensive evaluation model. This model selects problem type, scope of impact, and location of occurrence as three core evaluation indicators, assigning different weights to each indicator; for example, problem type has a weight of 0.5, scope of impact has a weight of 0.3, and location of occurrence has a weight of 0.2. Each indicator is further subdivided into multiple evaluation levels (such as high, medium, and low) with corresponding scores. The model uses fuzzy computation to comprehensively calculate the total priority score for an event.

[0058] Tiered push mechanism: Events are divided into high, medium, and low priorities based on the total score. High-priority events (e.g., total score ≥ 80) will be pushed to the responsible party simultaneously and instantly through multiple channels such as platform messages, associated mobile applications, and SMS; medium and low priority events will be pushed to the to-do list of the handling end in queue order.

[0059] Feedback and Self-Optimization: After completing the incident handling, the responsible party must feed back the handling results (such as the handling status and time taken) to the platform. This handling result data will serve as important feedback signals, flowing back to various AI models within the system. For example, it is used to update the multimodal feature library, as training data for the reinforcement learning model of the AI ​​algorithm dynamic scheduling module to optimize the scheduling strategy, and to adjust the weight parameters of each indicator in the problem priority classification model. This constitutes a closed loop of model self-learning and parameter optimization within the system, enabling the system to become increasingly intelligent with use.

[0060] III. Intelligent Route Planning System Adapting to Environment and Task This system aims to overcome the limitations of static route planning and achieve dynamic adaptation of routes to environmental changes, mission requirements, and equipment status. The various modules within the system are connected in a sequential manner and linked with environmental data in real time. The specific process is as follows: (I) Multi-source environmental perception fusion module This module is the sensory organ of the flight route planning system, responsible for integrating all dynamic and static data related to the flight environment.

[0061] The input data comes from a wide range of sources, including: 1. Environmental sensing data collected in real time by the drone's onboard sensors, such as wind speed, wind direction, temperature, and humidity.

[0062] 2. Equipment status data reported by the UAV flight control system, such as remaining battery power, number of satellite positioning points, and communication link quality.

[0063] 3. Real-time airspace data released by the platform's airspace management system, such as temporarily established no-fly zones, restricted flight zones, and other planned flight routes for drones.

[0064] 4. Geographic grid data in the platform's geographic information database, including high-precision terrain elevation models, 3D building models, airport locations, etc.

[0065] Data Processing and Output: The module employs spatiotemporal database technology to align timestamps and correlate spatial locations of the aforementioned multi-source data, constructing and continuously updating a dynamic environment, airspace, and geographic fusion database. This database serves as the unified situational awareness source for the entire flight route planning system, synchronizing in real time with all subsequent modules.

[0066] (II) Task-Route Parameter Intelligent Matching Module This module can intelligently generate basic parameters for the initial flight path with one click, based on the specific task and the performance of the drone, thus lowering the operational threshold.

[0067] Input data includes: fused database, user-specified job task type (such as orthophoto acquisition, power line inspection), and performance data of the selected drone model (such as maximum endurance, cruising speed, and camera parameters).

[0068] The core matching model employs a classification model built upon machine learning (such as XGBoost). This model is trained using historical data, including a large number of historical flight route planning records accumulated by the platform, as well as flight performance quality evaluation data after corresponding tasks, totaling approximately 350,000 records. The data is divided into training, validation, and test sets in a 7:2:1 ratio. During model training, the number of decision trees is set to 100, with a tree depth of 8 layers to balance model accuracy and complexity. The learning rate is set to 0.05 to control the training speed, and the cross-entropy loss function is used to measure prediction error. After training, the model can automatically output a set of optimal basic flight route parameters based on the input task type, UAV model, and operating area, including flight altitude, flight speed, forward overlap rate, lateral overlap rate, and camera shooting mode.

[0069] Output and Interaction: The generated parameters are presented to the user through a visual interface, allowing for fine-tuning based on experience. The confirmed parameter set serves as the foundation for the optimal flight path and is output to the next module. This model also supports online incremental learning to adapt to new drone models or mission types.

[0070] (III) Environmental Adaptive Dynamic Route Adjustment Module This module is responsible for dynamically and safely adjusting the predetermined flight path based on real-time environmental changes during flight.

[0071] The input data includes: the optimal basic parameters of the flight path output by the task-flight parameter intelligent matching module, real-time environmental / airspace data provided by the fusion database, and the real-time flight position data of the UAV.

[0072] The path planning algorithm employs a hybrid approach combining the A* algorithm and the artificial potential field method. The A* algorithm is used for efficient global optimal path search, while the artificial potential field method generates real-time repulsive forces against obstacles (such as temporary no-fly zones or other drones) or adverse environments (such as strong winds) that suddenly appear during flight, guiding the drone to perform smooth local detours.

[0073] Adjustment Trigger Mechanism: To avoid frequent and meaningless route adjustments due to minor environmental fluctuations, a route adjustment threshold model is preset within the module. This model defines clear adjustment trigger conditions; for example, dynamic route replanning is only initiated when the real-time wind speed consistently exceeds 8 m / s, the flight enters a temporarily issued restricted flight zone, or an unmodeled obstacle is detected ahead.

[0074] Output and Synchronization: The adjusted new flight path data will be synchronized to the platform's flight control center and the UAV's flight control system in real time, ensuring that the UAV immediately flies according to the newly planned path.

[0075] (iv) Multi-aircraft collaborative route planning and obstacle avoidance module This module is specifically designed for scenarios involving multiple drones working together, ensuring that the drone swarm completes its tasks efficiently and without conflict.

[0076] The input data consists of dynamic flight path data of each UAV, fused database data, and real-time status data of each UAV, all output by the environment adaptive dynamic flight path adjustment module.

[0077] The collaborative planning algorithm employs a distributed collaborative approach. This algorithm aims to maximize overall task completion efficiency by assigning each drone in the swarm a flight path free from spatial and temporal conflicts, and comprehensively considers the load capacity and remaining battery power of each drone to achieve load balancing for the task.

[0078] Dynamic obstacle avoidance technology: During flight, the module uses a perception technology that fuses ultrasonic and visual sensors to monitor the relative positions of multiple drones in real time. A preset safe distance threshold (e.g., 50 meters) is established. When the distance between any two drones is less than this threshold, the dynamic obstacle avoidance mechanism is immediately triggered. The obstacle avoidance mechanism calls the aforementioned path planning algorithm to replan a local detour path for drones with lower priority or easier adjustment, thereby ensuring flight safety.

[0079] Output and Synchronization: The final multi-drone collaborative conflict-free route planning results will be synchronized in real time to the terminal of each drone participating in the mission and visualized in the panoramic command module of the platform, so as to facilitate the commander's overall monitoring.

[0080] (v) Equipment Status - Route Energy Consumption Optimization Module This module focuses on the economy of flight and mission reliability, aiming to maximize the drone's endurance and prepare for unforeseen circumstances.

[0081] Input data includes: flight path data from the aforementioned modules, environmental data (especially wind speed and direction) from the fusion database, and real-time equipment status data of the UAV (such as current battery level and energy consumption rate per unit time).

[0082] Energy Consumption Modeling and Optimization: The module incorporates a sophisticated UAV energy consumption model. This model, through fitting experimental data, quantifies the relationship between multiple variables such as flight altitude, flight speed, and headwind speed and battery consumption rate. Based on this model, a dynamic programming algorithm is used to globally optimize a given flight path, finding the flight plan with the lowest total energy consumption while ensuring coverage of all mission waypoints.

[0083] Emergency Planning: Based on the optimized flight path, real-time battery level, and energy consumption predictions, the module automatically calculates and plans the safest return route and the nearest alternate landing point. Simultaneously, it marks critical waypoints on the flight path for resuming flight from interrupted points. When the drone's mission is interrupted due to insufficient battery or other reasons, upon takeoff again, the platform can automatically resume the unfinished route based on these marked points, achieving a successful restart.

[0084] The output includes energy-optimal route data, alternate landing point location data, and waypoint marker data for resuming flight after a breakpoint, and is then passed to the next module.

[0085] (vi) Real-time early warning and self-correction module for waypoint anomalies This module is the last intelligent line of defense for flight safety, monitoring and protecting the entire flight path execution process.

[0086] The input data includes the final flight path data output by the device status-flight energy consumption optimization module, real-time data from the fusion database, and real-time waypoint execution parameters fed back by the UAV flight control system.

[0087] Anomaly Detection and Handling: The module runs a real-time waypoint anomaly detection model. This model continuously compares the parameters of the waypoint the UAV is about to execute (such as planned altitude and speed) with the environmental data (such as terrain height and wind speed) and the UAV's extreme performance data in the current fused database. Once an anomaly is detected, such as the planned flight altitude being lower than the terrain height ahead or the planned speed exceeding the UAV's safe speed limit under current headwind conditions, the processing procedure is initiated.

[0088] For anomalies that can be corrected by software, the module's built-in self-correcting algorithm will automatically adjust waypoint parameters, such as raising the flight altitude to above a safe value.

[0089] For serious anomalies that cannot be handled automatically (such as waypoints located inside inaccessible obstacles), the module will immediately send a graded warning to the platform's flight control center and trigger an automatic hovering command on the UAV, waiting for operator intervention.

[0090] The output is a final, verified, and reliable flight path command stream, which is continuously sent to the UAV for execution; at the same time, any warning information will be highlighted on the command interface.

[0091] IV. Overall Platform Workflow The actual workflow of this platform is designed to fit user operating habits, and is divided into three stages: pre-flight configuration, intelligent operation during flight, and post-flight data feedback.

[0092] 1. Pre-flight configuration phase: Users create job tasks on the platform, selecting the task type, drone model, and operation area. The platform then automatically calls relevant modules from two major systems: the route planning system completes environmental perception fusion and intelligent matching of task-route parameters, generating and visualizing the initial route. After user confirmation or fine-tuning, the route is sent to the drone. Simultaneously, the identification and handling system begins loading corresponding multimodal perception and artificial intelligence (AI) algorithm resources.

[0093] 2. Intelligent Operation Phase During Flight: The UAV takes off as planned. The identification and handling system operates simultaneously, completing multi-source data fusion, dynamic AI scheduling, and complementary real-time and post-processing identification to achieve accurate problem identification and automated event generation and tiered push notifications. The route planning system dynamically adjusts the route based on real-time perceived environmental and airspace changes, manages multi-aircraft collaboration and obstacle avoidance, optimizes energy consumption, and monitors waypoint anomalies. The flight control center provides panoramic monitoring of the flight status and event handling progress, and operators can switch to manual mode at any time for intervention.

[0094] 3. Post-Flight Data Feedback Phase: After the mission concludes, the responsible party within the grid will report the incident handling results through the platform. Simultaneously, the system automatically collects complete flight path data, energy consumption data, anomaly records, and mission outcome quality data. This data serves as feedback signals, flowing into all relevant AI models across both systems (such as scheduling models, priority models, parameter matching models, and energy consumption models), driving a round of online incremental training and parameter optimization, thereby enabling continuous self-evolution of the platform. The platform automatically generates a comprehensive report for this mission, and all data is incorporated into the platform's big data analysis library to support future mission planning and decision-making.

[0095] In summary, this invention, through the deep integration and closed-loop optimization of two major intelligent systems, constructs a highly automated, adaptive, and continuously learning intelligent control and command platform for unmanned aerial vehicles (UAVs), effectively improving the efficiency, safety, and intelligence level of UAV applications.

[0096] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention.

[0097] This invention provides an intelligent management and command platform for unmanned aerial vehicles (UAVs), which can achieve high-precision intelligent recognition through multimodal fusion, dynamic algorithm scheduling, automated closed-loop process of recognition, event handling, and response, dynamic route planning that adapts to the environment, tasks, and equipment, multi-UAV collaborative obstacle avoidance, and energy consumption optimization, with high reliability.

[0098] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0099] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An intelligent control and command platform for unmanned aerial vehicles (UAVs), characterized in that, include: A multimodal artificial intelligence (AI) integrated intelligent recognition and real-time event triggering and handling system, and an intelligent route planning system that adapts to the environment and tasks; the two systems communicate with each other and link modules to jointly build a full-process intelligent closed-loop management and control system from multi-source perception, intelligent recognition, event triggering, route planning, dynamic adjustment to feedback optimization; The multimodal artificial intelligence (AI) fusion intelligent recognition and real-time event triggering and handling system includes a multimodal perception fusion module, an AI algorithm dynamic scheduling module, a real-time and post-processing AI recognition data complementarity module, an intelligent event real-time triggering module, and a problem priority intelligent classification module, which are connected in sequence to form a data closed loop. The intelligent route planning system that adapts to the environment and tasks includes a multi-source environmental perception fusion module that links with environmental data in real time, a task-route parameter intelligent matching module, an environment-adaptive dynamic route adjustment module, a multi-aircraft collaborative route planning and obstacle avoidance module, an equipment status-route energy consumption optimization module, and a waypoint anomaly real-time early warning and self-correction module. The platform includes both intelligent and manual modes, and the two systems are seamlessly integrated with the original basic modules of the regional-level UAV low-altitude integrated service platform.

2. The intelligent control and command platform for unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The multimodal perception fusion module uses spatiotemporal correlation data fusion technology to align and remove redundancy between UAV multi-source perception data and platform geographic airspace data, and uses a hybrid CNN and Transformer model to simultaneously extract local spatial features and global spatiotemporal correlation features to build a unified multimodal fusion feature library. The multi-source sensing data includes UAV visual data, laser ranging data, and environmental sensing data; the platform's geographic airspace data includes operational areas, restricted flight areas, and geographic grid data.

3. The intelligent control and command platform for unmanned aerial vehicles according to claim 1, characterized in that, The AI ​​algorithm dynamic scheduling module adopts a rule engine and reinforcement learning fusion architecture to build a scheduling model, and is trained through historical operation data of the platform to achieve dynamic algorithm scheduling based on task type and flight scenario; The scheduling model supports hot-plugging of algorithms, visual configuration, and online incremental training. Its reinforcement learning reward function is based on a weighted calculation of recognition accuracy and running time.

4. The intelligent control and command platform for unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The real-time-post AI recognition data complementarity module achieves bidirectional interaction between real-time frame-sampling data and post-high-definition data through a low-latency data transmission protocol, with transmission latency controlled within 100ms. The module establishes a feature library sharing mechanism, enabling real-time recognition and post-recognition to share the same multimodal feature library, and to complement each other's data in terms of false alarm filtering and recognition accuracy.

5. The intelligent control and command platform for unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The preset rules of the intelligent event real-time triggering module include a dynamic relational database of problem type, handling subject, and triggering conditions. The event information includes at least the problem type, shooting location, latitude and longitude, and associated media data, and is automatically bound to the platform grid responsible subject to realize the integration of event generation and responsibility traceability.

6. The intelligent control and command platform for unmanned aerial vehicles according to claim 1, characterized in that, The intelligent problem priority classification module uses a fuzzy comprehensive evaluation model to quantitatively evaluate events, taking problem type, scope of impact, and location of occurrence as evaluation indicators, and pushing them in a graded manner based on priority scores; The results of the handling are fed back to each artificial intelligence (AI) module to enable model self-learning and parameter optimization.

7. The intelligent control and command platform for unmanned aerial vehicles according to claim 1, characterized in that, The workflow of the environment- and task-adaptive intelligent route planning system is as follows: T1: The multi-source environmental perception fusion module collects UAV environmental sensor data, equipment status data, platform real-time airspace data and geographic grid data, and builds a dynamic environment, airspace and geographic fusion database and updates it in real time. T2: The task-route parameter intelligent matching module, based on the fused database, task type, and UAV performance data, outputs the optimal basic parameters of the route with one click through the parameter matching model, and supports manual fine-tuning; T3: The environment-adaptive dynamic flight path adjustment module, based on the optimal flight path parameters, the fusion database, and the UAV's real-time location data, uses a hybrid path planning algorithm and an adjustment threshold model to achieve real-time dynamic flight path adjustment. T4: When multiple drones work together, the multi-drone collaborative route planning and obstacle avoidance module realizes conflict-free route planning and dynamic obstacle avoidance during flight based on dynamic route data and the status data of each drone. T5: Equipment Status - Route Energy Consumption Optimization Module. Based on route data and UAV equipment status data, it achieves optimal energy consumption planning through energy consumption model and dynamic programming algorithm, sets alternate landing points and marks breakpoints for resuming flight. T6: The waypoint anomaly real-time warning and self-correction module is based on energy-optimal route data and fusion database. It verifies waypoint parameters in real time, realizes automatic correction and warning of anomalies, and triggers hovering and waits for manual intervention when it cannot be corrected.

8. The intelligent control and command platform for unmanned aerial vehicles according to claim 7, characterized in that, The environmental adaptive dynamic route adjustment module adopts a hybrid path planning algorithm that combines the A* algorithm with the artificial potential field method, and has a route adjustment threshold model. It triggers route adjustments based on wind speed, rainfall, and airspace status to avoid meaningless and frequent adjustments. The adjusted flight path data is synchronized to the platform's flight control center and the drone terminal in real time.

9. The intelligent control and command platform for unmanned aerial vehicles according to claim 7, characterized in that, The multi-aircraft collaborative route planning and obstacle avoidance module uses a distributed collaborative algorithm to achieve multi-aircraft load balancing and conflict-free route planning. It also employs obstacle avoidance technology that combines ultrasonic and visual methods to monitor the relative positions of multiple aircraft in real time and trigger a dynamic obstacle avoidance mechanism when the distance is ≤50m. The route planning results are synchronized to each UAV terminal and the platform's panoramic command module in real time.

10. The intelligent control and command platform for unmanned aerial vehicles according to claim 7, characterized in that, The device status-route energy consumption optimization module constructs an UAV energy consumption model, quantifies the relationship between flight altitude, speed, wind speed and battery consumption, and uses a dynamic programming algorithm to achieve optimal route energy consumption planning, automatically marks breakpoints and resume flight points and connects with the platform's breakpoint resume flight function. The waypoint anomaly real-time warning and self-correction module is equipped with a waypoint anomaly real-time verification model, which can automatically correct waypoint altitude, speed and yaw angle anomalies, and trigger hovering and warning when it cannot be corrected.

Images