A large-scale low-altitude aircraft distributed cooperative control system and method based on Spark

Abstract

This invention belongs to the field of low-altitude unmanned aerial vehicle (UAV) technology and provides a large-scale distributed collaborative control system and method for low-altitude aircraft based on Spark. The system includes a big data central control system based on Apache Spark technology and a multi-UAV cluster. The central control system includes a Spark Driver and several Spark Worker nodes. The Spark Driver is responsible for task reception, allocation, and scheduling decisions, while the Spark Worker nodes deploy real-time data processing and scheduling algorithms. The central control system collects UAV status data in real time through the Spark Streaming module, performs parallel analysis and matching of tasks and resources through the Spark SQL and GraphX ​​modules, and performs fault prediction and multi-objective optimization through the MLlib module, realizing dynamic scheduling and fault-tolerant decision-making for multi-UAV tasks. This invention significantly improves parallel processing capabilities and system robustness while maintaining distributed autonomy and scheduling intelligence, effectively supporting the collaborative operation needs of various low-altitude application scenarios such as logistics delivery and disaster response.

Description

Technical Field

[0001] This invention belongs to the field of low-altitude unmanned aerial vehicle (UAV) technology, specifically a large-scale distributed collaborative control system and method for low-altitude aircraft based on Spark. Background Technology

[0002] Low-altitude drone swarms are widely used in logistics, emergency rescue, and environmental monitoring, but traditional scheduling methods mostly rely on manual plans or centralized static calculations, which are difficult to cope with dynamic changes and sudden failures.

[0003] Existing solutions are often limited to collaboration of similar devices and lack real-time adaptability and fault tolerance. When faced with complex scenarios such as multi-task concurrency, environmental interference, and node failure, the system is prone to problems such as slow response, unreasonable resource allocation, and chain-like propagation of failures.

[0004] Therefore, how to achieve intelligent real-time scheduling and fault self-healing of UAV swarms in massive spatiotemporal data and highly dynamic environments is a key technical challenge that urgently needs to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a large-scale distributed collaborative control system and method for low-altitude aircraft based on Spark, which solves the problems in existing technologies where the system is prone to slow response, unreasonable resource allocation, and chain-like propagation of failures in complex scenarios such as multi-task concurrency, environmental interference, and node failure.

[0006] A large-scale distributed collaborative control system for low-altitude aircraft based on Spark is disclosed. The system includes a big data central control system based on Apache Spark technology and a multi-UAV cluster. The central control system includes a SparkDriver and several Spark Worker nodes. The SparkDriver is responsible for task reception, allocation, and scheduling decisions, while the SparkWorker nodes deploy real-time data processing and scheduling algorithms. The central control system collects UAV status data in real time through the Spark Streaming module, performs parallel analysis and matching of tasks and resources through the Spark SQL and GraphX ​​modules, and performs fault prediction and multi-objective optimization through the MLlib module, thereby realizing dynamic scheduling and fault-tolerant decision-making for multi-UAV tasks.

[0007] Preferably, the central control system further includes a state awareness and evaluation module, a task-resource matching and allocation module, a fault detection and reconstruction module, and a communication coordination module; the state awareness module maintains the real-time state of each UAV through Spark Streaming; the task-resource matching module calculates the fit degree based on Spark SQL and GraphX ​​and executes scheduling; the fault detection module uses MLlib to train a model to predict node faults; and the communication coordination module supports reliable links and data synchronization between the UAV and the central system.

[0008] Preferably, the multi-objective optimization employs an algorithm combining weighted priority scoring and Pareto sorting, which is solved in parallel iteratively in Spark MLlib to achieve a scheduling scheme that minimizes response time, energy consumption, and risk objectives.

[0009] Preferably, the system supports a variety of low-altitude application scenarios, including logistics delivery, emergency rescue, and forest patrol, and assigns complex tasks to a drone swarm for collaborative completion through task decomposition.

[0010] A Spark-based distributed cooperative control method for large-scale low-altitude aircraft, applied to the aforementioned Spark-based distributed cooperative control system for large-scale low-altitude aircraft, includes the following steps:

[0011] Real-time data acquisition: Spark Streaming is used to collect state streams from the drone cluster, the data is mapped to state RDDs and a global state table is maintained;

[0012] Task priority evaluation: Based on task urgency, time window, and spatial distribution, Spark SQL is used to compute and sort the priority of each task in parallel.

[0013] Fit calculation: For each pair of tasks and available drones, the fit function is computed in parallel using GraphX;

[0014] Task allocation: Based on the fitness results, parallel optimization algorithms (such as genetic algorithms and multi-objective weighting methods) are executed in the Spark environment to determine the optimal task-drone allocation scheme;

[0015] Status feedback and adjustment: During execution, the status of the task and the drone is continuously monitored. If a node failure is detected, fault-tolerant reconstruction is triggered.

[0016] Fault-tolerant reconfiguration: If a drone fails, its unfinished tasks are remarked as pending scheduling, and the process returns to the fitness calculation step for reallocation until all tasks are completed.

[0017] Preferably, in the task allocation step, the task allocation problem is modeled as a multi-objective optimization problem, and a genetic algorithm is executed in parallel using SparkMLlib. By dynamically adjusting the weights, a trade-off between response time and system stability is achieved.

[0018] A processor is configured to execute a Spark-based distributed cooperative control system for large-scale low-altitude aircraft as described above.

[0019] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned Spark-based distributed cooperative control system for large-scale low-altitude aircraft.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] This invention significantly improves the real-time performance and reliability of a multi-UAV distributed scheduling system by employing a Spark parallel computing architecture. On one hand, Spark Streaming and RDD mechanisms enable the system to process massive amounts of real-time UAV data without centralized bottlenecks, utilizing in-memory computing to accelerate iterative computation. Experiments show that as the number of nodes in the Spark cluster increases, the total computation time decreases approximately linearly, demonstrating excellent scalability. On the other hand, Spark's fault-tolerant design automatically recalculates lost partitions based on RDD system information when nodes fail, ensuring reliable execution of stream processing and scheduling computations. Furthermore, the Spark platform provides unified SQL, GraphX, and machine learning libraries, enabling seamless integration of complex task allocation, topology analysis, and fault prediction algorithms within the same framework, achieving end-to-end data-driven optimization. Therefore, this system, while maintaining distributed autonomy and scheduling intelligence, significantly improves parallel processing capabilities and system robustness, effectively supporting collaborative operation needs in various low-altitude application scenarios such as logistics delivery and disaster response. Attached Figure Description

[0022] Figure 1 This is a system structure diagram of the present invention;

[0023] Figure 2 This is a flowchart of the task awareness function of the present invention;

[0024] Figure 3 This is a flowchart of the state perception function of the present invention;

[0025] Figure 4 This is a flowchart of the algorithm of the present invention. Detailed Implementation

[0026] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0027] like Figures 1 to 4 As shown:

[0028] Figure 1 This is a schematic diagram of the system architecture of the present invention. The central control system shown in the diagram is based on the Apache Spark architecture, including a Spark Driver and multiple Spark Workers. The Spark Streaming module connects to the UAV cluster to achieve real-time acquisition and updating of flight status; the Spark SQL module is used to query and filter status data and task information; the GraphX ​​module is used to construct a task-UAV relationship graph and compute fitness in parallel; the MLlib module is used to train and apply fault prediction models and multi-objective optimization algorithms. The wireless communication module supports highly reliable data interaction between the central control system and the UAV cluster.

[0029] Figure 2 and Figure 3 The diagram shows the functional flowchart of the system of this invention. The system operation mainly includes six stages: First, the task perception module receives the upper-level flight tasks from the operation center and calls the task decomposition logic to generate executable subtasks. Then, the state perception module receives and parses the real-time state streams of each UAV through Spark Streaming to build an aircraft state database. Based on this, the task priority evaluation module sorts the tasks to be executed and evaluates the matching degree between the task and the aircraft through the task-resource fit calculation module (based on Spark SQL and GraphX) to form a fit matrix. After that, the system enters the multi-objective optimization and task allocation module, calls the MLlib parallel execution scheduling optimization algorithm, and outputs the optimal task allocation result. After the allocation is completed, the task scheduling instruction is sent to the UAV node through the communication coordination module, and the system continuously receives the task execution status through the feedback channel. When a node abnormality or task failure is detected, the fault tolerance decision module will automatically trigger the fault reconstruction process, dynamically reallocate the unfinished tasks, ensure that the tasks are not interrupted and maintain the stable operation of the system. The above modules work together under the Spark cluster to form a closed-loop "perception-decision-scheduling-feedback-recovery" functional flow.

[0030] Figure 4The diagram shows the flowchart of the task scheduling and fault-tolerant decision-making algorithm based on the Spark architecture of this invention. At the initial stage of system operation, the system receives the state stream reported by the drones via Spark Streaming and enters the state acquisition and update phase, storing parameters such as location information, energy, battery level, and health status into a state RDD. Subsequently, the system enters the task selection and priority ranking phase, where SparkSQL calculates factors such as urgency, time window, and spatial density for each task in parallel and sorts them in descending order of priority. Next, the system enters the fitness calculation phase, constructing a bidirectional graph structure between tasks and drones based on GraphX, calculating the fitness score between all available drones and tasks, and outputting a fitness matrix. Based on this, the system calls MLlib to enter the multi-objective optimization and task allocation phase, combining multiple objective functions such as response time, energy consumption, and task success rate, and using a genetic algorithm or weighted strategy to solve for the optimal allocation scheme in parallel within the cluster, generating the task-drone allocation result. After task allocation, the system continuously executes the task execution status monitoring and feedback analysis phase, sensing task progress and equipment status changes in real time. If a node failure or task anomaly is detected, the system immediately enters the fault detection and fault-tolerant reconstruction phase. The faulty node is removed from the communication graph, task dependencies are reconstructed, the task inheritance mechanism is triggered, and unfinished tasks are rescheduled to other available drones. Finally, the task execution status is updated, and the above process is repeated until all tasks are completed. This process fully utilizes Spark's in-memory computing, fault tolerance mechanisms, and parallel capabilities to achieve real-time performance, adaptability, and robustness in the task scheduling algorithm.

[0031] Example: This invention constructs a low-altitude multi-UAV distributed collaborative control system based on the Spark architecture, such as... Figure 1 As shown, the system mainly consists of a central control system (Spark Driver), several Spark worker nodes, a drone swarm, and a wireless communication module. The central control system is responsible for task reception and parsing, global scheduling decisions, and status monitoring; the Spark worker nodes deploy task scheduling algorithms, data processing, and fault prediction modules; and the drone swarm exchanges status and instructions with the Spark system in real time via wireless communication.

[0032] As can be seen from the above, the low-altitude multi-UAV distributed collaborative control system constructed by this invention, based on the Spark architecture, achieves efficient integration of task reception and parsing, global scheduling decision-making, and status monitoring through the close collaboration between the central control system (Spark Driver) and several Spark worker nodes, UAV clusters, and wireless communication modules. This system leverages the parallel processing and in-memory computing advantages of the Spark platform to significantly improve the real-time performance and reliability of multi-UAV distributed scheduling, ensuring decentralized bottleneck processing of massive amounts of real-time UAV data. Simultaneously, Spark's fault-tolerant design guarantees the reliable execution of stream processing and scheduling computation, effectively supporting the collaborative operation needs of various low-altitude application scenarios such as logistics delivery and disaster response.

[0033] Specifically, the low-altitude airspace multi-aircraft distributed collaborative control system proposed in this invention is based on the Apache Spark big data architecture. The overall structure consists of a central control unit, a Spark cluster computing platform, UAV nodes, and a wireless communication network. All components work collaboratively to achieve parallelized UAV task scheduling and intelligent fault-tolerant processing. The central control unit, as the scheduling and command core, operates based on the Spark Driver architecture and is responsible for receiving upper-level tasks, executing task decomposition, triggering resource matching and fault-tolerant strategies, and issuing control commands. The Spark cluster platform consists of multiple worker nodes, each deploying sub-modules such as Spark Streaming, Spark SQL, GraphX, and MLlib. It is responsible for real-time processing of UAV state streams, task and resource adaptation modeling, topology graph calculation and analysis, and fault prediction and multi-objective scheduling optimization. The UAV nodes, as front-end executors, periodically collect information such as flight position, battery level, payload capacity, and health status. Through a wireless communication self-organizing network with breakpoint resume and link redundancy capabilities, they transmit the status back to the central system in real time, while simultaneously receiving and executing scheduling commands. The system integrates multiple functional modules, including a task awareness and decomposition module (for receiving macro-level tasks and generating executable subtasks), a status awareness and evaluation module (relying on Spark Streaming to update the flight status database in real time), a task-resource matching and scheduling module (building an adaptation model and computing task allocation schemes in parallel using Spark SQL and GraphX), a fault detection and fault tolerance module (using MLlib for health assessment and fault prediction, automatically triggering a task inheritance mechanism upon failure), and a communication and coordination module (ensuring stable communication between task data and commands). The overall architecture leverages the parallel processing and in-memory computing advantages of the Spark platform to achieve a high degree of autonomy and real-time performance in UAV task scheduling, collaborative control, and fault-tolerant recovery.

[0034] As can be seen from the above, the low-altitude airspace multi-aircraft distributed collaborative control system proposed in this invention, based on the Apache Spark big data architecture, achieves highly parallelized UAV task scheduling and intelligent fault-tolerant processing through the collaborative work of the central control unit, Spark cluster computing platform, UAV nodes, and wireless communication network. Leveraging the parallel processing and in-memory computing advantages of the Spark platform, this system not only significantly improves the efficiency of real-time processing of UAV state streams and task resource adaptation modeling, but also enhances the system's robustness and response speed by integrating fault prediction and multi-objective scheduling optimization functions, ensuring the efficient collaboration and stable operation of the UAV cluster in complex environments.

[0035] Specifically, the scheduling process of this system relies on the distributed computing capabilities and memory scheduling mechanism of the Spark platform to achieve high parallelism and fault-tolerant autonomy. First, Spark Streaming receives the status streams periodically reported by each drone and parses the received JSON or binary data, updating the cluster state RDD in real time to ensure that the latest position, energy, and health indicators of each node are completely recorded. Then, Spark SQL is used to perform parallel queries on the current queue of tasks to be scheduled, calculating factors such as task urgency and spatial location, and arranging them in descending order of priority to form a scheduling sequence. In the task-resource adaptation phase, a task-drone bilateral graph structure is constructed based on GraphX, and adaptation indicators (such as a comprehensive score of distance, payload, endurance, and health) are calculated in parallel on each Worker node to form a global adaptation matrix. Subsequently, the adaptation results are aggregated and analyzed, and a multi-objective optimization algorithm is executed, using a genetic algorithm or Pareto sorting method, running in parallel in Spark MLlib to generate the optimal task-drone allocation scheme and update the global allocation state. Throughout the task execution process, Spark Streaming continuously monitors task feedback information (such as progress status and exceptions) and dynamically optimizes the scheduling strategy based on the feedback. When the system detects a failure in a drone node (such as connection interruption or abnormal status), it immediately triggers a fault-tolerant refactoring process. GraphX ​​recalculates the dependency relationship between the failed node and its tasks, and the unfinished tasks are added back to the scheduling queue. Spark SQL then reallocates a replacement node to ensure task continuity and system robustness.

[0036] As shown above, the scheduling process of this system fully utilizes the distributed computing capabilities and memory scheduling mechanism of the Spark platform, achieving high parallelism and fault-tolerant autonomy, significantly improving the efficiency and robustness of UAV task scheduling. Spark Streaming is used to receive and parse the UAV state stream in real time, ensuring real-time updates of the cluster state; Spark SQL is used to query the queue of tasks to be scheduled in parallel, quickly forming a priority-ordered sequence of tasks; a task-UAV dual-sided graph structure is constructed based on GraphX, and the fitness index is calculated in parallel to form a global fitness matrix; a multi-objective optimization algorithm is run in parallel in Spark MLlib to generate the optimal task-UAV allocation scheme. Simultaneously, the system continuously monitors task feedback information, dynamically optimizes the scheduling strategy, and immediately triggers a fault-tolerant refactoring process when a UAV node failure is detected, ensuring task continuity and system robustness.

[0037] Specifically, the key code segments of this system are as follows:

[0038] # Spark Streaming: Real-time UAV Status Stream Reception and Update from pyspark.sql import SparkSession from pyspark.streaming import StreamingContext from pyspark import SparkContext sc = SparkContext(appName="UAVStatusStreaming") ssc = StreamingContext(sc, 1) # 1-second batch ssc.checkpoint("checkpoint_dir") # Receive JSON status stream status_stream = ssc.socketTextStream("uav_host", 9999) # Status update function def update_status(new_values, last_state): return new_values[-1] if new_values ​​else last_state # Convert to (uav_id, status_dict) def parse_status(line): import json data = json.loads(line) return(data["uav_id"], data) status_pairs = status_stream.map(parse_status) status_state =status_pairs.updateStateByKey(update_status) # Task-resource fit calculation function def compute_fitness(task, uav): # Weight settings alpha, beta, gamma = 0.4, 0.3, 0.3 # Simplified example variable energy_ratio = uav["battery"] / uav["battery_max"] distance =haversine_distance(task["location"], uav["location"]) distance_score = 1.0 - distance / MAX_DISTANCE health_score = uav["health"] score = alpha * (uav["available_load"] / task["required_load"]) + \ beta * distance_score + \ gamma * health_score return score # Haversine distance calculation import math def haversine_distance(loc1, loc2): lon1, lat1 = loc1 lon2, lat2 = loc2 R = 6371.0 # Earth radius in km dlon = math.radians(lon2 - lon1) dlat = math.radians(lat2 - lat1) a = math.sin(dlat / 2) ** 2 + math.cos(math.radians(lat1)) * math.cos(math.radians(lat2)) * math.sin(dlon / 2) ** 2 c = 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a)) return R * c # Fault detection and task inheritance handling def detect_and_reassign_tasks(uav_status_dict, task_list): failed_uavs = [uid for uid, status in uav_status_dict.items() if status.get("status") == "FAILED"] reassigned = [] for task in task_list: if task["assigned_uav"] in failed_uavs: task["status"] = "待分配" reassigned.append(task) return reassigned# Adaptability sorting and task assignment schedulingdef assign_tasks(tasks, uavs): assignments = [] for task in tasks:candidates = [] for uav in uavs: if uav["status"] == "NORMAL" and uav["available_load"] >= task["required_load"]: score = compute_fitness(task, uav) candidates.append((uav["uav_id"], score)) if candidates: best_uav = sorted(candidates, key=lambda x: -x[1])[0][0] task["assigned_uav"] = best_uav task["status"] = "In progress" assignments.append((task["task_id"], best_uav))return assignments.

[0039] A Spark-based distributed cooperative control method for large-scale low-altitude aircraft, applied to the aforementioned Spark-based distributed cooperative control system for large-scale low-altitude aircraft, includes the following steps:

[0040] Real-time data acquisition: Spark Streaming is used to collect state streams from the drone cluster, the data is mapped to state RDDs and a global state table is maintained;

[0041] Task priority evaluation: Based on task urgency, time window, and spatial distribution, Spark SQL is used to compute and sort the priority of each task in parallel.

[0042] Fit calculation: For each pair of tasks and available drones, the fit function is computed in parallel using GraphX;

[0043] Task allocation: Based on the fitness results, parallel optimization algorithms (such as genetic algorithms and multi-objective weighting methods) are executed in the Spark environment to determine the optimal task-drone allocation scheme;

[0044] Status feedback and adjustment: During execution, the status of the task and the drone is continuously monitored. If a node failure is detected, fault-tolerant reconstruction is triggered.

[0045] Fault-tolerant reconfiguration: If a drone fails, its unfinished tasks are remarked as pending scheduling, and the process returns to the fitness calculation step for reallocation until all tasks are completed.

[0046] As shown above, this method effectively maintains the global state table by acquiring UAV state streams in real time using Spark Streaming and mapping them to state RDDs; it leverages Spark SQL for parallel computation of task priorities, ensuring tasks are rationally ordered according to urgency, time window, and spatial distribution; it significantly improves the accuracy of task allocation by utilizing the adaptability of GraphX ​​parallel computation tasks to UAVs; it executes parallel optimization algorithms in the Spark environment to determine the optimal task-UAV allocation scheme, achieving efficient resource utilization; and it continuously monitors task and UAV states, triggering fault-tolerant refactoring when node failures are detected, ensuring system robustness and uninterrupted task execution. This method significantly improves the efficiency and reliability of distributed collaborative control of multiple UAVs.

[0047] Specifically, the fit function includes: for the task and drones Calculate multi-factor fit As a linear combination of weights:

[0048] ;

[0049] in, , These are the drone's remaining energy and health index, respectively. For the distance between the mission and the drone; These are the weighting coefficients. .

[0050] As shown above, the suitability function, by calculating the weighted linear combination of the drone's remaining energy, health index, and distance to the task, can comprehensively evaluate the suitability of the drone to perform the task. This effectively improves the rationality of task allocation and the utilization efficiency of drone resources, ensuring that the task can be performed by the drone in the best condition and the most suitable position, thereby enhancing the collaborative operation capability and task completion quality of the entire drone swarm.

[0051] Specifically, the constraints on task allocation include: introducing binary decision variables. Indicates task Assign to drones Then it must satisfy:

[0052]

[0053] ,

[0054] in, For the task Resource requirements (such as load capacity, time window, etc.). For drones Capacity constraints.

[0055] As can be seen from the above, the task allocation constraint clarifies the allocation relationship between tasks and UAVs by introducing binary decision variables, and comprehensively considers the task's resource requirements (such as payload and time window) and the UAV's capacity constraints. This ensures that the task allocation meets both the actual operational needs and the physical and operational limitations of the UAVs, effectively avoiding the risks of resource overload and task failure, and improving the success rate of task execution and the overall stability of the system.

[0056] Specifically, in the task allocation step, the task allocation problem is modeled as a multi-objective optimization problem. A genetic algorithm is executed in parallel using SparkMLlib, and the weights are dynamically adjusted to achieve a trade-off between response time and system stability.

[0057] As shown above, by modeling the task allocation problem as a multi-objective optimization problem and using Spark MLlib to execute the genetic algorithm in parallel, the weights can be dynamically adjusted to balance response time and system stability. This approach significantly enhances the flexibility and adaptability of task allocation, ensuring that the system can respond quickly and maintain stable operation when facing complex and ever-changing environments and task requirements, thereby improving the overall task execution efficiency and system robustness.

[0058] Specifically, multi-objective optimization includes:

[0059] Set the target vector For example, total task completion time, total energy consumption, and system risk. Pareto frontier methods or weighted methods can be used for comprehensive optimization.

[0060]

[0061]

[0062] In multi-objective optimization, Spark is used to solve for approximate optimal solutions in parallel, and weights are adjusted in real time to achieve a balance between response and efficiency.

[0063] As shown above, modeling the task allocation problem as a multi-objective optimization problem and using Spark MLlib to execute genetic algorithms in parallel to dynamically adjust weights significantly enhances the system's decision-making flexibility and adaptability in complex and ever-changing environments. By optimizing the balance between response time and system stability, it ensures that the UAV swarm can execute tasks quickly and stably when facing diverse task requirements, thereby greatly improving the overall task execution efficiency and system robustness.

[0064] Working principle: Based on the Apache Spark big data architecture, the system collects UAV status data in real time through Spark Streaming and updates it to the global status table. It utilizes Spark SQL to perform parallel computing on task priorities and GraphX ​​to perform parallel computing on the suitability between tasks and UAVs. Then, in Spark MLlib, it uses parallel optimization algorithms (such as genetic algorithms) to perform multi-objective optimization and dynamically determine the optimal task-UAV allocation scheme. At the same time, the system continuously monitors the status of tasks and UAVs. Once a fault is detected, it immediately triggers a fault-tolerant refactoring process to reallocate unfinished tasks, ensuring that the system can execute tasks efficiently and stably in complex and ever-changing environments.

[0065] Application Example: The following section illustrates this system using a logistics delivery scenario. Assume several drone nodes are responsible for delivery tasks. The system receives drone status data such as GPS and load information via Spark Streaming. Whenever a new delivery task arises (e.g., goods moving from point A to point B), the system decomposes the task into location, transportation, and delivery sub-tasks. SparkSQL updates the task pool and the list of available drone resources in parallel and calculates task priorities. Then, GraphX ​​generates a task-drone fit graph in parallel, calculating the fit value between each delivery task and each drone. Next, multi-objective optimization (such as a genetic algorithm) is run on the Spark platform to derive the optimal task allocation scheme. For example, the drone closest to the task origination point and with sufficient energy is assigned to execute the task.

[0066] During simulation, if a drone malfunctions (e.g., communication loss or power failure), Spark Streaming detects the abnormal node status and reports it. The system then marks the remaining tasks for that drone as pending scheduling and calls the task inheritance module. GraphX ​​is used to recalculate the resource compatibility between the failed node and other nodes, and Spark SQL quickly identifies a suitable drone to take over. After rescheduling, the unfinished tasks are assigned to a backup drone to continue execution, ensuring uninterrupted task execution.

[0067] In summary, this invention significantly improves the real-time performance and reliability of a multi-UAV distributed scheduling system by employing a Spark parallel computing architecture. On one hand, Spark Streaming and RDD mechanisms enable the system to process massive amounts of real-time UAV data without centralized bottlenecks, utilizing in-memory computing to accelerate iterative computation. Experiments show that as the number of nodes in the Spark cluster increases, the total computation time decreases approximately linearly, demonstrating excellent scalability. On the other hand, Spark's fault-tolerant design automatically recalculates lost partitions based on RDD system information when nodes fail, ensuring reliable execution of stream processing and scheduling computations. Furthermore, the Spark platform provides unified SQL, GraphX, and machine learning libraries, enabling seamless integration of complex task allocation, topology analysis, and fault prediction algorithms within the same framework, achieving end-to-end data-driven optimization. Therefore, this system, while maintaining distributed autonomy and scheduling intelligence, significantly improves parallel processing capabilities and system robustness, effectively supporting collaborative operation needs in various low-altitude application scenarios such as logistics delivery and disaster response.

[0068] This application provides an electronic device applicable to the aforementioned Spark-based distributed cooperative control system for large-scale low-altitude aircraft, comprising:

[0069] Memory is used to protect computer programs and data;

[0070] A processor is used to run system programs.

[0071] This application provides a computer storage medium applicable to the aforementioned Spark-based distributed collaborative control system for large-scale low-altitude aircraft, and performs hierarchical confidentiality management of the system and data in accordance with confidentiality management requirements.

[0072] Those skilled in the art will understand that embodiments of this application can be provided as a system or a computer program product. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] This application is described with reference to flowchart illustrations and / or block diagrams of devices (systems) and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0078] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0080] The embodiments of the present invention are given for the purposes of illustration and description. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A large-scale distributed cooperative control system for low-altitude aircraft based on Spark, characterized in that: The system includes a big data central control system based on Apache Spark technology and a multi-UAV cluster. The central control system includes a Spark Driver and several Spark Worker nodes. The Spark Driver is responsible for task reception, allocation, and scheduling decisions, while the Spark Worker nodes deploy real-time data processing and scheduling algorithms. The central control system collects UAV status data in real time through the Spark Streaming module, performs parallel analysis and matching of tasks and resources through the Spark SQL and GraphX ​​modules, and performs fault prediction and multi-objective optimization through the MLlib module, thereby realizing dynamic scheduling and fault-tolerant decision-making for multi-UAV tasks.

2. The distributed cooperative control system and method for large-scale low-altitude aircraft based on Spark as described in claim 1, characterized in that: The central control system also includes a state awareness and evaluation module, a task-resource matching and allocation module, a fault detection and reconstruction module, and a communication coordination module. The state awareness module maintains the real-time state of each UAV through Spark Streaming. The task-resource matching module calculates the fit degree based on Spark SQL and GraphX ​​and executes scheduling. The fault detection module uses MLlib to train a model to predict node faults. The communication coordination module supports reliable links and data synchronization between UAVs and the central system.

3. The distributed cooperative control system and method for large-scale low-altitude aircraft based on Spark as described in claim 1, characterized in that: The multi-objective optimization employs an algorithm combining weighted priority scoring and Pareto sorting, which is solved in parallel iteratively in SparkMLlib to achieve a scheduling scheme that minimizes response time, energy consumption, and risk objectives.

4. The distributed cooperative control system and method for large-scale low-altitude aircraft based on Spark as described in claim 1, characterized in that: The system supports a variety of low-altitude application scenarios, including logistics delivery, emergency rescue, and forest patrol, and assigns complex tasks to drone swarms for collaborative completion through task decomposition.

5. A distributed cooperative control method for large-scale low-altitude aircraft based on Spark, characterized in that: An application to a Spark-based distributed cooperative control system for large-scale low-altitude aircraft as described in any one of claims 1-4 includes the following steps: S1. Use Spark Streaming to collect state streams from the drone cluster, map the data to state RDDs, and maintain a global state table; S2. Based on task urgency, time window, and spatial distribution, use Spark SQL to compute the priority of each task in parallel and sort them. S3. For each pair of tasks and available drones, use GraphX ​​to compute the fitness function in parallel; S4. Based on the fit results, execute a parallel optimization algorithm in the Spark environment to determine the optimal task-drone allocation scheme; S5. During execution, continuously monitor the status of the task and the drone. If a node failure is detected, trigger fault-tolerant refactoring. S6. If the drone fails, its unfinished tasks will be remarked as pending scheduling, and the process will return to the fitness calculation step for reallocation until all tasks are completed.

6. The large-scale distributed cooperative control method for low-altitude aircraft based on Spark as described in claim 5, characterized in that: In step S4, the task allocation problem is modeled as a multi-objective optimization problem. A genetic algorithm is executed in parallel using Spark MLlib, and the weights are dynamically adjusted to achieve a trade-off between response time and system stability.

7. A processor, characterized in that: It is configured to execute a Spark-based distributed cooperative control system for large-scale low-altitude aircraft according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements a Spark-based distributed cooperative control system for large-scale low-altitude aircraft as described in any one of claims 1 to 4.

Images