Air-ground unmanned equipment management and control system and method based on multi-agent collaborative learning

Abstract

The invention belongs to the technical field of automatic control and artificial intelligence, and particularly provides an air-ground unmanned equipment management and control system and method based on multi-agent collaborative learning. According to the method, the high self-adaptive capability of multi-agent collaborative learning is utilized, the dynamic change of the environment can be sensitively sensed, the response is made in time, and the collaborative strategy is continuously optimized through the continuous learning process, so that the network management and control capability of the unmanned equipment is remarkably improved, and the use requirements in the complex environment are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic control and artificial intelligence, and specifically relates to an air-to-ground unmanned equipment management and control system and method based on multi-agent collaborative learning. Background Art

[0002] In the past few years, the technology of unmanned aerial vehicles (UAVs) and unmanned vehicles (UGVs) has made significant progress. Due to their ability to perform tasks in harsh environments or when it is dangerous for human operators, they have been widely used in many fields such as military, agriculture, logistics, environmental monitoring, disaster relief, etc.

[0003] With the expansion of unmanned system applications, unmanned equipment represented by drones and unmanned vehicles will need to work together with other unmanned equipment in an intelligent manner when performing tasks in the future to improve the efficiency and safety of task execution. Although the independent operation technology of unmanned equipment represented by drones and unmanned vehicles is quite mature, the effect of managing and controlling them for intelligent collaborative work is far from ideal. At present, most of the unmanned equipment control operations rely on preset rules and simple response strategies or are completely intervened by humans, lacking sufficient flexibility and intelligence, and difficult to adapt to the rapidly changing operating environment. Summary of the invention

[0004] The technical problem to be solved by the present invention is: in order to improve the collaborative operation efficiency and adaptive ability of unmanned equipment in various environments, the present invention proposes an air-to-ground unmanned equipment control system and method based on multi-agent collaborative learning. The system utilizes the highly adaptive ability of multi-agent collaborative learning, can sensitively perceive the dynamic changes of the environment, and respond in time, and continuously optimizes the collaborative strategy through a continuous learning process, thereby significantly improving the network control ability of unmanned equipment to meet the use requirements in complex environments.

[0005] The present invention adopts the following technical solutions to solve the above technical problems: A control system for air-to-ground unmanned equipment based on multi-agent collaborative learning, the system comprising an equipment and perception layer, a data processing layer, an AI model layer and a planning and command layer; a one-way interface is provided between the equipment and perception layer and the data processing layer, through which the equipment and perception layer uploads real-time equipment data (including coordinates, speed, horizontal acceleration, vertical acceleration, altitude, temperature, humidity, wind speed, surrounding terrain map, and equipment status information) to the data processing layer; a one-way interface is provided between the data processing layer and the AI ​​model layer, through which the data processing layer uploads pre-processed data to the AI ​​model layer; a two-way interface is provided between the AI ​​model layer and the planning and command layer, through which the AI ​​model layer uploads optimized collaborative strategies to the planning and command layer, and at the same time, the planning and command layer feeds back task learning status to the AI ​​model layer; a two-way interface is provided between the planning and command layer and the equipment and perception layer, through which the planning and command layer sends task execution instructions and sensor management instructions to the equipment and perception layer, and at the same time, the equipment and perception layer can directly upload warning information to the planning and command layer when there is a status warning.

[0006] As a preferred solution of the present invention, the equipment and perception layer includes a data acquisition module, a status warning module, an unmanned equipment control module, and a sensor module; the data acquisition module is used to collect data collected from unmanned equipment and various sensors installed on the unmanned equipment, and transmit these data to the status warning module and the data receiving module of the data processing layer; the status warning module is used to analyze the data collected by the data acquisition module to identify potential risks and threats, and transmit the warning information to the planning and command layer to quickly respond to possible security issues; the unmanned equipment control module is used to directly control the unmanned equipment. The module is responsible for receiving and executing instructions, and feeds back real-time data of the execution results to the data acquisition module; the sensor module is responsible for the switch operation of each sensor and sensor data acquisition (unmanned equipment coordinates, speed, horizontal acceleration, vertical acceleration, altitude, temperature, humidity, wind speed, surrounding terrain map, equipment status, etc.), and transmits the data to the data acquisition module.

[0007] As a preferred solution of the present invention, the data processing layer includes a data receiving module, a data cleaning module, a feature extraction module and a data forwarding module; the data receiving module is used to receive the original data transmitted from the data acquisition module and the status warning module in the equipment and perception layer, and transmit the original data to the data cleaning module to prepare for further data processing and analysis; the data cleaning module is used to process the data transmitted by the data receiving module, remove noise and outliers in the data, and ensure the cleanliness and availability of the data; the feature extraction module is used to extract valuable data and features from the cleaned data, and transmit the data and features to the data forwarding module; the data forwarding module is used to send the processed data and features to the AI ​​model layer, ensuring the real-time and format consistency of the data, and supporting model learning and generation strategies.

[0008] As a preferred solution of the present invention, the AI ​​model layer includes a format data receiving module, a learning algorithm module, a strategy generation module and a simulation test module; the format data receiving module is used to collect and integrate the data transmitted from the data processing layer and transmit it to the learning algorithm module; the learning algorithm module is used to receive the data information transmitted by the format data receiving module, and at the same time receive the historical execution data collected by the execution monitoring module, use the multi-agent collaborative learning algorithm to process the data, and learn how to optimize the operation strategy; the strategy generation module is used to generate execution strategies according to the output of the learning algorithm module, and send these strategies to the planning and command layer task scheduling module; the simulation test module is used to test the newly generated strategies in a virtual environment to ensure their effectiveness and feasibility, and then feedback the results within this layer to optimize future strategy generation.

[0009] As a preferred solution of the present invention, the planning and command layer includes a task scheduling module, an instruction generation module, a sensor management module and an execution monitoring module. The task scheduling module is used to receive the strategies transmitted from the strategy generation module of the AI ​​model layer, and based on these strategies, coordinate all unmanned equipment in the system, formulate task plans and scheduling, optimize resource allocation, and transmit the task plan and scheduling information to the instruction generation module, while receiving the status warning information forwarded by the execution monitoring module, regenerate the scheduling instructions according to the warning content, and send them to the instruction generation module; the instruction generation module is used to convert the output of the task scheduling module into specific operation instructions, and transmit the instructions to the unmanned equipment execution module of the equipment and perception layer, and command the unmanned equipment to execute the instructions; the sensor management module is responsible for controlling the opening and closing of the sensor, the adjustment of the working mode, and the self-test and fault reporting of the sensor; the execution monitoring module is used to receive the warning information transmitted from the status warning module, and at the same time, through the video or picture transmitted by the data acquisition module, monitor the task execution process, ensure that it is carried out according to the established strategy, and collect the execution data and transmit it to the AI ​​model layer for further optimization and learning.

[0010] A method of the air-to-ground unmanned equipment control system based on multi-agent collaborative learning is as follows: Step 1: Use the unmanned equipment execution data collected by the data acquisition module in the equipment and perception layer and the sensor data installed on the unmanned equipment to upload the data to the data receiving module in the data processing layer, transmit the execution status information to the execution monitoring module in the planning and perception layer, and transmit the data to the status warning module in this layer to identify potential risks and threats, and transmit the status warning information to the execution monitoring module in the planning and command layer; Step 2: After receiving the data information from the equipment and perception layers, the data receiving module of the data processing layer first transmits the data to the data cleaning module, which processes the received data to remove noise and outliers to ensure the cleanliness and availability of the data. The feature extraction module extracts valuable data and features from the cleaned data and transmits these data and features to the data forwarding module, which sends the processed data and features to the AI ​​model layer to ensure the real-time and format consistency of the data and support model learning and strategy generation. Step 3: After receiving the data information from the data processing layer, the format data receiving module of the AI ​​model layer integrates the data and transmits it to the learning algorithm module. The learning algorithm module uses a multi-agent collaborative learning algorithm to learn how to optimize the operation strategy in combination with historical execution data. The strategy generation module generates execution strategies based on the output of the learning algorithm module and sends these strategies to the task scheduling module of the planning and command layer. Step 4: The simulation test module tests the strategies made by the strategy generation module in a virtual environment to ensure their effectiveness and feasibility, and feeds the results back to the learning algorithm module and the strategy generation module to optimize future strategy generation; Step 5: The task scheduling module of the planning and command layer receives the strategies from the AI ​​model layer, formulates task plans and scheduling based on these strategies, optimizes resource allocation, and receives the status warning information forwarded by the execution monitoring module. It regenerates the scheduling instructions according to the warning content and sends them to the instruction generation module; the instruction generation module converts the output of the task scheduling module into specific operation instructions, and transmits the instructions to the unmanned equipment execution module of the equipment and perception layer to command the unmanned equipment to perform tasks; Step 6: The execution monitoring module receives the execution status data transmitted from the data acquisition module, monitors the task execution process, and ensures that it is carried out in accordance with the established strategy; transmits the collected task execution data back to the AI ​​model layer for future strategy learning and optimization, thereby continuously improving the system's collaborative operation efficiency and adaptive capabilities; transmits the warning information from the status warning module to the task scheduling module, and the task scheduling module regenerates the execution instructions based on the warning information content; Step 7: The sensor management module receives the equipment status information and equipment surrounding environment information uploaded by the data acquisition module, analyzes the sensor needs according to the information and the type of task to be performed, controls the opening and closing of some sensors, adjusts the working mode, and performs self-test and fault reporting of sensors.

[0011] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: (1) The present invention significantly improves the efficiency of collaborative operation of unmanned equipment in different environments by introducing a multi-agent collaborative learning algorithm. This collaborative learning enables unmanned equipment to share environmental and task status information, so that the system can make better decisions. (2) The present invention realizes a self-learning control system. The mutual learning and experience sharing among multiple agents greatly improve the overall learning efficiency of the system. As the amount of network data accumulates, the deep reinforcement learning model can continuously extract knowledge from complex operating experience and continuously optimize the decision-making model to cope with increasingly complex task requirements; (3) The present invention allows for dynamic configuration of the tasks and behavior logic of unmanned equipment through an integrated multi-agent system. This programmable flexibility not only improves the system's ability to respond to emergencies, but also makes the adjustment of strategies more flexible, and can adjust the behavior strategy of unmanned equipment in real time according to the environment and task requirements; (4) The present invention utilizes collaborative learning to improve data processing and risk identification capabilities, and can exchange key safety information between multiple system nodes in real time, quickly identify and respond to potential risks. Compared with a single-node system, multi-agent collaboration improves the safety prevention capabilities and accident response speed of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a general architecture flow chart of an air-to-ground unmanned equipment control system and method based on multi-agent collaborative learning in an embodiment of the present invention; Figure 2 is a state warning judgment decision flow chart in an embodiment of the present invention; Figure 3 It is a diagram of the equipment and perception layer architecture and workflow in an embodiment of the present invention; Figure 4 It is a data processing layer architecture and workflow diagram in an embodiment of the present invention; Figure 5 It is the AI ​​model layer architecture and workflow diagram in the embodiment of the present invention; Figure 6 It is a command and planning layer architecture and workflow diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0014] In order to enable those skilled in the art to better understand the embodiments of the present invention, the technologies and terms involved in the present invention are introduced and explained below.

[0015] Multi-agent collaborative learning technology: Multi-agent collaborative learning is a multi-agent cooperative or collaborative behavior. When learning, the agent can not only learn by itself through trial and error (TAR), but also learn through cooperation or collaboration with other agents. In a multi-agent collaborative learning system, the learning process is distributed among various agents. Each agent can adopt different learning methods and different feedback mechanisms. The command and planning layer integrates the learning process of each agent into the learning process of the multi-agent system in an integrated manner. In this system, each unmanned equipment is an agent. They can not only learn and adapt to the environment individually, but also share information and learning experience with other agents to jointly optimize the task execution strategy.

[0016] Deep reinforcement learning technology: Deep reinforcement learning (DRL) is a machine learning method that combines deep learning (DL) and reinforcement learning (RL). It solves complex decision-making problems by using deep neural networks to approximate the policy or value function in reinforcement learning. In a multi-agent system, through deep reinforcement learning, different agents can learn how to cooperate more effectively to optimize task allocation and resource sharing. This coordination ability is difficult to achieve with traditional control methods, especially in multi-task and dynamic environments.

[0017] The equipment and perception layer, data processing layer and AI model layer can form a closed-loop control of the "perception-processing-instruction issuance-execution" process, which is helpful for autonomous learning and autonomous task allocation of unmanned equipment. However, the data perceived by each intelligent agent only contains a limited range of data information perceived by itself, so the machine learning of the AI ​​model layer only optimizes its own task execution strategy. Therefore, this model can only give the optimal strategy when executing tasks that require a single equipment. As a cross-domain collaborative system, the air-to-ground unmanned equipment control system requires the collaboration of multiple categories and models of unmanned equipment to achieve the best performance of the system.

[0018] In view of the problem that a single device cannot systematically plan the overall execution strategy of the system, the present invention specifically proposes to connect the planning and command layer with the AI ​​model layer to make each unmanned device an intelligent entity. Each unmanned device is equipped with a set of edge computing devices, which enables it to not only learn and adapt to the environment to perform tasks independently, but also share information and learning experience with other intelligent entities, and optimize the task execution strategy through the background command and planning layer. This not only simplifies the amount of background computing data and reduces the delay problem caused by calculation, but also coordinates the global unmanned equipment so that tasks can be executed faster and more efficiently.

[0019] like Figure 1 As shown, the present invention proposes an air-to-ground unmanned equipment management and control system based on multi-agent collaborative learning. The system can be divided into four parts: equipment and perception layer, data processing layer, AI model layer, planning and control layer. There are interfaces between the four layers to realize data uploading, command generation and self-supervised learning, forming a closed loop of air-to-ground unmanned equipment management.

[0020] The equipment and perception layer is responsible for directly controlling and managing unmanned equipment, collecting environmental data and equipment performance data, uploading the data to the data processing layer through a one-way interface, uploading the execution status data to the execution monitoring module of the planning and command layer, and judging whether there is a status warning. If there is a status warning, the warning information is directly uploaded to the planning and command layer, and the execution instructions and sensor instructions from the planning and command layer are received, so that the equipment and sensors execute instructions that meet the mission characteristics.

[0021] The data processing layer is supported by data preprocessing technology and feature extraction technology. It is responsible for receiving data from the equipment and perception layers, cleaning and preprocessing the data to remove noise and outliers, extracting key features from the processed data, and using the sliding window feature extraction method to extract the data features obtained by various sensors. The average, extreme value, local maximum point and degree of variation of various data in the window are calculated for the detection of position, motion state, environment and equipment status; image and video data features are extracted through color histograms, and the number of pixels of each color is extracted for color distribution analysis. The edge detection method identifies the edges of objects in the image for shape recognition. At the same time, the roughness and directionality of the image texture are analyzed, and the prepared data are formatted and sent to the AI ​​model layer to support further analysis and decision-making.

[0022] The AI ​​model layer is supported by machine learning and deep learning technologies, and multi-agent collaborative learning technologies. It is responsible for receiving formatted data from the data processing layer and historical execution data from the planning and command layer, and using multi-agent collaborative learning algorithms to process these data to optimize operational strategies. First, the environment and agents are set up, an agent agent is configured for each unmanned equipment, its available actions are specified, and the environmental model for the agent's operation is set up, including geography, weather, and other factors that affect the agent's decision-making; then the formatted data provided by the data processing layer is integrated, and the agent's observation of the environment and the task information received by the agent are updated according to the latest data; then each agent makes decisions and learns according to the location, motion status information, surrounding environment information and task instructions of the unmanned equipment collected by itself through reinforcement learning or other machine learning algorithms. The agent updates the operational strategy, formulates the subsequent movement route of the unmanned equipment, and transmits these operational strategies to the command and planning layer. At the same time, the simulation test module tests the effectiveness of these operational strategies in a virtual environment to ensure their effective application in actual applications.

[0023] The planning and command layer is supported by the task scheduling and resource management technology. It is responsible for formulating specific task plans and scheduling instructions based on the policy output provided by the AI ​​model layer and the complex event processing and task scheduling algorithms. First, the complex event processing system receives and analyzes data from various sources in real time, including sensor data and operational strategy recommendations from the AI ​​model layer, in order to quickly respond to any key events or changes that may affect ongoing or planned operations. Next, the task scheduling algorithm formulates a detailed task plan based on the information received from the complex event processing system and the pre-defined task priority and resource availability. This includes assigning tasks to specific unmanned equipment, setting a schedule for task execution, and predicting and allocating necessary resources. It also issues execution instructions to the equipment and perception layer, monitors the actual execution process and sensor status, and ensures that the task is carried out in accordance with the established strategy. It is also responsible for collecting execution data and feeding it back to the AI ​​model layer for decision optimization and adjustment. At the same time, it collects warning information from the status warning module, and quickly analyzes and evaluates this information based on the urgency and severity of the warning. Based on the evaluation, it regenerates task instructions, including adjusting the movement route of the unmanned equipment, changing the task priority, or reallocating the task to other equipment.

[0024] Based on the above control system, the present invention also proposes a method for controlling air-to-ground unmanned equipment based on multi-agent collaborative learning. Figure 1 The serial numbers ① to ⑦ marked in the figure correspond to the following steps 1 to 7, and their operating mechanism is as follows.

[0025] Step 1: The physical terminal (such as a ground station or remote control console) issues a task request to start the unmanned equipment to perform a specific task; after the unmanned equipment is started, the data acquisition module in the equipment and perception layer starts working; various environmental and status data are collected through the sensor module, such as the coordinates, speed, acceleration, altitude, temperature, humidity, wind speed, surrounding terrain map and equipment status of the unmanned equipment. These data are integrated into the data acquisition module and uploaded to the data processing layer through a one-way interface. At the same time, they are transmitted to the status warning module to determine whether there are security risks, such as severe weather conditions, rapid approach of unidentified objects, sudden changes in terrain obstacles, low equipment power, abnormal operation of equipment components, etc. If there are security risks, the status warning content will be immediately transmitted to the execution monitoring module of the planning and command layer. Step 2: The data receiving module of the data processing layer obtains the raw data from the equipment and perception layer. The data is transmitted to the data cleaning module for removing noise, processing missing values ​​and correcting data. The cleaned data ensures the accuracy and validity of the data and is passed to the feature extraction module. This module extracts key features from complex data sets through advanced data mining algorithms such as principal component analysis (PCA), k-means clustering, time series analysis algorithm and support vector machine (SVM). The extracted feature data is formatted and sent to the AI ​​model layer through the data forwarding module.

[0026] Step 3: The format data receiving module in the AI ​​model layer receives the feature data from the data processing layer, integrates it and transmits it to the learning algorithm module. The intelligent algorithm in the algorithm module (including multi-agent collaborative learning algorithm and deep learning algorithm) is loaded, and a comprehensive analysis is performed on the current task instructions and historical execution data. After the analysis is completed, the analysis results are transmitted to the strategy generation module. The strategy generation module generates task execution strategies based on the analysis results. These strategies include specific instructions, such as optimal movement path, speed adjustment, task allocation, etc., and sends these strategies to the task scheduling module of the planning and command layer.

[0027] Step 4: The simulation test module tests the strategies made by the strategy generation module in a virtual environment to ensure their effectiveness and feasibility. Microsoft AirSim is mainly used to provide a high-simulation environment. The simulation environment includes buildings, roads, traffic lights, etc. The environment also simulates realistic factors such as network delays and sensor errors. First, the operation strategy formulated by the strategy generation module is converted into a command sequence that can be recognized and executed by the simulation environment, and imported into the simulation environment. Then, a simulation test is carried out. The intelligent agent performs the predetermined task in the virtual environment according to the imported strategy. Through the simulation operation, it can be observed whether the behavior of the intelligent agent is executed as expected. At the same time, during the simulation execution process, various performance indicators are recorded in detail, such as the time required to complete the task, energy consumption, path optimization degree, and any possible conflicts or failures. After the simulation, the collected data is analyzed, and the results are fed back to the learning algorithm module and the strategy generation module to optimize future strategy generation. After the entire system is completed, all collected data and strategy results are verified again to continuously optimize and improve future task execution strategies.

[0028] Step 5: The task scheduling module of the planning and command layer receives the strategy from the AI ​​model layer, formulates task plans and scheduling strategies based on the strategy, optimizes resource allocation, and the instruction generation module converts the output of the task scheduling module into specific operation instructions and transmits the instructions to the unmanned equipment execution module of the equipment and perception layer to command the unmanned equipment to perform tasks.

[0029] Step 6: The execution monitoring module receives the execution status data transmitted from the data acquisition module, monitors the task execution process, and ensures that it is carried out in accordance with the established strategy; transmits the collected task execution data back to the AI ​​model layer for future strategy learning and optimization, thereby continuously improving the system's collaborative operation efficiency and adaptive capabilities; transmits the warning information from the status warning module to the task scheduling module, and the task scheduling module regenerates the execution instructions based on the warning information content.

[0030] Step 7: The sensor management module receives the equipment status information and equipment surrounding environment information uploaded by the data acquisition module, analyzes the sensor needs according to the information and the type of task to be performed, controls the opening and closing of some sensors, adjusts the working mode, and performs self-test and fault reporting of sensors.

[0031] like Figure 2As shown, the present invention proposes a method for timely response to status warning, so that the unmanned equipment can change the path as soon as possible under dangerous conditions, and ensure the safety of the unmanned equipment as much as possible. The main process is: after the data acquisition module collects the surrounding environment information, the status warning module performs warning judgment. If there is no safety warning, the data information is normally uploaded to the data processing layer for processing, and then the AI ​​model layer generates the initial strategy and uploads it to the planning and command layer, and the planning and command layer generates instructions; if there is a safety warning, the data information stops uploading to the data processing layer, and the warning information is directly uploaded to the planning and command layer. The planning and command layer readjusts the instruction content according to the warning information content to ensure the safety of the unmanned equipment itself and the smooth completion of the task.

[0032] like Figure 3 As shown, the equipment and perception layer of the present invention includes a data acquisition module, a status warning module, an unmanned equipment control module, and a sensor module. The data acquisition module collects data from the unmanned equipment and the sensors installed on the equipment, and transmits these data to the data processing layer and the planning and command layer; the status warning module analyzes all collected data to identify potential risks and threats, and transmits the warning information to the execution monitoring module of the planning and command layer. The unmanned equipment control module controls and manages the unmanned equipment, and feeds back the execution results to the data acquisition module; the sensor module is responsible for the switch operation of each sensor and the sensor data acquisition (unmanned equipment coordinates, speed, horizontal acceleration, vertical acceleration, height, temperature, humidity, wind speed, surrounding terrain map, equipment status, etc.), and transmits the data to the data acquisition module.

[0033] like Figure 4 As shown, the data processing layer of the present invention includes a data receiving module, a data cleaning module, a feature extraction module, and a data forwarding module. The data receiving module receives the original data from the equipment and perception layer, and transmits the original data to the data cleaning module; the data cleaning module processes the data from the data receiving module, removes noise and abnormal values ​​in the data, and ensures the cleanliness and availability of the data; the feature extraction module extracts valuable data and features from the cleaned data, and transmits the data and features to the data forwarding module; the data forwarding module sends the processed data and features to the AI ​​model layer to ensure the real-time and format consistency of the data, and support model learning and generation strategies.

[0034] like Figure 5As shown, the AI ​​model layer of the present invention includes a format data receiving module, a learning algorithm module, a strategy generation module, and a simulation test module. The format data receiving module collects and integrates the data transmitted from the data processing layer and transmits it to the learning algorithm module; the learning algorithm module receives the data information transmitted by the format data receiving module, and at the same time receives the historical execution data collected by the execution monitoring module, and uses the multi-agent collaborative learning algorithm to process the data; the strategy generation module generates execution strategies according to the output of the learning algorithm module, and sends these strategies to the planning and command layer task scheduling module; the simulation test module tests new strategies in a virtual environment, and feeds back the results within this layer to optimize future strategy generation.

[0035] like Figure 6 As shown, the command and planning layer of the present invention includes a task scheduling module, an instruction generation module, a sensor management module, and an execution monitoring module. The task scheduling module receives the strategies transmitted from the AI ​​model layer, formulates task plans and scheduling based on these strategies, and transmits the task plans and scheduling information to the instruction generation module; the instruction generation module converts the output of the task scheduling module into specific operation instructions, and transmits the instructions to the unmanned equipment execution module of the equipment and perception layer; the sensor management module analyzes the sensor needs according to the information uploaded by the data acquisition module and the execution task type, controls the opening and closing of some sensors, the adjustment of the working mode, and the self-test and fault reporting of the sensors; the execution monitoring module receives the execution status data transmitted from the data acquisition module, monitors the task execution process, and transmits the collected execution data to the AI ​​model layer for further optimization and learning. The execution and monitoring module also receives the warning information from the status warning module, and transmits the warning information to the task scheduling module, and the task scheduling module regenerates the execution instructions according to the content of the warning information.

[0036] Based on the above content, a specific usage example is given.

[0037] In an emergency mission in the city, two drones departed from the hospital and delivered medical supplies to three locations, A, B, and C. This mission was coordinated through the command and planning layer in the background to ensure that the two drones performed the mission independently and efficiently.

[0038] Before the mission begins, the two drones are configured for the mission through a physical terminal at the ground station, and all systems such as flight control and sensors are ensured to function properly. Each drone takes off autonomously according to the assigned mission: the first drone is responsible for going to location A and location B, while the second drone goes to location C.

[0039] During the flight, all flight data of the two drones, such as position, speed, and environmental status, are collected through their sensors and sent to the data processing layer in real time through a one-way interface. These data are used to monitor the operating status of the drones and are processed in real time through the AI ​​model layer to optimize the flight path and ensure that the mission is carried out as planned. If localized rainfall is detected on the path to location C, there may be a risk of delay. The system automatically adjusts the flight path to ensure that the drone can bypass the risk area and reach the destination safely and quickly. In addition, the task scheduling module continuously evaluates and optimizes the task plan based on the flight data to ensure that each drone can independently complete its respective tasks along the optimal route and that the overall cooperation and division of labor are optimal.

[0040] After completing the mission, the two drones successfully delivered the medical supplies to the designated locations. Throughout the process, the drones automatically performed all operations based on real-time data analysis and task scheduling in the background, demonstrating the efficiency and reliability of the system in emergency response and resource allocation. It not only efficiently solved the needs of emergency material transportation, but also verified the independent operation capabilities of multiple drones in complex tasks and the powerful support functions of the system.

[0041] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. An air-to-ground unmanned equipment control system based on multi-agent collaborative learning, characterized by: The system includes an equipment and perception layer, a data processing layer, an AI model layer, and a planning and command layer; There is a one-way interface between the equipment and perception layer and the data processing layer, through which the equipment and perception layer uploads real-time equipment data to the data processing layer; there is a one-way interface between the data processing layer and the AI ​​model layer, through which the data processing layer uploads preprocessed data to the AI ​​model layer; there is a two-way interface between the AI ​​model layer and the planning and command layer, through which the AI ​​model layer uploads optimized collaboration strategies to the planning and command layer, and at the same time, the planning and command layer feeds back task learning status to the AI ​​model layer; there is a two-way interface between the planning and command layer and the equipment and perception layer, through which the planning and command layer sends task execution instructions and sensor management instructions to the equipment and perception layer, and at the same time, the equipment and perception layer can directly upload warning information to the planning and command layer when there is a status warning.

2. The air-to-ground unmanned equipment control system based on multi-agent collaborative learning according to claim 1 is characterized by: The equipment and perception layer includes a data acquisition module, a status warning module, an unmanned equipment control module, and a sensor module; the data acquisition module is used to collect data collected from unmanned equipment and various sensors installed on the unmanned equipment, and transmit these data to the status warning module and the data receiving module of the data processing layer; The status warning module is used to analyze the data collected by the data acquisition module to identify potential risks and threats, and transmit the warning information to the planning and command level to quickly respond to existing security issues; The unmanned equipment control module is used to directly control the unmanned equipment. This module is responsible for receiving and executing instructions, and feeds back the execution results in real time to the data acquisition module; the sensor module is responsible for the switch operation of each sensor and sensor data collection. The data includes the coordinates, speed, horizontal acceleration, vertical acceleration, altitude, temperature, humidity, wind speed, surrounding terrain map, and equipment status of the unmanned equipment, and transmits the data to the data acquisition module.

3. The air-to-ground unmanned equipment control system based on multi-agent collaborative learning according to claim 1 is characterized by: The data processing layer includes a data receiving module, a data cleaning module, a feature extraction module and a data forwarding module; the data receiving module is used to receive the original data from the data acquisition module and the status warning module in the equipment and perception layer, and transmit the original data to the data cleaning module to prepare for further data processing and analysis; the data cleaning module is used to process the data transmitted by the data receiving module, remove noise and outliers in the data, and ensure the cleanliness and availability of the data; the feature extraction module is used to extract valuable data and features from the cleaned data, and transmit the data and features to the data forwarding module; the data forwarding module is used to send the processed data and features to the AI ​​model layer to ensure the real-time and format consistency of the data, and support model learning and generation strategies.

4. The air-to-ground unmanned equipment control system based on multi-agent collaborative learning according to claim 1 is characterized by: The AI ​​model layer includes a format data receiving module, a learning algorithm module, a strategy generation module and a simulation test module; the format data receiving module is used to collect and integrate the data transmitted from the data processing layer and transmit it to the learning algorithm module; The learning algorithm module is used to receive data information from the format data receiving module, and at the same time receive the historical execution data collected by the execution monitoring module, use the multi-agent collaborative learning algorithm to process the data, and learn how to optimize the operation strategy; The strategy generation module is used to generate execution strategies based on the output of the learning algorithm module and send these strategies to the planning and command level task scheduling module; The simulation testing module is used to test newly generated strategies in a virtual environment to ensure their effectiveness and feasibility, and then feed back the results within this layer to optimize future strategy generation.

5. The air-to-ground unmanned equipment control system based on multi-agent collaborative learning according to claim 1 is characterized by: The planning and command layer includes a task scheduling module, an instruction generation module, a sensor management module and an execution monitoring module; the task scheduling module is used to receive strategies from the strategy generation module of the AI ​​model layer, and based on these strategies, coordinate all unmanned equipment in the system, formulate task plans and schedules, optimize resource allocation, and transmit task plans and scheduling information to the instruction generation module. At the same time, it receives the status warning information forwarded by the execution monitoring module, regenerates the scheduling instructions according to the warning content, and sends them to the instruction generation module; the instruction generation module is used to convert the output of the task scheduling module into specific operation instructions, and transmit the instructions to the unmanned equipment execution module of the equipment and perception layer to command the unmanned equipment to execute the instructions; the sensor management module is responsible for controlling the opening and closing of the sensor, the adjustment of the working mode, and the self-test and fault reporting of the sensor; the execution monitoring module is used to receive the warning information transmitted from the status warning module, and at the same time monitor the task execution process through the video or pictures transmitted by the data acquisition module to ensure that it is carried out in accordance with the established strategy, and collect the execution data and transmit it to the AI ​​model layer for further optimization and learning.

6. A control method for an air-to-ground unmanned equipment control system based on multi-agent collaborative learning as described in any one of claims 1 to 5, characterized in that: The method specifically comprises the following steps: Step 1: Use the unmanned equipment execution data collected by the data acquisition module in the equipment and perception layer and the sensor data installed on the unmanned equipment to upload the data to the data receiving module in the data processing layer, transmit the execution status information to the execution monitoring module in the planning and perception layer, and transmit the data to the status warning module in this layer to identify potential risks and threats, and transmit the status warning information to the execution monitoring module in the planning and command layer; Step 2: After receiving the data information from the equipment and perception layers, the data receiving module of the data processing layer first transmits the data to the data cleaning module, which processes the received data to remove noise and outliers to ensure the cleanliness and availability of the data. The feature extraction module extracts valuable data and features from the cleaned data and transmits these data and features to the data forwarding module, which sends the processed data and features to the AI ​​model layer to ensure the real-time and format consistency of the data and support model learning and strategy generation. Step 3: After receiving the data information from the data processing layer, the format data receiving module of the AI ​​model layer integrates the data and transmits it to the learning algorithm module. The learning algorithm module uses a multi-agent collaborative learning algorithm to learn how to optimize the operation strategy in combination with historical execution data. The strategy generation module generates execution strategies based on the output of the learning algorithm module and sends these strategies to the task scheduling module of the planning and command layer. Step 4: The simulation test module tests the strategies made by the strategy generation module in a virtual environment to ensure their effectiveness and feasibility, and feeds the results back to the learning algorithm module and the strategy generation module to optimize future strategy generation; Step 5: The task scheduling module of the planning and command layer receives the strategies from the AI ​​model layer, formulates task plans and scheduling based on these strategies, optimizes resource allocation, and receives the status warning information forwarded by the execution monitoring module. It regenerates the scheduling instructions according to the warning content and sends them to the instruction generation module; the instruction generation module converts the output of the task scheduling module into specific operation instructions, and transmits the instructions to the unmanned equipment execution module of the equipment and perception layer to command the unmanned equipment to perform tasks; Step 6: The execution monitoring module receives the execution status data transmitted from the data acquisition module, monitors the task execution process, and ensures that it is carried out in accordance with the established strategy; the collected task execution data is transmitted back to the AI ​​model layer for future strategy learning and optimization, thereby continuously improving the system's collaborative operation efficiency and adaptive capabilities; The warning information from the status warning module is transmitted to the task scheduling module, and the task scheduling module regenerates the execution instruction according to the content of the warning information; Step 7: The sensor management module receives the equipment status information and equipment surrounding environment information uploaded by the data acquisition module, analyzes the sensor needs according to the information and the type of task to be performed, controls the opening and closing of some sensors, adjusts the working mode, and performs self-test and fault reporting of the sensors.

Images