An unmanned aerial vehicle control system based on intelligent robots

Abstract

The application provides an unmanned aerial vehicle control system based on an intelligent robot, comprising a remote management platform, an aerial vehicle, an intelligent robot and a honeycomb centralized management station, wherein the remote management platform comprises: an aerial vehicle configuration module, an environment sensing and task processing function of the aerial vehicle; an intelligent robot module, a sensor, a camera, a hardware control unit and a software control architecture configured for the intelligent robot; a site management module, internal monitoring, abnormal monitoring and collaborative management of the honeycomb centralized management station; a large model control module, closed loop control of the system by using multiple models; a monitoring and scheduling module, real-time monitoring, collaborative scheduling and fault response. The application can use robots to realize automatic closed loop management of aerial vehicle flight, reduce personnel intervention, realize efficient scheduling, automatic control and maintenance of the aerial vehicle, and provide a safe, intelligent and expandable aerial vehicle operation mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aircraft control technology, in particular to an unmanned aerial vehicle control system based on intelligent robots. BACKGROUND

[0002] An aircraft (Unmanned Aerial Vehicle, UAV) is a kind of aircraft driven by remote control or autonomous navigation technology, without carrying a pilot to drive. The aircraft realizes flight operation through sensors, navigation systems and communication modules. At present, the aircraft can be divided into military aircraft, commercial aircraft, consumer-level aircraft and other types according to the use. The use of aircraft can quickly complete many tasks that cannot be achieved by manpower in time, such as searching for signs of life or quickly transporting materials in disaster rescue, and using aircraft to replace personnel to perform tasks in high-risk environments (such as war, fire, chemical leakage sites, etc.) can greatly reduce the risk of casualties. It can also be applied in scientific research, entertainment and leisure fields, and has a wide range of application scenarios.

[0003] At present, the control mode of the aircraft is mainly artificial control, and the operator uses a remote control device (such as a handle) to manually control the flight, and the flight operation has the possibility of subjective error, and for complex tasks, the operator is required to be high, and the artificial response speed may not keep up with the changes of emergency situations. In addition, the current research on multi-aircraft cooperative flight is still in the experimental stage, and it is difficult to realize large-scale synchronous and efficient flight, resulting in limited scale of swarm intelligence control. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide an unmanned aerial vehicle control system based on intelligent robots, which can realize automatic closed-loop management of aircraft flight, realize efficient scheduling and automatic control of aircraft, and provide a safer, more intelligent and scalable aircraft operation mode.

[0005] To achieve the above purpose, the present application provides the following scheme: an unmanned aerial vehicle control system based on intelligent robots, comprising a remote management platform and an aircraft, an intelligent robot and a honeycomb centralized management station connected with the remote management platform, the remote management platform comprising:

[0006] An aircraft configuration module is configured to configure the sensor components and the first embedded chip of the aircraft, realize the environment perception and task processing functions of the aircraft; the aircraft includes but is not limited to unmanned aerial vehicles, manned aircraft, unmanned vehicles and unmanned ships;

[0007] An intelligent robot module is configured to select an intelligent robot with a moving mechanism and a mechanical arm, and configure sensors, cameras, hardware control units and software control architectures for the intelligent robot;

[0008] a site management module configured to deploy a plurality of monitoring cameras and environmental monitoring sensors in the cellular centralized management station, perform internal monitoring, equipment monitoring and internal anomaly detection of the cellular centralized management station, and clean, detect and manage the intelligent robot by numbering, so as to realize collaborative management among the aircraft, the cellular centralized management station and the intelligent robot;

[0009] a large model control module configured to use a collaborative control model to perform collaborative scheduling among the aircraft, the intelligent robot and the cellular centralized management station according to a task target and a requirement, use a flight control model to automatically generate an aircraft task path and an execution script, perform task sequence dynamic planning and abnormal situation decision of the aircraft, and use a data management model to perform algorithm implementation and information storage;

[0010] a monitoring and scheduling module configured to perform real-time monitoring on the aircraft, the intelligent robot and the cellular centralized management station, and perform collaborative scheduling and fault response among a plurality of the aircraft and a plurality of the cellular centralized management stations;

[0011] The aircraft configuration module, the intelligent robot module, the site management module, the large model control module and the monitoring and scheduling module are connected with each other.

[0012] Optionally, the cellular centralized management station comprises a cellular chamber, a plurality of open-cover storage boxes arranged in the cellular chamber, a plurality of the monitoring cameras and the environmental monitoring sensors, the open-cover storage box is provided with a pressure sensor for detecting whether the aircraft is stored and a travel switch sensor for detecting opening and closing of the box cover, the environmental monitoring sensors comprise a temperature and humidity sensor, a dust monitoring sensor and a noise sensor, and the open-cover storage box and the aircraft are both provided with a number.

[0013] Optionally, the cellular centralized management station comprises:

[0014] an internal management unit configured to use the monitoring cameras and the environmental monitoring sensors to collect site video data and site environmental data of the cellular centralized management station in real time, and perform environmental anomaly monitoring according to the site video data and the site environmental data;

[0015] a storage location management unit configured to bind the number of the open-cover storage box and the number of the corresponding aircraft, establish a number mapping table, track the position and state of the aircraft in real time, and establish a storage box state table to mark the use state of each open-cover storage box;

[0016] A safety management unit is configured to, after the aircraft lands in the designated area, the intelligent robot separates the battery of the aircraft and charges the aircraft, and puts the aircraft back into the corresponding open-cover storage box for automatic recording and binding verification.

[0017] Optionally, the large model control module comprises:

[0018] A cooperative control unit is configured to construct a cooperative task allocation model based on graph theory, and utilize the cooperative task allocation model to perform multi-agent cooperative scheduling, dynamic task planning, cooperative execution and result feedback among the aircraft, the intelligent robot and the honeycomb centralized management station.

[0019] A flight control model unit is configured to construct a flight control model, and utilize the flight control model to perform task decomposition, path planning and abnormal decision of the aircraft.

[0020] A data management model unit is configured to combine algorithms to perform data structured management on the aircraft, the intelligent robot and the honeycomb centralized management station.

[0021] Optionally, the cooperative control unit comprises:

[0022] A cooperative task modeling subunit is configured to acquire state and task demand data of the aircraft, the intelligent robot and the honeycomb centralized management station, utilize a cooperative task allocation model based on graph theory to decompose the overall task into multiple subtasks, and perform task allocation based on the ability, state and geographical location of each node.

[0023] A cooperative scheduling algorithm subunit is configured to introduce a distributed protocol based on MAS, and adopt a dynamic allocation strategy based on reinforcement learning to construct a cooperative scheduling algorithm of conflict detection-coordination decision-instruction issuing, and realize multi-aircraft cooperative flight, obstacle avoidance and operation cooperation.

[0024] A task dynamic planning subunit is configured to, when environmental changes, path blockage or equipment abnormalities occur, combine A algorithm and a deep learning prediction model to perform task re-planning.

[0025] A cooperative execution and result feedback subunit is configured to report the progress and state of the aircraft, the intelligent robot and the honeycomb centralized management station in real time, and perform logging operation and safety check on the reported progress and state in real time.

[0026] Optionally, the flight control model unit comprises:

[0027] The task decomposition unit is configured to introduce a path planning algorithm and an environment adaptation model into a deep learning model to obtain a flight control model, decompose a task according to a task target input by a user to obtain a plurality of subtasks, dynamically adjust priorities of the subtasks according to an urgency of the task, sort the subtasks to obtain a subtask list, and generate operation instructions according to the subtask list.

[0028] The path planning unit is configured to input sensor fusion data of the aircraft and a dynamic three-dimensional map obtained by the aircraft into the flight control model to obtain a task path and a flight time, and perform multi-constraint optimization of total energy consumption of the task path in combination with an environment variable.

[0029] The abnormality decision unit is configured to perform abnormality judgment and fault prediction according to the sensor fusion data, and perform potential fault judgment in combination with a time sequence model and a classification model to realize fault prediction.

[0030] Optionally, a multi-constraint optimization expression of total energy consumption of the task path is as follows:

[0031]

[0032] wherein, E tatal is the total energy consumption of the task path, P hover is hovering power consumption, P move (v, a) is power consumption of the aircraft moving at a speed and an acceleration, ΔE env is an influence of an environment variable on energy consumption.

[0033] Optionally, the data management model unit comprises:

[0034] The data acquisition subunit is configured to obtain aircraft sensor data, robot operation state, and site environment data and site scheduling data to obtain multi-source data, and perform integrity and real-time checking on the multi-source data by using a buffer + write-ahead log.

[0035] The data management subunit is configured to write the multi-source data into a distributed time sequence database, write structured business data and control logs into a relational database, and construct multi-dimensional index and time stamp index in the distributed time sequence database and the relational database to perform multi-condition combined query.

[0036] The algorithm support subunit is configured to periodically schedule tasks to perform desensitization, abnormal value detection, missing data filling, and labeling preprocessing on the multi-source data, and perform batch analysis on the multi-source data by using Spark or Flink.

[0037] Optionally, the monitoring and scheduling module comprises:

[0038] A real-time monitoring unit is configured to deploy a YOLO target detection model, collect monitoring data of the monitoring camera and the visual sensor on the aircraft, obtain comprehensive monitoring video, and transmit the comprehensive monitoring video to the YOLO target detection model for abnormality identification by using a real-time streaming protocol, and then collect monitoring data of the sensor assembly and robot state data to obtain comprehensive monitoring data, and determine whether an abnormal state exists according to the comprehensive monitoring data and a preset abnormal data threshold;

[0039] A task scheduling unit is configured to calculate a task urgency degree of a target device, a remaining power of the aircraft and a device distance by weighting, obtain a task priority, perform task allocation and execution by using a distributed task scheduling framework according to the task priority, set 5G communication as a main communication channel and satellite communication as a backup communication channel, and automatically switch the communication mode to the backup communication channel when the main communication channel has a delay exceeding a set communication delay threshold.

[0040] A fault response unit is configured to locate a fault device according to the abnormality identification result and the abnormal state determination result, and call a backup device to respond to the task.

[0041] Optionally, the expression of the task priority is as follows:

[0042]

[0043] wherein, P task is the task priority, U urg , U power , U dist are the task urgency degree, the remaining power of the aircraft and the device distance respectively, W urg , W power , W dist are weight factors, and ∑W i is a total weight factor.

[0044] The present application provides an unmanned aircraft control system based on an intelligent robot, and discloses the following technical effects:

[0045] 1. The present application divides the system into four blocks, i.e., a remote management platform, an intelligent robot, an aircraft and a honeycomb centralized management station, can realize closed-loop management of aircraft flight, reduce manual control, reduce the possibility of subjective errors in flight operation, realize efficient scheduling and automatic control of the aircraft, and provide a safer, more intelligent and scalable aircraft operation mode.

[0046] 2, The remote management platform can combine large models, automatically generate aircraft task paths and execution scripts according to task targets, determine execution sequences and make decisions in abnormal situations.

[0047] 3, The intelligent robot can freely move in the honeycomb centralized management station, complete flexible grabbing, placing and cleaning of the aircraft, and the like.

[0048] 4, The honeycomb centralized management station can one-to-one store and manage the aircraft, detect the effective storage state of the aircraft in the box through the pressure sensor, prevent the aircraft from being incorrectly placed, detect the specific angle and completion state of the cover opening and closing through the travel switch sensor, and make the cover opening and closing type storage box have maintenance capability, so that the state of the aircraft is always optimal.

[0049] The technical solutions of the present application will be further described in detail below with the help of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other accompanying drawings can be obtained by those skilled in the art without creative labor.

[0051] Figure 1 The system architecture diagram provided for the embodiments of the present application;

[0052] Figure 2 The structure diagram of the honeycomb centralized management station provided for the embodiments of the present application;

[0053] Figure 3 The structure diagram of the cover opening and closing type storage box provided for the embodiments of the present application;

[0054] Figure 4 The composition architecture diagram of the remote management platform provided for the embodiments of the present application;

[0055] Explanation of reference signs: 1, honeycomb chamber; 2, open cover storage box. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0058] As shown in the drawings, Figure 1 The present application provides an unmanned aerial vehicle control system based on intelligent robots, which comprises a remote management platform, an aerial vehicle connected with the remote management platform, an intelligent robot and a honeycomb centralized management station.

[0059] The aerial vehicle comprises an aerial vehicle body, a sensor assembly arranged on the aerial vehicle body and a first embedded chip connected with the sensor assembly.

[0060] IMU (Inertial Measurement Unit), which can adopt a high-precision IMU module such as Bosch BMI270 or MPU-9250, has a three-axis accelerometer, a three-axis gyroscope, and integrates temperature compensation and low-power design. Provides real-time linear acceleration and angular velocity data, supports the calculation of aerial vehicle attitude angle (pitch, roll, yaw).

[0061] Visual sensor, which can select a binocular stereo camera + depth camera such as Intel Realsense D435i, is used to provide depth perception and obstacle distance measurement, and then realize short-distance obstacle avoidance and environment construction.

[0062] Laser radar, which can select a rotating laser radar such as Velodyne PuckVLP-16 or low-cost Lidar like Livox Mid-360. Laser radar can detect obstacles at a long distance (more than 100 meters) and is used for navigation in complex terrain.

[0063] GPS module, which can select a high-precision RTK-GPS module such as Ublox ZED-F9P, supports centimeter-level positioning. Can accurately estimate the geographical position and track the flight path.

[0064] As shown in the drawings, Figure 2 , Figure 3As shown, the honeycomb centralized management station comprises a honeycomb room 1, a plurality of open-cover storage boxes 2 arranged inside the honeycomb room 1, a plurality of monitoring cameras and environmental monitoring sensors, a pressure sensor for detecting whether an aircraft is stored in the open-cover storage box 2, a travel switch sensor for detecting the opening and closing of the box cover, and a humidity sensor, a dust monitoring sensor and a noise sensor.

[0065] The dust monitoring sensor, such as a GP2Y1010AU0F dust sensor, is used to detect the dust concentration in the air inside the honeycomb station to prevent dust accumulation on the aircraft body from affecting mission efficiency; the noise sensor ensures that the honeycomb station environment operates within the specified noise value.

[0066] The honeycomb centralized management station comprises:

[0067] The internal management unit is used to collect real-time site video data and site environmental data of the honeycomb centralized management station by using the monitoring camera and the environmental monitoring sensor, and to perform environmental anomaly monitoring according to the site video data and the site environmental data, such as excessively high temperature and humidity or excessive dust concentration.

[0068] The storage location management unit is used to bind the open-cover storage box 2 number and the corresponding aircraft number, establish a number mapping table, track the aircraft position and state in real time, and establish a storage box state table to mark the use state (idle / occupied / fault) of each open-cover storage box 2.

[0069] The security management unit is used to separate the battery and charge the aircraft by the intelligent robot after the aircraft lands in the designated area, and to put the aircraft back into the corresponding open-cover storage box 2 for automatic recording and binding verification.

[0070] As shown in the figure, Figure 4 The remote management platform comprises an aircraft configuration module, an intelligent robot module, a site management module, a large model control module and a monitoring and dispatching module connected to each other.

[0071] 1. Aircraft configuration module

[0072] The sensor assembly and the first embedded chip are used to configure the aircraft to realize the environmental perception and task processing functions of the aircraft; the aircraft includes but is not limited to unmanned aerial vehicles, manned aircraft, unmanned cars and unmanned ships.

[0073] 2. Intelligent robot module

[0074] A smart robot with a moving mechanism and a mechanical arm is selected, and sensors, cameras, hardware control units and software control architectures are configured for the smart robot.

[0075] For example, a laser radar and an ultrasonic sensor are deployed at the front end of the moving mechanism of the smart robot for movement and obstacle avoidance, a camera, a jet nozzle and an NFC scanning device or a two-dimensional code reader are deployed on the mechanical arm for target recognition, grabbing, cleaning and number identification.

[0076] A second embedded chip responsible for decision-making and communication, a microcontroller for controlling the moving mechanism and the mechanical arm, and a communication module such as a 5G module and a LoRa module are deployed in the smart robot.

[0077] In the smart robot, a distributed control framework is constructed based on the ROS framework for parallel processing of tasks such as parallel operation execution task scheduling, sensor data reading and real-time mechanical arm control.

[0078] 3, site management module

[0079] A plurality of monitoring cameras and environmental monitoring sensors are deployed in the honeycomb centralized management station for internal monitoring, equipment monitoring and internal anomaly detection of the honeycomb centralized management station, and the smart robot is cleaned, detected and managed by number, realizing collaborative management between the aircraft, the honeycomb centralized management station and the smart robot.

[0080] 4, large model control module

[0081] A collaborative control model is used to perform collaborative scheduling between the aircraft, the smart robot and the honeycomb centralized management station according to task targets and requirements, a flight control model is used to automatically generate aircraft task paths and execution scripts, task sequence dynamic planning and abnormal situation decision-making of the aircraft are performed, and a data management model is used for algorithm implementation and information storage; the large model control module includes:

[0082] 4.1 collaborative control unit

[0083] A collaborative task allocation model based on graph theory is constructed, and the collaborative task allocation model is used for multi-agent collaborative scheduling, dynamic task planning, collaborative execution and result return between the aircraft, the smart robot and the honeycomb centralized management station; the collaborative control unit includes:

[0084] 4.11 collaborative task modeling subunit

[0085] The state and task requirement data of the aircraft, the intelligent robot and the honeycomb centralized management station are acquired, the overall task is decomposed into multiple sub-tasks by using a collaborative task allocation model based on graph theory, and task allocation is performed based on the capability, state and geographical position of each node.

[0086] 4.12 Collaborative scheduling algorithm subunit

[0087] A MAS-based distributed protocol is introduced, and a dynamic allocation strategy based on reinforcement learning is adopted to construct a collaborative scheduling algorithm for conflict detection-coordination decision-instruction issuance, so as to realize multi-aircraft collaborative flight, obstacle avoidance and operation collaboration.

[0088] 4.13 Task dynamic planning subunit

[0089] When environmental changes, path blockages or equipment abnormalities occur, the A algorithm and a deep learning prediction model are combined for task re-planning.

[0090] 4.14 Collaborative execution and result feedback subunit

[0091] The progress and state of the aircraft, the intelligent robot and the honeycomb centralized management station are reported in real time, and the real-time reported progress and state are logged and safety checked.

[0092] 4.2 Flight control model unit

[0093] A flight control model is constructed, and the flight control model is used for task decomposition, path planning and abnormal decision of the aircraft. The flight control model unit comprises:

[0094] 4.21 Task decomposition unit

[0095] A path planning algorithm and an environment adaptation model are introduced into a deep learning model (with task decomposition and path planning capability) to obtain a flight control model. The flight control model decomposes a task according to a user input task target to obtain multiple sub-tasks, dynamically adjusts the priority of the sub-tasks according to the urgency of the task, sorts the sub-tasks to obtain a sub-task list, and generates operation instructions according to the sub-task list.

[0096] Flight control model:

[0097] Input: User target task, such as delivering an item to location A.

[0098] Output: Path planning: aircraft waypoint coordinates, flight height, time window, etc.; execution script: specific flight steps and backup strategies for controlling the aircraft.

[0099] Decomposition task: the user inputs the target on the platform, such as the inspection area or the delivery task, and the large model analyzes the task and splits it into a list of executable sub-tasks, and generates specific operation instructions.

[0100] Path planning: environmental perception data: such as map, obstacle information, weather conditions, input the environmental perception data into the large model, the large model assigns the aircraft waypoints, calculates the optimal path and flight time. Use dynamic path optimization mechanism, such as real-time obstacle avoidance strategy adjustment, to ensure task completion efficiency.

[0101] 4.22 Path planning unit

[0102] For inputting the sensor fusion data and the dynamic three-dimensional map into the flight control model, obtaining the task path and flight time, and combining the environmental variables (wind speed, air flow, dynamic obstacles, etc.) to perform multi-constraint optimization (minimize energy consumption) of the total energy consumption of the task path; the multi-constraint optimization expression of the total energy consumption of the task path is:

[0103]

[0104] Where, E tatal is the total energy consumption of the task path, P hover is the hovering power consumption, P move (v, a) is the power consumption of the aircraft moving at speed and acceleration, ΔE env is the influence of environmental variables on energy consumption.

[0105] 4.23 Abnormal decision unit

[0106] For abnormal judgment and fault prediction according to the sensor fusion data, and combining the time sequence model (LSTM) and the classification model to make potential fault judgment, such as sensor abnormality, insufficient power, to realize fault prediction.

[0107] 4.3 Data management model unit

[0108] For data structured management of the aircraft, the intelligent robot and the honeycomb centralized management station in combination with algorithms. The data management model unit includes:

[0109] 4.31 Data acquisition sub-unit

[0110] For obtaining aircraft sensor data, robot operation status, and site environment data and site scheduling data, obtaining multi-source data, and using buffer + write-ahead log to check the integrity and real-time of the multi-source data.

[0111] 4.32 Data management sub-unit

[0112] The multi-source data is written into a distributed time series database, and structured business data and control logs are written into a relational database, and multi-dimensional indexes and timestamp indexes are constructed in the distributed time series database and the relational database for multi-condition combined queries.

[0113] 4.33 Algorithm support subunit

[0114] Periodically scheduled tasks are used to desensitize, detect outliers, fill in missing data, and labelize the preprocessing of the multi-source data, and batch analysis of the multi-source data is performed using Spark or Flink.

[0115] 5, Monitoring and scheduling module

[0116] The aircraft, the intelligent robot and the honeycomb centralized management station are monitored in real time, and collaborative scheduling and fault response between multiple aircraft and multiple honeycomb centralized management stations are performed; the monitoring and scheduling module comprises:

[0117] 5.1 Real-time monitoring unit

[0118] A YOLO target detection model (YOLOv5 or YOLOv8) is deployed, monitoring data of the monitoring camera and the visual sensor on the aircraft is collected, comprehensive monitoring video is obtained, and the comprehensive monitoring video is transmitted to the YOLO target detection model for abnormal identification using the real-time streaming protocol, obstacles such as unremoved obstacles and dynamically moving objects at the aircraft takeoff and landing point are identified, and equipment faults such as aircraft tilting or accidental falling are detected.

[0119] Monitoring data of the sensor assembly and robot state data (movement path, mechanical arm state, load information) are also collected, comprehensive monitoring data is obtained, and whether there is an abnormal state is determined according to the comprehensive monitoring data and a preset abnormal data threshold (flight speed exceeds the standard, battery power is insufficient, or sensor temperature is too high). Time series database and dynamic alarm rules can also be used to connect equipment status and sudden abnormalities to scheduling cooperation to achieve real-time monitoring closed loop.

[0120] 5.2 Task scheduling unit

[0121] The task urgency of the target device, the remaining power of the aircraft and the distance of the device are weighted to obtain the task priority, and the distributed task scheduling framework such as Apache Airflow is used to perform task allocation and execution according to the task priority, 5G communication is set as the main communication channel, satellite communication is set as the backup communication channel, and when the main communication channel delay exceeds the set communication delay threshold, the communication mode is automatically switched to the backup communication channel.

[0122] The expression of the task priority is:

[0123]

[0124] wherein P task is the task priority, U urg , U power , U dist are the task urgency, the remaining power of the aircraft and the distance of the device respectively, W urg , W power , W dist are the weight factors of the task urgency, the remaining power of the aircraft and the distance of the device respectively, and i is the sum of the weight factors.

[0125] 5.3 Fault response unit

[0126] For locating the fault device (such as aircraft A fault) according to the abnormal identification result and the abnormal state judgment result, and calling the standby device from the idle device to respond to the task to continue the task.

[0127] 5.4 Data storage

[0128] Distributed storage architecture: using time series databases such as InfluxDB or TimescaleDB to store key data of aircrafts:

[0129] Task data: including task instructions, aircraft path points, flight speed, flight status updates, etc.

[0130] Environmental data: such as wind speed, air pressure, temperature, humidity, etc.

[0131] Maintenance records: such as aircraft health status (battery power, component abnormalities) and storage box operation status.

[0132] Therefore, the unmanned aerial vehicle control system based on intelligent robots can realize automatic closed-loop management of aircraft flight, efficient scheduling and automatic control of aircraft, and provide a safer, more intelligent and expandable aircraft operation mode.

[0133] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0134] The principles and implementation manners of the present application are described by using specific examples in the present application, and the above examples are only used to help understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the present specification should not be understood as the limitation of the present application.

Claims

1. A control system for an unmanned aerial vehicle based on an intelligent robot, characterized in that, This includes a remote management platform and aircraft, intelligent robots, and a honeycomb-style centralized management station connected to the remote management platform. The remote management platform includes: An aircraft configuration module is used to configure the sensor components and the first embedded chip of the aircraft to realize the environmental perception and task processing functions of the aircraft; the aircraft includes, but is not limited to, unmanned aircraft, manned aircraft, unmanned vehicles and unmanned ships; The intelligent robot module is used to select an intelligent robot with a mobile mechanism and a robotic arm, and to configure sensors, cameras, hardware control units and software control architecture for the intelligent robot; The site management module is used to deploy multiple surveillance cameras and environmental monitoring sensors in the honeycomb centralized management station to perform internal monitoring, equipment monitoring and internal anomaly detection of the honeycomb centralized management station, and to clean, detect and manage the intelligent robot by numbering, so as to realize collaborative management between the aircraft, the honeycomb centralized management station and the intelligent robot; The large model control module is used to utilize a collaborative control model to perform collaborative scheduling between the aircraft, the intelligent robot, and the honeycomb centralized management station according to mission objectives and requirements. It uses a flight control model to automatically generate aircraft mission paths and execution scripts, perform dynamic planning of aircraft mission sequence and decision-making for abnormal situations, and then uses a data management model to implement algorithms and store information. A monitoring and scheduling module is used to monitor the aircraft, the intelligent robot, and the honeycomb centralized management station in real time, and to perform coordinated scheduling and fault response among multiple aircraft and multiple honeycomb centralized management stations; the monitoring and scheduling module includes: The real-time monitoring unit is used to deploy the YOLO target detection model, collect monitoring data from the monitoring camera and the visual sensor on the aircraft to obtain comprehensive monitoring video, and use a real-time streaming protocol to transmit the comprehensive monitoring video to the YOLO target detection model for anomaly identification. It also collects monitoring data from the sensor components and robot status data to obtain comprehensive monitoring data. Based on the comprehensive monitoring data and a preset anomaly data threshold, it determines whether there is an abnormal state. The task scheduling unit is used to calculate the task urgency of the target device, the remaining power of the aircraft, and the distance to the device in a weighted manner to obtain the task priority. Based on the task priority, the distributed task scheduling framework is used to allocate and execute tasks, and 5G communication is set as the main communication channel and satellite communication is set as the backup communication channel. When the delay of the main communication channel exceeds the set communication delay threshold, the communication mode is automatically switched to the backup communication channel. The fault response unit is used to locate the faulty equipment and call the backup equipment to respond to the task based on the anomaly identification results and the anomaly status judgment results. The expression for task priority is: ； in, As a task priority, These are mission urgency, remaining vehicle battery power, and equipment range, respectively. All are weighting factors. This is the sum of the weighting factors; The aircraft configuration module, the intelligent robot module, the site management module, the large model control module, and the monitoring and scheduling module are interconnected.

2. The unmanned aerial vehicle control system based on an intelligent robot according to claim 1, characterized in that, The honeycomb-style centralized management station includes a honeycomb chamber, multiple open-top storage boxes located inside the honeycomb chamber, multiple monitoring cameras, and environmental monitoring sensors. Each open-top storage box is equipped with a pressure sensor for detecting whether an aircraft is stored and a limit switch sensor for detecting the opening and closing of the box lid. The environmental monitoring sensors include a temperature and humidity sensor, a dust monitoring sensor, and a noise sensor. Both the open-top storage boxes and the aircraft are numbered.

3. The unmanned aerial vehicle control system based on an intelligent robot according to claim 2, characterized in that, The honeycomb-style centralized management station includes: The internal management unit is used to collect site video data and site environmental data of the honeycomb centralized management station in real time using the surveillance camera and the environmental monitoring sensor, and to perform environmental anomaly monitoring based on the site video data and the site environmental data. The storage location management unit is used to bind the number of the open-top storage box to the corresponding aircraft number, establish a number mapping table, track the location and status of the aircraft in real time, and establish a storage box status table to mark the usage status of each of the open-top storage boxes. The safety management unit is used so that when the aircraft lands in a designated area, the intelligent robot can separate and charge the aircraft's battery, and then put the aircraft back into the corresponding open-top storage box for automatic recording and binding verification.

4. The unmanned aerial vehicle control system based on an intelligent robot according to claim 3, characterized in that, The large model control module includes: The collaborative control unit is used to construct a graph theory-based collaborative task allocation model, and to use the collaborative task allocation model to perform multi-agent collaborative scheduling, dynamic task planning, collaborative execution and result turnaround among the aircraft, the intelligent robot and the honeycomb centralized management station; The flight control model unit is used to construct a flight control model and to perform mission decomposition, path planning, and anomaly decision-making for the aircraft using the flight control model. The data management model unit is used to combine algorithms to perform structured data management of the aircraft, the intelligent robot, and the honeycomb centralized management station.

5. The unmanned aerial vehicle control system based on an intelligent robot according to claim 4, characterized in that, The collaborative control unit includes: The collaborative task modeling subunit is used to acquire the status and task requirements data of the aircraft, the intelligent robot and the honeycomb centralized management station, and to decompose the overall task into multiple sub-tasks using a graph theory-based collaborative task allocation model, and to allocate tasks based on the capabilities, status and geographical location of each node. The cooperative scheduling algorithm subunit is used to introduce a distributed protocol based on MAS and adopt a dynamic allocation strategy based on reinforcement learning to construct a cooperative scheduling algorithm of conflict detection-coordination decision-instruction issuance, so as to realize multi-aircraft cooperative flight, obstacle avoidance and operation coordination. The task dynamic planning subunit is used to replan the task by combining the A algorithm and a deep learning prediction model when environmental changes, path blockages, or equipment malfunctions occur. The collaborative execution and result feedback unit is used to report the progress and status of the aircraft, the intelligent robot, and the honeycomb centralized management station in real time, and to log and perform security verification on the real-time reported progress and status.

6. The unmanned aerial vehicle control system based on an intelligent robot according to claim 5, characterized in that, The flight control model unit includes: The task decomposition unit is used to introduce path planning algorithms and environment adaptation models into the deep learning model to obtain a flight control model. The flight control model decomposes the task according to the task objective input by the user to obtain multiple sub-tasks. According to the urgency of the task, the priority of the sub-tasks is dynamically adjusted and sorted to obtain a sub-task list. Then, operation instructions are generated according to the sub-task list. The path planning unit is used to input the sensor fusion data and the dynamic 3D map acquired by the aircraft into the flight control model to obtain the mission path and flight time, and to perform multi-constraint optimization of the total energy consumption of the mission path based on environmental variables. An anomaly decision unit is used to make anomaly judgments and fault predictions based on the sensor fusion data, and to make potential fault judgments by combining time series models and classification models, thereby achieving fault prediction.

7. A control system for an unmanned aerial vehicle based on an intelligent robot according to claim 6, characterized in that, The multi-constraint optimization expression for the total energy consumption of the task path is: ； in, This represents the total energy consumption of the task path. For hovering power consumption, The power consumption of the aircraft as it moves with speed and acceleration. The impact of environmental variables on energy consumption.

8. A control system for an unmanned aerial vehicle based on an intelligent robot according to claim 7, characterized in that, The data management model unit includes: The data acquisition subunit is used to acquire aircraft sensor data, robot operation status, site environment data, and site scheduling data to obtain multi-source data, and uses a buffer + pre-write log to check the integrity and real-time performance of the multi-source data. The data management subunit is used to write the multi-source data into a distributed time-series database, write the structured business data and control logs into a relational database, and build multi-dimensional indexes and timestamp indexes in the distributed time-series database and the relational database to perform multi-condition combined queries. The algorithm supports sub-units for periodically scheduling tasks to perform desensitization, outlier detection, missing value imputation, and labeling preprocessing on the multi-source data, and to perform batch analysis on the multi-source data using Spark or Flink.

Images