Multi-level industrial unmanned aerial vehicle control system and method

Abstract

The invention discloses a multi-level industrial unmanned aerial vehicle control system and method, and relates to the field of unmanned aerial vehicle control, and the system comprises an intelligent agent decision-making layer which is used for carrying out the environment perception through a multi-mode large model according to the current task instruction, the current visual situation information and the current inertial navigation information of an industrial unmanned aerial vehicle, and generating a current action instruction; and the flight control layer is used for controlling the action of the industrial unmanned aerial vehicle based on the current action instruction. According to the invention, the autonomous decision-making and execution capability of the industrial unmanned aerial vehicle is improved through hierarchical control, and the industrial unmanned aerial vehicle can achieve the efficient execution of a multi-stage industrial task in a complex dynamic situation.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control, and in particular to a multi-level industrial UAV control system and method. Background Technology

[0002] With the development of technology, drones are increasingly being used in disaster relief, environmental monitoring, and other fields. These fields often require drones to perform complex tasks in unknown and dynamically changing environments, such as obstacle avoidance, target search and tracking. Industrial drones are mainly used in agriculture and forestry plant protection, logistics, security patrols, and oil and gas extraction, among many other industries. Traditional drone systems typically use relatively simple control logic and limited automation capabilities, relying mostly on preset flight paths and manual remote control, which shows significant limitations when facing complex and constantly changing real-world environments. Especially when performing multi-stage industrial-grade tasks, drones lacking high-level intelligent decision-making and rapid response mechanisms struggle to ensure the safety and effectiveness of mission execution.

[0003] Despite significant progress in drone technology research, existing technologies still have several shortcomings. Firstly, the vast majority of industrial drones lack high-level intelligent decision-making capabilities. Their so-called intelligence currently only manifests in predefined flight paths and mission settings, lacking the ability to make on-the-spot decisions and thus exhibiting weak adaptability to complex environments. Secondly, existing drone intelligent systems based on artificial intelligence and other technologies remain at the algorithm and simulation levels, lacking flexibility and precision in responding to commands. This makes them difficult to implement in real-world industrial drone missions, particularly when performing multi-stage tasks requiring continuous decision-making and action adjustments.

[0004] Against this backdrop, numerous scholars have conducted research on intelligent methods for industrial drones, but some shortcomings remain. For example, application-level multimodal dialogue models can handle relevant industrial tasks based on aerial imagery, but the illusion problem of large models has not been fundamentally solved. While reinforcement learning-based drone racing research can reach or surpass human performance in single tasks, it lacks complex industrial-level domain knowledge and cannot cope with the complex situations and unexpected events in industrial task scenarios. Therefore, current intelligent technologies are difficult to apply in practice and no truly executable systems have been proposed, because intelligent technologies involve high computational demands and high latency, making direct application difficult and requiring matching with engineering-level system architectures. Summary of the Invention

[0005] The purpose of this application is to provide a multi-level industrial drone control system and method, which can enhance the autonomous decision-making and execution capabilities of industrial drones and enable them to efficiently execute multi-stage industrial-grade tasks under complex dynamic situations.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a multi-level industrial unmanned aerial vehicle (UAV) control system, including: The intelligent agent decision layer is used to perceive the environment and generate current action commands based on the current mission instructions, current visual situation information and current inertial navigation information of the industrial drone using a multimodal large model. The flight control layer is used to control the actions of the industrial drone based on the current action command.

[0007] In one embodiment, the agent decision layer includes: The planning module is used to generate task planning information using a multimodal large model based on the current task instructions, current visual situation information, current inertial navigation information, and memory of the industrial UAV; the memory stores the historical task information and historical action information of the industrial UAV. The action module is used to generate current action commands using a multimodal large model based on the task planning information, the current visual situation information, the current inertial navigation information, the memory library, and the action library; the action library stores the action functions of the industrial UAV, and each action function corresponds to a function of the industrial UAV.

[0008] In one embodiment, the intelligent agent decision layer further includes a memory module, configured to generate current task information and current action information based on the current task instruction, the task planning information, the current action instruction, the current visual situation information, and the current inertial navigation information, and store them in a memory bank.

[0009] In one embodiment, the system further includes an inter-layer data bus; the agent decision layer transmits the current action command to the flight control layer via the inter-layer data bus.

[0010] In one embodiment, the current task instruction is a task instruction in natural language form.

[0011] In one embodiment, the multimodal large model is LLaVa.

[0012] In one embodiment, the current visual situation information is acquired by a camera mounted on an industrial drone; the current inertial navigation information is acquired by a lidar sensor mounted on an industrial drone.

[0013] In one embodiment, the intelligent agent decision-making layer is deployed on an onboard computer, and the flight control layer is deployed on a flight control unit on an industrial drone.

[0014] In one embodiment, when the industrial drone is a multi-rotor drone, the current action command is the rotational speed of each rotor motor; when the industrial drone is a fixed-wing drone, the current action command is the deflection amount and throttle amount of each control surface.

[0015] Secondly, this application provides a multi-level industrial drone control method, including: using a multimodal large model to perform environmental perception and generate current action commands based on the current mission command, current visual situation information and current inertial navigation information of the industrial drone; and controlling the actions of the industrial drone based on the current action commands.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a multi-level industrial unmanned aerial vehicle (UAV) control system and method. The intelligent agent decision-making layer, based on multimodal large-scale model intelligent agent technology, enables rapid environmental perception, understanding of complex situations, and generation of multi-level, multi-stage action commands. The flight control layer, closely integrated with the commands of the intelligent agent decision-making layer, is responsible for the precise motion control of the industrial UAV. This enhances the autonomous decision-making and execution capabilities of the industrial UAV and enables it to efficiently execute multi-stage industrial-grade tasks under complex dynamic situations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a multi-level industrial unmanned aerial vehicle (UAV) control system provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram illustrating the practical application process of a multi-level industrial unmanned aerial vehicle (UAV) control system provided in an embodiment of this application.

[0020] Figure 3 This is a flowchart illustrating a multi-level industrial unmanned aerial vehicle (UAV) control method provided in one embodiment of this application.

[0021] Figure 4 This is a hierarchy diagram showing the relationships between layers and nodes in one embodiment of this application.

[0022] Figure 5 This is a schematic diagram illustrating the deployment method of the large model in one embodiment of this application.

[0023] Figure 6This is a flowchart illustrating the process of completing a task in one embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The purpose of this application is to improve the ability of industrial drones to quickly perceive unknown environments, construct complex decision-making processes, and handle multi-stage tasks, thereby enhancing the autonomy and intelligence of industrial drones in performing tasks. This will save on personnel operating costs while significantly improving the accuracy of task execution and the complexity of the tasks they can handle.

[0026] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] In one exemplary embodiment, such as Figure 1 As shown, a multi-level industrial unmanned aerial vehicle (UAV) control system is provided, including an intelligent agent decision-making layer 11, a flight control layer 12, and an inter-layer data bus 13. The intelligent agent decision-making layer 11 is deployed on an onboard computer, and the flight control layer 12 is deployed on the flight control unit of the industrial UAV.

[0028] In addition to the intelligent agent decision layer 11 and the flight control layer 12, the multi-level industrial drone control system also includes other layers, such as the actuator layer, the perception layer, and the estimator layer, which perform specific tasks depending on the specific scenario.

[0029] The intelligent agent decision layer 11 is used to perform environmental perception and generate current action commands based on the current mission instructions, current visual situation information and current inertial navigation information of the industrial drone using a multimodal large model.

[0030] The intelligent agent decision layer 11 can receive situational awareness, execution feedback, and situational information about the current state of the industrial drone from other layers, perform analysis and decision-making within the layer, and then generate control commands for other layers, similar to the central decision-making function of the brain. The intelligent agent decision layer 11 is essentially a function implemented using programming languages ​​such as C++ and Python.

[0031] In a specific application example, the agent decision layer 11 includes: a planning module, an action module, and a memory module.

[0032] The planning module generates task planning information using a multimodal large model based on the industrial UAV's current mission instructions, current visual situational awareness, current inertial navigation information, and memory. Its expression is: .in, The task planning information was broken down and described step-by-step in natural language. This is the current task instruction. Current visual situation information in image form. This refers to the current inertial navigation information, primarily the current position coordinates. For all the data stored in the memory bank, For multimodal large models, The fixed prompt word template is for the planning module. The purpose of the prompt word template is to integrate the discrete input data into a structured graphic prompt that is friendly to large models.

[0033] The action module is used to generate the current action command using a multimodal large model based on the task planning information, the current visual situation information, the current inertial navigation information, the memory bank, and the action bank. Its expression is: .in, This is the current action command, presented in natural language. It includes the action functions from the action library used by the current action and the parameter values ​​passed to them. For all data stored in the action library, This is a fixed prompt template for the action module.

[0034] The memory module is used to generate current task information and current action information based on the current task instruction, the task planning information, the current action instruction, the current visual situation information, and the current inertial navigation information, and store them in the memory bank. Its expression is: .in, The information that should be stored in the memory bank at present includes current task information and current action information, in natural language form. This is a fixed prompt word template corresponding to the memory module.

[0035] The memory bank stores historical mission information and historical action information of the industrial drone. Specifically, the memory bank includes stage information memory and historical action memory. Stage information memory is historical mission information of completed stages, including images and text descriptions. Historical action memory is a record of calling action functions, including the call time, the called function, and the parameters passed.

[0036] The action library stores the action functions of the industrial drone, that is, all the action functions that the industrial drone can call. Each action function corresponds to a function of the industrial drone, which is implemented by one or more actuators in the flight control layer 12 or other layers.

[0037] In this embodiment, the multimodal large model used by the planning module, action module, and memory module can be the same pre-trained model instance, such as LLaVa (Large Language and Vision Assistant). LLaVa itself has general visual understanding and language generation capabilities, and does not require independent retraining or fine-tuning for the three specific sub-tasks of planning, action, and memory.

[0038] In the planning, action, and memory modules, the process is accomplished by inputting structured and task-oriented prompts generated from different prompt templates into the same multimodal large model. The input and output of the multimodal large model vary according to specific task requirements, thereby achieving the technical objective of driving multiple functional modules with a single general large model.

[0039] In this embodiment, the current task instruction is a task instruction in natural language form. The current visual situation information is acquired by a camera mounted on the industrial drone. The current inertial navigation information is acquired by a lidar sensor mounted on the industrial drone.

[0040] When the industrial drone is a multi-rotor drone, the current action command is the rotational speed of each rotor motor. When the industrial drone is a fixed-wing drone, the current action command is the deflection and throttle of each control surface.

[0041] The inter-layer data bus 13 is used to transmit messages between layers, typically in the form of publish and subscribe. In this application, the agent decision layer 11 transmits the current action command to the flight control layer 12 through the inter-layer data bus 13. The agent decision layer 11 is the top layer, and the flight control layer 12 and other layers are the bottom layers. Data transmission between the top and bottom layers, as well as between the bottom layers, is carried out through the inter-layer data bus 13.

[0042] The flight control layer 12 is used to control the actions of the industrial drone based on the current action command.

[0043] The flight control layer 12 is typically a standalone hardware device that receives action commands from the agent decision layer 11 and uses internally deployed position estimation algorithms and PID control algorithms to calculate and execute the desired movement to a target point. Its expression is: .in, For current action instructions The desired action, depending on the action type, can be the desired location coordinates or the desired drone attitude. The control algorithm used by flight control layer 12 These are the underlying control commands for industrial drones. Depending on the drone configuration, they contain different control quantities. On multi-rotor drones, these are the rotational speeds of each rotor motor, while on fixed-wing drones, they are the deflection and throttle values ​​of each control surface.

[0044] This application demonstrates the ability to autonomously generate multi-stage industrial-grade tasks under complex dynamic conditions derived from multiple sensors, with each stage executing the tasks separately. This is significant for improving the autonomy, intelligence, and operational efficiency of industrial drones. This multi-layered system can autonomously generate and execute multi-stage industrial-grade tasks, significantly enhancing the operational capabilities and adaptability of industrial drones and meeting increasingly complex application demands.

[0045] Specifically, the intelligent agent decision layer 11 can leverage multimodal large-scale model intelligent agent technology to achieve rapid environmental perception, understanding of complex situations, and multi-level, multi-stage decision generation, and stably convert the generated specific decisions into executable instructions for other layers. The flight control layer 12, closely integrated with the instructions of the intelligent agent decision layer 11, is responsible for the precise motion control of the industrial drone. Such a system elevates the independence and adaptability to complex environments of industrial drone applications to a new level, and is crucial for improving the operational efficiency, safety, and reliability of drones.

[0046] like Figure 2 As shown, the actual application process of the multi-level industrial unmanned aerial vehicle control system provided in this application includes the following steps (1) to (5).

[0047] (1) Based on the current scenario, initialize the action library and memory library and receive task instructions. First, declare the data types of the two databases. The action library adds the required action functions to the action library according to the current task requirements and the actual payload of the industrial UAV. The memory library is empty during initialization. The multi-level industrial UAV control system receives the current task instructions in natural language form. The current task instructions specify the final goal of the task, but are usually relatively brief and can be decomposed into multiple steps for execution. During the operation of the multi-level industrial UAV control system, the perception and computation information required by the intelligent agent decision layer 11 and the flight control layer 12, such as real-time images, position and distance estimation, etc., are acquired and processed by the perception layer, estimator layer, etc. of the corresponding functions in other layers, and then transmitted to the corresponding layer through the inter-layer data bus 13.

[0048] (2) Input the current task instructions into the planning module in the agent decision layer 11. The planning module is driven by an arbitrary multimodal large model. The planning module accesses the current visual situation information and current inertial navigation information of the industrial UAV, and at the same time calls the current memory bank for reference comparison to generate task planning information for each stage.

[0049] (3) Input the task planning information generated by the planning module into the action module in the intelligent agent decision layer 11. The action module is driven by an arbitrary multimodal large model. The action module accesses the current visual situation information and current inertial navigation information of the industrial UAV, and at the same time calls the current memory bank for reference comparison, guides the specific execution of the task, judges the completion status of the task planning at the current stage, and generates the current action command.

[0050] (4) Input all the information obtained in the above steps into the memory module in the agent decision layer 11. The memory module is driven by an arbitrary multimodal large model and stores all the above information into the memory bank.

[0051] (5) Transmit the current action instructions Data is transmitted to the flight control layer 12 via the inter-layer data bus 13. The flight control layer 12 can use any industrial UAV control algorithm currently available, such as PID (Proportional-Integral-Derivative) control, to control the actions of the industrial UAV.

[0052] Repeat steps (2) to (5) above until the planning module determines that the current task instruction received in step (1) has been completed.

[0053] Based on the same inventive concept, this application also provides a multi-level industrial drone control method using the above-mentioned multi-level industrial drone control system. The solution provided by this method is similar to the solution described in the above system. Therefore, the specific limitations in one or more multi-level industrial drone control method embodiments provided below can be found in the limitations of the multi-level industrial drone control system above, and will not be repeated here.

[0054] like Figure 3 As shown, the multi-level industrial drone control method includes the following steps 31 and 32.

[0055] Step 31: Based on the current mission instructions, current visual situation information and current inertial navigation information of the industrial UAV, use a multimodal large model to perceive the environment and generate current action instructions.

[0056] Step 32: Based on the current action command, control the actions of the industrial drone.

[0057] This application also provides an execution control process for the mission instruction of "rescuing trapped people", which specifically includes the following steps ① to ⑦.

[0058] ① Construct an industrial drone hardware system. On any selected open-source drone rack, a T265 camera is mounted as the forward-looking camera, and a USB (Universal Serial Bus) camera is mounted as the downward-looking camera. An rpLidar lidar sensor is mounted. A Pixhawk flight control unit is mounted. An Intel onboard computer is mounted. A microphone and megaphone are mounted for communication. The onboard computer is essential for this application; that is, any embodiment using this application requires an onboard computer. Other components are selected according to mission requirements. The above components are selected because this embodiment requires forward-looking navigation, downward-looking search, radar ranging, basic flight control, and communication with stranded individuals. The algorithm model of this embodiment is deployed in the cloud, and the specific deployment architecture is as follows... Figure 4 As shown.

[0059] ② Run the data bus on the onboard computer. ROS (Robot Operating System) is a general-purpose system that uses a ROS master bus. In this embodiment, ROS is installed on the UAV's onboard computer. On the inter-layer data bus 13, nodes of each layer of the multi-level industrial UAV control system are registered through the node registration interface. For example... Figure 5 As shown.

[0060] 1) Register agent nodes. Subscribe to messages for Downward Image, Forward Image, and Depth. Publish messages for Commands. Agent nodes contain various general-purpose agents based on large models. Deploy these general-purpose agents based on large models on a cloud server and access them through the server's Internet API (Application Programming Interface).

[0061] 2) Register the controller node. This includes three sub-nodes: Actuator Control, Position Control, and Pose Control. Subscribed messages include Commands, PositionInfo, and Pose Info. Published messages include Target Pose, TargetActuator State, and Target Position. The controller node carries a general-purpose Flight Control Unit (FCU), which can translate commands into corresponding expectations.

[0062] 3) Register sensor nodes. Includes rpLidar (Light Detection and Ranging) sub-nodes. Publish the message Scan Info (Light Detection and Ranging Information).

[0063] 4) Register camera nodes. This includes three child nodes: RealSense2_D435i (depth camera), USB_Camera (downward-facing camera), and RealSense2_T265 (forward-facing camera). Published messages include Forward Image, Depth, Downward Image, and Local Position / Pose.

[0064] 5) Register the estimator node. It contains two child nodes: Pose Estimator and Position Estimator. Subscribe to messages such as Scan Info and Local Position / Pose. Publish messages such as Position Info and Pose Info.

[0065] 6) Register the executor node. This includes child nodes such as the megaphone. Subscribe to messages such as Commands, TargetActuator State, Target Position, and Target Pose.

[0066] ③ Define the industrial task to be performed. In this embodiment, the task is to rescue trapped people. The task instruction in natural language form, "As an industrial drone, your task is to rescue trapped people," is input as a parameter into the agent's decision layer 11. Initialize all databases. In this embodiment, both the memory database and the action database are deployed using MySQL.

[0067] ④ Input the task instructions into the planning module of the agent's decision layer 11. In this embodiment, the planning module is driven by the open-source multimodal large model LLaVa deployed in the cloud. LLaVa is a novel multimodal AI model that combines natural language processing and computer vision technologies. In this embodiment, the number of parameters selected is 13B. The prompts for interacting with the large model and the task planning information returned by the large model are shown in Table 1. The information returned by the large model is stored as task planning variables.

[0068] Table 1 Input and output information of the large model in the planning module

[0069] ⑤ The task plan is input into the action module in the agent's decision layer 11. In this embodiment, the action module is also driven by the open-source multimodal large model LLaVa deployed in the cloud. The prompts for interacting with the large model and the action instruction information returned by the large model are shown in Table 2. The information returned by the model is stored as action instruction variables.

[0070] Table 2 Input and output information of the large model in the action module

[0071] ⑥ Input all the aforementioned information into the memory module in the agent's decision layer 11. In this embodiment, the memory module is also driven by the open-source multimodal large model LLaVa deployed in the cloud. The prompts for interacting with the large model and the memory information to be stored returned by the large model are shown in Table 3. The information returned by the model is stored in the memory bank. Specifically, a new entry is added to the memory bank, and the current storage time is added as a data attribute when storing.

[0072] Table 3 Input and output information of the large model in the memory module

[0073] ⑦ The agent node publishes the message "FlyTo<5,8,10>", meaning it flies to the position (5,8,10), which is received by the controller node. After the controller node calculates this flight command (the calculation process is performed by the built-in control law, and the method can be arbitrarily selected), it publishes the flight command to the actuator node, and the industrial drone successfully takes flight.

[0074] In the process of fulfilling the mission instruction of "rescuing trapped people", such as Figure 6As shown, by repeating steps ④ to ⑦, namely "planning, action, memory, and execution," the industrial drone breaks down the task into three specific steps: "moving towards the trapped people, searching for an exit, and guiding the people to escape," until it successfully helps the trapped people escape and completes the mission. Specifically, it first takes aerial images to locate the trapped people, then issues movement commands through intelligent agent nodes, moves to the designated location through controller nodes, executes actions through actuator nodes, and receives and broadcasts distress calls. Next, it issues exploration commands through intelligent agent nodes, finds an exit through controller nodes, issues guidance commands through intelligent agent nodes, and guides the trapped people to escape through actuator nodes.

[0075] Compared to current control methods for intelligent unmanned aerial vehicle (UAV) systems, this application enables industrial UAVs to autonomously complete industrial tasks. Specifically, unlike applications such as automatic flight path generation and obstacle avoidance, this application allows industrial UAVs to be controlled by a large-scale intelligent agent brain, generating multi-stage complex tasks and completing long-sequence dynamic objectives. This application provides a multi-layered design, allowing each layer to focus on its own tasks, thus solving the instability problem in multimodal large-scale intelligent agent technology and exhibiting a robust and stable system architecture.

[0076] In summary, this application can not only significantly improve the autonomous decision-making and execution capabilities of industrial drones, but also enable the efficient execution of multi-stage industrial-grade tasks under complex dynamic situations. It has a profound impact and important research value for further expanding the operating range and application scenarios of industrial drones in complex environments.

[0077] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0078] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0079] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-level industrial unmanned aerial vehicle (UAV) control system, characterized in that, The system includes: The intelligent agent decision layer is used to perceive the environment and generate current action commands based on the current mission instructions, current visual situation information and current inertial navigation information of the industrial drone using a multimodal large model. The flight control layer is used to control the actions of the industrial drone based on the current action command.

2. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, The intelligent agent decision-making layer includes: The planning module is used to generate task planning information using a multimodal large model based on the current task instructions, current visual situation information, current inertial navigation information, and memory of the industrial UAV; the memory stores the historical task information and historical action information of the industrial UAV. The action module is used to generate current action commands using a multimodal large model based on the task planning information, the current visual situation information, the current inertial navigation information, the memory library, and the action library; the action library stores the action functions of the industrial UAV, and each action function corresponds to a function of the industrial UAV.

3. The multi-level industrial unmanned aerial vehicle control system according to claim 2, characterized in that, The intelligent agent decision-making layer also includes: The memory module is used to generate current task information and current action information based on the current task instruction, the task planning information, the current action instruction, the current visual situation information, and the current inertial navigation information, and store them in the memory bank.

4. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, The system also includes an inter-layer data bus; the agent decision layer transmits the current action command to the flight control layer through the inter-layer data bus.

5. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, The current task instruction is a task instruction in natural language form.

6. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, The multimodal large model is LLaVa.

7. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, The current visual situation information is obtained by a camera mounted on an industrial drone; the current inertial navigation information is obtained by a lidar sensor mounted on an industrial drone.

8. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, The intelligent agent decision-making layer is deployed on an onboard computer, and the flight control layer is deployed on the flight control unit of the industrial drone.

9. The multi-level industrial unmanned aerial vehicle control system according to claim 1, characterized in that, When the industrial drone is a multi-rotor drone, the current action command is the rotational speed of each rotor motor; when the industrial drone is a fixed-wing drone, the current action command is the deflection and throttle of each control surface.

10. A multi-level industrial unmanned aerial vehicle (UAV) control method, characterized in that, The method applies the multi-level industrial unmanned aerial vehicle control system according to any one of claims 1-9, and the method includes: Based on the current mission instructions, current visual situation information and current inertial navigation information of the industrial drone, a multimodal large model is used to perceive the environment and generate current action instructions; Control the actions of the industrial drone based on the current action command.

Images