Graphical arrangement and management system for unmanned aerial vehicle tasks

Abstract

The invention provides an unmanned aerial vehicle task graphical arrangement and management system, which analyzes a task component model through a task graphical arrangement module, combines and arranges task components displayed in a graphical manner through dragging and connecting a visual interaction mode, displays the task components in the graphical manner, and performs visual deployment and operation management on tasks. A task model file is generated, a task deployment management module generates operation instructions corresponding to task installation, loading, starting, stopping and deleting operation controls according to user operations in the task model file, and a task operation management module receives and responds to the operation instructions sent by the task deployment management module, analyzes an unmanned aerial vehicle task description model and sends the unmanned aerial vehicle task description model to the task deployment management module. The execution result sent by each task component is received in real time, and the interface is called to control installation, loading, starting, stopping, deleting and receiving of the execution result according to the switching logic between the task components in the task component model, so that the problem that in the prior art, complex steps are needed to be put into use is solved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) mission development and management technology, specifically to a graphical orchestration and management system for UAV missions. Background Technology

[0002] Currently, drones are widely used in various fields such as emergency rescue, power line inspection, surveying and mapping, and agricultural plant protection. These fields have different operating environments and application requirements. Commercial drones with single functions need to purchase multiple devices according to different application scenarios, which increases the cost of use.

[0003] Currently, UAV mission development systems that support secondary development are typically designed based on scripts or programming languages. While they can adapt to different mission requirements to some extent, whenever mission requirements change, users need to define the UAV's mission flow and execution logic by writing scripts in a specific format or adjusting mission code files. This requires complex steps such as compilation, debugging, and deployment before the system can be put into use. This process has a high technical threshold, creating operational obstacles for non-technical personnel, limiting the widespread application of UAVs, severely impacting their ability to respond quickly to new missions, and reducing the overall operational efficiency of UAVs.

[0004] Specifically, this task development method based on scripts or programming languages ​​has the following obvious technical drawbacks:

[0005] (1) Poor readability and visualization: Scripts or programming languages ​​are essentially textual expressions that lack intuitive task flow displays. Ordinary operators find it difficult to understand complex task logic, resulting in low task design efficiency.

[0006] (2) Insufficient flexibility and scalability: Since the task logic is directly hard-coded in the script or code, once the task requirements change, it must be modified, compiled and tested again, making it difficult to reuse and combine task modules, which limits the system's adaptability in diverse task scenarios.

[0007] (3) High development and maintenance costs: Using scripts or programming languages ​​to write tasks relies on professional technical personnel. Non-professional technical personnel find it difficult to independently complete the construction of complex tasks, which increases the cost of training and human resources. At the same time, writing errors can easily lead to task execution failures, and the debugging process is cumbersome.

[0008] (4) It is not conducive to quickly responding to new task requirements: When faced with new task requirements, the method of writing tasks using scripts or programming languages ​​cannot quickly splice existing task modules to form new tasks, resulting in a slow response speed.

[0009] In summary, although task development systems based on scripts or programming languages ​​can achieve basic task development functions, they still have significant shortcomings in terms of ease of operation, task maintainability, module reusability, and rapid response capabilities. These problems restrict the operational efficiency of UAVs and limit their application scope. Summary of the Invention

[0010] The purpose of this invention is to design a graphical arrangement and management system for unmanned aerial vehicle (UAV) missions.

[0011] The present invention provides a graphical orchestration and management system for unmanned aerial vehicle (UAV) missions, comprising:

[0012] The task component encapsulation and model building module is used to encapsulate the UAV's task software module into task components, and to describe the parameters and interfaces of the task components, build a task component model, and build a UAV task description model based on the switching logic between task components in the task component model, combined with basic task information and a list of task components.

[0013] The task graphical orchestration module is used to parse task component models, and combine and orchestrate task components through drag-and-drop and connection visual interactive methods. It also provides a graphical display of task components, visualizes task deployment and operation management, and generates task model files.

[0014] The task deployment management module is used to generate operation instructions corresponding to the task installation, loading, starting, stopping, and deletion operation controls based on user operations in the task model file;

[0015] The task operation management module is used to receive and respond to operation instructions issued by the task deployment management module, parse the UAV task description model, receive the execution results sent by each task component in real time, and call the interface to control the installation, loading, start, stop, deletion and execution results of task components according to the switching logic between task components in the task component model.

[0016] Preferably, the parameter description includes: using the value attribute in the entrance element to specify the startup entry point or execution script path of the task component, the value attribute in the url element to describe the network address of the drone node where the task component is located and its listening port number, and the value attribute in the platform_url element to describe the network address of the drone node and the communication port number of the task operation management module.

[0017] Preferably, the parameter description includes: using the outs element to describe the return value type that may be generated after the task component is executed, the port element to indicate the executable result or status code, if the outs element does not contain any port element, it means that the task component is the last execution component in the entire task process, and the params attribute set is used to define the parameter set that the task component supports graphical configuration, each param sub-element corresponds to a configurable parameter, including parameter name, type and value.

[0018] Preferably, the interface description includes: using ComponentRequest to pass parameters to the task component interface, including an array of type Param called params; and using ComponentResponse to return parameters to the task component interface, including the result identifier flag of the interface call and specific information info.

[0019] Preferably, the UAV mission description model includes: using several sub-elements of the components element to describe all mission components that constitute the mission, and the flows element having multiple ordinal elements to describe the switching logic between mission components.

[0020] This invention proposes a graphical orchestration and management system for UAV missions. By encapsulating common, single-function mission modules in UAVs into standardized mission components, these components can be combined visually via drag-and-drop and connection to quickly construct UAV missions that meet new requirements. The system is simple to operate, requires no code development, and can be used even by non-technical personnel. Furthermore, the system enables remote deployment and management of new missions, significantly improving the flexibility and adaptability of UAVs in diverse application scenarios, enhancing their rapid response to new missions, and improving overall operational efficiency. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the code describing the parameters of the UAV mission component model provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the code describing the interface of the UAV mission component model provided in an embodiment of the present invention;

[0023] Figure 3 This is a partial code diagram of the UAV mission description model provided in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the graphical arrangement of UAV missions provided in an embodiment of the present invention;

[0025] Figure 5 This is a partial code diagram of the interface description model for the task operation management module provided in an embodiment of the present invention;

[0026] Figure 6 A sequence diagram of the functions of the UAV mission graphical orchestration and management system provided in this embodiment of the invention;

[0027] Figure 7 This is a schematic diagram illustrating the deployment of various modules in the graphical UAV mission orchestration and management system provided in this embodiment of the invention. Detailed Implementation

[0028] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0029] This invention provides a graphical orchestration and management system for unmanned aerial vehicle (UAV) missions, comprising the following steps:

[0030] The design of UAV mission component models and mission models involves encapsulating general-purpose, single-function mission software modules in UAVs, such as takeoff, waypoint flight, obstacle avoidance flight, target search, and landing.

[0031] The task component model consists of two parts: parameter description defined using Extensible Markup Language (XML) and interface description described using an interface description language. This improves the model's compatibility and readability, enables efficient management of task components by the task execution management system, and responds to button operations in the task deployment management module.

[0032] like Figure 1 As shown, the component element represents a task component. The name attribute is used to identify the name of the task component, which is convenient for users to identify and manage. The uuid (Universally Unique Identifier) ​​attribute is used to uniquely identify the task component. In the same task, the uuid value of each task component is unique, ensuring that there is no ambiguity in internal system references.

[0033] The `value` attribute of the `entrance` element specifies the entry point or execution script path for the task component, supporting program calls or service startup. The `value` attribute of the `url(Uniform Resource Locator, url)` element describes the network address and listening port number of the UAV node where the task component resides, facilitating the location of the specific task component during remote procedure calls. The `value` attribute of the `platform_url` element describes the network address of the UAV node and the communication port number of the task operation management module, facilitating communication between the ground station computer and the UAV's onboard computer. The `nums` attribute of the `ins` element indicates the number of input ports accepted by the task component. When `nums` is 0, it indicates that this task component is the first execution component in the entire task flow.

[0034] The `outs` element describes the possible return value types that a task component might produce upon completion, i.e., the output ports. Each `port` sub-element represents a possible execution result or status code. If `outs` does not contain any `port` elements, it indicates that this task component is the last one executed in the entire task flow. `params` is a collection of attributes used to define the set of parameters that the task component supports for graphical configuration. Each `param` sub-element corresponds to a configurable parameter, containing the parameter name, type, and value.

[0035] Unmanned aerial vehicle (UAV) mission component interface description model, such as Figure 2 As shown, the UAV mission component interface description model is described using the interface description language protobuf (Protocol Buffers, protobuf). The mission component provides two interfaces, start and stop, to control its lifecycle. The parameters passed to the mission component interface are of type ComponentRequest, and the returned parameters are of type ComponentResponse. ComponentRequest contains an array of type Param, params, used to pass parameters to the mission component interface. Param contains three fields, all of type string: name (parameter name), type (parameter type), and value (parameter value). ComponentResponse contains two fields: flag (an integer identifier indicating the result of the interface call) and info (a string containing specific information).

[0036] The UAV mission description model describes the names of all mission components and their execution order within a mission. Specifically, the model defines the switching logic between these components, dynamically switching to the appropriate subsequent task component based on its execution result after the completion of a given component. For compatibility and readability, the UAV mission model also uses XML for structured description. A UAV mission is composed of multiple mission components.

[0037] like Figure 3 As shown, the UAV mission description model comprises three parts: basic mission information, a list of mission components, and the switching logic between mission components. In the mission description model, the `task` element contains three attributes: `name` (mission name), `time` (mission creation time), and `uuid` (unique identifier). The `components` element contains several sub-elements `component`, which describe all the mission components that make up the mission. Each `component` element has two attributes: `name` (mission component name) and `uuid` (unique identifier). The `flows` element describes the mission execution flow, consisting of the switching logic between multiple mission components, and is composed of multiple `ordinal` elements. Each `ordinal` element describes one flow of the mission. The `value` attribute describes the flow number, defining the order of the mission flows. The `flow` element describes the switching logic between mission components; a flow contains the switching logic of one or more mission components, i.e., one or more flows. Each flow element contains three attributes: the previous_component_uuid attribute is the UUID of the previously executed task component, the previous_component_result attribute is the execution result of the previous task component, and the component_uuid attribute is the UUID of the next task component to be executed.

[0038] The UAV mission graphical orchestration and management system consists of three modules: mission graphical orchestration module, mission deployment management module, and mission operation management module.

[0039] The task graphical orchestration module follows the MVC (Model-View-Controller) design philosophy. After parsing the task component parameter description model, it graphically displays the task components and provides visual interactive methods such as drag-and-drop and connection to combine and orchestrate them, ultimately generating a task model file, such as... Figure 4 As shown.

[0040] The task deployment management module provides a graphical interface, including operation controls for installing, loading, starting, stopping, and deleting tasks, making it easy for users to visually deploy and manage tasks.

[0041] The task execution management module is responsible for receiving and responding to operation instructions issued by the task deployment management module, specifically implementing functions such as install, load, start, stop, and delete for tasks. Furthermore, the task execution management module needs to receive the execution results (notices) sent by each task component in real time. Based on the logical relationships between components in the task description model, it controls the start and stop of task components by calling the start and stop interfaces in the task component interface model. The task execution management model interface description model is as follows: Figure 5 As shown.

[0042] The task execution management model interface description model is described by the interface description language protobuf, which provides six interfaces: install, load, start, stop, delete, and notice.

[0043] The `install` interface sends the task model file, the task component parameter description model file, and the corresponding task component executable files from the ground station computer to the UAV's onboard computer. The `install` interface takes a parameter of type `TaskInstallRequest` as input. `TaskInstallRequest` contains four fields: `name` (task name), `path` (relative path to each file in the task), `index` (index of the file segment), and `content` (partial content of the file segment corresponding to the `index`). The return value is a parameter of type `TaskResponse`, containing two fields: `flag` (an identifier of the interface call result) and `info` (specific information). Due to network maximum transmission unit limitations, large task files need to be divided into several smaller data packets, and the entire file fragment installation process is completed through multiple calls to the `install` interface. Therefore, the `install` interface uses a bidirectional streaming communication mechanism to accommodate this fragmented transmission requirement. Furthermore, to further improve the performance and efficiency of the `install` interface, a multi-threaded parallel sending mechanism is employed to ensure that the data transmission process is both fast and reliable. This design allows the task graphical orchestration module to monitor the task installation progress in real time.

[0044] The `load` interface loads all task components into memory and activates them, but these components do not execute any specific business logic. The `load` interface takes a parameter of type `TaskRequest` as input, which contains a field named `name` representing the task. It returns a parameter of type `TaskResponse`. Since a task contains multiple components that need to be run sequentially, and the success of the interface call is only known after a certain period, the `load` interface employs a server-side streaming communication mechanism. This design allows the task graphical orchestration module to obtain real-time task loading progress.

[0045] The `start` interface initiates the execution of specific business functions within a task. Based on the task component execution strategy defined in the task model file, and considering the result of the previous task component's completion, it calls the `start` interface of the next task component to launch it. The `start` interface takes a parameter of type `TaskRequest` as input and returns a parameter of type `TaskResponse`. Because a task contains multiple task components and there are dependencies between them, the success of the interface call can only be determined after a certain period. Therefore, the `start` interface employs a server-side streaming communication mechanism. This design allows the task graphical orchestration module to monitor and obtain the execution progress of each task component in real time, ensuring the observability and controllability of the entire task execution process.

[0046] The `stop` interface terminates task execution and releases occupied resources. The `stop` interface takes a parameter of type `TaskRequest` as input and returns a parameter of type `TaskResponse`. Since a task contains multiple task components, the success of the interface call can only be determined after a certain period of time. Therefore, the `stop` interface uses a server-side streaming communication mechanism. This design allows the task graphical orchestration module to monitor the task's stopping progress in real time.

[0047] The `delete` interface deletes task files from the UAV's onboard computer. The `delete` interface takes a parameter of type `TaskRequest` as input and returns a parameter of type `TaskResponse`. Considering the complexity of the task file directory structure and the potential number of files, and the fact that the success of the interface call may only be known after a certain period of time, the `delete` interface employs a server-side streaming communication mechanism. This design allows the task graphical orchestration module to view the task deletion progress in real time.

[0048] The `notice` interface is used to receive the execution results of task components. The input to the `notice` interface is a parameter of type `TaskNoticeRequest`, which contains two fields: `uuid` (a unique identifier for the task component) and `result` (the execution result of the task component's business function). The return value is a parameter of type `TaskResponse`. All task components that are loaded into memory and activated will periodically call the `notice` interface to send their execution results to the task management module.

[0049] The sequence diagram of the UAV mission graphical orchestration and management system includes functions such as mission orchestration, deployment, and management, as shown in the figure. Figure 6 As shown.

[0050] The task graphical orchestration and task deployment management modules are installed on the ground station computer, while the task operation management module is deployed on the UAV's onboard computer. Interface calls between the task deployment management module, the task operation management module, and task components are implemented using Remote Procedure Call (RPC) middleware, such as gRPC (Google Remote Procedure Call) or CORBAR (Common Object Request Broker Architecture), to support efficient remote interface calls between modules under wireless network communication conditions. Figure 7 As shown.

[0051] The core protection point of this invention is: the design and construction of the task component model, which is used to standardize general and single-function task modules. The task model formed based on this component model supports the expression and organization of complex task logic, and provides a user-oriented graphical orchestration and management system.

[0052] This invention proposes a graphical orchestration and management system for unmanned aerial vehicle (UAV) missions. By encapsulating various single-function, highly versatile mission modules into standardized mission components, users can flexibly combine these basic components through an intuitive graphical interface. This enables rapid development, deployment, and efficient management of UAV missions, forming a complete UAV mission orchestration, deployment, and operation management system. This system significantly improves the flexibility and adaptability of UAVs in complex and ever-changing application scenarios, effectively enhancing their response speed and execution efficiency to new or unexpected missions. Furthermore, the standardized component design not only lowers the technical threshold for mission development and maintenance but also greatly improves system scalability and reusability, demonstrating promising engineering application prospects and widespread application value.

Claims

1. A graphical orchestration and management system for unmanned aerial vehicle (UAV) missions, characterized in that, include: The task component encapsulation and model building module is used to encapsulate the UAV's task software module into task components, and to describe the parameters and interfaces of the task components, build a task component model, and build a UAV task description model based on the switching logic between task components in the task component model, combined with basic task information and a list of task components. The task graphical orchestration module is used to parse task component models, and combine and orchestrate task components through drag-and-drop and connection visual interactive methods. It also provides a graphical display of task components, visualizes task deployment and operation management, and generates task model files. The task deployment management module is used to generate operation instructions corresponding to the task installation, loading, starting, stopping, and deletion operation controls based on user operations in the task model file; The task operation management module is used to receive and respond to operation instructions issued by the task deployment management module, parse the UAV task description model, receive the execution results sent by each task component in real time, and call the interface to control the installation, loading, start, stop, deletion and execution results of task components according to the switching logic between task components in the task component model.

2. The graphical orchestration and management system for unmanned aerial vehicle (UAV) missions as described in claim 1, characterized in that, The parameter descriptions include: specifying the startup entry point or execution script path of the task component using the value attribute in the entrance element; describing the network address of the drone node where the task component is located and its listening port number using the value attribute in the url element; and describing the network address of the drone node and the communication port number of the task operation management module using the value attribute in the platform_url element.

3. The graphical orchestration and management system for unmanned aerial vehicle (UAV) missions as described in claim 1, characterized in that, The parameter description includes: the outs element describes the return value type that may be generated after the task component is executed; the port element indicates the executable result or status code; if the outs element does not contain any port element, it means that the task component is the last execution component in the entire task process; the params attribute set is used to define the parameter set that the task component supports graphical configuration; each param sub-element corresponds to a configurable parameter, including parameter name, type, and value.

4. The graphical orchestration and management system for unmanned aerial vehicle (UAV) missions as described in claim 1, characterized in that, The interface description includes: using ComponentRequest to pass parameters to the task component interface, including an array of type Param called params; and using ComponentResponse to return parameters to the task component interface, including the result identifier flag of the interface call and specific information info.

5. The graphical orchestration and management system for unmanned aerial vehicle (UAV) missions as described in claim 1, characterized in that, The UAV mission description model includes: using several sub-elements of the components element to describe all mission components that constitute the mission, and the flows element having multiple ordinal elements to describe the switching logic between mission components.

Images