Intelligent robot control algorithm scheduling method and system based on dynamic directed acyclic graph

Abstract

The application discloses a kind of intelligent robot control algorithm scheduling method and system based on dynamic directed acyclic graph.The scheduling method includes receiving the change of robot state or environment data, traversing conditional directed acyclic graph from root node, when executing to end node, dynamically generating intelligent robot control algorithm scheduling instance;Wherein, conditional directed acyclic graph contains at least one condition node, for activating corresponding model branch according to input data.This application scheduling method can effectively speed up the deployment of robot control algorithm when facing different scenarios, and reduce the complexity of deployment and implementation.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and more specifically to a scheduling method and system for intelligent robot control algorithms based on dynamic directed acyclic graphs. Background Technology

[0002] With the development of embodied intelligence technology, robots are transitioning from "digital intelligence" to "physical intelligence." In this process, reinforcement learning strategy models endow robots with motion control capabilities, while multimodal visual detection models enable them to perceive their environment.

[0003] However, in engineering practice, due to limitations in edge computing power and detection accuracy, a single model cannot cover all motion control or vision inspection scenarios. Currently, for motion control strategy models, a state machine approach is typically used, which involves switching between different robot states. When the robot switches to a different state, the corresponding strategy model for that state is invoked to control the robot's motion behavior. For vision inspection, custom development is often employed.

[0004] However, when controlling robot motion based on state machines, it works well when only considering state changes triggered by data from the robot's motors and sensors such as the IMU, and switching between algorithms is required. However, incorporating visual inspection needs to address more complex scenarios, which presents the following challenges:

[0005] State explosion: Complex scenarios require defining a massive number of state combinations (such as "walking + face detected + obstacle present + low battery"), resulting in bloated code;

[0006] Poor flexibility: The state transition logic needs to be reconstructed when adding a new sensor dimension;

[0007] Weak parallel processing capability: It is difficult to execute multiple tasks concurrently within the same state (e.g., running face recognition and obstacle avoidance algorithms simultaneously while walking requires custom development).

[0008] Therefore, how to solve the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0009] In view of this, and in view of the shortcomings of traditional robot motion behavior control based on state machines, this invention provides a scheduling method and system for intelligent robot control algorithms based on dynamic directed acyclic graphs.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] First, this invention provides a scheduling method for intelligent robot control algorithms based on dynamic directed acyclic graphs, the steps of which include:

[0012] Receive changes in robot state or environmental data, traverse the conditional directed acyclic graph starting from the root node, and dynamically generate a scheduling instance of the intelligent robot control algorithm when the execution reaches the end node.

[0013] The conditional directed acyclic graph contains at least one conditional node, which is used to activate the corresponding model branch based on the input data.

[0014] Optionally, the model branches include a policy model and a visual model.

[0015] Optionally, sub-condition nodes are connected to the subgraph branches of the strategy model and the visual model, and semantics are executed in parallel among the sub-condition nodes.

[0016] Optionally, the execution semantics include at least one of sequential execution, parallel execution, and nested execution; wherein the nested execution applies only to visual model branches.

[0017] Second, the present invention provides a scheduling system for intelligent robot control algorithms based on dynamic directed acyclic graphs. The system has a built-in scheduling configuration file, which is used to implement the scheduling method for intelligent robot control algorithms based on dynamic directed acyclic graphs as described above.

[0018] Preferably, the system also includes:

[0019] The configuration file editing unit is used to edit and modify the scheduling configuration file as needed;

[0020] The configuration file loading unit is used to interpret the edited and modified scheduling configuration file and load it into memory;

[0021] The configuration file management unit is used to trigger the scheduling configuration file in memory and generate a scheduling instance when it receives changes in robot status or environmental data.

[0022] Preferably, the system also includes:

[0023] The instance parsing unit is used to expand the scheduling instance according to the topology and generate scheduling instructions based on node semantics.

[0024] The execution engine unit is used to receive the scheduling instructions and realize unified scheduling execution.

[0025] As can be seen from the above technical solutions, the present invention discloses a scheduling method and system for intelligent robot control algorithms based on dynamic directed acyclic graphs. Compared with the prior art, under the same hardware conditions, the present application can accelerate the deployment and implementation speed of robot models, improve implementation efficiency, and save implementation costs when facing different scenarios. At the same time, it can reduce the complexity of robot model deployment and implementation, and promote the application and implementation efficiency of robots in real-world scenarios. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0027] Figure 1 This is a flowchart of the intelligent robot control algorithm scheduling method based on dynamic directed acyclic graph of the present invention;

[0028] Figure 2 This is a structural diagram of the intelligent robot control algorithm scheduling system based on dynamic directed acyclic graphs according to the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0031] In intelligent robots, the biggest challenge compared to traditional robots is how to autonomously adapt to dynamically changing scenarios, thus overcoming the limitation of traditional robots that can only handle deterministic scenarios with fixed rules. Changes in the scenario mean changes in the robot's state and the data it acquires about its surrounding environment; it is precisely these changes in data that drive the changes in the scenario.

[0032] Traditionally, Directed Acyclic Graphs (DAGs) are often used to determine the dependencies and execution order of tasks in deterministic scenarios. Therefore, if we can extend the traditional DAG by adding conditional nodes, we can use conditional DAGs to generalize and represent all sets of tasks that an intelligent robot should perform in different scenarios, as well as the dependencies between tasks.

[0033] Based on this, the present invention extends the classic directed acyclic graph (DAG) and proposes the idea of ​​dynamic finite acyclic graph, as well as constructs a method and system for scheduling algorithms such as motion control algorithm and visual detection algorithm for intelligent robots.

[0034] Example 1:

[0035] This invention provides a scheduling method for intelligent robot control algorithms based on dynamic directed acyclic graphs (DAGs). By extending the traditional DAG and introducing conditional nodes, a model scheduling configuration file for conditional DAGs is formed, replacing the previous state machine. This transforms the need for scheduling and executing various types of model algorithms under different conditions into specific execution nodes activated by conditional nodes. At the same time, considering the actual situation of intelligent robot algorithm scheduling, three different execution semantics are formed: concurrent execution, sequential execution, and nested execution.

[0036] Specifically, the scheduling steps include:

[0037] Receive changes in robot state or environmental data, traverse the conditional directed acyclic graph starting from the root node root_node, and dynamically generate a scheduling instance of the intelligent robot control algorithm when the execution reaches the end node end_node.

[0038] In this invention, the conditional directed acyclic graph contains at least one conditional node, which is used to activate the corresponding model branch based on the input data.

[0039] In one embodiment, such as Figure 1 , Figure 1 This is a flowchart illustrating the scheduling method of this application;

[0040] root_node: This represents the root node of the entire conditional DAG. Whenever the robot's state or the data obtained from the environment changes, the data is input into root_node, and the entire conditional DAG is traversed starting from root_node to dynamically create a model scheduling and running instance based on the DAG.

[0041] `type_condition_node` represents the condition node for the model type. In this invention, different types of nodes are classified, such as policy models based on reinforcement learning and vision models. Currently, in the model scheduling configuration file of this conditional DAG, this embodiment supports two types by default: policy models based on reinforcement learning (RL_graph_node) and vision models (VI_graph_node). When the model to be called is a policy model, the branch of RL_graph_node will be entered based on the condition judgment; similarly, when the model to be called is a vision model, the branch of VI_graph_node will be entered based on the condition judgment.

[0042] RL_graph_node: Represents a subgraph branch of the policy model, under which all condition nodes are conditions for the policy model.

[0043] VI_graph_node: Represents a subgraph branch of the visual model, and all condition nodes under it are conditions for the visual model.

[0044] It should be noted that this application supports the configuration and scheduling of both policy models and vision models by default. When other types of algorithms also need to be scheduled, it is only necessary to define a new graph_node.

[0045] In this embodiment, there are two semantics in the execution of a specific model: one is that multiple models are executed in parallel, and the other is that multiple models are executed sequentially in a certain order.

[0046] For vision-based models, there is a third semantic: nested execution semantics. This semantic refers to the subgraph resulting from the recognition of a previous parent node. Subsequent nodes will then perform further recognition and detection on this subgraph. This approach addresses the challenge of multiple recognition iterations in visual detection. To address these three semantics, each is defined in the model scheduling configuration file of this invention.

[0047] In the defined semantics, condition_node represents the condition that a model node can be activated. Through condition_node, different combinations of states in the state machine can be transformed into independent discrete conditions, and the corresponding model can be activated by activating the condition node.

[0048] `execution_node` represents a model node with sequential execution semantics. When this node is reached, the model corresponding to it will be scheduled for execution. All subsequent nodes of each `execution_node` form a sequential execution semantic relationship with this node.

[0049] nested_execution_node: Represents a nested execution semantic. Once activated, the execution_node or nested_execution_node under the subsequent condition node connected to nested_execution_node can use the subgraph identified by nested_execution_node as input. nested_execution_node can only exist in the DAG of VI_graph_node.

[0050] In this application, multiple condition_nodes can be connected starting from RL_graph_node and VI_graph_node. In this case, the condition_nodes execute in parallel, without any order of precedence, and can be executed simultaneously. After an execution_node or nested_execution_node finishes execution, multiple condition_nodes can be connected afterward, and these condition_nodes execute concurrently.

[0051] When all nodes have reached end_node, the dynamically created DAG instance will have finished executing.

[0052] In this embodiment, all conditions and models are defined in the configuration file, thus forming a complete conditional DAG covering different scenarios. Whenever the robot's state or the data obtained from the environment changes, the robot traverses the root node and dynamically calculates based on the actual data of each device in the current conditional DAG to form the DAG instance to be executed. The system then schedules and executes all models on this DAG instance to complete the complete model scheduling.

[0053] Example 2:

[0054] This embodiment provides a scheduling system for intelligent robot control algorithms based on dynamic directed acyclic graphs. The system has a built-in scheduling configuration file, which is used to implement the scheduling method for intelligent robot control algorithms based on dynamic directed acyclic graphs as described above.

[0055] In a preferred embodiment, the system structure is as follows: Figure 2 As shown;

[0056] like Figure 2 The system also includes:

[0057] The configuration file editing unit is used to edit and modify the scheduling configuration file as needed. In one embodiment, when the execution conditions of a model change in a certain scenario or when a new model needs to be added, the user can edit or modify the model scheduling configuration file based on conditional DAG through this editing unit, either initially or in subsequent robot applications. This invention supports both online and offline editing methods.

[0058] The configuration file loading unit is used to interpret the edited and modified scheduling configuration file and load it into memory. Specifically, when the system starts or the conditional DAG is edited or modified online, this unit will automatically detect the changes in the file, interpret the file, load it into memory, form a static scheduling model structure based on the conditional DAG, and store it in memory.

[0059] The configuration file management unit is used to trigger the scheduling configuration file in memory and generate a scheduling instance when it receives changes in robot state or environmental data. During robot operation, as the scene and operating conditions change, the robot's state and the data from each sensor also change. When this changed data is passed to this unit, it triggers the calculation of the variables and conditions of the entire DAG starting from the root node in the static structure of the conditional DAG model scheduling in memory. Branches that do not meet the conditions are automatically discarded, and only branches with the condition set to True are retained, thus forming the DAG that needs to be actually executed, i.e., the DAG-based model scheduling execution instance.

[0060] To further optimize the above technical solution, the system also includes:

[0061] The instance parsing unit expands the scheduling instance according to the topology and generates scheduling instructions based on node semantics. After obtaining the running instance, this unit is responsible for expanding the entire instance graph from top to bottom, starting from the root node of the DAG instance, and sending the corresponding model call commands to the unified model execution engine. For concurrent execution, multiple commands are sent to the unified model execution engine simultaneously; for sequential execution, the next node's model execution command is sent after the previous model has finished executing; for parent-child model calls, the detection results of the parent node model are automatically sent to the child model when the child model is executed, thus realizing the execution semantics of parent-child nodes.

[0062] The execution engine unit receives the scheduling instructions and implements unified scheduling execution. This invention, through this unified execution engine unit, achieves the scheduling and execution of the strategy model and the visual model according to the model specified in the model invocation command, and feeds back the results to the DAG-based model scheduling and execution engine after execution.

[0063] The embodiments disclosed in this application, when facing different scenarios, can accelerate the deployment and implementation speed of robot models under the same hardware conditions, improve implementation efficiency, and save implementation costs; at the same time, they can reduce the complexity of robot model deployment and implementation, and promote the application and implementation efficiency of robots in real-world scenarios.

[0064] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0065] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic directed acyclic graph based intelligent robot control algorithm scheduling system, characterized in that, The built-in scheduling configuration file is used to implement a dynamic directed acyclic graph-based intelligent robot control algorithm scheduling method; the method comprises: Receiving changes in robot state or environment data, traversing the conditional directed acyclic graph from the root node, and generating an intelligent robot control algorithm scheduling instance when execution reaches the end node; The conditional directed acyclic graph contains at least one conditional node for activating the corresponding model branch according to the input data; The system further comprises: A configuration file editing unit for editing and modifying the scheduling configuration file on demand; A configuration file loading unit for interpreting the edited and modified scheduling configuration file to form a model scheduling static structure of the backbone conditional DAG and loading it into the memory; A configuration file management unit for triggering the scheduling configuration file in the memory to generate a scheduling instance when receiving changes in robot state or environment data; An instance analysis unit for expanding the scheduling instance according to the topological structure and generating scheduling instructions according to the node semantics, which is responsible for expanding the entire instance graph from the root node of the DAG instance and sending corresponding model scheduling instructions to the model unified execution engine; An execution engine unit for receiving the scheduling instructions, implementing unified scheduling execution, and feeding back the results to the DAG-based model scheduling execution engine after execution.

2. The intelligent robot control algorithm scheduling system of claim 1, wherein, The model branch includes a strategy model and a vision model.

3. The intelligent robot control algorithm scheduling system of claim 2, wherein, The subgraph branches of the strategy model and the vision model are connected with sub-condition nodes, and the execution semantics of each sub-condition node are executed in parallel.

4. The intelligent robot control algorithm scheduling system of claim 3, wherein, The execution semantics at least include one of sequential execution, parallel execution, and nested execution; wherein, The nested execution is only applicable to the vision model branch.

Images