System for controlling robot task decision-making on the basis of semantic network and knowledge base

By introducing a decision behavior tree construction method based on semantic networks and knowledge bases into the robot system, combining large language models and knowledge bases, the problems of low decision-making control efficiency and insufficient independent decision-making capabilities in the existing technology are solved, and rapid and efficient task decision-making and diversity improvement are achieved.

WO2025102453A1PCT designated stage expired Publication Date: 2025-05-22ZHEJIANG LAB

Patent Information

Application Number
PCT/CN2023/136721
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-11-13
Filing Date
2023-12-06
Publication Date
2025-05-22

AI Technical Summary

Technical Problem

In the prior art, the robot's task decision control relies on fixed programmatic scripts or large language models, cannot make quick and efficient decisions, and is highly cost-effective; the scale of the traditional knowledge base limits the robot's autonomous decision-making control capabilities and diversity, resulting in overly programmatic and rigid operation behavior.

Method used

Design a system based on semantic networks and knowledge bases, combine large language models and knowledge bases, build task decision behavior trees through semantic network modules, large language model modules, knowledge base modules and task decision-making modules, and use semantic networks and large language models to build decision tree nodes and fill parameter.

Benefits of technology

It realizes the rapid and efficient robot task decision-making, avoids redundant actions and behaviors, improves the robot's independent decision-making control capabilities and diversity, and reduces training costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023136721_22052025_PF_FP_ABST
    Figure CN2023136721_22052025_PF_FP_ABST
Patent Text Reader

Abstract

A system for controlling robot task decision-making on the basis of a semantic network and a knowledge base, the system comprising: a semantic network module, which is used for providing a semantic network for constructing a task decision-making behavior tree, wherein the semantic network is supplemented and expanded by means of question-answering results of a large language model and a knowledge base; a large language model module, which is used for using the large language model to perform knowledge question-answering and output the question-answering results; a knowledge base module, which is used for providing knowledge for the construction of a task decision-making behavior tree; a task decision-making module, which is used for constructing the task decision-making behavior tree on the basis of the semantic network module, the large language model module and the knowledge base module, and specifically, constructing nodes of the decision-making tree by means of querying the semantic network, performing node parameter filling by means of the question-answering results of the large language model, and performing node parameter filling by means of the knowledge base; and a variable storage module, which is used for storing temporary variables of when the task decision-making behavior tree is constructed. On the basis of the system, a decision-making behavior tree is constructed more accurately.
Need to check novelty before this filing date? Find Prior Art

Description

A system for controlling robot task decisions based on semantic network and knowledge base Technical Field

[0001] The present invention belongs to the technical field of robots and artificial intelligence, and in particular relates to a system for controlling robot task decisions based on a semantic network and a knowledge base. Background Art

[0002] In existing technologies, robot control relies more on fixed programmed scripts, knowledge symbols or newly developed large models. From the perspective of technical application, programmed scripts and knowledge symbols cannot adapt to environmental changes. Large language models can handle robot task control in dynamic scenarios, but they cannot make decisions quickly and efficiently, and the training cost is huge.

[0003] Traditional knowledge bases only store knowledge about objects, people, common sense, and actions. These knowledge is then used to plan tasks using pre-built robot planning methods. During this process, knowledge is stored as isolated nodes, and parameter attributes must be synchronized and injected only when the robot is performing tasks. Furthermore, the size of the knowledge base limits the robot's autonomous decision-making and control capabilities and diversity, making its operational behaviors overly programmed and rigid.

[0004] At the same time, the robot task planning output of the prefabricated model is a sequence of behaviors. This results in a large number of redundant actions and behaviors when constructing complex robot tasks, and these behaviors and tasks will not be fully covered during runtime.

[0005] To this end, there is an urgent need to improve and innovate based on knowledge bases, graph computing, and large language models, and to design a new solution based on the big data semantic network foundation of robot operations to apply semantic networks to robot behavior decision-making.

[0006] Patent document CN114153943A discloses a system and method for constructing a robot behavior tree based on a knowledge graph. The system includes: a knowledge graph and a behavior tree. The knowledge graph is constructed according to the AOG format through a set of directed connected action nodes. The action nodes include action categories and their corresponding set of action features and action objects. The directed connections connect the action nodes based on the compared action features. The behavior tree uses the subject and object as the action objects according to the task of the subject-predicate-object structure, classifies the predicate into a set of action features corresponding to the action category, constructs the corresponding behavior according to the action features, constructs the behavior according to the behavior, and swaps the order of each behavior node in the behavior tree according to the directed connection of the sub-action nodes. The method includes: S1, constructing an object knowledge base; S2, constructing an action knowledge base; S3, constructing a knowledge graph; S4, constructing a behavior tree.

[0007] The above technical solution constructs a behavior tree based on the knowledge graph, but many behaviors do not exist in the knowledge graph, which may cause the behavior tree construction to fail or be inaccurate.

[0008] Summary of the Invention

[0009] In view of the above, the purpose of the present invention is to provide a system for controlling robot task decision-making based on semantic network and knowledge base, which combines semantic network, large language model and knowledge base to generate decision behavior tree for robot task.

[0010] To achieve the above-mentioned object of the invention, the present invention provides a system for controlling robot task decision-making based on semantic network and knowledge base, comprising a semantic network module, a large language model module, a knowledge base module, a task decision module and a variable storage module;

[0011] The semantic network module is used to provide a semantic network for constructing a task decision behavior tree, and the semantic network is supplemented and expanded by the question-answering results and knowledge base of the large language model;

[0012] The large language model module is used to use the large language model to perform knowledge question answering and output question answering results;

[0013] The knowledge base module is used to provide knowledge for constructing a task decision behavior tree;

[0014] The task decision module is used to construct a task decision behavior tree based on the semantic network module, the large language model module, and the knowledge base module. Specifically, the nodes of the decision tree are constructed by querying the semantic network, the node parameters are filled in according to the question and answer results of the large language model, and the node parameters are filled in according to the knowledge base.

[0015] The variable storage module is used to store temporary variables when constructing a task decision behavior tree.

[0016] Preferably, the semantic network is a directed connected graph structure, which sequentially connects associated semantic nodes, and uses in-degree and directed connections to represent the weights and possibilities between semantic nodes. Each semantic node includes an identifier and a parameter list, wherein the directed connection relationship includes a relationship type and relationship attributes, and the relationship attributes include routing possibilities, execution order, and conditional descriptions. The semantic nodes include semantic nodes with single semantics, semantic nodes with decorative information, semantic nodes with conditions, and semantic nodes that allow targets to be sets, so as to adapt to the construction of task decision behavior trees.

[0017] Preferably, the large language model pre-stores knowledge, which can perform question-answering search based on the input question and output question-answering results when expanding the semantic network and constructing the task decision behavior tree. At the same time, it can search based on the task decomposition problem and provide multiple possible results for task planning. The question-answering results and possible results are connected to the semantic network as expanded knowledge, or added to the nodes of the task decision behavior tree as supplementary knowledge.

[0018] Preferably, the knowledge base is refined knowledge, including an action knowledge base and an object knowledge base, wherein the action knowledge base includes actions, action types, action features, and action objects; the object knowledge base includes objects and object attributes.

[0019] Preferably, the step of constructing a task decision behavior tree includes:

[0020] Determine the predicate and target object corresponding to the task based on the task understanding command and task classification results;

[0021] Searching the object instance corresponding to the target object in the object knowledge base according to the target object as the target object instance of the action;

[0022] Classify predicates into action categories in the action knowledge base, and construct a set of behaviors based on a set of task semantics corresponding to the action categories;

[0023] Search the semantic network for the semantic node set corresponding to the behavior set, then construct a task decision behavior tree based on the semantic node set, and add node parameters to the action nodes in the task decision behavior tree, where the node parameters come from at least one of the target object instances searched from the object knowledge base, the dynamically perceived scene instances, and the question-and-answer results obtained through multiple rounds of dialogue based on the large language model.

[0024] Preferably, searching for a semantic node set corresponding to the behavior set in the semantic network and then constructing a task decision behavior tree according to the semantic node set includes:

[0025] The semantic node set corresponding to the behavior includes a single semantic node, a semantic node with decorative information, a semantic node with conditions attached, and a semantic node with a set as the target;

[0026] When constructing a decision behavior tree based on the topological structure corresponding to the semantic node set, when the semantic node is a single semantic node, an action node is generated and executed sequentially; when the semantic node is a semantic node with attached conditions, a branch is generated for the semantic node with attached conditions, specifically a selection node is generated, and a condition node and an action node of the branch are connected under the selection node; when the semantic node is a semantic node with attached decoration information, a decoration node is generated for the semantic node with attached decoration information, and a sequence node and an action node are generated under the decoration node; when the semantic node is a semantic node that allows the target to be a set, multiple decoration nodes are generated for the semantic node with the target being a set, and each decoration node is expanded and generated in the above manner;

[0027] The action nodes, sequence node selection nodes, condition nodes, and decoration nodes generated according to the topological structure corresponding to the semantic node set constitute the task decision behavior tree.

[0028] Preferably, during the specific construction, when a semantic node is attached with a condition, the rule knowledge is queried, a selection node is constructed based on the rule knowledge, and a condition node is generated under the selection node based on the rule knowledge.

[0029] Preferably, when constructing the task decision behavior tree, the execution order of each sub-behavior in the task decision behavior tree is swapped according to the execution order of the sub-action nodes.

[0030] Preferably, when constructing the task decision behavior tree, a task decision behavior tree verification process is also included, and the verification conditions are:

[0031] The sub-behaviors under each behavior must all correspond to sequential nodes or selection nodes. Parallel nodes can coexist with sequential nodes. Selection nodes must contain conditional nodes and the conditional node judgment Boolean parameters contained in the conditional nodes. Any behavior must contain an action identification parameter to complete the action.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] This invention incorporates the design thinking of complex semantic networks based on action knowledge, creating dependencies between action knowledge. This allows the robot's behavior to be constructed solely through semantic networks; the resulting task decision-making behavior tree is superior to language and script control sequences in terms of control logic structure. Behavior trees can represent both the order and the hierarchy of actions, and the same behavior plan will not appear repeatedly. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] 1 is a schematic diagram of the structure of a system for controlling robot task decision-making based on a semantic network and a knowledge base provided by an embodiment;

[0036] FIG2 is a schematic diagram of generating an action node from a terminal semantic node according to an embodiment;

[0037] FIG3 is a schematic diagram of a semantic node generating a behavior tree for implementing the provided terminal attached condition;

[0038] FIG4 is a schematic diagram of a behavior tree generated by a semantic node with decoration information provided by an embodiment;

[0039] FIG5 is a schematic diagram of generating a behavior tree for semantic nodes of a set as a target provided by an embodiment;

[0040] FIG6 is an overall flow chart of generating a task decision behavior tree provided by an embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0042] As shown in FIG1 , the system 100 for controlling robot task decision-making based on semantic network and knowledge base provided in the embodiment includes a semantic network module 110 , a large language model module 120 , a knowledge base module 130 , a task decision module and a variable storage module 140 .

[0043] The semantic network model 110 is used to provide a semantic network for constructing a task decision behavior tree. This semantic network is supplemented and expanded by the question-answering results and knowledge base of the large language model. The semantic network corresponds to the decomposition structure of the task, establishing action nodes M based on the robot's minimum controllable actions. Using task decomposition, a task is broken down into multiple required subtasks, some of which have conditional judgments and selection values.

[0044] A semantic network is a directed graph structure that sequentially connects semantic nodes. In-degree and directed connections represent the weights and probabilities between semantic nodes. Each semantic node, when acting as an action node, includes a unique action identifier and a parameter list (vars) for its target object. A directed connection consists of a relationship type and relationship attributes. Attributes include routing probability (prop), execution order (seq), and conditional description (condition). For the routing probability (prop), if a semantic node's subordinate nodes lack an execution order parameter, the prop is used as the sorting criterion. When acting as a conditional relationship, the prop determines the probability of the node being placed in the behavior tree, effectively constraining a random number within a certain range. The relationship type is the association attribute between semantic nodes, including is, action, transform, and space. Regardless of the type of relationship, connections are constrained by the execution order (seq) from smallest to largest. The conditional description (condition) indicates the conditions for decomposing the relationship. This requires creating decorator and conditional nodes in the corresponding decision behavior tree to address this issue.

[0045] Directed links can be in two different states: feasibility and necessity. When in the feasibility state, the route probability prop indicates the selection strength of the action node. Semantic nodes include single semantic nodes, semantic nodes with decorative information, semantic nodes with conditions, and semantic nodes that allow targets to be sets, to adapt to the construction of task decision behavior trees. Single semantics encompasses various action semantics.

[0046] The large language model module 120 is used to conduct knowledge Q&A using the large language model and output Q&A results. The large language model pre-stores knowledge and can perform Q&A searches based on input questions and output Q&A results when expanding the semantic network and constructing the task decision behavior tree. It also searches based on the task decomposition question and provides multiple possible outcomes for task planning. The Q&A results and multiple possible outcomes are then connected to the semantic network as expanded knowledge or added to nodes in the task decision behavior tree as supplementary knowledge.

[0047] The knowledge base module 130 provides the knowledge needed to construct the task decision behavior tree. This refined knowledge is used to store real-time dynamic scene knowledge and includes an action knowledge base and an object knowledge base. The object knowledge base includes scene objects and their attributes. These objects include person information {People} and environmental object information {Env}. Both contain attributes such as person names, person titles, person coordinates, person features, object names, object coordinates, object features, historical trajectories, and social gestures. When constructing the decision behavior tree, the object knowledge base searches for the corresponding target object based on the robot information and task classification results, which serves as the basis for populating the action action object parameter vars.

[0048] The action knowledge base includes actions, their motion types (motions), action features, and action objects. Action features refer to the robot hardware components required to execute an action and a table of necessary parameters for executing the action. A type identifier is constructed for each action category ID, a semantic association is constructed for the action features, and a target object is associated with the action object. The target object is an object instance created in the object knowledge base based on its data format in computer memory. It has a unique object identifier and uses a unique action ID to act on the target object and the object parameter list vars. The object parameter list vars includes the location, target object, target person, language content, grasping space pose information, interaction goals, preprocessing functions, etc., as shown in Table 1.

[0049] Table 1

[0050] The task decision module 140 is used to construct a task decision behavior tree based on the semantic network module, the large language model module, and the knowledge base module. Specifically, it constructs the nodes of the decision tree by querying the semantic network, fills the node parameters through the question and answer results of the large language model, and fills the node parameters through the knowledge base.

[0051] Specifically, combining tasks and using semantic networks, a decision-making behavior tree is constructed through selected conditional rules. The conditional rule calculation is related to the robot's perception information. When the real-time conditional rule calculation determines that the current conditional rule can be executed, a multi-way selection branch behavior is constructed to obtain a real-time conditional judgment result of Boolean type through the synchronous condition-rule-execution feature. When generating targets for the decision-making behavior tree, a target object query is performed for the behavior based on the corresponding subject of the behavior. Position information, action category motion, description information, language information, job type, and target list are injected into the object parameter list of the action node. The action category motion is injected into the behavior as content. The task type and a random number are used as the unique identifier ID of the decision-making behavior tree and cached in the database. For actions where the parameter object target cannot be determined at all, the behavior tree construction is determined to have failed.

[0052] More specifically, as shown in Figure 6, constructing a task decision behavior tree includes: determining the predicate and target object corresponding to the task based on the task understanding command and the task classification result; searching the object instance corresponding to the target object in the object knowledge base as the target object instance of the action; classifying the predicate into the action category in the action knowledge base, and constructing a behavior set based on a set of task semantics corresponding to the action category; searching the semantic node set corresponding to the behavior set in the semantic network, and then constructing the task decision behavior tree according to the semantic node set, and adding node parameters to the action nodes in the task decision behavior tree, wherein the node parameters come from at least one of the target object instance searched from the object knowledge base, the dynamically perceived scene instance, and the question and answer results obtained through multiple rounds of dialogue based on the large language model.

[0053] When constructing a decision behavior tree based on the topological structure corresponding to the semantic node set, as shown in Figure 2, when the semantic node is a single semantic node, an action node is generated and executed sequentially; as shown in Figure 3, when the semantic node is a conditional semantic node, a branch is generated for the conditional semantic node, specifically a selection node is generated, and a condition node and an action node of the branch are connected under the selection node; as shown in Figure 4, when the semantic node is a semantic node with decorative information, a decoration node is generated for the semantic node with decorative information, and a sequence node and an action node are generated under the decoration node; as shown in Figure 5, when the semantic node is a semantic node that allows the target to be a set, multiple decoration nodes are generated for the semantic node whose target is a set, and each decoration node is expanded and generated in the above manner; the action nodes, sequence nodes, selection nodes, condition nodes, and decoration nodes generated based on the topological structure corresponding to the semantic node set constitute a task decision behavior tree.

[0054] Specifically, a semantic node set corresponding to a behavior set is searched in the semantic network. This semantic node set represents an action node and a sub-behavior set, where sub-behaviors are composed of action nodes. For this semantic node set, the action is split into action nodes of different behaviors based on whether the semantic node relationship type has attached conditions. A selection node is added above the action node. When a task has decoration information, a decoration node is constructed and added with decoration attributes. A selection node is added to the decoration node, and conditional nodes are placed below the selection node in sequence. Sequential nodes with conditions are also expanded according to this pattern. Both sequential and conditional nodes must be processed according to this pattern until a sub-behavior tree is constructed. During construction, the execution order of each sub-behavior in the task decision behavior tree is adjusted based on the execution order of the sub-action nodes. During the process of building the robot behavior tree, if there is a failure to obtain feature data from the scene, the node type is out of range, or an empty node is generated when generating sequential nodes and selection nodes, it will be recorded as a behavior tree construction failure.

[0055] In this example, the process of constructing a behavior tree based on a semantic network is as follows: according to the task trigger data, robot data is found in the robot database, and an object entity is found in the object or dynamic scene library as the action object. Construction fails when the object entity does not exist in the scene; the task is classified into a specific action by a classifier, and this action corresponds to the label list features of the action features in the action knowledge base, that is, the action features in the action node M; the parameter list vars of the action object is filled. The specific parameters are obtained from the obtained scene information and the dest information in the additional supplementary information, and the language content is obtained from the corpus of the corresponding target in the knowledge base; the corresponding behavior is constructed according to the action features, and the use of movement, behavior or language dialogue is selected according to the behavior characteristics. The completed behavior is recorded as an action node in the behavior tree.

[0056] For already constructed action nodes, the subtask is centered in the semantic network, and the corresponding subgraph is found. This subgraph represents all next-order nodes containing the action node. Based on whether the edges in the semantic network description contain conditional information, decoration information, or prop variables, they are mapped into three types of behavior tree logic nodes: selection nodes, sequence nodes, and parallel nodes. (Semantic network relationships are selection nodes when they have conditional information; otherwise, they are sequence nodes. Nodes with the same sequence number within a sequence node are collectively classified as parallel nodes.) For leaf nodes, a sequence node is constructed. The first node under the sequence node is the corresponding conditional node, followed by the corresponding action nodes. Nodes with the same sequence number are designated as parallel nodes. For all {Behavior}sel selection behavior nodes are constructed. During construction, the conditional judgment cond(env) is performed to obtain a real-time Boolean conditional judgment result. For all {Behavior}seq sequence behavior nodes are constructed. During construction, they are sorted, and the order of the behavior tree subnodes is swapped from largest to smallest according to the seq sorting method.

[0057] When injecting parameters into a decision behavior tree, the following method is used for any node: the subject always uses the subject s from the task understanding. Based on the subject s, the environment and character target are queried, and the location information, the target object ID (i.e., the action category motion), and the description information are injected into the target parameter list of the action node. The action ID is injected into the behavior as content, and the predicate v from the task understanding information is used as the behavior tree name. For behaviors for which parameters cannot be determined, the knowledge base is searched to find actions with similar features for processing. For actions for which the object parameters cannot be determined at all, the behavior cannot be constructed, and the entire behavior tree construction is deemed to have failed.

[0058] In the embodiment, a task decision behavior tree verification process is also performed, and the verification conditions are:

[0059] The sub-behaviors under each behavior must all correspond to sequential nodes or selection nodes. Parallel nodes can coexist with sequential nodes. Selection nodes must contain conditional nodes and the conditional node judgment Boolean parameters contained in the conditional nodes. Any behavior must contain an action identification parameter to complete the action.

[0060] The variable storage module 150 is used to store temporary variables when constructing the task decision behavior tree. The temporary variables include scene information read during the dynamic perception process, question-answering results from multiple rounds of dialogue using a large language model, and knowledge read from the knowledge base.

[0061] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A system for controlling robot task decisions based on semantic networks and knowledge bases, It is characterized in that It includes semantic network module, large language model module, knowledge base module, task decision module and variable storage module; The semantic network module is used to provide a semantic network for constructing a task decision behavior tree, and the semantic network is supplemented and expanded by the question-answering results and knowledge base of the large language model; The large language model module is used to use the large language model to perform knowledge question answering and output question answering results; The knowledge base module is used to provide knowledge for constructing a task decision behavior tree; The task decision module is used to construct a task decision behavior tree based on the semantic network module, the large language model module, and the knowledge base module, specifically by querying the semantic network to construct nodes of the decision tree, filling in node parameters through the question and answer results of the large language model, and filling in node parameters through the knowledge base; The variable storage module is used to store temporary variables when constructing a task decision behavior tree.

2. The system for controlling robot task decision based on semantic network and knowledge base according to claim 1, It is characterized in that The semantic network is a directed connected graph structure, which sequentially connects associated semantic nodes, uses in-and-out degrees and directed connections to represent the weights and possibilities between semantic nodes, and each semantic node includes an identifier and a parameter list, wherein the directed connection relationship includes a relationship type and a relationship attribute, and the relationship attributes include routing possibilities, execution order, and condition descriptions. The semantic nodes include semantic nodes with single semantics, semantic nodes with attached decorative information, semantic nodes with attached conditions, and semantic nodes that allow targets to be sets, so as to adapt to the construction of task decision behavior trees.

3. The system for controlling robot task decision based on semantic network and knowledge base according to claim 2, It is characterized in that Directed links include two different states: feasibility connection and necessity connection. When the possibility connection state appears, the routing possibility prop indicates the selection strength of the action node.

4. The system for controlling robot task decision based on semantic network and knowledge base according to claim 1, It is characterized in that The large language model pre-stores knowledge and can perform question-and-answer search based on input questions and output question-and-answer results when expanding the semantic network and constructing the task decision behavior tree. At the same time, it can search based on the task decomposition problem and provide multiple possible results for task planning. The question-and-answer results and multiple possible results are connected to the semantic network as expanded knowledge, or added to the nodes of the task decision behavior tree as supplementary knowledge.

5. The system for controlling robot task decision based on semantic network and knowledge base according to claim 1, It is characterized in that The knowledge base is refined knowledge, including an action knowledge base and an object knowledge base, wherein the action knowledge base includes actions, action types to which actions belong, action features, and action objects; the object knowledge base includes objects and object attributes.

6. The system for controlling robot task decision based on semantic network and knowledge base according to claim 1, It is characterized in that The construction of the task decision behavior tree includes: Determine the predicate and target object corresponding to the task based on the task understanding command and task classification results; According to the target object, an object instance corresponding to the target object is searched in the object knowledge base as the target object instance of the action; Classify the predicates into action categories in the action knowledge base, and construct a set of behaviors based on a set of task semantics corresponding to the action categories; Search the semantic node set corresponding to the behavior set in the semantic network, then build a task decision behavior tree according to the semantic node set, and add node parameters to the action nodes in the task decision behavior tree, where the node parameters come from the target object instances searched from the object knowledge base, the dynamically perceived scene instances, and the question-answering results obtained through multiple rounds of dialogue based on the large language model. One less.

7. The system for controlling robot task decision based on semantic network and knowledge base according to claim 6, It is characterized in that The step of searching for a semantic node set corresponding to the behavior set in the semantic network and then constructing a task decision behavior tree according to the semantic node set includes: The semantic node set corresponding to the behavior includes a single semantic node, a semantic node with decorative information, a semantic node with conditions, and a semantic node with a set as the target; When constructing a decision behavior tree according to the topological structure corresponding to the semantic node set, when the semantic node is a single semantic node, an action node is generated and executed sequentially; when the semantic node is a semantic node with attached conditions, a branch is generated for the semantic node with attached conditions, specifically a selection node is generated, and a condition node and an action node of the branch are connected under the selection node; when the semantic node is a semantic node with attached decoration information, a decoration node is generated for the semantic node with attached decoration information, and a sequence node and an action node are generated under the decoration node; when the semantic node is a semantic node that allows the target to be a set, multiple decoration nodes are generated for the semantic node whose target is a set, and each decoration node is expanded and generated in the above manner; The action nodes, sequence node selection nodes, condition nodes, and decoration nodes generated according to the topological structure corresponding to the semantic node set constitute the task decision behavior tree.

8. The system for controlling robot task decision based on semantic network and knowledge base according to claim 7, It is characterized in that During the specific construction, when the semantic node is attached with conditions, the rule knowledge is queried, the selection node is constructed based on the rule knowledge, and the conditional node is generated under the selection node based on the rule knowledge.

9. The system for controlling robot task decision based on semantic network and knowledge base according to claim 7, It is characterized in that When constructing a task decision behavior tree, the execution order of each sub-behavior in the task decision behavior tree is swapped according to the execution order of the sub-action nodes.

10. The system for controlling robot task decision based on semantic network and knowledge base according to claim 7, It is characterized in that It also includes the task decision behavior tree verification process, and the verification conditions are: The sub-behaviors under each behavior must correspond to all sequential nodes or selection nodes. Parallel nodes can coexist with sequential nodes. Selection nodes must contain conditional nodes and the conditional node judgment Boolean parameters contained in the conditional nodes. Any behavior must contain an action identification parameter to complete the action.

Citation Information

Patent Citations

  • System and method for constructing robot behavior tree based on knowledge graph

    CN114153943A

  • Automatic mechanical arm assembling method based on multiple knowledge bases

    CN114310883A

  • Method and device for training generative large language model based on knowledge base feedback

    CN117009490A

  • Artificial intelligence server for setting language of robot and method for the same

    US20190392820A1

  • Semantic Robotic System

    US20200298401A1

Cited By

  • Man-machine cooperation method and system based on robot process automation

    CN120258747A

  • Assembly mechanical arm task planning method and device

    CN120307305A

  • Building robot task planning and skill learning method and system

    CN120363218A

  • Robot low-code editable task sequence generation system based on large model

    CN120370756A

  • A low-code editable task sequence generation system for robots based on large models

    CN120370756B