A method and apparatus for reasoning task in dynamic scenario based on knowledge base

Abstract

The application discloses a kind of method and device for reasoning task under dynamic scene based on knowledge base, comprising: on the basis of semantic network expressing knowledge base is divided into action relationship subnetwork, state change relationship subnetwork, subordinate relationship subnetwork and preposition relationship subnetwork according to relationship type, the reasoning of the behavior tree missing judgment of task is carried out according to action relationship subnetwork, the missing data such as target object and state-related perception data is obtained by reasoning through all subnetworks, and then the reasoning of behavior tree is realized.The mode can avoid using task knowledge or structured robot task design language, complete the reasoning planning of task using general semantic network, realize the operation problem of robot, and the reasoning process does not need too much artificial editing and intervention.As a technical user, you only need to provide complete semantic network and real-time perception data, and you can complete the autonomous control of robot behavior.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent robot task behavior planning and knowledge base combination, and particularly relates to a method and device for reasoning tasks in a dynamic scene based on a knowledge base. BACKGROUND

[0002] Robots are physical agents that complete tasks by manipulating the physical world, and have the ability to interact with the physical world similar to humans, or even surpass humans in some aspects. However, like an individual alone, a single robot system has been difficult to meet the increasingly complex task requirements of modern society due to its own limitations in information processing, decision-making, and task execution capabilities.

[0003] Robots need task planning to sequence actions to achieve goals that cannot be achieved by a single action. Task planning refers to a strategy for sequencing actions to ensure efficient and complete completion of tasks under the constraints. Task planning is more inclined to high-level decision-making rather than process implementation. For simple tasks, it is generally not necessary to strictly distinguish the process of task planning. For complex tasks, robots need task planning algorithms to sequence actions to achieve goals that cannot be achieved by a single action.

[0004] Off-the-shelf task planners can be used by intelligent robots to solve various planning problems. However, there are many different planners, each with different strengths and weaknesses, and there is no general rule to determine which planner is best suited for a given problem.

[0005] Existing task planning includes state machine task decomposition, task decomposition using pre-made scripts, and task planning using behavior tree control. Among them, a state machine represents a finite number of states and the transitions and actions between these states. The state stores information about the past, that is, it reflects the changes in inputs from the start of the system to the present moment. Transitions represent state changes, described by conditions that cause transitions to occur. Actions are descriptions of activities to be performed at a given time. The advantage of state machine task decomposition is easy to understand and can quickly implement basic functions. However, it has two disadvantages. First, it lacks flexibility and cannot implement tasks with uncertain processes. Second, for complex tasks, the workload is huge when there are many combinations, and all processes need to be exhausted. Third, it is not easy to maintain. If there is a change in the execution process, the content to be modified can be very large.

[0006] A behavior tree is a hierarchical node tree that controls the decision-making process of an entity. Within the scope of the tree, the leaves are actual commands to control the entity, and the branches are various types of utility nodes that control the direction along the tree to achieve the most suitable command sequence for the situation.

[0007] At present, the behavior control of robots mainly depends on rule constraints, and when the robots do not receive instructions, the robots cannot generate behavior instructions spontaneously, or the robots execute some repetitive tasks according to the pre-prepared automatic instructions of the robot designers, and in the case of only ontology knowledge graph, the relationship between ontology nodes cannot be obtained, and the specific relationship and parameters of unknown state task planning cannot be obtained. SUMMARY

[0008] In view of the above, the purpose of the present application is to provide a method and device for reasoning tasks in a dynamic scene based on a knowledge base, to realize task planning and obtain missing data in a dynamic scene of a robot.

[0009] To achieve the above-mentioned purpose of the application, an embodiment provides a method for reasoning tasks in a dynamic scene based on a knowledge base, comprising the following steps:

[0010] Obtain a semantic network expressing the knowledge base, and divide the semantic network into an action relationship sub-network, a state change relationship sub-network, a subordinate relationship sub-network and a preposition relationship sub-network according to the relationship types;

[0011] When the behavior tree corresponding to the target task cannot be searched in the task planning graph, at least one of the following is performed: target object reasoning, task reasoning and state and perception related reasoning in the semantic network;

[0012] For target object reasoning, determine the action set of the target task, and reason the target object in the action relationship sub-network, the subordinate relationship sub-network and the preposition relationship sub-network according to the actions in the action set;

[0013] For task reasoning, search the path in the action relationship sub-network according to the two endpoints of the missing segment of the behavior tree, determine the missing segment according to the search result, and realize task reasoning;

[0014] For state and perception related reasoning, determine the ontology of the state by searching in the state change relationship sub-network and the subordinate relationship sub-network, trigger the perception device to collect the perception data related to the ontology according to the ontology, and perform reasoning according to the perception data.

[0015] Preferably, after the action relationship sub-network is divided, a mapping from tasks to action sets is constructed, comprising:

[0016] Mark the actions related to a single task in the action relationship sub-network, and mark the actions related to a combined task in the action relationship sub-network, then perform task semantic clustering on the marked actions, and each clustering cluster contains all actions corresponding to a task semantic, so that a mapping from each task to an action set is obtained, and then a mapping set is obtained.

[0017] Preferably, the action of determining the action set of the target task comprises searching for a task matching the target task in the mapping set and taking the action set of the task mapping as the action set of the target task.

[0018] Preferably, the action of reasoning the target object according to the action in the action set in the action relationship sub-network and the subordinate relationship sub-network comprises:

[0019] The object operated by the action in the action relationship sub-network is determined according to each action, and the multi-level expansion of the object is performed in the subordinate relationship sub-network to obtain new target objects, and the new target objects are filtered according to the perception situation.

[0020] Preferably, the action of performing the multi-level expansion of the object in the subordinate relationship sub-network comprises:

[0021] The sub-class object having the subordinate relationship with the object in the subordinate relationship sub-network is searched as the new target object; the parent class object having the subordinate relationship with the object in the subordinate relationship sub-network is searched, and other sub-class objects of the parent class object are taken as the new target object.

[0022] Preferably, the action of filtering the new target object according to the perception situation comprises: after searching the new target object through the parent class object of the object, the new target object that cannot appear in the scene is deleted according to the perceived object instance in the scene.

[0023] Preferably, the action of reasoning the target object according to the action in the action set in the action relationship sub-network, the subordinate relationship sub-network and the preposition relationship sub-network comprises:

[0024] When the target object set cannot be reasoned in the subordinate relationship sub-network, the target object is reasoned in the preposition relationship sub-network according to the preposition, comprising: at least 3-hop search is performed in the preposition relationship sub-network according to the preposition relationship based on the object operated by the action, and each object searched in each hop is taken as a new target object.

[0025] Preferably, the action of searching the path in the action relationship sub-network according to the two endpoints of the missing segment of the behavior tree and determining the missing segment according to the search result comprises:

[0026] The behavior tree containing the missing segment is mapped into a sub-semantic network, wherein the behavior of the behavior tree is mapped into a relationship in the sub-semantic network, and the action target of the behavior is mapped into a node of the sub-semantic network.

[0027] The action of finding the two end points of the missing fragment in the sub-semantic network, finding the two objects corresponding to the two end point actions in the action relationship sub-network according to the two end point actions, and searching for multiple connected paths between the two objects in the action relationship sub-network, each connected path representing a series of action plans between the two objects;

[0028] According to the matching rule, it is judged whether each connected path can match the task planning graph, wherein the matching rule is that all edge relationships in the connected path must be classified into a sub-task or an action plan; the nodes in the connected path must exist in the scene or be generalized to the instances in the scene through the subordination relationship sub-network;

[0029] The shortest connected path that meets the matching rule is screened, the edges in the shortest connected path are converted into behaviors, and the nodes in the shortest connected path are converted into real objects in the scene to obtain a missing fragment of the behavior tree,

[0030] The missing fragment of the behavior tree is supplemented into the behavior tree to complete the task reasoning of the behavior tree.

[0031] Preferably, the ontology of the state is determined by searching in the state change relationship sub-network and the subordination relationship sub-network, and the perception device related to the ontology is triggered to collect perception data related to the ontology according to the ontology, comprising:

[0032] On the basis of the target object and the corresponding unknown sensing parameter label, the state related to the target object is searched in the state change relationship sub-network, and the ontology of the state is searched in the subordination relationship sub-network, and the perception device corresponding to the ontology is linked according to the parameter description of the ontology, and the instantiation data of the perception device to the ontology parameter is obtained, and the label of the instantiation data is the unknown sensing parameter label, so that the behavior parameter is filled by querying the instantiation data under the label, and the reasoning of the missing information is completed.

[0033] Preferably, the method further comprises: searching for a behavior tree corresponding to a target task in a task planning graph, specifically comprising: classifying the target task into a task node in the task planning graph, and searching all connected sub-task nodes in turn from the task node, and grouping all the obtained sub-task nodes and edges to form a task tree, and determining the action set of the sub-task according to the mapping from the task to the action set, and constructing the behavior tree according to the action set of the sub-task.

[0034] Preferably, the method further comprises: verifying the behavior tree obtained by reasoning, and the verification rule comprises: verifying whether the logic of the behavior tree is correct according to the international specification of the behavior tree; and judging whether the task requirements can be met, i.e. whether the object instances exist in the scene and the robot can perform the actions described in the behavior tree to complete the task.

[0035] Preferably, the method further comprises: converting the behavior tree with the verification success into a data structure and transmitting the data structure to a behavior execution hub of the robot for driving the robot by using an execution logic and behavior parameters of the behavior tree.

[0036] To achieve the above object, the embodiment further provides a device for reasoning task in dynamic scene based on knowledge base, comprising a division unit, a searching unit and a reasoning unit.

[0037] The division unit is configured to acquire a semantic network representing the knowledge base, and divide the semantic network into an action relationship sub-network, a state change relationship sub-network, a subordinate relationship sub-network and a preposition relationship sub-network according to relationship types.

[0038] The searching unit is configured to search a behavior tree for a target task in a task planning graph.

[0039] The reasoning unit is configured to perform at least one of target object reasoning, task reasoning and state and perception related reasoning in the semantic network when the behavior tree corresponding to the target task cannot be searched in the task planning graph.

[0040] Preferably, the reasoning unit comprises a target object reasoning module, a task reasoning module and a state and perception related reasoning module.

[0041] The target object reasoning module is configured to determine an action set of the target task, and reason a target object in the action relationship sub-network, the subordinate relationship sub-network and the preposition relationship sub-network according to actions in the action set.

[0042] The task reasoning module is configured to search a path in the semantic network according to two end points of a missing segment of the behavior tree, determine the missing segment according to a search result, and implement task reasoning.

[0043] The state and perception related reasoning module is configured to determine an ontology of a state by searching in the state change relationship sub-network and the subordinate relationship sub-network, trigger a perception device to collect perception data related to the ontology according to the ontology, and perform reasoning according to the perception data.

[0044] Compared with the prior art, the present application has at least the following beneficial effects:

[0045] On the basis of dividing the semantic network expressing the knowledge base into an action relation sub-network, a state change relation sub-network, a subordinate relation sub-network and a preposition relation sub-network according to the relation types, the behavior tree missing judgment reasoning of the task is performed according to the action relation sub-network, the missing data of the target object and the state-related perception data are obtained through reasoning of all the sub-networks, and then the reasoning of the behavior tree is realized. In this way, the task knowledge or the structured robot task design language can be avoided, the general semantic network is used to complete the reasoning planning of the task, the operation problem of the robot is realized, the reasoning process does not need too much human editing and intervention, and as a technical user, the autonomous control of the robot behavior can be completed by providing the complete semantic network and the real-time perception data. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0047] Figure 1 is a flowchart of the method for reasoning a task in a dynamic scene based on a knowledge base provided by the embodiments;

[0048] Figure 2 is a structural schematic diagram of a robot decision system provided by the embodiments;

[0049] Figure 3 is an exemplary task tree provided by the embodiments;

[0050] Figure 4 is a schematic diagram of a task that cannot be decomposed in a behavior tree provided by the embodiments;

[0051] Figure 5 is another flowchart of the method for reasoning a task in a dynamic scene based on a knowledge base provided by the embodiments;

[0052] Figure 6 is a structural schematic diagram of an apparatus for reasoning a task in a dynamic scene based on a knowledge base provided by the embodiments;

[0053] Figure 7 is a structural schematic diagram of a reasoning unit provided by the embodiments. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer, the following will further describe the present application in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the protection scope of the present application.

[0055] At present, when a robot uses a behavior tree to control a task planning mode, the following problems are encountered: (1) task decomposition exceeds the pre-decomposition range, resulting in task planning failure; (2) target loss or accurate target does not exist in the task planning process. To solve this problem, the embodiment provides a method and device for reasoning tasks in a dynamic scene based on a knowledge base, the core idea of which is to improve the task decomposition, decision-making and control of the robot through the semantic network expressing the knowledge base, and to enhance the work capacity of the robot in an unknown environment.

[0056] As an expression form of the knowledge base, the semantic network provides a knowledge set of subject-predicate-object-adverb combinations, and also has a dedicated computing and query engine to provide knowledge. The semantic network as a whole presents a graph structure composed of nodes and edges.

[0057] The preconditions for implementing the method for reasoning tasks in a dynamic scene based on a knowledge base include: having a complete semantic network of the robot field, the home field and the environment field, converting the environment in which the robot is located into a scene graph, and storing it in the form of a knowledge graph. The behavior tree planning strategy uses a pre-prepared task planning graph to decompose behaviors, queries specific information in the scene on the execution node to complete the generation of the behavior tree, and finally realizes the behavior tree.

[0058] Based on the above preconditions, the method for reasoning tasks in a dynamic scene based on a knowledge base provided by the embodiment returns to the semantic network to reason feasible solutions when the behavior tree planning process encounters tasks that cannot be decomposed, and obtains the latest results through comprehensive sorting; a complete behavior tree and task planning scheme are completed in combination with the knowledge base, as shown in Figure 1 , specifically including the following steps:

[0059] S1, obtain a semantic network expressing a knowledge base, and divide the semantic network into an action relationship sub-network, a state change relationship sub-network, a subordinate relationship sub-network and a preposition relationship sub-network according to the relationship types.

[0060] In the embodiment, the knowledge base provides attribute information in the robot scene, including the ontological characteristics of the robot, the environmental motion information of the robot, the action control knowledge of the robot, and the behavior information of the robot. Such a knowledge base is converted into a semantic network Graph, and the subject-predicate, predicate and state descriptions in the semantic network Graph are explicitly described.

[0061] In order to accelerate the construction of the reasoning and behavior tree, it is necessary to divide the semantic network Graph into sub-networks. The semantic network Graph is point-edge relationship data for language expression, in which the edge relationship can be described as an action relationship v expressing an action, an affiliation relationship o expressing affiliation, a preposition relationship p expressing a preposition, and a state change relationship s expressing a state. Therefore, according to these relationships, the semantic network Graph is divided into an action relationship sub-network Graph-v representing the action relationship, a state change relationship sub-network Graph-s representing the state change relationship, an affiliation relationship sub-network Graph-o representing the affiliation relationship of things, and a preposition relationship sub-network Graph-p representing the preposition.

[0062] In the embodiment, after the action relationship sub-network is divided, the action relationship sub-network is processed to construct the mapping of the task to the action set, including: marking the actions associated with a single task in the action relationship sub-network, marking the actions associated with a combined task in the action relationship sub-network, and then performing task semantic clustering on the marked actions. Each clustering cluster contains all actions corresponding to a task semantic, so that the mapping of each task to the action set is obtained, and then a mapping set is obtained.

[0063] Any arbitrary verb has a probability of being associated with an action primitive of the robot, for example, the verb "deliver" is associated with the action primitive "send" executable by the robot, wherein the action primitive refers to an action that cannot be further divided for robot task planning, and a single verb primitive can be considered as a single task. Other verbs can also be associated with a combined task of the action primitive of the robot. In the embodiment, the above two types (associated with an action primitive and associated with a task combination) of verbs v are marked in Graph-v. Because each association relationship is not unique, a semantic clustering algorithm is used to classify the verbs into an action primitive, and the verbs corresponding to an action primitive are mapped to a semantic action set, for example, task->{Verbs}, wherein verbs represent all action sets that can be mapped to the task task.

[0064] In the embodiment, based on the above divided sub-networks, the sub-networks are added to the robot decision system, and the robot decision system with the sub-networks is used to implement task reasoning in a dynamic scenario. Figure 2As shown, it includes a plurality of perception sensors, a data backplane, a task construction algorithm, a semantic network, Graph-v, Graph-s, Graph-p, Graph-o, and a task planning graph. The perception sensors are used to collect perception instance data in the robot scene, the data backplane is used to convert the perception instance data into an execution parameter table of the behavior tree, and record the data changes of the behavior tree execution. The task construction algorithm includes a method of constructing a behavior tree through graph inference. The task planning graph is a huge task planning network, and a subtask is searched through a specific task to complete the construction of the behavior tree.

[0065] In the above system, based on the target task T and the action object O, the behavior action of the robot in the dynamic scene is planned, the semantic network is used as the input of inference, and the inference planning of a simple task (one task and one target) is completed.

[0066] S2, search the behavior tree corresponding to the target task in the task planning graph.

[0067] In the embodiment, when the inference planning of the target task is performed, the behavior tree corresponding to the target task is searched in the task planning graph, including: classifying the target task into a task node in the task planning graph, and searching all connected subtask nodes in turn from the task node, and all subtask nodes and edges obtained are combined to form a task tree Task-Tree, as shown in the following table. Figure 3 As shown, if it cannot be classified into any task node in the task planning graph, or any subtask node cannot be searched, it is considered to be failed, and the task inference in the semantic network needs to be performed.

[0068] The embodiment also determines whether all subtasks in the task tree Task-Tree can be implemented by the robot, specifically, by comparing the subtasks with the previously planned tasks that can be implemented by the robot to determine. After determining the subtasks that can be actually executed, the action set of the subtask is determined according to the mapping from the task to the action set, and the behavior tree is constructed according to the action set of the subtask, and the nodes in the behavior tree represent actions.

[0069] S3, when the behavior tree corresponding to the target task cannot be searched in the task planning graph, at least one of the target object inference and the task inference in the semantic network and the state and perception related inference is performed.

[0070] In the embodiment, when the behavior tree corresponding to the target task cannot be searched in the task planning graph, the semantic network is used for calculation and reasoning, so as to attempt to solve the problem from the task tree to the behavior tree. In the specific implementation, the search is jumped in the four sub-networks of graph-s, graph-o, graph-p and graph-v from the two angles of breadth and depth, to find the relationship between actions and actions, tasks and tasks, tasks and target objects, and objects and objects, and at least one of the target object reasoning and the task reasoning and the state and perception related reasoning is implemented.

[0071] For the case that the target object cannot be determined when the behavior tree of the target task is constructed, the target object reasoning is used to determine the target object. For the target object reasoning, the action set of the target task is determined, and the target object is reasoned in the action relationship sub-network, the subordinate relationship sub-network and the preposition relationship sub-network according to the action in the action set.

[0072] In the embodiment, first, the action set of the target task is searched in the mapping set according to the mapping set, and the action set of the task is mapped as the action set of the target task. Then, the target object is reasoned in the action relationship sub-network and the subordinate relationship sub-network according to the action in the action set, including: determining the object operated by the action in the action relationship sub-network according to each action, and performing multi-level expansion of the object in the subordinate relationship sub-network to obtain a new target object, and performing screening and filtering operation on the new target object according to the perception condition.

[0073] In the embodiment, the multi-level expansion of the object in the subordinate relationship sub-network includes: searching for a subclass object having a subordinate relationship with the object in the subordinate relationship sub-network as a new target object. In one implementation, starting from graph-v in parallel, the operation target of each action v is determined as a noun object O in graph-v, and O is entered into graph-o for generalization, that is, O is generalized in graph-o and all other objects having a subordinate relationship with O are taken as new target objects, to obtain a set {O} composed of the new target objects, and the set {O} is returned to graph-v, so that the expansion of the (v, O) combination becomes (v, {O}), and the search return based on graph-v is expanded. Each time graph-v expands the target object through graph-o, the reference count is increased by 1, and then the reference count is used for sorting to screen the target objects.

[0074] In an embodiment, the multi-level expansion of the object in the subordination sub-network further comprises: searching for parent class objects having a subordination relationship with the object in the subordination sub-network, and taking other subclass objects of the parent class objects as new target objects. In an embodiment, for the parallel search of the multiple target objects, starting from a target object O, searching for all parent class objects of the target object O in the graph-o, and then taking the subclass objects of the parent class objects as new target objects to construct a new set {O}, at this time, the scene target is equivalent to being searched and expanded through the graph-o. The manner of determining the new target objects based on the parent class objects can be repeatedly used to gradually expand the range of the set {O}.

[0075] In an embodiment, after the parallel search of the multiple target objects and the new target objects, the new target objects that cannot appear in the scene are deleted according to the perceived object instances in the scene. Specifically, after the new target objects are determined based on the parent class objects each time, the instances in the scene perception need to be checked for the new target objects, and the objects that cannot exist in the scene are timely removed from the set {O}; each time the new target objects are determined based on the parent class objects, the reference count is increased by one.

[0076] When the target object set {O} cannot be inferred in the graph-o, the preposition state of the graph-p is used for inference. The preposition relationship p describes the position dependence relationship of the space, and only the “contains”, “above and below”, “left and right”, and “contains” are inferred when the object position is inferred. In the graph-p, the inference of the target object based on the preposition comprises: taking the object of the action operation as the basis, and performing at least 3-hop search in the graph-p based on the preposition relationship, and each object searched in each hop is taken as a new target object. In an embodiment, the next hop node {O-p} corresponding to the preposition relationship P in the graph-p can represent that the objects are close or dependent in space, and similar objects existing in the scene are searched in {O-p}; when the similar objects do not exist, taking each element O in {O-p} as a target, repeating the process and limiting the search hop length to be less than 3.

[0077] In this way, the replacement objects and similar objects in the task planning can be searched through the graph-v, the graph-o, and the graph-p to assist the task planning decision to further calculate. In an embodiment, the new target objects determined in the graph-o are sorted in ascending order according to the reference count, so as to ensure that the final result is as few as possible. The search result avoids too large differences between the target and the expected result.

[0078] When the task decomposition cannot be completely performed in the task planning graph, the task inference needs to be performed, and the missing part needs to be completed in the semantic network to find the subgraph and the composite behavior tree planning. For example,Figure 4 As shown, root is the root node, b1-b6 are behaviors, and the question mark is a missing part.

[0079] In an embodiment, task reasoning includes searching for a path in the action relationship subnetwork according to the two endpoints of the missing segment of the behavior tree, determining the missing segment according to the search result, and implementing task reasoning. Specifically, it includes:

[0080] Because the behavior tree itself is a directed acyclic graph, the behavior tree containing the missing segment is mapped to a sub-semantic network, wherein the behaviors of the behavior tree are mapped to the relationships in the sub-semantic network, and the action targets of the behaviors are mapped to the nodes of the sub-semantic network; the actions of the two endpoints of the missing segment in the sub-semantic network are found, the two objects O1 and O2 corresponding to the two endpoints of the actions in the graph-v are found according to the two endpoints of the actions, and a plurality of connected paths between the two objects in the graph-v search are found to form a path set {path}, each connected path path(O1->R-O2) represents a series of action plans R between the two objects; whether each connected path can be matched to the task planning graph is determined according to the matching rule, and when determining, the shortest path in {path} is started, and the matching rule is: all edge relationships in the connected path must be able to be classified into a sub-task or an action plan; the nodes in the connected path must exist in the scene or be generalized to the instances in the scene in the graph-o; the shortest connected path that satisfies the matching rule is screened, the edges in the shortest connected path are converted into behaviors, and the nodes in the shortest connected path are converted into real objects in the scene, which is reversely mapped into a behavior tree segment, i.e., the missing segment of the behavior tree; the missing segment of the behavior tree is supplemented to the part that cannot be decomposed in the behavior tree, and the task reasoning of the behavior tree is completed.

[0081] In the task planning process, when the task action target needs a precondition, and the condition is an environmental perception parameter, the parameter cannot be found in the current perception data, and state and perception related reasoning is needed. In an embodiment, the state and perception related reasoning includes: determining the ontology of the state by searching in the state change relationship subnetwork and the dependent relationship subnetwork, triggering the perception device to collect the perception data related to the ontology according to the ontology, and performing reasoning according to the perception data.

[0082] Based on the target object O and the corresponding unknown sensing parameter label, the content related to the attribute is searched in the semantic network graph-s, the attribute expression is searched, and then the mapping relationship between the attribute table and the scene is established. If the current state cannot complete the state constraint, a feedback is triggered to search the ontology of the knowledge graph, and the perception that can meet the current behavior planning set is found. In an embodiment, after searching the state s related to the target object O in the graph-s, the ontology of the state s is searched in the graph-o. If the ontology exists, the corresponding sensor device is linked according to the parameter description of the ontology, and the instantiation data of the sensor device to the ontology parameter is obtained. The label of the instantiation data is the unknown sensing parameter label label. In this way, the behavior parameter is filled by querying the instantiation data under the label, and the missing information is inferred.

[0083] As shown in Figure 5 Based on the above S1-S3, the method for reasoning task in dynamic scene based on knowledge base provided by the embodiment further includes the following steps:

[0084] S4, the behavior tree obtained by reasoning is verified.

[0085] In the embodiment, the verification rule for verifying the behavior tree obtained by reasoning includes: (1) according to the international specification of the behavior tree, verifying whether the logic of the behavior tree is correct; (2) judging whether it can meet the task requirement, that is, the object instance really exists in the scene, the robot can execute the action described in the behavior tree, and the task is completed.

[0086] S5, the robot is driven according to the behavior tree verified successfully.

[0087] In the embodiment, the behavior tree verified successfully is converted into a data structure such as XML or JSON, and then transmitted to the behavior execution center of the robot, and the robot is driven by using the execution logic and behavior parameters of the behavior tree.

[0088] Based on the same inventive concept, as shown in Figure 6 The embodiment further provides an apparatus 600 for reasoning task in dynamic scene based on knowledge base, which includes a division unit 610, a searching unit 620 and a reasoning unit 630. The division unit 610 is used for obtaining a semantic network expressing a knowledge base, and dividing the semantic network into an action relationship sub-network, a state change relationship sub-network, a subordinate relationship sub-network and a preposition relationship sub-network according to relationship types. The searching unit 620 is used for searching a behavior tree for a target task in a task planning graph. The reasoning unit 630 is used for performing at least one of target object reasoning and task reasoning in the semantic network and state and perception related reasoning when the behavior tree corresponding to the target task cannot be searched in the task planning graph.

[0089] As Figure 7 shown, the reasoning unit 630 includes a target object reasoning module 710, a task reasoning module 720, and a state and perception related reasoning module 730; the target object reasoning module 710 is configured to determine a set of actions of a target task, and reason about target objects in the action relationship sub-network, the subordinate relationship sub-network, and the preposition relationship sub-network according to the actions in the set of actions; the task reasoning module 720 is configured to search a path in the semantic network according to two endpoints of a missing segment of a behavior tree, determine the missing segment according to a search result, and implement task reasoning; and the state and perception related reasoning module 730 is configured to determine an ontology of a state by searching in a state change relationship sub-network and a subordinate relationship sub-network, trigger a perception device to collect perception data related to the ontology according to the ontology, and perform reasoning according to the perception data.

[0090] It should be noted that the device for reasoning about a task in a dynamic scenario based on a knowledge base provided in the above embodiments should be illustrated by the division of the above functional units and modules, and the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of a terminal or a server is divided into different functional units and modules to complete all or part of the above described functions. In addition, the device for reasoning about a task in a dynamic scenario based on a knowledge base provided in the above embodiments and the method for reasoning about a task in a dynamic scenario based on a knowledge base belong to the same concept, and the specific implementation process is detailed in the method for reasoning about a task in a dynamic scenario based on a knowledge base, which will not be described here.

[0091] The method and device for reasoning about a task in a dynamic scenario based on a knowledge base provided in the embodiments use a knowledge base of a semantic network structure, convert an information missing problem in a robot autonomous behavior construction process into a semantic retrieval problem, and perform a given task planning of a robot according to a perception data backplane and a main task tree of the robot. The robot operation problem can be implemented without using task knowledge or a structured robot task design language, and a general semantic network is used to complete the reasoning planning of a task, and the reasoning process does not need too much human editing and intervention. As a technical user, only a complete semantic network and real-time perception data need to be provided, and the autonomous control of the robot behavior can be completed.

[0092] The specific embodiments described above have described the technical solutions and advantages of the present application in detail. It should be understood that the above description is only the most preferred embodiment of the present application, and is not used to limit the present application. Any modification, supplement, and equivalent replacement made within the principle range of the present application should be included in the protection range of the present application.

Claims

1. A method for reasoning task in dynamic scenario based on knowledge base, characterized in that, The method comprises the following steps: obtaining a semantic network expressing a knowledge base, and dividing the semantic network into an action relationship sub-network, a state change relationship sub-network, a subordinate relationship sub-network and a preposition relationship sub-network according to relationship types; when a behavior tree corresponding to a target task cannot be searched in a task planning graph, performing at least one of target object reasoning, task reasoning and state and perception related reasoning under the semantic network; for the target object reasoning, determining an action set of the target task, and reasoning the target object according to actions in the action set in the action relationship sub-network, the subordinate relationship sub-network and the preposition relationship sub-network; for the task reasoning, performing path searching according to two endpoints of a missing segment of the behavior tree in the action relationship sub-network, determining the missing segment according to a searching result, and implementing the task reasoning, including: mapping the behavior tree containing the missing segment into a sub-semantic network, wherein actions of the behavior tree are mapped into relationships in the sub-semantic network, and action targets of the actions are mapped into nodes of the sub-semantic network; finding two objects corresponding to actions of the two endpoints in the action relationship sub-network according to the two actions of the two endpoints, and searching a plurality of connected paths between the two objects in the action relationship sub-network, wherein each connected path represents a series of action planning between the two objects; judging whether each connected path can be matched to the task planning graph according to a matching rule, wherein the matching rule is that all edge relationships in the connected path must be classified into a sub-task or action planning; nodes in the connected path must exist in a scene or be generalized to instances in the scene in the subordinate relationship sub-network; screening a shortest connected path satisfying the matching rule, converting edges in the shortest connected path into actions, and converting nodes in the shortest connected path into real objects in the scene to obtain the missing segment of the behavior tree; and supplementing the missing segment of the behavior tree into the behavior tree to complete the task reasoning of the behavior tree; for the state and perception related reasoning, determining an ontology of a state by searching in the state change relationship sub-network and the subordinate relationship sub-network, triggering a perception device to collect perception data related to the ontology according to the ontology of the robot, and performing reasoning according to the perception data.

2. The method for knowledge base based reasoning task under dynamic scenario according to claim 1, characterized in that, After the action relationship sub-network is divided, a mapping of a task to an action set is constructed, including: labeling actions associated with a single task in the action relationship sub-network, labeling actions associated with a combined task in the action relationship sub-network, and then performing task semantic clustering on the labeled actions, wherein all actions included in each clustering cluster correspond to a task semantic, so that a mapping of each task to an action set is obtained, and a mapping set is further obtained.

3. The method for knowledge base based reasoning task under dynamic scenario according to claim 2, characterized in that, The determining of the action set of the target task includes searching a task matching the target task in the mapping set, and taking an action set mapped by the task as the action set of the target task.

4. The method for knowledge base based reasoning task under dynamic scenario according to claim 1, characterized in that, The reasoning of the target object according to the actions in the action set in the action relationship sub-network, the subordinate relationship sub-network and the preposition relationship sub-network includes: Determine the object of the action operation in the action relationship sub-network according to each action, and perform multi-level expansion of the object in the subordination sub-network to obtain new target objects, and filter the new target objects according to the perception situation.

5. The method for knowledge base based reasoning task under dynamic scenario according to claim 4, characterized in that, The multi-level expansion of the object in the subordination sub-network comprises: Searching for a subclass object having a subordination relationship with the object in the subordination sub-network as a new target object; searching for a parent class object having a subordination relationship with the object in the subordination sub-network, and taking other subclass objects of the parent class object as new target objects.

6. The method for knowledge base based reasoning task under dynamic scenario according to claim 5, characterized in that, The filtering operation of the new target objects according to the perception situation comprises: after searching for the new target objects through the parent class object of the object, deleting the new target objects that cannot appear in the scene according to the perceived object instances in the scene.

7. The method for knowledge base based reasoning task under dynamic scenario according to claim 1 or 4, characterized in that, Reasoning target objects in the action relationship sub-network, the subordination sub-network and the preposition relationship sub-network according to the actions in the action set comprises: When the target object set cannot be reasoned in the subordination sub-network, reasoning the target objects in the preposition relationship sub-network according to prepositions comprises: taking the object of the action operation as the basis, searching at least 3 hops in the preposition relationship sub-network according to the preposition relationship, and taking each hop object as a new target object.

8. The method for knowledge base based reasoning task under dynamic scenario according to claim 1, characterized in that, The method further comprises: searching for a state ontology in the state change relationship sub-network and the subordination sub-network, triggering a perception device to collect perception data related to the ontology according to the ontology, and filling the behavior parameters by querying the instantiation data under the label. Taking the target object and the corresponding unknown sensing parameter label as the basis, searching for a state related to the target object in the state change relationship sub-network, searching for a state ontology in the subordination sub-network, linking to the corresponding perception device according to the parameter description of the ontology, and obtaining the instantiation data of the perception device on the ontology parameters, the label of the instantiation data being the unknown sensing parameter label, so that the behavior parameters are filled by querying the instantiation data under the label, and the missing information is reasoned.

9. The method for knowledge base based reasoning task under dynamic scenario according to claim 2, characterized in that, The method further comprises: searching for a behavior tree corresponding to the target task in the task planning graph, specifically comprising: classifying the target task into a task node in the task planning graph, and searching all connected sub-task nodes in turn from the task node, and composing a task tree with all the obtained sub-task nodes and edges, determining the action set of the sub-task according to the mapping from the task to the action set after determining that the task tree can be actually executed, and constructing the behavior tree according to the action set of the sub-task.

10. The method for knowledge base based reasoning task under dynamic scenario according to claim 1 or 9, characterized in that, The method further comprises: verifying the reasoned behavior tree, and the verification rule comprises: verifying whether the logic of the behavior tree is correct according to the international specification of the behavior tree; and determining whether the task requirements can be met, i.e., whether the object instances exist in the scene and the robot can execute the actions described in the behavior tree to complete the task.

11. The method for knowledge base based reasoning task under dynamic scenario according to claim 10, characterized in that, The method further comprises: converting the verified behavior tree into a data structure and transmitting it to the behavior execution center of the robot, and driving the robot using the execution logic and behavior parameters of the behavior tree.

12. An apparatus for reasoning tasks in a dynamic scenario based on a knowledge base, the apparatus comprising: The system comprises a division unit, a searching unit and a reasoning unit. The division unit is configured to obtain a semantic network representing a knowledge base, and divide the semantic network into an action relationship sub-network, a state change relationship sub-network, a subordinate relationship sub-network, and a preposition relationship sub-network according to relationship types; The searching unit is configured to search a behavior tree for a target task in a task planning graph; The reasoning unit is configured to perform at least one of target object reasoning, task reasoning, and state and perception related reasoning under the semantic network when the behavior tree corresponding to the target task cannot be searched in the task planning graph; The reasoning unit includes a target object reasoning module, a task reasoning module, and a state and perception related reasoning module; The target object reasoning module is configured to determine an action set of the target task, and reason a target object in the action relationship sub-network, the subordinate relationship sub-network, and the preposition relationship sub-network according to actions in the action set; The task reasoning module is configured to search a path in the semantic network according to two endpoints of a missing segment of the behavior tree, determine the missing segment according to a search result, and implement task reasoning, including: mapping the behavior tree containing the missing segment into a sub-semantic network, wherein behaviors of the behavior tree are mapped into relationships in the sub-semantic network, and action targets of the behaviors are mapped into nodes of the sub-semantic network; finding actions of the two endpoints in the sub-semantic network, finding two objects corresponding to the actions of the two endpoints in the action relationship sub-network according to the actions of the two endpoints, and searching a plurality of connected paths between the two objects in the action relationship sub-network, wherein each connected path represents a series of action plans between the two objects; judging whether each connected path can be matched to the task planning graph according to a matching rule, wherein the matching rule is that all edge relationships in the connected path must be classified into a sub-task or an action plan, and nodes in the connected path must exist in a scene or be generalized to instances in the scene in the subordinate relationship sub-network; screening a shortest connected path that satisfies the matching rule, converting edges in the shortest connected path into behaviors, converting nodes in the shortest connected path into real objects in the scene, and obtaining the missing segment of the behavior tree; and supplementing the missing segment of the behavior tree to the behavior tree, and completing task reasoning of the behavior tree; The state and perception related reasoning module is configured to search for an ontology of a state in the state change relationship sub-network and the subordinate relationship sub-network, trigger a perception device to collect perception data related to the ontology according to the ontology of the robot, and perform reasoning according to the perception data.

Images