Task arrangement method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a task arrangement method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: receiving a user instruction forwarded by an intelligent robot; determining capability information corresponding to the user instruction, generating a task sequence based on the user instruction and the capability information, and sending the task sequence to the intelligent machine; an observation result sent by the intelligent robot is received, and the observation result is a result obtained by the intelligent robot executing a first operation behavior indicated by the task sequence in the physical environment and observing the physical environment; according to the observation result, a task instruction is generated and sent to the intelligent robot, and the task instruction is used for instructing the intelligent robot to perform a second operation behavior so as to complete a task target corresponding to the user instruction. The technical problem that matching errors and cross-scene application obstacles occur during task planning due to incomplete and non-standardized skill description of the intelligent robot is solved.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and robotics, and more specifically, to a task orchestration method, apparatus, non-volatile storage medium, and electronic device. Background Technology

[0002] Embodied intelligence, a cutting-edge field at the intersection of artificial intelligence and robotics, emphasizes that intelligent agents rely on their physical bodies to dynamically interact with their environment, thereby achieving autonomous learning and evolution. Its system employs a closed-loop learning mechanism of "perception (sensors) → decision-making (algorithms) → action (actuators) → environmental feedback," enabling it to adjust strategies in real time within dynamic environments and adapt to changing scenarios.

[0003] At the hardware level of embodied intelligent robots, execution units (such as robotic arms, mobile chassis, and end effectors) suffer from the limitation of atomicity of action: any hardware can only perform basic actions that are "single-dimensional, limited in scope, and have clearly defined parameters," and cannot directly complete complex tasks that are cross-dimensional and have long processes. This limitation dictates that complex tasks must be broken down into "atomic action sequences" that can be executed by the hardware. At the same time, the robot's perception modules (cameras, LiDAR, force sensors) and cognition modules (large models, planning algorithms) have an upper limit to their information processing capabilities—complex tasks contain "multimodal information, multiple constraints, and multiple target conflicts," which can easily exceed the real-time processing range of the system. However, by breaking down tasks, the information dimensions and decision-making difficulty can be controlled within the processing capabilities of the cognition modules. Therefore, the decomposition of complex tasks is a necessary prerequisite for embodied intelligent robots to perform tasks.

[0004] In the traditional field of AI planning, Hierarchical Task Networks (HTNs) are an important method for task decomposition. The core idea is to break down complex tasks layer by layer into subtasks and atomic actions, forming a tree structure until the subtasks are directly executable. However, traditional HTN methods rely excessively on manually designed rules and limited domain knowledge, lacking flexibility and generalization ability, and exhibiting poor adaptability in complex and dynamic scenarios. With the development of large-scale model technology, this situation has improved: large models can automatically learn task decomposition patterns and execution strategies. Methods that directly generate task sequences based on large models have become one of the mainstream technical approaches in the field of embodied intelligence—this approach fully utilizes the natural language understanding and generation capabilities of large models, directly transforming complex task instructions into executable action sequences.

[0005] However, existing task decomposition based on large models relies on prompt words. Robot skills, as a core component of prompt words, suffer from key problems such as missing information dimensions and chaotic description standards. These issues directly restrict the accuracy of complex task planning and the scalability of the system, becoming a core bottleneck for technology implementation. Specifically, this manifests as follows:

[0006] (1) Insufficient skill information coverage leads to task-skill mismatch. The current prompts describe robot skills at the "function name level" and lack key information such as "capability parameters, applicable scenarios, and constraints", which makes it impossible for the large model to accurately judge the suitability of skills and sub-tasks, thus causing planning errors;

[0007] (2) Lack of a unified skill description method leads to confusion in cross-scene / cross-device recognition. The industry has not yet established a unified standard for describing robot skills. Different developers and device manufacturers have significant differences in their descriptions of the same skill, which makes it difficult for large models to form a unified skill cognition and adapt to the task requirements of multiple devices and multiple scenarios.

[0008] There is currently no effective solution to the above problems. Summary of the Invention

[0009] This application provides a task orchestration method, apparatus, non-volatile storage medium, and electronic device to at least solve the technical problems of matching errors and cross-scenario application obstacles caused by incomplete and non-standardized descriptions of intelligent robot skills during task planning.

[0010] According to one aspect of this application, a task orchestration method is provided, comprising: receiving a user instruction forwarded by an intelligent robot; determining capability information corresponding to the user instruction; generating a task sequence based on the user instruction and the capability information; and sending the task sequence to the intelligent robot, wherein the task sequence is used to define the operational flow that the intelligent robot must follow to execute the user instruction; receiving observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in observing the physical environment while performing a first operational action indicated by the task sequence; generating a task instruction based on the observation results; and sending the task instruction to the intelligent robot, wherein the task instruction is used to instruct the intelligent robot to perform a second operational action to complete the task objective corresponding to the user instruction.

[0011] Optionally, determining the capability information corresponding to the user instruction includes: obtaining atomic capability knowledge source documents, and dividing the atomic capability knowledge source documents into blocks according to semantic relevance or functional modules to obtain multiple skill knowledge blocks; converting the skill knowledge blocks into vector representations and storing the vector representations in a vector database; performing semantic enhancement and parameter completion processing on the user instruction, and generating a search query statement based on the processed user instruction; performing a similarity search in the vector database based on the search query statement to obtain initial search results; and performing executability filtering on the initial search results to obtain capability information.

[0012] Optionally, the initial search results are subjected to executability screening to obtain capability information, including: obtaining physical environment state information associated with user instructions, wherein the physical environment state information is used to characterize the current state of the physical environment in which the intelligent robot is located when performing the task; performing executability evaluation on the skill knowledge blocks in the initial search results based on the matching degree between the skill execution constraints defined in the skill knowledge blocks in the initial search results and the physical environment state information, wherein the skill execution constraints are used to determine the conditions required for the atomic capabilities corresponding to the skill knowledge blocks to be successfully executed; obtaining the semantic relevance index of the skill knowledge blocks in the initial search results, and fusing the semantic relevance index and the executability evaluation results to obtain a comprehensive fit index; and determining the skill knowledge block corresponding to the highest comprehensive fit index in the initial search results as capability information.

[0013] Optionally, based on user instructions and capability information, a task sequence is generated, including: analyzing the user instructions and capability information using a generative model to obtain a task sequence consisting of multiple atomic capabilities arranged in an execution logical order, wherein the task sequence is a task chain required to instruct the intelligent robot to execute user instructions.

[0014] Optionally, a generative model is used to analyze user instructions and capability information to obtain a task sequence composed of multiple atomic capabilities arranged in an execution logical order. This includes: analyzing user instructions using a generative model to determine the target state that the task indicated by the user instruction needs to achieve, and determining the initial state of the task based on the physical environment state information associated with the user instruction; parsing the preconditions and expected effects described by each atomic capability in the capability information, and analyzing the preconditions and expected effects using a generative model to obtain the state dependencies and causal order relationships between atomic capabilities; constructing a directed skill graph based on the state dependencies and causal order relationships, where the nodes of the directed skill graph are atomic capabilities and the edges are state reachability relationships; in the directed skill graph, starting from the initial state and ending at the target state, determining the sequence of atomic capabilities that can transform the initial state into the target state through a heuristic search strategy integrated in the generative model; verifying whether the state transitions in the atomic capability sequence meet the logical constraints of the preconditions and expected effects; and if the verification fails, re-evaluating the capability information context or adjusting the atomic capability sequence using the generative model until a task sequence is generated.

[0015] Optionally, based on the observation results, a task instruction is generated, including: if the current execution step of the task sequence fails, analyzing the observation results using a generative model to determine the category of cause for failure, wherein the category of cause includes physical environment interference, skill execution deviation, or target state error; selecting an adjustment strategy corresponding to the cause category from a predefined strategy set, wherein the strategy set includes skill parameter adjustment, atomic ability replacement, execution step rollback, and re-determining the task sequence; and generating a task instruction to correct or skip the current failure state based on the selected adjustment strategy and the physical environment state information associated with the user instruction.

[0016] Optionally, before analyzing the observation results using a generative model to determine the category of reasons for failure when the current execution step of the task sequence fails, the method further includes: comparing the observation results with the expected state corresponding to the current execution step in the task sequence to obtain state difference information; and determining whether the current execution step of the task sequence has been successfully executed based on the state difference information.

[0017] According to another aspect of this application, a task orchestration apparatus is also provided, comprising: a first receiving module for receiving user instructions forwarded by an intelligent robot; a first generating module for determining capability information corresponding to the user instructions, generating a task sequence based on the user instructions and capability information, and sending the task sequence to the intelligent robot, wherein the task sequence is used to define the operational flow that the intelligent robot must follow to execute the user instructions; a second receiving module for receiving observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in observing the physical environment while performing the first operational behavior indicated by the task sequence; and a second generating module for generating task instructions based on the observation results and sending the task instructions to the intelligent robot, wherein the task instructions are used to instruct the intelligent robot to perform a second operational behavior to complete the task objective corresponding to the user instructions.

[0018] According to another aspect of this application, a non-volatile storage medium is also provided, the storage medium including a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the above task orchestration method.

[0019] According to another aspect of this application, an electronic device is also provided, comprising: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes the above-described task orchestration method during runtime.

[0020] According to another aspect of this application, a computer program is also provided, wherein when the computer program is executed by a processor, it implements the above-described task scheduling method.

[0021] According to another aspect of this application, a computer program product is also provided, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-described task orchestration method.

[0022] In this application, the following methods are employed: receiving user instructions forwarded by an intelligent robot; determining the capability information corresponding to the user instructions; generating a task sequence based on the user instructions and capability information; sending the task sequence to the intelligent robot, whereby the task sequence defines the operational flow that the intelligent robot must follow to execute the user instructions; receiving observation results sent by the intelligent robot, whereby the observation results are the results obtained by the intelligent robot in observing the physical environment while performing the first operational behavior indicated by the task sequence; generating task instructions based on the observation results; and sending the task instructions to the intelligent robot, whereby the task instructions instruct the intelligent robot to perform a second operational behavior to complete the task objective corresponding to the user instructions. This approach improves the accuracy and efficiency of task planning, thereby achieving the technical effect of intelligent robots autonomously adapting to and efficiently executing complex tasks in dynamic environments. Furthermore, it solves the technical problems of matching errors and cross-scenario application obstacles caused by incomplete and non-standardized descriptions of intelligent robot skills during task planning. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1 This is a flowchart of a task orchestration method according to an embodiment of this application;

[0025] Figure 2 This is a flowchart of another task orchestration method according to an embodiment of this application;

[0026] Figure 3 This is a framework diagram of a task orchestration system according to an embodiment of this application;

[0027] Figure 4 This is a robot skill description method and RAG module architecture diagram according to an embodiment of this application;

[0028] Figure 5 This is a structural diagram of a task orchestration apparatus according to an embodiment of this application;

[0029] Figure 6 This is a hardware structure block diagram of a computer terminal according to a task orchestration method according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] According to an embodiment of this application, a method embodiment of a task orchestration method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] Figure 1 This is a flowchart of a task orchestration method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0034] Step S102: Receive user instructions forwarded by the intelligent robot.

[0035] In step S102, the user conveys specific task requirements to the embodied intelligent robot via voice, text, or a preset format input device. Upon receiving the command, the intelligent robot's built-in perception module, such as a microphone, touchscreen, or dedicated user interface, captures and records the command content. The intelligent robot performs preliminary processing on the user's command, such as speech recognition to text conversion and format standardization, to ensure the command can be accurately parsed and understood. After completing this series of preprocessing steps, the intelligent robot forwards the user's command to the intelligent agent, i.e., the location of the generative model (e.g., a large language model).

[0036] Step S104: Determine the capability information corresponding to the user instruction, generate a task sequence based on the user instruction and capability information, and send the task sequence to the intelligent robot. The task sequence is used to define the operation process that the intelligent robot needs to follow to execute the user instruction.

[0037] In step S104, after receiving the user instruction forwarded by the intelligent robot, deep semantic analysis is performed to understand the task requirements and environmental conditions behind the instruction. The skill knowledge base is queried to determine the capability information corresponding to the user instruction.

[0038] The skill knowledge base stores detailed skill description documents. Each document includes fields such as skill identification, version information, skill category, function name, detailed description, parameter description, input / output format, example application, execution prerequisites, expected results, execution duration, resource dependencies, parallelism, interruptibility, error handling mechanism, priority, constraints, and related skills' pre- and post-requirements.

[0039] Based on user instructions and retrieved capability information, the generative model begins generating a task sequence. In this process, the model progressively breaks down high-level instructions into a series of executable low-level actions, each corresponding to an atomic skill in the skill knowledge base. The model considers the execution order of skills, dependencies, resource constraints, and possible error handling strategies to ensure that the generated task sequence is not only logically coherent but also conforms to the actual limitations of the physical world. The task sequence is presented in a structured format, containing the skill name, execution order, expected result, and associated environmental and resource states for each subtask. The task sequence is then sent back to the intelligent robot. Upon receiving the sequence, the intelligent robot executes the subtasks sequentially, each subtask guiding the robot to perform a specific physical operation or information processing activity.

[0040] Step S106: Receive the observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in performing the first operation behavior indicated by the task sequence in the physical environment and observing the physical environment.

[0041] For example, when an intelligent robot begins performing a subtask such as "picking up a water glass," it uses its integrated sensing modules, such as cameras, force sensors, and position sensors, to meticulously observe the physical environment. This process involves identifying the position of the water glass, assessing its stability, and detecting real-time changes in the distance between the robot and the water glass to ensure the safety and accuracy of the operation.

[0042] After the intelligent robot completes its first action, namely "picking up the water cup," it updates the environmental state, including but not limited to confirming the current position of the water cup, recording the robot's holding position, and the position and state of other objects in the environment. The intelligent agent receives the observations, analyzes and interprets them, evaluates the quality of the sub-task completion, confirms changes in the environmental state, and identifies any potential problems or risks.

[0043] Step S108: Based on the observation results, generate task instructions and send the task instructions to the intelligent robot. The task instructions are used to instruct the intelligent robot to perform a second operation to complete the task objective corresponding to the user instructions.

[0044] The first operational action refers to the specific action performed by the intelligent robot after receiving a task sequence generated by the intelligent agent, based on the first N defined operations in the task sequence. The first operational action marks the beginning of task execution and aims to directly respond to the initial needs mentioned in the user's instructions. For example, if the user's instruction is "Please pick up the water glass on the table and bring it to the living room sofa," then the first operational action could be "identify and locate the water glass," that is, using the intelligent robot's perception module to locate the water glass pointed to by the user in the physical environment, and then executing the atomic skill of "picking up the water glass" to ensure that the water glass is safely retrieved.

[0045] The second action is a subsequent action performed by the intelligent robot after completing the first action and feeding back the observation results to the agent, based on a new task instruction generated by the agent based on the observation results. The second action involves further processing of the environmental state and advancing the task objective. In the example above, after completing the first action, the intelligent robot feeds back the information that the water cup has been picked up and the current environmental state (such as the relative position of the robot and the sofa) to the agent. Based on the observation results, the agent generates a new task instruction, instructing the intelligent robot to "move to the designated location near the living room sofa" and "smoothly place the water cup." The second action thus embodies the subsequent steps in the task sequence, and its goal is to gradually achieve the final task objective set by the user instruction, that is, to successfully deliver the water cup to the designated location.

[0046] Through steps S102 and S104, this application achieves deep analysis and precise matching of user instructions, ensuring that the robot selects the most suitable set of atomic skills when performing tasks. This can significantly improve the accuracy and efficiency of task planning and avoid task execution inaccuracies caused by incomplete skill descriptions or inconsistent standards.

[0047] In step S106, while the robot performs its first operational action in the physical environment, it simultaneously observes and provides status feedback on the environment in real time. This enhances the agent's real-time perception of dynamic changes in the environment, ensuring that the robot can make timely adjustments based on the actual state of the environment. This mechanism greatly improves the flexibility and robustness of the robot's task execution.

[0048] In step S108, the agent dynamically generates task instructions for the next stage based on observations obtained from the robot, rather than pre-setting a fixed execution flow. This iterative instruction generation based on real-time feedback enables the robot to respond more intelligently to uncertainties and unexpected events, improving the success rate and efficiency of task execution.

[0049] The above solution forms a closed-loop control and optimization process, from receiving user instructions to generating and executing task instructions, and then receiving environmental feedback. This closed-loop mechanism ensures the continuity of information flow and the real-time response of the decision-making process, enabling the robot to continuously optimize its behavior during execution and achieve more efficient and accurate task completion.

[0050] The following are Figure 1 The steps shown are illustrated and explained by way of example.

[0051] According to some optional embodiments of this application, determining the capability information corresponding to a user instruction can be achieved through the following methods: obtaining atomic capability knowledge source documents, and dividing the atomic capability knowledge source documents into blocks according to semantic relevance or functional modules to obtain multiple skill knowledge blocks; converting the skill knowledge blocks into vector representations and storing the vector representations in a vector database; performing semantic enhancement and parameter completion processing on the user instruction, and generating a search query statement based on the processed user instruction; performing a similarity search in the vector database based on the search query statement to obtain initial search results; and performing executability screening on the initial search results to obtain capability information.

[0052] Specifically, the feasibility screening of the initial search results to obtain capability information can be achieved through the following methods: First, obtain the physical environment state information associated with the user's instructions, whereby the physical environment state information characterizes the current state of the physical environment in which the intelligent robot performs the task. Second, based on the matching degree between the skill execution constraints defined in the skill knowledge blocks in the initial search results and the physical environment state information, perform a feasibility assessment on the skill knowledge blocks in the initial search results, whereby the skill execution constraints determine the conditions required for the successful execution of the atomic capabilities corresponding to the skill knowledge blocks. Third, obtain the semantic relevance index of the skill knowledge blocks in the initial search results, and fuse the semantic relevance index and the feasibility assessment results to obtain a comprehensive adaptation index. Finally, in the initial search results, the skill knowledge block corresponding to the highest comprehensive adaptation index is identified as the capability information.

[0053] In this embodiment, the embodied intelligent robot's built-in perception module collects and analyzes physical environment state information in real time. This physical environment state information includes, but is not limited to, object positions, environmental layout, lighting conditions, and obstacle distribution. The physical environment state information can be obtained by scanning the surrounding environment using a sensor array.

[0054] Based on the initial results retrieved from the vector database—skill knowledge blocks—the skill execution constraints defined in each skill knowledge block are analyzed one by one. These constraints include the environmental states that must be met before skill execution (e.g., object distance, ambient lighting), the physical conditions required during skill execution (e.g., robotic arm load limits, battery power thresholds), and the expected target state after skill execution (e.g., object position change, completion of a specific action). These constraints are then compared with the latest state information obtained from the physical environment to evaluate the executability of each skill knowledge block in this environment.

[0055] To quantify the correlation between skill knowledge blocks and user instructions, a semantic relevance index for the skill knowledge blocks is calculated. This semantic relevance index can be based on natural language processing techniques, such as word embedding and semantic similarity scoring, to measure the semantic consistency between the skill description and the user instructions. Simultaneously, the previously obtained executability assessment results are fused with the semantic relevance index to generate a comprehensive adaptation index.

[0056] By comparing the comprehensive fit index of all skill knowledge blocks, the skill knowledge block with the highest value is selected as the best-matching capability information. This process ensures that the identified skills are not only highly semantically related to user instructions but also practically feasible in the current physical environment, providing the most suitable task execution plan for the intelligent robot.

[0057] Specifically, dividing the atomic capability knowledge source documents into blocks according to semantic relevance or functional modules to obtain multiple skill knowledge blocks can be achieved through the following methods:

[0058] First, all atomic ability knowledge source documents are formatted in a uniform way to ensure that each document contains complete skill description fields, such as skill ID, skill name, function description, execution conditions, expected effects, and other attributes.

[0059] Secondly, natural language processing techniques (such as word embedding and text clustering algorithms) are used to perform in-depth analysis of the content of each document and extract key semantic tags. These tags can be the category to which the skill belongs (such as "movement," "grasping," "perception"), the physical characteristics of the skill (such as "requires force sensors," "involves robotic arm operation"), or the functional description of the skill (such as "object recognition," "location and navigation"). Through tag extraction, a clear understanding of the central theme and skill attributes of the document can be obtained.

[0060] Next, based on the extracted semantic tags, documents with similar attributes or functions are categorized into blocks. For example, all skill documents involving "object recognition" are grouped into one block, while "movement" skill documents are grouped into another. This block-segmentation process can be based on manually set rules, such as keyword matching based on skill names or function descriptions, or it can be automatically clustered using machine learning methods, such as using the K-means algorithm or DBSCAN algorithm to group documents based on the similarity of their content.

[0061] Finally, the document sets in each block are integrated into an independent skill knowledge block. When constructing skill knowledge blocks, the granularity of the blocks can be further refined. For example, the "object recognition" skill knowledge block can be divided into two sub-blocks: "stationary object recognition" and "moving object recognition," to reflect the specific application of the skill in different scenarios.

[0062] On the other hand, determining the capability information corresponding to a user command can also be achieved through the following methods:

[0063] S1, acquire physical environment state information associated with user commands. The physical environment state information includes geometric state sub-information, physical attribute sub-information and body state sub-information collected in real time by robot sensors.

[0064] S2, for each skill knowledge block in the initial search results, perform the following operation:

[0065] S21, parse the set of structured skill execution constraints in the skill knowledge block, where each constraint is represented as a triple of <parameter, relational operator, threshold>.

[0066] S22, substitute the real-time data corresponding to the parameters in the physical environment state information into the triplet to perform relational operations and obtain the satisfaction score of the constraint condition;

[0067] S23, aggregate the satisfaction scores of all constraints of the skill knowledge block to generate the executability evaluation result of the skill knowledge block;

[0068] S3. For each skill knowledge block, its semantic similarity-based relevance index and its executability assessment result are input into an adaptive weighted fusion function to calculate a comprehensive fit index. The adaptive weighted fusion function dynamically adjusts the weights assigned to the executability assessment result according to the task security level or execution urgency implied in the user instructions.

[0069] S4. All skill knowledge blocks are sorted according to the comprehensive adaptation index, and a safety veto rule is applied: if the executability assessment result of any skill knowledge block is lower than the preset safety threshold, it is excluded from the final candidate set.

[0070] S5 selects the top few skill knowledge blocks with the highest comprehensive fit index from the candidate set that has passed the security veto screening, and uses them as capability information.

[0071] In some optional embodiments of this application, a task sequence can be generated based on user instructions and capability information by the following method: analyzing the user instructions and capability information using a generative model to obtain a task sequence consisting of multiple atomic capabilities arranged in an execution logical order, wherein the task sequence is a task chain required to instruct the intelligent robot to execute user instructions.

[0072] Specifically, by analyzing user instructions and capability information using a generative model, a task sequence consisting of multiple atomic capabilities arranged in execution logical order can be obtained. This can be achieved as follows: Analyze user instructions using a generative model to determine the target state that the task indicated by the user instruction needs to achieve, and determine the initial state of the task based on the physical environment state information associated with the user instruction; parse the preconditions and expected effects described by each atomic capability in the capability information, and analyze the preconditions and expected effects using a generative model to obtain the state dependencies and causal order relationships between atomic capabilities; construct a directed skill graph based on the state dependencies and causal order relationships, where the nodes of the directed skill graph are atomic capabilities and the edges are state reachability relationships; in the directed skill graph, starting from the initial state and ending at the target state, determine the sequence of atomic capabilities that can transform the initial state into the target state using a heuristic search strategy integrated into the generative model; verify whether the state transitions in the atomic capability sequence meet the logical constraints of the preconditions and expected effects; if the verification fails, re-evaluate the capability information context or adjust the atomic capability sequence using the generative model until a task sequence is generated.

[0073] In this embodiment, assuming the received user instruction is "Please move the water glass from the living room to the kitchen sink," an integrated generative model is used to perform deep analysis on this instruction. First, key information in the instruction is parsed to confirm the target state as "the water glass is in the kitchen sink," and the initial state as "the water glass is in an unknown location in the living room." By analyzing the state information of the physical environment, such as the layout of the living room and kitchen, and the relative positions of the water glass and sink, an environmental model for task execution is constructed.

[0074] The analysis of atomic ability descriptions within the ability information involves atomic skills such as "locating an object," "moving to a target," "picking up," and "putting down." It analyzes the prerequisites for these skills; for example, the "locating an object" skill requires sufficient and unobstructed lighting, while the "picking up" skill requires the object to be within the robot's operating range. It also analyzes the expected effects of the skills; for instance, after executing the "picking up" skill, the robot should hold the water cup, while the "putting down" skill requires the water cup to be placed in a designated location and the robot no longer holding it.

[0075] Based on the state dependencies and causal ordering relationships between atomic capabilities, a directed skill graph is constructed. In the graph, each node represents an atomic capability, and edges represent state reachability relationships, i.e., whether the execution of one atomic capability can lead to the fulfillment of the execution condition of the next atomic capability. For example, the "Locate Object" node is connected to the "Move to Target" node through an edge, indicating that after successfully locating the water cup, the robot can move to the vicinity of the water cup.

[0076] In a directed skill graph, starting from an initial state node and ending at a target state node, a heuristic search strategy from generative models is used to find a path along which a sequence of atomic abilities can efficiently move a water glass from the living room to the kitchen sink. During the search, each state transition is verified to ensure it meets the preconditions and expected effects of the atomic abilities, guaranteeing the feasibility and logical consistency of the selected path.

[0077] If, during the search process, a state transition along a path is found to violate logical constraints or execution conditions—for example, the robot cannot pick up a cup because it is too large—the generative model will be triggered to re-evaluate the capability information within the context. This involves re-analyzing the match between the current environment and the skill, or adjusting the atomic capability sequence, such as finding a smaller cup or using a more suitable grasping tool. This process will continue iterating until a sequence of atomic capabilities that both conforms to logical constraints and can transform the initial state into the target state is generated. Ultimately, this forms a task sequence that guides the robot to complete the task proposed by the user.

[0078] Through the above steps, the intelligent robot can not only accurately understand the final goal required by the user's instructions, but also intelligently plan the execution sequence of a series of atomic capabilities according to the actual state of the physical environment, ensuring the efficient, safe and accurate execution of the task.

[0079] As some optional embodiments of this application, generating task instructions based on observation results can be achieved through the following methods: when the current execution step of the task sequence fails, the observation results are analyzed using a generative model to determine the category of cause for failure, wherein the category of cause includes physical environment interference, skill execution deviation, or target state error; an adjustment strategy corresponding to the category of cause is selected from a predefined strategy set, wherein the strategy set includes skill parameter adjustment, atomic ability replacement, execution step rollback, and re-determining the task sequence; based on the selected adjustment strategy and the physical environment state information associated with the user instruction, a task instruction for correcting or skipping the current failure state is generated.

[0080] In this embodiment, the current task is assumed to be "assisting the user in safely moving a water glass from the living room to the kitchen sink." During execution, the agent encounters a failure in the atomic ability to "pick up the water glass." Analysis of the observations reveals that the robot's robotic arm failed to grasp the water glass, leaving it in place. This failure triggers a deep analysis of the observations using a generative model to determine the root cause of the failure.

[0081] Model analysis determined that the failure was due to "skill execution deviation," meaning the robot failed to achieve the expected result when performing the "pick up" skill. Further analysis revealed that the skill execution deviation might be caused by specific factors such as improper setting of the robotic arm's gripping force or the friction between the cup and the table exceeding expectations.

[0082] For the cause category of "skill execution deviation," the "skill parameter adjustment" strategy is selected from the predefined strategy set. This involves adjusting the force parameters when the robotic arm grips the cup and optimizing the contact angle between the gripper and the cup to improve the gripping success rate.

[0083] Based on the selected adjustment strategy and physical environment information, new task instructions are generated. These include modifying the robotic arm's gripping force to a higher value to enhance gripping stability, and adjusting the robotic arm's angle on the next attempt to ensure vertical alignment with the water cup and reduce the impact of friction. Furthermore, parameters of other related skills, such as the speed and path selection for "moving to the target location," can be dynamically adjusted according to current environmental conditions, such as the stability of the water cup and the smoothness of the tabletop.

[0084] Upon receiving the revised task instruction, the "pick up the water cup" atomic ability is re-executed, while real-time status information during skill execution is monitored. If the adjustment is successful, the robotic arm will successfully pick up the water cup, verifying whether this action meets expectations—that is, whether the water cup is stably held by the robotic arm without tipping over or being damaged. If the adjusted skill execution still fails, the process will revert to the strategy selection phase, considering "atomic ability replacement," i.e., finding alternative gripping skills, such as using a suction cup or a more sophisticated gripper to improve the success rate of gripping.

[0085] If adjusting the skill parameters fails to resolve the issue, you can choose to "roll back the execution step," which reverts to the "locate object" step to reconfirm the cup's position and orientation, ensuring the accuracy of subsequent actions. In some cases, such as when model analysis reveals that the cup was slightly moved in a previous attempt, causing inaccurate positional information, rolling back the execution step is necessary.

[0086] If the problem persists after multiple attempts to adjust, the "redetermine task sequence" strategy is activated. This involves reassessing user instructions and the physical environment, generating entirely new task sequences using generative models, and potentially replanning the entire operation process. This could include adjusting the robot's path to the water cup, using different object recognition methods, or even changing the target state, such as moving the water cup to an intermediate position before transferring it to the kitchen sink.

[0087] Through the above analysis, strategy selection, and task instruction generation, the intelligent robot can intelligently respond to failures during task execution. By adjusting skill parameters, replacing atomic capabilities, reverting execution steps, or even replanning the entire task sequence, it can overcome obstacles and ultimately successfully complete the task objectives set by the user.

[0088] In some optional embodiments of this application, before analyzing the observation results using a generative model to determine the cause of failure when the current execution step of the task sequence fails, the following steps may also be performed: comparing the observation results with the expected state corresponding to the current execution step in the task sequence to obtain state difference information; and determining whether the current execution step of the task sequence has been successfully executed based on the state difference information.

[0089] For example, an intelligent robot is assigned the task: "Move the water glass from the living room to the kitchen sink." During this task, the current step is "picking up the water glass." After performing this step, the robot collects observations through its integrated sensing system (such as cameras and force sensors). These observations include specific information such as whether the water glass is held in the robotic arm's gripper and the stability of the glass during gripping.

[0090] The observed results are compared with the expected state corresponding to the "pick up the water cup" step in the task sequence. The expected state includes a confirmation signal that the robotic arm holds the water cup, the contact force between the water cup and the table decreasing to zero (indicating that the water cup has been picked up), and the change in the water cup's position coordinates (from the table to the robotic arm's gripping position). By comparing the observed results with the expected state, it is possible to accurately determine whether the current execution step has achieved the expected goal and identify possible state differences, such as the water cup not being picked up or the robotic arm being unstable.

[0091] If the observed results match the expected state—that is, the robotic arm successfully grips the water cup, the water cup's position coordinates are updated correctly, and the water cup is in a stable state—then the "pick up the water cup" step is considered successfully completed. Conversely, if the observed results show any state difference, such as the water cup still not leaving the table, the position coordinates not changing, or the water cup tilting during the robotic arm's gripping process, the current step is considered a failure, and the specific state difference information is recorded as a basis for subsequent analysis and adjustment.

[0092] Figure 2 This is a flowchart of another task orchestration method according to an embodiment of this application. Figure 3 This is a framework diagram of a task orchestration system according to an embodiment of this application, wherein, Figure 3 The task orchestration system shown is used for execution Figure 2 The system shown is based on the method described.

[0093] like Figure 2 As shown, the method includes the following steps:

[0094] Step S201: The embodied intelligent robot, in its activated state, acquires interaction commands.

[0095] In step S202, the embodied intelligent robot sends the received user instructions to the large language model and requests the generation of an executable task sequence.

[0096] In step S203, the large language model retrieves relevant skills by combining the RAG mechanism with the atomic skill base.

[0097] In step S204, the large language model generates a task sequence (action chain) and returns it to the embodied intelligent robot.

[0098] In step S205, the embodied intelligent robot performs actions and observes changes in the environment, feeding the observation results back to the large language model.

[0099] Figure 4 This is a robot skill description method and RAG module architecture diagram according to an embodiment of this application, by... Figure 4 It can be seen that the following preparatory work needs to be performed before step S201.

[0100] First, a knowledge foundation for robot skills is constructed through multi-level atomic skill classification and standardized structured description. Information such as skill function definitions, execution parameters, constraints, dependencies, and expected effects are transformed into a unified format to achieve standardized expression of skill knowledge, providing a structured knowledge source for subsequent retrieval and reasoning. The knowledge source document for each skill can be saved using structured data storage file formats such as JSON, XML, and YAML. The formats are as follows:

[0101] {

[0102] "robot": "",

[0103] "skill_id": "",

[0104] "version": "",

[0105] "skill_category": "",

[0106] "skill_name": "",

[0107] "function_name": "",

[0108] "description": "",

[0109] "param_description": "",

[0110] "input_format": [],

[0111] "output_format": [],

[0112] example: "",

[0113] "precondition": [],

[0114] "postcondition": [],

[0115] "execution_duration": [],

[0116] "resource_dependency": [],

[0117] "parallelizable": "",

[0118] "interruptible": "",

[0119] "error_handling": "",

[0120] "priority": 1,

[0121] "constraints": [],

[0122] "related_skills": {

[0123] "pre_skills": [],

[0124] "post_skills": []

[0125] }

[0126] }

[0127] The meanings of the above fields are shown in the table below:

[0128]

[0129]

[0130] It should be noted that the above steps include the following core processes and components:

[0131] 1. Knowledge preprocessing process.

[0132] Import and Segmentation: The structured skill knowledge processed by the "Unified Skill Description Model" is imported into the system and segmented into blocks according to semantic relevance or functional modules to ensure that the granularity of knowledge units is adapted to the retrieval scenario.

[0133] Constructing an index and vector database: The segmented skill knowledge is encoded into high-dimensional vectors, and features such as meta-information are extracted to construct a vector index, which is finally stored in a vector database, thereby realizing an efficient retrieval and storage structure for skill knowledge.

[0134] 2. Search-Generation Process.

[0135] Action Command & Query Enhancement: Receive user action commands from the robot, enhance the semantics of the commands, complete the parameters (such as adding context and clarifying constraints), and generate query statements adapted to vector retrieval.

[0136] Retrieval and Reordering: Based on the enhanced query, the most relevant skill knowledge blocks are retrieved from the vector database, and the retrieval results are reordered according to relevance and suitability to select the optimal knowledge subset.

[0137] LLM (Large Language Model) and the generation of task chains: The large language model receives a rearranged subset of skill knowledge, combines it with the context of action instructions to reason and plan, and combines atomic skills in a logical order to generate an executable task chain (i.e., the sequence of steps for the robot to complete the target task).

[0138] The robot's atomic skills are processed through a "unified skill description model" and input into the RAG module to complete the knowledge "import-blocking-index construction-vector storage". When an action command is input, the RAG module obtains accurate skill knowledge through "query enhancement-retrieval rearrangement", and then generates a task chain through LLM reasoning. Finally, it realizes the intelligent transformation of "skill knowledge → task execution process", providing technical support for the task planning and execution of embodied intelligent robots.

[0139] The following example, “helping the user get a water cup,” illustrates in detail how the large language model generates task sequences (action chains).

[0140] Step 1: Collect and store the original information of core atomic skills through the Embodied Intelligence Unified Description Model, including "identifying objects," "picking up a water glass," "moving to the target location," and "putting down the water glass." Taking the "picking up a water glass" skill as an example, the corresponding file is as follows:

[0141]

[0142]

[0143] Step 2: Establish a vector database. Extract fields such as description, precondition, and postcondition for each file from the structured knowledge base as metadata. Call the text embedding model to convert the segmented files into 4096-dimensional vectors, store them in the Faiss vector library, and create an IndexFlatL2 index to complete the vector preparation work before retrieval.

[0144] Step 3: The user inputs their task request, "Please get me the water glass on the table," through the device's voice interface (which supports dialect recognition). The speech is then converted to text and sent to the query enhancement module. The module calls the model to parse and enhance the query, and sends the generated enhanced retrieval vector to the retrieval and reordering module, triggering the skill retrieval process.

[0145] Step 4: The retrieval and rearrangement module sends the retrieval results to the LLM. The retrieval results and task instructions are combined into prompt words, and the LLM generates action chain results according to the prompt words.

[0146] The prompt template is as follows:

[0147]

[0148]

[0149] The LLM-generated task chain results are as follows:

[0150]

[0151]

[0152] Step 5: Action Chain Execution and Exception Feedback The execution module of the embodied intelligence system calls the hardware driver function corresponding to the function_name of each skill in the order of the action chain.

[0153] In the above steps, firstly, the inaccuracy of task planning is addressed—related technologies only describe skills at the "functional name level," lacking key information such as execution conditions. This embodiment focuses on "modularization, hierarchicalization, and standardization," using unified documentation to cover all dimensions of atomic skill information, providing accurate basis for LLM and avoiding task-skill mismatch. Secondly, the system scalability problem is solved—related technologies lack unified standards, and the descriptions of similar skills vary greatly. This embodiment uses standardized specifications to ensure consistent skill expression across different devices, significantly reducing cross-scenario / cross-device adaptation costs. Finally, the quality of task orchestration is optimized—related technologies rely solely on prompt words. This embodiment introduces RAG technology to retrieve accurate skill information, assisting LLM in scientifically decomposing tasks and combining skills, improving the rigor and feasibility of orchestration.

[0154] Figure 5 This is a structural diagram of a task orchestration apparatus according to an embodiment of this application, such as... Figure 5 As shown, the device includes:

[0155] The first receiving module 52 is used to receive user instructions forwarded by the intelligent robot.

[0156] The first generation module 54 is used to determine the capability information corresponding to the user instruction, generate a task sequence based on the user instruction and capability information, and send the task sequence to the intelligent robot. The task sequence is used to define the operation process that the intelligent robot needs to follow to execute the user instruction.

[0157] The second receiving module 56 is used to receive the observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in performing the first operation behavior indicated by the task sequence in the physical environment and observing the physical environment.

[0158] The second generation module 58 is used to generate task instructions based on the observation results and send the task instructions to the intelligent robot. The task instructions are used to instruct the intelligent robot to perform a second operation to complete the task objective corresponding to the user instruction.

[0159] Optionally, determining the capability information corresponding to the user instruction includes the following steps: obtaining atomic capability knowledge source documents, and dividing the atomic capability knowledge source documents into blocks according to semantic relevance or functional modules to obtain multiple skill knowledge blocks; converting the skill knowledge blocks into vector representations and storing the vector representations in a vector database; performing semantic enhancement and parameter completion processing on the user instruction, and generating a search query statement based on the processed user instruction; performing a similarity search in the vector database based on the search query statement to obtain initial search results; and performing executability screening on the initial search results to obtain capability information.

[0160] Optionally, the initial search results are subjected to executability screening to obtain capability information, specifically including the following steps: obtaining physical environment state information associated with user instructions, wherein the physical environment state information is used to characterize the current state of the physical environment in which the intelligent robot performs the task; based on the matching degree between the skill execution constraints defined in the skill knowledge blocks in the initial search results and the physical environment state information, the executability of the skill knowledge blocks in the initial search results is evaluated, wherein the skill execution constraints are used to determine the conditions required for the atomic capabilities corresponding to the skill knowledge blocks to be successfully executed; obtaining the semantic relevance index of the skill knowledge blocks in the initial search results, and fusing the semantic relevance index and the executability evaluation results to obtain a comprehensive adaptation index; in the initial search results, the skill knowledge block corresponding to the highest comprehensive adaptation index is determined as capability information.

[0161] Optionally, based on user instructions and capability information, a task sequence is generated, specifically including the following steps: using a generative model to analyze the user instructions and capability information to obtain a task sequence consisting of multiple atomic capabilities arranged in the order of execution logic, wherein the task sequence is a task chain required to instruct the intelligent robot to execute user instructions.

[0162] Optionally, a generative model is used to analyze user instructions and capability information to obtain a task sequence composed of multiple atomic capabilities arranged in execution logical order. Specifically, this includes the following steps: analyzing user instructions using a generative model to determine the target state that the task indicated by the user instruction needs to achieve, and determining the initial state of the task based on the physical environment state information associated with the user instruction; parsing the preconditions and expected effects described by each atomic capability in the capability information, and analyzing the preconditions and expected effects using a generative model to obtain the state dependencies and causal order relationships between atomic capabilities; constructing a directed skill graph based on the state dependencies and causal order relationships, where nodes in the directed skill graph are atomic capabilities and edges are state reachability relationships; in the directed skill graph, starting from the initial state and ending at the target state, determining the sequence of atomic capabilities that can transform the initial state into the target state using a heuristic search strategy integrated in the generative model; verifying whether the state transitions in the atomic capability sequence meet the logical constraints of the preconditions and expected effects; if the verification fails, re-evaluating the capability information context or adjusting the atomic capability sequence using the generative model until a task sequence is generated.

[0163] Optionally, based on the observation results, a task instruction is generated, specifically including the following steps: when the current execution step of the task sequence fails, the observation results are analyzed using a generative model to determine the category of cause for failure, where the category of cause includes physical environment interference, skill execution deviation, or target state error; an adjustment strategy corresponding to the category of cause is selected from a predefined strategy set, where the strategy set includes skill parameter adjustment, atomic ability replacement, execution step rollback, and re-determining the task sequence; based on the selected adjustment strategy and the physical environment state information associated with the user instruction, a task instruction is generated to correct or skip the current failure state.

[0164] Optionally, if the current execution step of the task sequence fails, before analyzing the observation results using a generative model to determine the category of the cause of failure, the following steps can also be performed: compare the observation results with the expected state corresponding to the current execution step in the task sequence to obtain state difference information; based on the state difference information, determine whether the current execution step of the task sequence has been executed successfully.

[0165] It should be noted that the above Figure 5 Each module can be a program module (e.g., a set of program instructions that implements a specific function) or a hardware module. For the latter, it can take the following forms, but is not limited to them: each of the above modules is represented by a processor, or the functions of each of the above modules are implemented by a processor.

[0166] It should be noted that, Figure 5 Preferred embodiments of the shown examples can be found in [reference needed]. Figure 1 The relevant descriptions of the embodiments shown will not be repeated here.

[0167] Figure 6 A hardware block diagram of a computer terminal for implementing a task orchestration method is shown. Figure 6 As shown, the computer terminal 60 may include one or more processors 602 (shown as 602a, 602b, ..., 602n in the figure) 602 (processor 602 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 604 for storing data, and a transmission module 606 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the computer terminal 60 may also include... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0168] It should be noted that the aforementioned one or more processors 602 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 60. As described in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0169] The memory 604 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the task orchestration method in this embodiment. The processor 602 executes various functional applications and data processing by running the software programs and modules stored in the memory 604, thereby realizing the above-mentioned task orchestration method. The memory 604 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 604 may further include memory remotely located relative to the processor 602, and these remote memories can be connected to the computer terminal 60 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0170] The transmission module 606 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 60. In one example, the transmission module 606 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 606 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0171] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 60.

[0172] It should be noted here that, in some optional embodiments, the above... Figure 6 The computer terminal shown may include hardware components (including circuitry), software components (including computer code stored on a computer-readable medium), or a combination of both hardware and software components. It should be noted that... Figure 6 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computer terminal.

[0173] It should be noted that, Figure 6 The computer terminal shown is used to execute Figure 1 The task orchestration method shown above means that the relevant explanations in the command execution method also apply to this electronic device, and will not be repeated here.

[0174] This application embodiment also provides a non-volatile storage medium, which includes a stored program, wherein the program, when running, controls the device where the storage medium is located to execute the above task orchestration method.

[0175] A non-volatile storage medium performs the following functions: receiving user instructions forwarded by an intelligent robot; determining the capability information corresponding to the user instructions, generating a task sequence based on the user instructions and capability information, and sending the task sequence to the intelligent robot, wherein the task sequence is used to define the operation flow that the intelligent robot must follow to execute the user instructions; receiving observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in observing the physical environment while performing the first operation action indicated by the task sequence; generating task instructions based on the observation results, and sending the task instructions to the intelligent robot, wherein the task instructions are used to instruct the intelligent robot to perform a second operation action to complete the task objective corresponding to the user instructions.

[0176] This application also provides an electronic device, including a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the above-described task orchestration method during runtime.

[0177] The processor is used to run a program that performs the following functions: receiving user instructions forwarded by the intelligent robot; determining the capability information corresponding to the user instructions, generating a task sequence based on the user instructions and capability information, and sending the task sequence to the intelligent robot, wherein the task sequence is used to define the operation flow that the intelligent robot must follow to execute the user instructions; receiving observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in observing the physical environment while performing the first operation behavior indicated by the task sequence; generating task instructions based on the observation results, and sending the task instructions to the intelligent robot, wherein the task instructions are used to instruct the intelligent robot to perform a second operation behavior to complete the task objective corresponding to the user instructions.

[0178] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0179] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0180] In the above embodiments of this application, the information collected is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with relevant laws, regulations and standards, take necessary protective measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0185] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A task orchestration method, characterized in that, include: Receive user commands forwarded by the intelligent robot; The capability information corresponding to the user instruction is determined, a task sequence is generated based on the user instruction and the capability information, and the task sequence is sent to the intelligent robot. The task sequence is used to define the operation flow that the intelligent robot needs to follow to execute the user instruction. The system receives observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in observing the physical environment after performing the first operation behavior indicated by the task sequence in the physical environment; Based on the observation results, a task instruction is generated and sent to the intelligent robot. The task instruction is used to instruct the intelligent robot to perform a second operation to complete the task objective corresponding to the user instruction.

2. The method according to claim 1, characterized in that, Determining the capability information corresponding to the user instruction includes: Obtain the source document of atomic ability knowledge, and divide the source document of atomic ability knowledge into blocks according to semantic relevance or functional modules to obtain multiple skill knowledge blocks; The skill knowledge blocks are converted into vector representations, and the vector representations are stored in a vector database; The user command is semantically enhanced and parameter completed, and a search query statement is generated based on the processed user command. Based on the search query, a similarity search is performed in the vector database to obtain initial search results; The initial search results are subjected to executability screening to obtain the capability information.

3. The method according to claim 2, characterized in that, The initial search results are subjected to executability screening to obtain the capability information, including: Obtain physical environment state information associated with the user instruction, wherein the physical environment state information is used to characterize the current state of the physical environment in which the intelligent robot is located when performing the task; Based on the matching degree between the skill execution constraints defined in the skill knowledge blocks of the initial search results and the physical environment state information, the executability of the skill knowledge blocks in the initial search results is evaluated, wherein the skill execution constraints are used to determine the conditions required for the atomic capabilities corresponding to the skill knowledge blocks to be successfully executed: Obtain the semantic relevance index of the skill knowledge blocks in the initial search results, and fuse the semantic relevance index with the executability assessment results to obtain a comprehensive adaptation index; In the initial search results, the skill knowledge block corresponding to the highest comprehensive adaptation index is identified as the capability information.

4. The method according to claim 1, characterized in that, Based on the user instructions and the capability information, a task sequence is generated, including: By analyzing the user instructions and the capability information using a generative model, a task sequence consisting of multiple atomic capabilities arranged in an execution logical order is obtained, wherein the task sequence is a task chain required to instruct the intelligent robot to execute the user instructions.

5. The method according to claim 4, characterized in that, By analyzing the user instructions and the capability information using a generative model, a task sequence consisting of multiple atomic capabilities arranged in logical execution order is obtained, including: The generative model is used to analyze the user instructions to determine the target state that the task indicated by the user instructions needs to achieve, and the initial state of the task is determined based on the physical environment state information associated with the user instructions. The preconditions and expected effects described by each atomic ability in the capability information are analyzed, and the generative model is used to analyze the preconditions and expected effects to obtain the state dependencies and causal order relationships between the atomic abilities. Based on the state dependencies and the causal order relationships, a directed skill graph is constructed, wherein the nodes of the directed skill graph are the atomic abilities, and the edges are the state reachability relationships; In the directed skill graph, starting from the initial state and ending at the target state, a sequence of atomic abilities that can transform the initial state into the target state is determined by a heuristic search strategy integrated into the generative model. Verify whether the state transitions in the atomic capability sequence meet the preconditions and the logical constraints of the expected effect; If the verification fails, the capability information is re-evaluated in context or the atomic capability sequence is adjusted using the generative model until the task sequence is generated.

6. The method according to claim 1, characterized in that, Based on the observations, task instructions are generated, including: If the current execution step of the task sequence fails, the observation results are analyzed using a generative model to determine the category of reasons for the failure, wherein the category of reasons includes physical environmental interference, skill execution deviation, or target state error; Select the adjustment strategy corresponding to the reason category from the predefined strategy set, wherein the strategy set includes skill parameter adjustment, atomic ability replacement, execution step rollback, and re-determining the task sequence; Based on the selected adjustment strategy and the physical environment state information associated with the user instruction, a task instruction is generated to correct or skip the current failure state.

7. The method according to claim 6, characterized in that, Before analyzing the observations using a generative model to determine the category of reasons for failure when the current execution step of the task sequence fails, the method further includes: The observation results are compared with the expected state corresponding to the current execution step in the task sequence to obtain state difference information; Based on the state difference information, it is determined whether the current execution step of the task sequence has been successfully executed.

8. A task orchestration device, characterized in that, include: The first receiving module is used to receive user instructions forwarded by the intelligent robot; The first generation module is used to determine the capability information corresponding to the user instruction, generate a task sequence based on the user instruction and the capability information, and send the task sequence to the intelligent robot. The task sequence is used to define the operation process that the intelligent robot needs to follow to execute the user instruction. The second receiving module is used to receive the observation results sent by the intelligent robot, wherein the observation results are the results obtained by the intelligent robot in observing the physical environment after performing the first operation behavior indicated by the task sequence in the physical environment; The second generation module is used to generate task instructions based on the observation results and send the task instructions to the intelligent robot. The task instructions are used to instruct the intelligent robot to perform a second operation to complete the task objective corresponding to the user instruction.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, the device containing the non-volatile storage medium is controlled to perform the task orchestration method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the task orchestration method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the task orchestration method according to any one of claims 1 to 7.

Images