Human-machine collaborative task processing method, device, equipment, medium and program product

Abstract

Embodiments of the present application provide a human-robot collaborative task processing method, device, equipment, medium and program product, which are applied to a computer device, create a simulation environment, obtain a task target and collect an observable scene state graph, judge the completion progress of the task target, if the task target is not completed, input the observable scene state graph into a belief model, output global scene estimation information of a robot and a virtual person, generate sub-target instructions and motion sequences by using a senior planner, the robot and the virtual person execute the motion sequences, generate an updated simulation environment, collect an updated observable scene state graph, if the updated task target is not completed, re-execute the step of inputting the observable scene state graph into the belief model and outputting the global scene estimation information until the task target is completed, the scene estimation information output by the belief model guides the current behavior of an intelligent agent and deduces the behavior target of the other party, and the effect of completing the task by the robot and the assisted person is improved.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method, apparatus, equipment, medium, and program product for human-machine collaborative task processing. Background Technology

[0002] With the development of robotics and information technology, robots have expanded from the industrial sector to fields such as home services, making home service robots an integral part of our daily lives. To effectively assist humans in the home environment, home service robots can perform various complex tasks such as navigation, object recognition, and robotic arm operation. Combining deep learning technology with training robots using large amounts of training data can improve their perception and recognition capabilities, thereby assisting in completing more tasks. However, deep learning requires a large amount of training data, and training robots directly in real-world environments is costly and poses safety risks. Robot simulation, as a branch of virtual reality technology, enables the testing and training of robots in a computer-simulated environment by creating a digital simulation space.

[0003] In existing technologies, humanoid intelligent agents infer the tasks that the assisted person wants to complete by observing the behavior of the assisted person and then assist the assisted person in completing the corresponding tasks.

[0004] However, existing technologies cannot train or test the robot's ability to actively collaborate. When the robot and the assisted user both want to complete the same task, conflicts arise, leading to a decrease in the effectiveness of the task completion by both the robot and the assisted user. Summary of the Invention

[0005] The human-machine collaborative task processing method, apparatus, equipment, medium, and program products provided in this application are used to improve the efficiency of robots and assisted users in completing tasks.

[0006] In a first aspect, embodiments of this application provide a human-computer collaborative task processing method, applied to a computer device, comprising:

[0007] Create a simulation environment for human-machine collaboration, wherein the simulation environment includes robots and virtual humans;

[0008] The task objective is obtained and an observable scene state diagram of the simulation environment is collected to determine the progress of the task objective.

[0009] If the task objective is not achieved, the observable scene state diagram is input into the belief model to output global scene estimation information for the robot and the virtual human.

[0010] The global scene estimation information of the robot and the virtual human and the task objective are input into the high-level planner to output sub-objective instructions;

[0011] Generate a motion sequence according to the sub-target instructions;

[0012] The robot and the virtual human execute the motion sequence to generate an updated simulation environment;

[0013] Collect the observable scene state map of the updated simulation environment to obtain a new observable scene state map;

[0014] Identify whether the task objective in the new observable scene state graph has been completed;

[0015] If the task objective in the new observable scene state diagram is not completed, the step of "inputting the observable scene state diagram into the belief model to output the global scene estimation information of the robot and the virtual human" is repeated until the motion sequence of the robot and the virtual human is completed.

[0016] In one possible implementation, the step of inputting the observable scene state diagram into the belief model to output global scene estimation information for the robot and the virtual human includes: generating scene observation information for the robot and scene observation information for the virtual human based on the observable scene state diagram; updating the belief model based on the robot's scene observation information; sampling the updated belief model to obtain global scene estimation information for the robot; updating the belief model based on the virtual human's scene observation information; and sampling the updated belief model to obtain global scene estimation information for the virtual human.

[0017] In one possible implementation, the step of inputting the global scene estimation information of the robot and the virtual human and the task objective into a high-level planner to output sub-objective instructions includes: parsing the task objective through the high-level planner to obtain multiple sub-objectives; determining the type of the sub-objectives; if the type of the sub-objectives requires the robot and the virtual human to cooperate to complete, then inputting the global scene estimation information of the robot and the virtual human and the sub-objectives into a Monte Carlo tree search model to output sub-objective instructions.

[0018] In one possible implementation, the step of inputting the global scene estimation information of the robot and the virtual human and the sub-target into a Monte Carlo tree search model to output sub-target instructions includes: creating a Monte Carlo tree search model for the robot and the virtual human, wherein the Monte Carlo tree search model includes multiple layers of nodes, and the multiple layers of nodes record the task rounds of the virtual human or the robot; setting the task round of the root node in the Monte Carlo tree search model as the task round of the virtual human, and setting the next layer of task rounds of the virtual human's task round as the robot's task round; when the virtual human is in the task round... During a task round, the virtual human, based on its global scene estimation information and the sub-target, derives and generates the sub-target to be executed by the virtual human and the sub-target to be executed by the robot in the next round; when in the robot's task round, the robot, based on its global scene estimation information and the sub-target, derives and generates the sub-target to be executed by the robot and the sub-target to be executed by the virtual human in the next round; obtains the sub-target to be executed by the robot in each task round and generates the robot's sub-target instruction; obtains the sub-target to be executed by the virtual human in each task round and generates the virtual human's sub-target instruction.

[0019] In one possible implementation, generating a motion sequence based on the sub-target instruction includes: extracting text information from the sub-target instruction using a low-level planner based on a regular expression; recognizing the text information to generate action information and item information to be interacted with; and generating a motion sequence based on the action information and the item information to be interacted with.

[0020] Secondly, embodiments of this application provide a human-computer collaborative task processing device, applied to a computer device, comprising:

[0021] A creation module is used to create a simulation environment for human-machine collaboration, wherein the simulation environment includes robots and virtual humans;

[0022] The acquisition module is used to acquire the task objective and collect the observable scene state diagram of the simulation environment in order to determine the progress of the task objective.

[0023] The first output module is used to input the observable scene state map into the belief model if the task objective is not completed, so as to output the global scene estimation information of the robot and the virtual human.

[0024] The second output module is used to input the global scene estimation information of the robot and the virtual human and the task objective into the high-level planner to output sub-objective instructions;

[0025] The first generation module is used to generate a motion sequence according to the sub-target instruction;

[0026] The second generation module is used for the robot and the virtual human to execute the motion sequence to generate an updated simulation environment;

[0027] The acquisition module is used to acquire the observable scene state map of the updated simulation environment to obtain a new observable scene state map.

[0028] The identification module is used to identify whether the task objective in the new observable scene state graph has been completed;

[0029] The third output module is used to re-execute the step of "inputting the observable scene state diagram into the belief model to output the global scene estimation information of the robot and the virtual human" if the task objective in the new observable scene state diagram is not completed, until the motion sequence of the robot and the virtual human is completed.

[0030] In one possible implementation, the first output module includes: a first generation unit, configured to generate scene observation information of a robot and scene observation information of a virtual human based on the observable scene state diagram; a first update unit, configured to allow the robot to update the belief model based on the robot's scene observation information; a first sampling unit, configured to allow the robot to sample using the updated belief model to obtain global scene estimation information of the robot; a second update unit, configured to allow the virtual human to update the belief model based on the virtual human's scene observation information; and a second sampling unit, configured to allow the virtual human to sample using the updated belief model to obtain global scene estimation information of the virtual human.

[0031] Thirdly, embodiments of this application provide a human-machine collaborative task processing device, including: a memory and a processor;

[0032] The memory stores computer-executed instructions;

[0033] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0034] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0035] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0036] The human-machine collaborative task processing method, apparatus, device, medium, and program products provided in this application create a human-machine collaborative simulation environment, acquire task objectives and collect observable scene state diagrams, determine the progress of task objective completion, and if the task objective is not completed, input the observable scene state diagram into a belief model. The belief model outputs global scene estimation information for the robot and virtual human, guides the current behavior of the robot and virtual human respectively and infers the behavioral objectives of the other party through the global scene estimation information, uses an advanced planner to generate sub-objective instructions and motion sequences, executes the motion sequences, generates an updated simulation environment, and collects the observable scene state diagram of the updated simulation environment. If the task objective in the updated observable scene state diagram is not completed, the steps of inputting the observable scene state diagram into the belief model and outputting global scene estimation information are repeated until the task objective is completed. By using the scene estimation information output by the belief model to guide the current behavior of the intelligent agent and infer the behavioral objectives of the other party, the effectiveness of the robot and the assisted party in completing the task is improved. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0038] Figure 1 This is a schematic diagram of the system structure of a computer device provided in an embodiment of this application;

[0039] Figure 2 A flowchart illustrating the human-computer collaboration task processing method provided in this application;

[0040] Figure 3 A flowchart of active collaborative task planning and execution based on a robot simulation environment is provided for embodiments of this application;

[0041] Figure 4 A schematic diagram of a hierarchical planner provided in an embodiment of this application;

[0042] Figure 5 A schematic diagram illustrating the human-computer collaboration task objective provided in an embodiment of this application;

[0043] Figure 6 This is a schematic diagram illustrating the task of moving the utility table, provided in an embodiment of this application.

[0044] Figure 7 A visual schematic diagram is randomly generated to illustrate the scene layout of the four basic tasks provided in the embodiments of this application.

[0045] Figure 8 This is a schematic diagram of the output processing flow of the advanced planner provided in the embodiments of this application;

[0046] Figure 9 A schematic diagram of the Monte Carlo tree search model provided in the embodiments of this application;

[0047] Figure 10 A schematic diagram illustrating the instruction execution process provided in an embodiment of this application;

[0048] Figure 11 A schematic diagram of the human-machine collaborative task processing device provided in this application;

[0049] Figure 12 A schematic diagram of the structure of the human-machine collaborative task processing device provided in the embodiments of this application.

[0050] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] With the development of robotics and information technology, robots have expanded from the industrial sector to fields such as home services, making home service robots an integral part of our daily lives. To effectively assist humans in the home environment, home service robots can perform various complex tasks such as navigation, object recognition, and robotic arm operation. Combining deep learning technology with training robots using large amounts of training data can improve their perception and recognition capabilities, thereby assisting in completing more tasks. However, deep learning requires a large amount of training data, and training robots directly in real-world environments is costly and poses safety risks. Robot simulation, as a branch of virtual reality technology, enables the testing and training of robots in a computer simulation environment by creating digital simulation environments. In existing technologies, humanoid intelligent agents infer the tasks the assisted individual wants to complete by observing their behavior and then assist the assisted individual in completing the corresponding tasks. However, existing solutions cannot train or test the robot's proactive collaborative capabilities. When the robot and the assisted individual simultaneously want to complete the same task, conflicts arise, leading to a decrease in the effectiveness of both parties in completing the task.

[0053] To address the aforementioned technical problems, this application proposes the following technical concept: The inventors considered creating a human-machine collaborative simulation environment. Virtual humans and robots are set up within this environment. By acquiring the observable scene state map and task objectives of the simulation environment, the progress of task objective completion is determined. If the task objective is not completed, a belief model is created. The observable scene state map is input into the belief model, which outputs global scene estimation information for the robot and virtual human. This global scene estimation information guides the current behavior of the robot and virtual human and infers the other's behavioral objectives. The global scene estimation information is input into a high-level planner, which outputs sub-objective instructions and generates motion sequences. The robot and virtual human execute these motion sequences, generating an updated simulation environment. By acquiring the updated observable scene state map of the simulation environment, it is determined whether the task objective in the new observable scene state map has been completed. If the task objective is not completed, the steps of inputting the observable scene state map into the belief model and outputting global scene estimation information are repeated until the task objective is completed. Detailed embodiments are described below.

[0054] Figure 1 This is a schematic diagram of the system architecture of the computer device provided in an embodiment of this application. Figure 1 As shown, the computer device includes: a receiving device 101, a processor 102, and a display device 103.

[0055] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the item recognition method. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0056] In the specific implementation process, the receiving device 101 can be an input / output interface or a communication interface, and can acquire the task target.

[0057] The processor 102 can generate motion sequences and generate observable scene state diagrams.

[0058] The display device 103 can be used to display the above-mentioned observable scene state diagram, etc.

[0059] The display device can also be a touch screen, used to receive user commands while displaying the above content, so as to realize the operation interaction with the user.

[0060] It should be understood that the aforementioned processor can be implemented by reading instructions from memory and executing those instructions, or it can be implemented through chip circuitry.

[0061] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0062] Figure 2 The flowchart illustrating the human-computer collaborative task processing method provided in this application is as follows: Figure 2 As shown, the method includes:

[0063] S201: Create a simulation environment for human-machine collaboration, in which the simulation environment includes robots and virtual humans.

[0064] Figure 3 This is a flowchart illustrating the active collaborative task planning and execution process based on a robot simulation environment, as provided in the embodiments of this application.

[0065] In this embodiment, the simulation environment includes a high-level planner and a low-level planner.

[0066] Figure 4 A schematic diagram of a hierarchical planner provided in an embodiment of this application.

[0067] like Figure 4 As shown, the high-level planner is used to plan which actions need to be performed, while the low-level planner focuses on performing the specific actions.

[0068] For example, in the task of setting a table, which requires placing several plates and cups, the high-level planner will plan instructions based on the task objective, such as placing plate 1 on the table and cup 2 on the table. The low-level planner will parse the actions and the items to be interacted with, and then execute the actions based on the actions and the items to be interacted with.

[0069] During training and testing, the virtual human's high-level and low-level planners share the task flow with the robot's high-level and low-level planners. During testing, developers can manually control the virtual human's interface to perform task collaboration.

[0070] S202: Obtain the task objective and collect observable scene state diagrams of the simulation environment to determine the progress of the task objective.

[0071] Specifically, an observable scene state diagram of the simulation environment is obtained, and the status of the virtual human, robot, and items involved in the assisted task is used to determine whether the task objective has been achieved.

[0072] In this embodiment, the types of task objectives are divided into three categories: objectives that do not require assistance, objectives that can be completed independently, and objectives that must be completed in cooperation with others.

[0073] For example, a goal that does not require assistance could be a virtual person reading on a sofa; a goal that can be accomplished independently could be setting the dining table; and a goal that requires collaboration with others could be moving the table.

[0074] Figure 5 This is a schematic diagram of a human-computer collaboration task objective provided in an embodiment of this application.

[0075] like Figure 5 As shown, where Figure 5 Scenario A shows a virtual person sitting on a sofa watching TV, representing a goal that does not require assistance. Scenario B shows a virtual person picking up an item on a table, representing a goal that can be accomplished independently. Scenario C shows a virtual person and a robot lifting and moving a table together, representing a goal that can be accomplished in cooperation with others.

[0076] Specifically, in the simulation environment, four tasks and corresponding sub-goals are set according to the type of task objective. Depending on the different task objectives, the robot provides assistance to the virtual human. Among them, setting the dining table, collecting parameters, and reading and relaxing can be completed independently by the robot or the virtual human, while moving the clutter table requires the robot and the virtual human to work together.

[0077] Table 1 is a record table of four human-machine collaborative tasks and their sub-objective types.

[0078] Table 1. Four human-machine collaborative tasks and their sub-objective types

[0079]

[0080] S203: If the task objective is not achieved, the observable scene state map is input into the belief model to output global scene estimation information for the robot and the virtual human.

[0081] Specifically, the robot and virtual human update their belief models based on scene observation information, and obtain global scene estimation information for the robot and virtual human by sampling through the updated belief models.

[0082] S204: Input the global scene estimation information and task objectives of the robot and virtual human into the advanced planner to output sub-objective instructions.

[0083] Specifically, the high-level planner parses the task objective to obtain multiple sub-objectives, inputs the global scene estimation information and sub-objectives into the Monte Carlo tree search model, and outputs sub-objective instructions.

[0084] S205: Generate motion sequences based on sub-target instructions.

[0085] Specifically, a low-level planner extracts text information based on regular expressions, identifies the text information to generate action information and information about items to be interacted with, and generates a motion sequence.

[0086] S206: Robots and virtual humans execute motion sequences to generate an updated simulation environment.

[0087] For example, Figure 6 This is a schematic diagram illustrating the task of moving the utility table, provided in an embodiment of this application.

[0088] like Figure 6 As shown, Figure 6 In scenario A, the robot discovers the clutter on the table and takes away the cup; in scenario B, the robot takes away the small box on the table; in scenario C, the virtual human discovers and takes away the last cluttered item, the cup, and places it in the living room; in scenario D, the virtual human lifts the table, preparing to move it, but the robot is not in position; in scenario E, the robot is in position and lifts and moves the table together with the virtual human; in scenario F, the robot and the virtual human put the table down together.

[0089] S207: Collect the updated observable scene state map of the simulation environment to obtain a new observable scene state map.

[0090] In this embodiment, the actions of the virtual human or robot will continuously change the state of the simulation environment, generating new observable scene state diagrams.

[0091] S208: Identify whether the task objective in the new observable scene state graph has been completed.

[0092] Specifically, the states of robots, virtual people, and items in the task objective are determined in the new observable scene state diagram to determine whether the task objective has been completed.

[0093] S209: If the task objective in the new observable scene state diagram is not completed, the step of "inputting the observable scene state diagram into the belief model to output global scene estimation information for the robot and the virtual human" is repeated until the motion sequence of the robot and the virtual human is completed to the task objective.

[0094] In this embodiment, existing training methods for human-computer collaborative tasks include the watch-and-help method and the NOPA (Neurally-guided Online Probabilistic Assistance) method.

[0095] In this embodiment, seven basic test scenarios are set. Table 2 lists all the tasks to be tested and their generated data, and shows the index number of the basic test scenario that can be used to generate task data.

[0096] Table 2 Scene Setup Generation Details

[0097]

[0098] Figure 7 A visual schematic diagram is generated randomly for the scene layout of the four basic tasks provided in the embodiments of this application.

[0099] like Figure 7 As shown, Figure 7 Scenario A involves setting up a dining table, with only a fruit platter and condiments left on it, and the tableware scattered in containers such as the refrigerator and cabinets in the kitchen; Scenario B involves collecting the tableware, which is placed on the dining table, refrigerator, etc. in the kitchen and needs to be put in the dishwasher; Scenario C involves reading and relaxing, with snacks and drinks scattered throughout the scene, and the virtual person eventually sits on the sofa to read; Scenario D involves moving a clutter table, which is randomly placed in the room and needs to be moved to the next room.

[0100] Table 3 shows the execution results for the four tasks. Efficiency is measured by the average number of steps completed per task. In the task of moving the clutter table, the sub-objective of cleaning the table is considered when calculating the number of steps. The results in the table are categorized into no robot intervention, stochastic reinforcement learning method, watch-and-help method, NOPA method, and the proposed method. The results show that the stochastic reinforcement learning methods (watch-and-help and NOPA) fail to converge in the task of moving the clutter table due to the large action space and its sensitivity to sequence, often failing due to timeouts. Under reinforcement learning methods, the robot's behavior may interfere with task completion, such as mistakenly taking books it shouldn't.

[0101] Table 3. Average number of single-objective execution steps for the active assistance method on four types of combined tasks.

[0102]

[0103] As can be seen from the above embodiments, by creating a human-machine collaborative simulation environment, acquiring the task objective and collecting the observable scene state map, judging the progress of the task objective completion, if the task objective is not completed, the observable scene state map is input into the belief model, the belief model outputs global scene estimation information for the robot and the virtual human, and the global scene estimation information guides the current behavior of the robot and the virtual human and infers the behavioral objective of the other party, using the advanced planner to generate sub-objective instructions and motion sequences, the robot and the virtual human execute the motion sequences, generating an updated simulation environment, collecting the observable scene state map of the updated simulation environment, if the task objective in the updated observable scene state map is not completed, the steps of inputting the observable scene state map into the belief model and outputting global scene estimation information are repeated until the task objective is completed. The scene estimation information output by the belief model guides the current behavior of the intelligent agent and infers the behavioral objective of the other party, improving the efficiency of the robot and the assisted party in completing the task.

[0104] In one embodiment of this application, step S203 includes:

[0105] S2031: Generate scene observation information for the robot and scene observation information for the virtual human based on the observable scene state diagram.

[0106] Figure 8 This is a schematic diagram of the output processing flow of the advanced planner provided in the embodiments of this application.

[0107] In this embodiment, the scene observation information observed by the robot and virtual human at a set time is a portion of the information in the entire simulation environment.

[0108] S2032: The robot updates its belief model based on the robot's scene observation information.

[0109] In this embodiment, the belief model is the agent's probability estimate of the distribution of items in the scene.

[0110] For example, the robot believes that the probability of the book being in the living room is 30%, in the kitchen is 20%, and in the bedroom is 50%.

[0111] S2033: The robot samples using the updated belief model to obtain global scene estimation information for the robot.

[0112] Specifically, the robot updates the scene by executing sub-goals, obtains scene observation information in the new scene, updates the belief model with the scene observation information in the new scene, and samples from the updated belief model when executing the next sub-goal to obtain global scene estimation information.

[0113] S2034: The virtual human updates its belief model based on the virtual human's scene observation information.

[0114] In this embodiment, unobserved items are assigned an average estimate. When an item is observed, its position in the belief model is updated to its exact location. When an item is out of sight, its estimate gradually regresses to the initial average estimate.

[0115] The initialization of the belief model is based on a global state graph, which provides information on the existence of all items in the scene, but does not specify their locations. In order to use this belief information, a global state estimate of the scene needs to be obtained through sampling, including the possible locations of items, the on / off states of appliances, and the on / off states of furniture, thereby generating a detailed global state graph.

[0116] S2035: The virtual human is sampled through an updated belief model to obtain global scene estimation information for the virtual human.

[0117] Specifically, the virtual human updates the scene by performing sub-goals, obtains scene observation information in the new scene, updates the belief model with the scene observation information in the new scene, and samples from the updated belief model when performing the next sub-goal to obtain global scene estimation information.

[0118] As can be seen from the above embodiments, the belief model is updated by the scene observation information of the robot and the scene observation information of the virtual human. The robot and the virtual human sample from the updated belief model to obtain their respective global scene estimation information. The global scene estimation information guides their current behavior and infers the other's behavior goal, thus realizing the collaboration between the robot and the virtual human.

[0119] In one embodiment of this application, step S204 includes:

[0120] S2041: The task objective is analyzed by the advanced planner, resulting in multiple sub-objectives.

[0121] In this embodiment, the task objective includes multiple sub-objectives.

[0122] For example, the task objective of moving the clutter table includes the sub-objectives of: picking up a mug from the clutter table, moving the clutter table to the target location, and placing milk on the target clutter table.

[0123] S2042: Determine the type of sub-target.

[0124] In this embodiment, the types of sub-goals include goals that do not require help, goals that can be completed independently, and goals that must be completed in cooperation with others.

[0125] S2043: If the type of sub-objective requires the cooperation of robots and virtual humans to complete, then input the global scene estimation information of robots and virtual humans and the sub-objective into the Monte Carlo tree search model to output the sub-objective instruction.

[0126] In this embodiment, during the execution of collaborative tasks, the robot and the virtual human need to update their beliefs based on observations and sample from these beliefs to obtain a global state estimate of the scene. This estimated information is used to guide the current behavior of the agents and to infer the behavioral goals of the other agent.

[0127] As can be seen from the above embodiments, by dividing the task objective into sub-objectives and determining the type of the sub-objectives, if the sub-objective is an objective that requires cooperation to complete, the Monte Carlo tree search model generates sub-objective instructions for the virtual human and the robot based on the global scene estimation information and the sub-objectives, so that the robot and the virtual human can cooperate to complete the task.

[0128] In one embodiment of this application, step S2043 includes:

[0129] S301: Create a Monte Carlo tree search model for robots and virtual humans, wherein the Monte Carlo tree search model includes multiple layers of nodes, and the multiple layers of nodes record the task rounds of the virtual human or robot.

[0130] Figure 9 This is a schematic diagram of the Monte Carlo tree search model provided in an embodiment of this application.

[0131] S302: Set the task turn of the root node in the Monte Carlo tree search model to the task turn of the virtual human, and set the next level task turn of the virtual human to the task turn of the robot.

[0132] In this embodiment, the virtual human is defined as agent A, and the robot is defined as agent B.

[0133] For example, in a table-moving task, the cups and remote control on the table must be cleared before the table can be lifted, and only then can the task of moving the table to the target location be performed.

[0134] Specifically, the root node is set to the turn of agent A, the second-level node is set to the turn of agent B, and so on, so that the nodes of the tree are expanded in turn by agent A executing sub-goals.

[0135] S303: When in the virtual human's task round, the virtual human derives and generates the sub-goals to be executed by the virtual human and the sub-goals to be executed by the robot in the next round based on the virtual human's global scene estimation information and sub-goals.

[0136] Specifically, the virtual human derives its own sub-goals based on its beliefs and task objectives, while simultaneously speculating on the possible sub-goals of agent B.

[0137] S304: When in the robot's task round, the robot derives and generates the sub-objectives to be executed by the robot and the sub-objectives to be executed by the virtual human in the next round based on the robot's global scene estimation information and sub-objectives.

[0138] Specifically, the robot derives its own sub-goals based on its beliefs and task objectives, while also speculating on possible sub-goals of agent A.

[0139] S305: Obtain the sub-goals to be executed in each task round of the robot and generate the sub-goal instructions for the robot.

[0140] Specifically, the nodes with the best UCT (Upper Confidence Bound Apply to Trees) value in the Monte Carlo tree search model that belong to the agent's round B are traversed, the sub-goals executed by the node are recorded, and sub-goal instructions are generated.

[0141] S306: Obtain the sub-goals to be executed in each task round of the virtual human, and generate the sub-goal instructions for the virtual human.

[0142] Specifically, the node with the optimal UCT value in the Monte Carlo tree search model that belongs to round A of the agent is traversed, the sub-goal executed by the node is recorded, and the sub-goal instruction is generated.

[0143] As can be seen from the above embodiments, by creating a Monte Carlo tree search model, setting the root node in the model as the virtual human's turn, and the virtual human's next-level task turn as the robot's turn, and executing the loop settings sequentially, when in the virtual human's task turn, the virtual human deduces the sub-goal to be executed by the virtual human and the sub-goal to be executed by the robot in the next turn, and the robot deduces the sub-goal to be executed by the robot and the sub-goal to be executed by the virtual human in the next turn. The sub-goals of all turns of the robot and the virtual human are generated into corresponding sub-goal instructions. By the robot and the virtual human mutually deduce the sub-goals that they and each other may execute and generate target instructions, task conflicts between the robot and the virtual human when performing tasks are avoided, and the efficiency of collaborative task execution is improved.

[0144] In one embodiment of this application, step S205 includes:

[0145] S2051: Extract text information from sub-target instructions using regular expressions via a low-level planner.

[0146] Specifically, the low-level planner receives and parses instructions, and uses regular expressions to extract key text information from the text of the instructions.

[0147] The text information includes, but is not limited to, the action to be performed, the item category to be interacted with, and the instance ID.

[0148] S2052: Recognize text information and generate action information and item information to be interacted with.

[0149] Specifically, the low-level planner identifies specific interactive items from the constructed state graph and sends action information and item information to the action executor.

[0150] S2053: Generate a motion sequence based on the action information and the information of the item to be interacted with.

[0151] Specifically, the motion actuator generates motion sequences based on the basic modules in the simulation environment.

[0152] Figure 10 This is a schematic diagram of the instruction execution process provided in an embodiment of this application.

[0153] As can be seen from the above embodiments, by recognizing the text information in the instructions through the low-level planner, the actions and objects to be interacted with recorded in the text information are parsed out, and a motion sequence is generated based on the actions and objects to be interacted with, so as to avoid conflicts in the execution actions of the robot and the intelligent agent.

[0154] Figure 11 A schematic diagram of the human-machine collaborative task processing device provided in this application is shown below. Figure 11 As shown, the human-machine collaborative task processing device 11 provided in this embodiment includes: a creation module 111, an acquisition module 112, a first output module 113, a second output module 114, a first generation module 115, a second generation module 116, an acquisition module 117, an identification module 118, and a third output module 119.

[0155] Create module 111 to create a simulation environment for human-machine collaboration, which includes robots and virtual humans.

[0156] The acquisition module 112 is used to acquire the task objective and collect the observable scene state diagram of the simulation environment in order to determine the progress of the task objective.

[0157] The first output module 113 is used to input the observable scene state map into the belief model if the task objective is not completed, so as to output global scene estimation information for the robot and the virtual human.

[0158] The second output module 114 is used to input the global scene estimation information and task objectives of the robot and virtual human into the advanced planner to output sub-objective instructions.

[0159] The first generation module 115 is used to generate motion sequences according to sub-target instructions.

[0160] The second generation module 116 is used for the robot and virtual human to execute motion sequences to generate an updated simulation environment.

[0161] The acquisition module 117 is used to acquire the observable scene state map of the updated simulation environment to obtain a new observable scene state map.

[0162] The identification module 118 is used to identify whether the task objective in the new observable scene state graph has been completed.

[0163] The third output module 119 is used to re-execute the step of "inputting the observable scene state diagram into the belief model to output global scene estimation information of the robot and the virtual human" if the task objective in the new observable scene state diagram is not completed, until the motion sequence of the robot and the virtual human is completed to the task objective.

[0164] In one possible implementation, the first output module 113 includes:

[0165] The first generation unit 131 is used to generate scene observation information for the robot and scene observation information for the virtual human based on the observable scene state diagram.

[0166] The first update unit 132 is used for the robot to update the belief model based on the robot's scene observation information.

[0167] The first sampling unit 133 is used for the robot to sample through the updated belief model to obtain global scene estimation information of the robot.

[0168] The second update unit 134 is used for the virtual human to update the belief model based on the virtual human's scene observation information.

[0169] The second sampling unit 135 is used for sampling by the virtual human through the updated belief model to obtain global scene estimation information of the virtual human.

[0170] In one possible implementation, the second output module 114 includes:

[0171] The parsing unit 141 is used to parse the task objective through the high-level planner to obtain multiple sub-objectives.

[0172] Judgment unit 142 is used to determine the type of sub-target.

[0173] Output unit 143 is used to input the global scene estimation information of the robot and the virtual human and the sub-target into the Monte Carlo tree search model to output the sub-target instruction if the type of the sub-target requires the cooperation of the robot and the virtual human.

[0174] In one possible implementation, the output unit 143 includes:

[0175] Create sub-unit 1431 to create a Monte Carlo tree search model for robots and virtual humans, wherein the Monte Carlo tree search model includes multiple layers of nodes, and the multiple layers of nodes record the task rounds of virtual humans or robots.

[0176] The sub-unit 1432 is used to set the task turn of the root node in the Monte Carlo tree search model as the task turn of the virtual human, and set the next level task turn of the virtual human as the task turn of the robot.

[0177] The first generation subunit 1433 is used to, when in the virtual human's task round, deduce and generate the sub-objectives to be executed by the virtual human and the sub-objectives to be executed by the robot in the next round based on the virtual human's global scene estimation information and sub-objectives.

[0178] The second generation sub-unit 1434 is used to, when in the robot's task round, deduce and generate the sub-objectives to be executed by the robot and the sub-objectives to be executed by the virtual human in the next round based on the robot's global scene estimation information and sub-objectives.

[0179] The first acquisition subunit 1435 is used to acquire the sub-targets to be executed in each task round of the robot and generate the sub-target instructions for the robot.

[0180] The second acquisition subunit 1436 is used to acquire the sub-targets to be executed in each task round of the virtual human and generate the sub-target instructions of the virtual human.

[0181] In one possible implementation, the first generation module 115 includes:

[0182] Extraction unit 151 is used to extract text information from subtarget instructions based on regular expressions by a low-level planner.

[0183] The recognition unit 152 is used to recognize text information and generate action information and item information to be interacted with.

[0184] The second generation unit 153 is used to generate a motion sequence based on the action information and the information of the item to be interacted with.

[0185] The human-computer collaborative task processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0186] Figure 12 This is a schematic diagram of the structure of a human-machine collaborative task processing device provided in an embodiment of this application. Figure 12As shown, the human-machine collaborative task processing device 120 provided in this embodiment includes at least one processor 1201 and a memory 1202. Optionally, the human-machine collaborative task processing device 120 further includes a communication component 1203. The processor 1201, memory 1202, and communication component 1203 are connected via a bus 1204.

[0187] In the specific implementation process, at least one processor 1201 executes computer execution instructions stored in memory 1202, causing at least one processor 1201 to execute the above-described human-computer collaborative task processing method.

[0188] The specific implementation process of processor 1201 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0189] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0190] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0191] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0192] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described human-computer collaborative task processing method.

[0193] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned human-computer collaborative task processing method.

[0194] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0195] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0196] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0201] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A human-computer collaborative task processing method, characterized in that, Applied to computer equipment, including: Create a simulation environment for human-machine collaboration, wherein the simulation environment includes robots and virtual humans; The task objective is obtained and an observable scene state diagram of the simulation environment is collected to determine the progress of the task objective. If the task objective is not achieved, the observable scene state diagram is input into the belief model to output global scene estimation information for the robot and the virtual human. The global scene estimation information of the robot and the virtual human and the task objective are input into the high-level planner to output sub-objective instructions; Generate a motion sequence according to the sub-target instructions; The robot and the virtual human execute the motion sequence to generate an updated simulation environment; Collect the observable scene state map of the updated simulation environment to obtain a new observable scene state map; Identify whether the task objective in the new observable scene state graph has been completed; If the task objective in the new observable scene state graph is not completed, the step of "inputting the observable scene state graph into the belief model to output the global scene estimation information of the robot and the virtual human" is repeated until the motion sequence of the robot and the virtual human is completed.

2. The method according to claim 1, characterized in that, The step of inputting the observable scene state diagram into the belief model to output global scene estimation information for the robot and the virtual human includes: The scene observation information of the robot and the scene observation information of the virtual human are generated based on the observable scene state diagram. The robot updates the belief model based on the scene observation information of the robot; The robot obtains global scene estimation information by sampling through an updated belief model; The virtual human updates the belief model based on the scene observation information of the virtual human; The virtual human is sampled using an updated belief model to obtain global scene estimation information.

3. The method according to claim 1, characterized in that, The step of inputting the global scene estimation information of the robot and the virtual human and the task objective into the high-level planner to output sub-objective instructions includes: The advanced planner parses the task objective to obtain multiple sub-objectives; Determine the type of the sub-target; If the sub-objective requires the robot and the virtual human to cooperate to complete, then the global scene estimation information of the robot and the virtual human and the sub-objective are input into the Monte Carlo tree search model to output the sub-objective instruction.

4. The method according to claim 3, characterized in that, The step of inputting the global scene estimation information of the robot and the virtual human and the sub-target into the Monte Carlo tree search model to output sub-target instructions includes: Create a Monte Carlo tree search model for the robot and the virtual human, wherein the Monte Carlo tree search model includes multiple layers of nodes, and the multiple layers of nodes record the task rounds of the virtual human or the robot; The task round of the root node in the Monte Carlo tree search model is set as the task round of the virtual human, and the next level task round of the virtual human is set as the task round of the robot. When the virtual human is in a task round, the virtual human derives and generates the sub-objective to be executed by the virtual human and the sub-objective to be executed by the robot in the next round based on the virtual human's global scene estimation information and the sub-objective. When the robot is in a task round, the robot derives and generates the sub-objective to be executed by the robot and the sub-objective to be executed by the virtual human in the next round based on the robot's global scene estimation information and the sub-objective. Obtain the sub-objectives to be executed in each task round of the robot, and generate the sub-objective instructions for the robot; Obtain the sub-goals to be executed in each task round of the virtual human, and generate the sub-goal instructions for the virtual human.

5. The method according to any one of claims 1 to 4, characterized in that, The step of generating a motion sequence according to the sub-target instruction includes: The text information in the sub-target instructions is extracted using a low-level planner based on regular expressions; The text information is recognized to generate action information and item information to be interacted with; A motion sequence is generated based on the action information and the information of the item to be interacted with.

6. A human-machine collaborative task processing device, characterized in that, Applied to computer equipment, including: A creation module is used to create a simulation environment for human-machine collaboration, wherein the simulation environment includes robots and virtual humans; The acquisition module is used to acquire the task objective and collect the observable scene state diagram of the simulation environment in order to determine the progress of the task objective. The first output module is used to input the observable scene state map into the belief model if the task objective is not completed, so as to output the global scene estimation information of the robot and the virtual human. The second output module is used to input the global scene estimation information of the robot and the virtual human and the task objective into the high-level planner to output sub-objective instructions; The first generation module is used to generate a motion sequence according to the sub-target instruction; The second generation module is used for the robot and the virtual human to execute the motion sequence to generate an updated simulation environment; The acquisition module is used to acquire the observable scene state map of the updated simulation environment to obtain a new observable scene state map. The identification module is used to identify whether the task objective in the new observable scene state graph has been completed; The third output module is used to re-execute the step of "inputting the observable scene state diagram into the belief model to output the global scene estimation information of the robot and the virtual human" if the task objective in the new observable scene state diagram is not completed, until the motion sequence of the robot and the virtual human is completed.

7. The apparatus according to claim 6, characterized in that, The first output module includes: The first generation unit is used to generate scene observation information of the robot and scene observation information of the virtual human based on the observable scene state diagram. The first update unit is used for the robot to update the belief model based on the robot's scene observation information; The first sampling unit is used by the robot to sample through the updated belief model to obtain global scene estimation information of the robot; The second updating unit is used for the virtual human to update the belief model based on the scene observation information of the virtual human. The second sampling unit is used for the virtual human to sample through the updated belief model to obtain global scene estimation information of the virtual human.

8. A human-machine collaborative task processing device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the human-computer collaborative task processing method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the human-computer collaborative task processing method as described in any one of claims 1 to 5.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the human-computer collaborative task processing method according to any one of claims 1 to 5.

Images