Virtual machine, motion control method and intelligent agent applied to intelligent agent

Abstract

The present invention discloses a virtual machine, an action control method and an intelligent agent applied to an intelligent agent, and belongs to the field of artificial intelligence technology. The intelligent agent comprises a perception component, an intelligent agent body and an execution component, and a planner and a virtual machine are run on the intelligent agent body; the virtual machine comprises a decoding unit, an interpretation unit and an execution unit, the decoding unit is used to output a first decoded action according to a first action code sent by the planner, and the first action code is a binary code generated by the planner according to data collected by the perception component and a virtual machine specification followed by the virtual machine; the interpretation unit is used to translate the first decoded action into at least one native operation and output the native operation to the execution unit; the execution unit is used to call the execution component to execute the native operation, so that the planner can have universality, and the cross-intelligent agent operation of the same planner can be realized, which is beneficial to reducing the development cost of the intelligent agent and improving the development efficiency.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular, relates to a virtual machine applied to an intelligent body, an action control method and an intelligent body. Background Art

[0002] With the continuous leap forward of artificial intelligence technology, agents have penetrated into every aspect of our lives. Agents are a core concept in the field of artificial intelligence, which refers to entities that can perceive the environment and achieve specific goals by performing actions. Agents exist in various forms, such as smart assistants on mobile phones, smart speakers that can interact with voice, robots that can walk and operate, and drones that fly freely in the air to perform tasks. These agents not only enrich our lives, but also greatly improve work efficiency and convenience.

[0003] An agent usually consists of a perception component, a planner, and an execution component. The perception component perceives information such as sound and images from the environment, and the planner makes plans based on the perceived information and calls the execution component to respond to the perceived information. However, most planners are currently deeply bound to specific agents during design and development, and cannot be migrated to other agents for use. For example, a planner developed for a robot cannot be migrated to an aircraft to drive the aircraft to fly, making it impossible to achieve cross-agent operation of the same planner, which in turn leads to high development costs and low development efficiency for various agents. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a virtual machine, an action control method and an intelligent agent applied to an intelligent agent, which can realize the cross-agent operation of the same planner, which is conducive to reducing the development cost of the intelligent agent and improving the development efficiency.

[0005] In a first aspect, the present application provides a virtual machine applied to an intelligent agent, the intelligent agent comprising a perception component, an intelligent agent body and an execution component, a planner and the virtual machine running on the intelligent agent body, the virtual machine comprising:

[0006] a decoding unit, configured to output a first decoding action according to a first action code sent by the planner, wherein the first action code is a binary code generated by the planner according to data collected by the perception component and a virtual machine specification followed by the virtual machine;

[0007] an interpretation unit, configured to translate the first decoding action into at least one native operation, and output the native operation to an execution unit;

[0008] The execution unit is used to call the execution component to execute the native operation.

[0009] In some embodiments, the decoding unit is specifically used for:

[0010] Detecting whether the data format of the first action code is legal;

[0011] If the data format of the first action code is legal, decoding the first action code;

[0012] When the decoded action name matches the name of an action in the action set supported by the virtual machine, and the decoded action parameters are complete and within a specified range, the action name and the action parameters are output as the first decoded action.

[0013] In some embodiments, the interpretation unit is specifically used to:

[0014] Translating the first decoding action into at least one native operation based on a preset rule, or calling a trained model to translate the first decoding action into at least one native operation;

[0015] The native operation is output to the execution unit.

[0016] In some embodiments, when the native operation is translated from the trained model, the execution unit is further configured to:

[0017] Feedback the execution result of each native operation to the interpretation unit;

[0018] The interpretation unit is also used to control the trained model to optimize model parameters according to the execution result.

[0019] In some embodiments, the virtual machine further includes a scheduling unit, configured to:

[0020] receiving the first decoding action output by the decoding unit;

[0021] According to the currently executed action, the action priority of the first decoding action and the preset action conflict relationship, conflict detection arbitration is performed on the first decoding action to obtain an arbitration result;

[0022] When the arbitration result indicates that the execution is rejected, feeding back the arbitration result to the planner, and not transmitting the first decoding action to the interpretation unit;

[0023] When the arbitration result indicates immediate execution, transmitting the first decoding action to the interpretation unit;

[0024] When the arbitration result indicates queuing for execution, the first decoding action is transmitted to the interpretation unit after the waiting condition is met.

[0025] In some embodiments, the execution unit and the interpretation unit are both further configured to send error information generated during their own work to the planner;

[0026] The decoding unit is further used to output a second decoding action according to a second action code sent by the planner, where the second action code is generated by the planner according to the received feedback information and is used to indicate cancellation of the first decoding action currently being executed, and the feedback information includes the error information;

[0027] The scheduling unit is further configured to control the interpretation unit to stop translating the first decoding action and control the execution unit to stop executing the native operation according to the second decoding action.

[0028] In a second aspect, the present application provides a motion control method applied to an intelligent agent, wherein the intelligent agent includes a perception component, an intelligent agent body, and an execution component, and a planner and any one of the virtual machines described above are running on the intelligent agent body. The motion control method applied to the intelligent agent includes:

[0029] Collecting data of the internal and external environment of the agent through the perception component, and sending the collected data to the planner;

[0030] Generate a first action code by the planner according to the data collected by the perception component and the virtual machine specification followed by the virtual machine, and send the first action code to the decoding unit;

[0031] outputting, by the decoding unit, a first decoding action according to the first action code;

[0032] By means of the interpretation unit, the first decoding action is translated into at least one native operation, and the native operation is sent to the execution unit;

[0033] The execution component is called through the execution unit to execute the native operation.

[0034] In some embodiments, the virtual machine specification includes a basic action set, a plurality of extended action sets, and an interface for detecting whether each of the extended action sets is supported, and the planner generates a first action code according to the data collected by the perception component and the virtual machine specification followed by the virtual machine, specifically including:

[0035] Through the planner, the interface is called to detect the extended action set supported by the virtual machine, and a first action code is generated according to the extended action set supported by the virtual machine, the basic action set and the data collected by the perception component.

[0036] In some embodiments, outputting a first decoding action according to the first action code by the decoding unit includes:

[0037] By means of the decoding unit, detecting whether the data format of the first action code is legal;

[0038] If the data format of the first action code is legal, decoding the first action code;

[0039] When the decoded action name matches the name of an action in the action set supported by the virtual machine, and the decoded action parameters are complete and within a specified range, the action name and the action parameters are output as the first decoded action.

[0040] In some embodiments, translating the first decoding action into at least one native operation by the interpretation unit includes:

[0041] Through the interpretation unit, the first decoding action is translated into at least one native operation based on a preset rule, or the trained model is called to translate the first decoding action into at least one native operation.

[0042] In some embodiments, when the native operation is translated by the trained model, the action control method applied to the agent further includes:

[0043] Feedback the execution result of each native operation to the interpretation unit through the execution unit;

[0044] Through the interpretation unit, the trained model is controlled to optimize model parameters according to the execution result.

[0045] In some embodiments, the virtual machine further includes a scheduling unit, and the motion control method applied to the agent further includes:

[0046] The first decoding action output by the decoding unit is received by the scheduling unit, and conflict detection arbitration is performed on the first decoding action according to the action currently being executed, the action priority of the first decoding action, and a preset action conflict relationship to obtain an arbitration result;

[0047] When the arbitration result indicates that the execution is rejected, the arbitration result is fed back to the planner through the scheduling unit, and the first decoding action is not transmitted to the interpretation unit;

[0048] When the arbitration result indicates immediate execution, transmitting the first decoding action to the interpretation unit through the scheduling unit;

[0049] When the arbitration result indicates queuing for execution, the first decoding action is transmitted to the interpretation unit through the scheduling unit after the waiting condition is met.

[0050] In some embodiments, the motion control method applied to an intelligent agent further includes:

[0051] Sending error information generated during the work of the planner to the planner through the execution unit and the interpretation unit;

[0052] Generate a second action code according to the received feedback information by the planner, and send the second action code to the decoding unit, wherein the second action code is used to indicate cancellation of the first decoding action currently being executed, and the feedback information includes the error information;

[0053] outputting, by the decoding unit, a second decoding action according to the second action code;

[0054] Through the scheduling unit, the interpretation unit is controlled to stop translating the first decoding action according to the second decoding action, and the execution unit is controlled to stop executing the native operation.

[0055] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned motion control methods applied to an intelligent body.

[0056] In a fourth aspect, the present application provides an intelligent agent, including a perception component, an intelligent agent body and an execution component, wherein the intelligent agent body includes a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the program, it implements any of the above-mentioned motion control methods applied to the intelligent agent.

[0057] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned motion control methods applied to an intelligent body.

[0058] The embodiments of the present application provide a virtual machine, an action control method, an intelligent agent, a storage medium and a computer program product applied to an intelligent agent. The intelligent agent includes a perception component, an intelligent agent body and an execution component. A planner and a virtual machine are run on the intelligent agent body; the virtual machine includes a decoding unit, an interpretation unit and an execution unit. The decoding unit is used to output a first decoded action according to a first action code sent by the planner, and the first action code is a binary code generated by the planner according to data collected by the perception component and a virtual machine specification followed by the virtual machine; the interpretation unit is used to translate the first decoded action into at least one native operation and output the native operation to the execution unit; the execution unit is used to call the execution component to execute the native operation, that is, by setting a virtual machine with abstract action translation capability in the intelligent agent, and the virtual machine and the planner interact through action codes, so that the differences between different execution components are transparent to the planner, and the decoupling of the planner from the specific action execution details is realized, so that not only can the planner have universality, but also the cross-agent operation of the same planner can be realized, and the planner can also be developed and trained on a simulation platform and then seamlessly migrated to the intelligent agent in the real world, which is conducive to reducing the development cost of the intelligent agent and improving the development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0060] Figure 1 is a schematic diagram of the structure of an intelligent agent provided in an embodiment of the present application;

[0061] Figure 2 is a schematic diagram of the structure of a virtual machine applied to an intelligent agent provided in an embodiment of the present application;

[0062] Figure 3 It is a schematic diagram showing the combination relationship between the planner, virtual machine and intelligent agent provided in the embodiment of the present application;

[0063] Figure 4 is a flow chart of a motion control method applied to an intelligent body provided in an embodiment of the present application;

[0064] Figure 5 It is another structural diagram of the intelligent agent provided in an embodiment of the present application.

[0065] Description of reference numerals:

[0066] 100, 300 - intelligent agent, 101, 301 - perception component, 102, 302 - intelligent agent body, 103, 303 - execution component, 104 - planner, 105 - virtual machine, 1051 - decoding unit, 1052 - interpretation unit, 1053 - execution unit, 1054 - scheduling unit, 3021 - processor, 3022 - memory. DETAILED DESCRIPTION

[0067] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0068] An intelligent agent usually includes a perception component, a planner, and an execution component. The perception component includes various types of data collectors, such as microphones for collecting sound and cameras for collecting images. The execution component includes motors and solenoid valves that drive mechanical parts (such as wheels and joints), and speakers that control sound. For different types of intelligent agents, the action space of their execution components is quite different. For example, a smart speaker can only respond through a speaker, while a robot can affect the real environment through operations such as walking and grasping. At the same time, different intelligent agents of the same type have different action spaces provided by their actuators due to differences in the simulation environment and the real environment, as well as differences in hardware manufacturers.

[0069] However, due to the differences in the action spaces provided by the actuators of different agents, most planners currently need to be deeply bound to a specific agent during design, that is, they need to be developed and trained based on the action space provided by the actuators of the specific agent, resulting in a planner designed for one agent being difficult to use to drive another agent. For example, a planner developed for a robot cannot be transferred to an aircraft to control the flight of the aircraft. In this case, different agents of the same type and different types of agents need to develop and train their own planners. Therefore, a planner developed for one agent, or the knowledge learned on one agent, cannot be transferred to other agents, resulting in high development costs and low development efficiency for various types of agents.

[0070] To solve at least one of the above technical problems, an embodiment of the present application provides a virtual machine, an action control method, an intelligent agent, a storage medium and a computer program product applied to an intelligent agent.

[0071] See also Figure 1 and Figure 2 , Figure 1 is a schematic diagram of the structure of an intelligent agent provided in an embodiment of the present application, Figure 2 It is a structural diagram of a virtual machine applied to an intelligent agent provided in an embodiment of the present application. Among them, the intelligent agent 100 includes a perception component 101, an intelligent agent body 102 and an execution component 103, and a planner 104 and a virtual machine 105 are running on the intelligent agent body 102. The virtual machine 105 includes: a decoding unit 1051, which is used to output a first decoding action according to a first action code sent by the planner 104, and the first action code is a binary code generated by the planner 104 according to the data collected by the perception component 101 and the virtual machine specification followed by the virtual machine 105; an interpretation unit 1052, which is used to translate the first decoding action into at least one native operation, and output the native operation to the execution unit 1053; the execution unit 1053, which is used to call the execution component 103 to execute the native operation.

[0072] Among them, the planner 104 and the virtual machine 105 run independently in the intelligent body 100, and the two interact through action codes (such as the first action code mentioned above and the second action code described below). The action code can be obtained by encoding in formats such as json and protobuf, and the action code is of indefinite length. The virtual machine specification is similar to the CPU architecture (such as X64, ARM, RISC and other CPU architectures), which mainly includes three parts: a basic action set, several extended action sets and an interface for detecting whether any extended action set is supported by the virtual machine 105, wherein, for the basic action set, all virtual machines 105 that implement the specification must support all actions in the set. For the extended action set, the virtual machine 105 that implements the specification can optionally support these extended action sets, that is, different virtual machines 105 that implement the same virtual machine specification can support different extended action sets. The virtual machines 105 in different intelligent bodies 100 can implement the same or different virtual machine specifications.

[0073] Since there are many types of agents 100, in order to enable the planner 104 to run across agents 100, the planner 104 should not use a specific virtual machine 105 as an action running platform when designing, but should use a certain virtual machine specification as an action running platform, so that the decoupling between the agent 100 and the planner 104 can be achieved based on the virtual machine specification, so that the planner 104 can run on multiple agents 100. For example, see Figure 3 , Figure 3 It is a schematic diagram showing the combination relationship between the planner 104, virtual machine 105 and agent 100 provided in an embodiment of the present application. The virtual machine #1 running in agents #1 and #2 complies with virtual machine specification #1, and the virtual machine #2 running in agent #3 also complies with virtual machine specification #1. Planners #1, #2, #3, and #4 all use virtual machine specification #1 as their action running platform. For these planners 104, they can run on any of agents #1, #2, and #3.

[0074] Specifically, the perception component 101 is mainly used to collect data from the internal and external environment of the agent 100, and send the collected data to the planner 104. The perception component 101 may include various types of data collectors, such as microphones for collecting sound, cameras for collecting images, etc. The execution component 103 may include motors and solenoid valves for driving mechanical parts (such as wheels, joints, etc.), speakers for controlling sound, etc. Different agents 100 may be provided with different perception components 101 and execution components 103.

[0075] The planner 104 is mainly used to plan actions. The actions used in planning can be divided into a strongly dependent action set and an optional action set. The actions in the strongly dependent action set are strongly dependent on the planner 104, and the absence of these actions will cause the planner 104 to fail to run. The actions in the optional action set are only dependent on a certain optional function of the planner 104, or when missing, the planner 104 can replace them in other ways (for example, by combining the actions in the strongly dependent action set). At the same time, the actions supported by the virtual machine 105 can be divided into a streamlined action set and a complex action set. The actions in the streamlined action set are orthogonal to each other and are indivisible minimum units. Here, "orthogonal" means that each action in the set is independent, and there is no overlap or redundancy between each other. The effect of the actions in the complex action set is equivalent to a sequence composed of the actions in the streamlined action set.

[0076] Based on the above division of action sets, the action sets can be used to evaluate the compatibility of the virtual machine 105 (agent 100) and the planner 104, and the versatility of the planner 104. Generally, if the planner 104 is to be able to run across agents 100, the planner 104 needs to be compatible with the virtual machine 105 in the agent 100. When the action set supported by the virtual machine 105 includes the strongly dependent action set of the planner 104, the virtual machine 105 is compatible with the planner 104. Further, when the intersection of the action set supported by the action virtual machine 105 and the optional action set of the planner 104 is not empty, the planner 104 can support additional functions or achieve better results by using these actions in the intersection. For example, if the strongly dependent action set of a planner 104 includes movement actions, and the optional action set includes flight actions, it means that the planner 104 must run on an agent 100 with movement capabilities, but will support additional functions or perform better on an agent 100 with flight capabilities.

[0077] When each planner 104 performs action planning, it needs to first confirm which actions are supported by the virtual machine 105 in the agent 100, such as calling the interface provided by the virtual machine specification to detect which extended action sets are supported by the virtual machine 105. The actions supported by the virtual machine 105 are the basic action set and all the actions in the detected extended action set. These actions can be used as the actions used by the planner 104. At the same time, the planner 104 should give priority to the actions in the streamlined action set. Only when the planner 104 cannot complete a complex task or the effect is poor due to defects in planning speed, accuracy, computing resources, etc., can the action in the complex action set be used to reduce the planning difficulty.

[0078] In some embodiments, the decoding unit 1051 is specifically used for:

[0079] Detecting whether the data format of the first action code is legal;

[0080] If the data format of the first action code is legal, decoding the first action code;

[0081] When the decoded action name matches the name of the action in the action set supported by the virtual machine 105, and the decoded action parameters are complete and within the specified range, the action name and the action parameters are output as the first decoded action.

[0082] Specifically, each action code (such as the first action code mentioned above and the second action code mentioned below) involves an action name and action parameters. The structure of the action code is shown in ActionCode:

[0083] struct ActionCode {

[0084] uint64_t id;

[0085] std::string name;

[0086] std::string args;

[0087] };

[0088] Among them, id is the unique identifier of the action code, name is the action name, and args is the action parameter. Different actions have different parameters, such as the parameter of TurnAction is the degree, and the parameter of MoveAction is the coordinate. It should be pointed out that for Cancel Action, its action parameter is the id of the action to be canceled.

[0089] The decoding unit 1051 is responsible for decoding the action code into an action (such as TurnAction or MoveAction mentioned above) and action parameters. It is responsible for checking whether the action code format is legal (i.e., whether it conforms to the structure of ActionCode mentioned above), whether the action name is supported (i.e., whether it matches the name of the action in the action set supported by the virtual machine 105), whether the action parameters are missing and within the specified range, and giving the specific error cause when an error is detected. If there is no error, the decoding is successful.

[0090] In some embodiments, the interpretation unit 1052 is specifically used to:

[0091] Translate the first decoding action into at least one native operation based on a preset rule, or call a trained model to translate the first decoding action into at least one native operation;

[0092] The native operation is output to the execution unit 1053 .

[0093] Among them, native operations are operations that can be directly executed by agent 100 (executor component 103). Agent 100 affects the environment and its own state by executing native operations. For example, assuming that agent A is a robot with a full body and agent B is an aircraft, agent A can provide native operations such as walking, and agent B can provide native operations such as flying. Decoding actions (such as the first decoding action mentioned above) are abstract actions defined by the virtual machine specification, which abstract the execution details of the action. Decoding actions need to be translated into native operations before they can be executed. For example, the moving action is translated into the native operation "walking" on agent A and translated into the native operation "flying" on agent B.

[0094] The preset rules and the trained models need to be set in advance, and are both used to translate abstract actions into native operations, wherein the preset rules are set according to the actions in the virtual machine specification followed by the virtual machine 105 and the actual execution component 103 in the agent 100, and the trained model needs to be manually set up with some sample training. By translating the decoded actions into native operations, the differences in the action spaces of different execution components 103 can be made transparent to the planner 104, and the planner 104 can be decoupled from the specific action execution details.

[0095] For example, for the two intelligent agents 100, a bipedal robot and an aircraft, when the planner 104 requires the intelligent agent 100 to move to a certain position, the bipedal robot does so by walking, and the aircraft does so by flying. If the planner 104 directly controls the robot hardware, then two planners 104 need to be developed for these two intelligent agents 100. By using the action virtual machine 105, the planner 104 only needs to generate an abstract movement action, and the virtual machine 105 is responsible for translating the abstract movement action into an operation on specific hardware (walking or flying). In this way, as long as the virtual machine 105 of the intelligent agent 100 implements the action required by the planner 104, the planner 104 can run on the intelligent agent 100.

[0096] At the same time, since the planner 104 and the intelligent agent 100 are decoupled, the planner 104 can also be developed and trained on the simulation platform, and then seamlessly migrated to the intelligent agent 100 in the real world after development, thereby reducing development and training costs and improving development and training efficiency.

[0097] In some embodiments, when the native operation is translated by the trained model, the execution unit 1053 is further used to: feedback the execution result of each native operation to the interpretation unit 1052. At this time, the interpretation unit 1052 is also used to control the trained model to optimize the model parameters according to the execution result.

[0098] Among them, the execution unit 1053 needs to record the execution status of each native operation, and each native operation has 4 execution states: queue, start, success, and failure, among which success and failure are terminal states. When the native operation enters the failure state, the specific error code and the cause of the error need to be recorded. The decoding action is considered to be executed only when and only when all native operations interpreted by the interpretation unit 1052 reach the terminal state. The execution unit 1053 can feedback the execution result of each native operation to the interpretation unit 1052, and the trained model can optimize its own model parameters based on the execution results, that is, the interpretation unit 1052 can use the execution results of the execution unit 1053 to dynamically optimize the interpretation strategy to adapt to the hardware parameter differences of different intelligent agents 100.

[0099] Furthermore, the virtual machine 105 further includes a scheduling unit 1054, which is used to:

[0100] Receiving a first decoding action output by the decoding unit 1051;

[0101] According to the currently executed action, the action priority of the first decoding action and the preset action conflict relationship, conflict detection arbitration is performed on the first decoding action to obtain an arbitration result;

[0102] When the arbitration result indicates that the execution is rejected, the arbitration result is fed back to the planner 104, and the first decoding action is not transmitted to the interpretation unit 1052;

[0103] When the arbitration result indicates immediate execution, transmitting the first decoding action to the interpretation unit 1052;

[0104] When the arbitration result indicates queuing for execution, the first decoding action is transmitted to the interpretation unit 1052 after the waiting condition is met.

[0105] Among them, the interpretation and execution of different decoding actions are performed concurrently, and the interpretation and execution of the same decoding action are also performed concurrently, that is, when the interpretation unit 1052 only interprets part of the native operations, the execution unit 1053 can start to execute these native operations, thereby giving full play to the hardware capabilities of the intelligent agent 100. For the decoding actions that are successfully decoded, the decoding unit 1051 can submit them to the scheduling unit 1054, and the scheduling unit 1054 is responsible for conflict detection arbitration for the received decoding actions according to the priority of the actions, the actions in progress, and the inherent conflict relationship between the actions.

[0106] There are three arbitration results: immediate execution, queue waiting for execution, and return to refuse execution. Usually, when there is no conflict between the decoding action and the running action, the arbitration result is immediate execution. At this time, the scheduling unit 1054 immediately transmits the first decoding action to the interpretation unit 1052. When there is a conflict between the decoding action and the action being run, the scheduling unit 1054 can generate an arbitration result of returning a refusal to execute or queuing for execution based on the agreed rules. If the arbitration result is queuing for execution, the scheduling unit 1054 can wait for the condition to be met (for example, the action being run is completed) before transmitting the first decoding action to the interpretation unit 1052; if the arbitration result is a refusal to execute, there is no need to further interpret the decoding action at this time, that is, the scheduling unit 1054 does not need to transmit the first decoding action to the interpretation unit 1052, and directly returns the arbitration result to the planner 104, and the planner 104 will further decide whether to retry the action based on the arbitration result. If retry is required, the conflict relationship needs to be resolved. To this end, the planner 104 can wait for the execution of the action being run to be completed and resend the corresponding action code for retry, or it can also cancel the action being run (for example, the planner 104 generates an action code indicating the cancellation of the action being run and sends it to the virtual machine 105) for retry, or it can be other ways, which are not limited here.

[0107] In some embodiments, the execution unit 1053 and the interpretation unit 1052 are also used to send error information generated during their own work to the planner 104;

[0108] The decoding unit 1051 is further configured to output a second decoding action according to a second action code sent by the planner 104, wherein the second action code is generated by the planner 104 according to the received feedback information and is used to indicate cancellation of the first decoding action currently being executed, wherein the feedback information includes the error information;

[0109] The scheduling unit 1054 is further used to control the interpretation unit 1052 to stop translating the first decoding action according to the second decoding action, and to control the execution unit 1053 to stop executing the native operation.

[0110] Among them, the interpretation unit 1052 may generate errors during the interpretation of the action, and the execution unit 1053 may generate errors during the execution of the native operation. For example, for a grasping action, if other objects are already held in the hand, the interpretation unit 1052 will generate an error. If the object fails to be picked up because it is too far from the hand or too heavy, the execution unit 1053 will generate an error. The interpretation unit 1052 or the execution unit 1053 will feed back the error information generated during its own work to the planner 104, so that the planner 104 can respond to environmental changes in a timely manner.

[0111] During the execution of native operations by the execution unit 1053, if the planner 104 receives an error message sent by the virtual machine 105, or receives other feedback information, such as receiving environmental change information, resulting in the previously planned action no longer needing to be executed, or receiving a higher priority task and target, and needs to terminate the previously planned action to execute a higher-level task, the planner 104 usually generates an action code (such as the second action code mentioned above) indicating the cancellation of a certain action and sends it to the virtual machine 105. The action parameter obtained after decoding is the action ID (name) that needs to be canceled. This cancellation action does not need to be further sent to the interpretation unit 1052 for interpretation after decoding, but is directly sent to the scheduling unit 1054. When the scheduling unit 1054 receives the cancellation action, it will immediately cancel the action associated with the action ID. After the action is canceled, the action will stop running, that is, the interpretation unit 1052 stops interpreting the action, and the execution unit 1053 stops executing the native operation that has been interpreted but not submitted, and cancels the native operation that has been submitted for execution.

[0112] As can be seen from the above, the virtual machine 105 applied to the intelligent agent 100 provided in the embodiment of the present application, the intelligent agent 100 includes a perception component 101, an intelligent agent body 102 and an execution component 103, and a planner 104 and a virtual machine 105 are running on the intelligent agent body 102; the virtual machine 105 includes a decoding unit 1051, an interpretation unit 1052 and an execution unit 1053, the decoding unit 1051 is used to output a first decoding action according to a first action code sent by the planner 104, and the first action code is a binary code generated by the planner 104 according to the data collected by the perception component 101 and the virtual machine specification followed by the virtual machine 105; the interpretation unit 1052 is used to translate the first decoding action into at least one native operation, and send it to the execution unit 1053. The element 1053 outputs the native operation; the execution unit 1053 is used to call the execution component 103 to execute the native operation, that is, by setting a virtual machine 105 with the ability to translate abstract actions in the intelligent agent 100, and the virtual machine 105 and the planner 104 interact through action codes, so that the differences between different execution components 103 are transparent to the planner 104, and the decoupling of the planner 104 from the specific action execution details is realized, so that not only the planner 104 can be universal, but also the same planner 104 can be run across intelligent agents 100, and the planner 104 can be developed and trained on the simulation platform and then seamlessly migrated to the intelligent agent 100 in the real world, which is conducive to reducing the development cost of the intelligent agent 100 and improving the development efficiency. Furthermore, by enabling the virtual machine 105 to support the concurrent operation of multiple actions, and supporting the concurrent operation of the interpretation unit 1052 and the execution unit 1053 of the same action, the hardware capabilities of the intelligent agent 100 can be fully utilized.

[0113] Furthermore, by allowing the interpretation unit 1052 and the execution unit 1053 to feed back the errors generated during their own work to the planner 104 in real time, the planner 104 can respond to environmental changes in a timely manner, thereby improving the intelligence of the agent 100. Furthermore, by allowing the interpretation unit 1052 to dynamically optimize its own interpretation strategy using the execution results of the execution unit 1053, the virtual machine 105 can adapt to the hardware parameter differences of different agents 100, improve the translation accuracy of the action, and further improve the action control accuracy of the agent 100.

[0114] According to the virtual machine described in the above embodiment, the present application embodiment also provides a method for controlling the motion of an intelligent agent. Figure 4 , Figure 41 is a flow chart of a motion control method applied to an intelligent agent provided in an embodiment of the present application. The intelligent agent includes a perception component, an intelligent agent body, and an execution component. A planner and a virtual machine described in any of the above embodiments are run on the intelligent agent body. The motion control method applied to the intelligent agent can be executed by a functional module or functional entity in the intelligent agent. Specifically, the motion control method applied to the intelligent agent may include the following steps 201-205, wherein:

[0115] 201. Collect data of the internal and external environment of the agent through the perception component, and send the collected data to the planner;

[0116] 202. Generate a first action code through the planner according to the data collected by the perception component and the virtual machine specification followed by the virtual machine, and send the first action code to the decoding unit;

[0117] 203. Outputting a first decoding action according to the first action code through the decoding unit;

[0118] 204. Translate the first decoding action into at least one native operation through the interpretation unit, and send the native operation to the execution unit;

[0119] 205. Call the execution component through the execution unit to execute the native operation.

[0120] In some embodiments, the virtual machine specification includes a basic action set, several extended action sets, and an interface for detecting whether each extended action set is supported. In this case, the above step 202 may specifically include:

[0121] Through the planner, the interface is called to detect the extended action set supported by the virtual machine, and a first action code is generated according to the extended action set supported by the virtual machine, the basic action set and the data collected by the perception component.

[0122] In some embodiments, the above step 203 may specifically include:

[0123] By means of the decoding unit, detecting whether the data format of the first action code is legal;

[0124] If the data format of the first action code is legal, decoding the first action code;

[0125] When the decoded action name matches the name of the action in the action set supported by the virtual machine, and the decoded action parameters are complete and within the specified range, the action name and the action parameters are output as the first decoded action.

[0126] In some embodiments, the above step 204 may specifically include:

[0127] Through the interpretation unit, the first decoding action is translated into at least one native operation based on a preset rule, or a trained model is called to translate the first decoding action into at least one native operation.

[0128] In some embodiments, when the native operation is translated by the trained model, the action control method applied to the agent may further include:

[0129] Feedback the execution result of each native operation to the interpretation unit through the execution unit;

[0130] Through the interpretation unit, the trained model is controlled to optimize model parameters according to the execution result.

[0131] In some embodiments, the virtual machine further includes a scheduling unit, and the motion control method applied to the agent may further include:

[0132] The scheduling unit receives the first decoding action output by the decoding unit, and performs conflict detection arbitration on the first decoding action according to the currently executed action, the action priority of the first decoding action, and the preset action conflict relationship to obtain an arbitration result;

[0133] When the arbitration result indicates that the execution is rejected, the arbitration result is fed back to the planner through the scheduling unit, and the first decoding action is not transmitted to the interpretation unit;

[0134] When the arbitration result indicates immediate execution, transmitting the first decoding action to the interpretation unit through the scheduling unit;

[0135] When the arbitration result indicates queuing for execution, the first decoding action is transmitted to the interpretation unit through the scheduling unit after the waiting condition is met.

[0136] In some embodiments, the motion control method applied to an intelligent agent may further include:

[0137] Sending error information generated during the work of the planner to the planner through the execution unit and the interpretation unit;

[0138] Generate a second action code according to the received feedback information through the planner, and send the second action code to the decoding unit, wherein the second action code is used to indicate cancellation of the first decoding action currently being executed, and the feedback information includes the error information;

[0139] outputting, by the decoding unit, a second decoding action according to the second action code;

[0140] Through the scheduling unit, the interpretation unit is controlled to stop translating the first decoding action according to the second decoding action, and the execution unit is controlled to stop executing the native operation.

[0141] It should be noted that the specific details of each module unit of the virtual machine in the above-mentioned motion control method applied to the intelligent body have been described in detail in the above-mentioned virtual machine embodiment, and will not be repeated here.

[0142] As can be seen from the above, the action control method applied to an intelligent agent provided in the embodiment of the present application, the intelligent agent includes a perception component, an intelligent agent body and an execution component, a planner and a virtual machine are running on the intelligent agent body, and the virtual machine includes a decoding unit, an interpretation unit and an execution unit; through the perception component, data in the internal and external environment of the intelligent agent is collected, and the collected data is sent to the planner; through the planner, a first action code is generated according to the data collected by the perception component and the virtual machine specification followed by the virtual machine, and the first action code is sent to the decoding unit; through the decoding unit, a first decoded action is output according to the first action code sent by the planner; through the interpretation unit, the first The decoded action is translated into at least one native operation, and the native operation is sent to the execution unit; through the execution unit, the execution component is called to execute the native operation, that is, by setting a virtual machine with abstract action translation capability in the agent, and the virtual machine and the planner interact through action codes, the differences between different execution components are transparent to the planner, and the planner is decoupled from the specific action execution details, which not only makes the planner universal and can realize cross-agent operation of the same planner, but also the planner can be developed and trained on the simulation platform and then seamlessly migrated to the real-world agent, which is conducive to reducing the development cost of the agent and improving development efficiency.

[0143] In some embodiments, Figure 5 As shown, the embodiment of the present application also provides an intelligent agent 300, including a perception component 301, an intelligent agent body 302 and an execution component 303, wherein the intelligent agent body 302 includes a processor 3021, a memory 3022 and a computer program stored in the memory 3022 and executable on the processor 3021, and when the program is executed by the processor 3021, each process of the above-mentioned embodiment of the motion control method applied to the intelligent agent is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0144] It should be noted that the agents in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices. Those skilled in the art will understand that Figure 5 The intelligent agent structure shown in the figure does not constitute a limitation of the intelligent agent. The intelligent agent may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components, which will not be repeated here.

[0145] The memory 3022 can be used to store software programs and various data. The memory 3022 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 3022 may include a volatile memory or a non-volatile memory, or the memory 3022 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 3022 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0146] The processor 3021 may include one or more processing units; the processor 3021 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 3021.

[0147] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned embodiment of the action control method applied to an intelligent body are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0148] The processor is the processor in the agent described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0149] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned action control method applied to an intelligent body.

[0150] The processor is the processor in the agent described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0151] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0153] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0154] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0155] In the description of the present application, “plurality” means two or more.

[0156] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0157] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A virtual machine applied to an intelligent agent, characterized in that: The intelligent agent includes a perception component, an intelligent agent body and an execution component. A planner and the virtual machine are running on the intelligent agent body. The virtual machine includes: a decoding unit, configured to output a first decoding action according to a first action code sent by the planner, wherein the first action code is a binary code generated by the planner according to the data collected by the perception component and the virtual machine specification followed by the virtual machine, the planner uses the virtual machine specification as an action running platform, the virtual machine specification includes a basic action set, a plurality of extended action sets and an interface for detecting whether any extended action set is supported by the virtual machine, and all virtual machines implementing the virtual machine specification must support all actions in the basic action set and optionally support actions in the extended action set; an interpretation unit, configured to translate the first decoding action into at least one native operation, and output the native operation to an execution unit; The execution unit is used to call the execution component to execute the native operation.

2. The virtual machine applied to an intelligent agent according to claim 1, characterized in that: The decoding unit is specifically used for: Detecting whether the data format of the first action code is legal; If the data format of the first action code is legal, decoding the first action code; When the decoded action name matches the name of an action in the action set supported by the virtual machine, and the decoded action parameters are complete and within a specified range, the action name and the action parameters are output as the first decoded action.

3. The virtual machine applied to an intelligent agent according to claim 1, characterized in that: The interpretation unit is specifically used for: Translating the first decoding action into at least one native operation based on a preset rule, or calling a trained model to translate the first decoding action into at least one native operation; The native operation is output to the execution unit.

4. The virtual machine applied to an intelligent agent according to claim 3, characterized in that: When the native operation is translated from the trained model, the execution unit is further used for: Feedback the execution result of each native operation to the interpretation unit; The interpretation unit is also used to control the trained model to optimize model parameters according to the execution result.

5. The virtual machine applied to an intelligent agent according to any one of claims 1 to 4, characterized in that: The virtual machine also includes a scheduling unit, which is used to: receiving the first decoding action output by the decoding unit; According to the currently executed action, the action priority of the first decoding action and the preset action conflict relationship, conflict detection arbitration is performed on the first decoding action to obtain an arbitration result; When the arbitration result indicates that the execution is rejected, feeding back the arbitration result to the planner, and not transmitting the first decoding action to the interpretation unit; When the arbitration result indicates immediate execution, transmitting the first decoding action to the interpretation unit; When the arbitration result indicates queuing for execution, the first decoding action is transmitted to the interpretation unit after the waiting condition is met.

6. The virtual machine applied to an intelligent agent according to claim 5, characterized in that: The execution unit and the interpretation unit are both used to send error information generated during their own work to the planner; The decoding unit is further used to output a second decoding action according to a second action code sent by the planner, where the second action code is generated by the planner according to the received feedback information and is used to indicate cancellation of the first decoding action currently being executed, and the feedback information includes the error information; The scheduling unit is further configured to control the interpretation unit to stop translating the first decoding action and control the execution unit to stop executing the native operation according to the second decoding action.

7. A motion control method applied to an intelligent agent, characterized in that: The intelligent agent includes a perception component, an intelligent agent body and an execution component, a planner and a virtual machine as described in any one of claims 1 to 6 are running on the intelligent agent body, and the action control method applied to the intelligent agent includes: Collecting data of the internal and external environment of the agent through the perception component, and sending the collected data to the planner; Generate a first action code by the planner according to the data collected by the perception component and the virtual machine specification followed by the virtual machine, and send the first action code to the decoding unit; outputting, by the decoding unit, a first decoding action according to the first action code; By means of the interpretation unit, the first decoding action is translated into at least one native operation, and the native operation is sent to the execution unit; The execution component is called through the execution unit to execute the native operation.

8. The motion control method applied to an intelligent agent according to claim 7, characterized in that: The generating of the first action code according to the data collected by the perception component and the virtual machine specification followed by the virtual machine by the planner specifically includes: Through the planner, the interface is called to detect the extended action set supported by the virtual machine, and a first action code is generated according to the extended action set supported by the virtual machine, the basic action set and the data collected by the perception component.

9. The motion control method applied to an intelligent agent according to claim 7, characterized in that: Outputting a first decoding action according to the first action code by the decoding unit includes: By means of the decoding unit, detecting whether the data format of the first action code is legal; If the data format of the first action code is legal, decoding the first action code; When the decoded action name matches the name of an action in the action set supported by the virtual machine, and the decoded action parameters are complete and within a specified range, the action name and the action parameters are output as the first decoded action.

10. The motion control method applied to an intelligent body according to claim 7, characterized in that: The step of translating the first decoding action into at least one native operation by the interpretation unit includes: Through the interpretation unit, the first decoding action is translated into at least one native operation based on a preset rule, or the trained model is called to translate the first decoding action into at least one native operation.

11. The motion control method applied to an intelligent agent according to claim 10, characterized in that: When the native operation is translated by the trained model, the action control method applied to the agent further includes: Feedback the execution result of each native operation to the interpretation unit through the execution unit; Through the interpretation unit, the trained model is controlled to optimize model parameters according to the execution result.

12. The motion control method applied to an intelligent agent according to any one of claims 7 to 11, characterized in that: The virtual machine further includes a scheduling unit, and the motion control method applied to the agent further includes: The first decoding action output by the decoding unit is received by the scheduling unit, and conflict detection arbitration is performed on the first decoding action according to the action currently being executed, the action priority of the first decoding action, and a preset action conflict relationship to obtain an arbitration result; When the arbitration result indicates that the execution is rejected, the arbitration result is fed back to the planner through the scheduling unit, and the first decoding action is not transmitted to the interpretation unit; When the arbitration result indicates immediate execution, transmitting the first decoding action to the interpretation unit through the scheduling unit; When the arbitration result indicates queuing for execution, the first decoding action is transmitted to the interpretation unit through the scheduling unit after the waiting condition is met.

13. The motion control method applied to an intelligent agent according to claim 12, characterized in that: The motion control method applied to the intelligent body also includes: Sending error information generated during the work of the planner to the planner through the execution unit and the interpretation unit; Generate a second action code according to the received feedback information by the planner, and send the second action code to the decoding unit, wherein the second action code is used to indicate cancellation of the first decoding action currently being executed, and the feedback information includes the error information; outputting, by the decoding unit, a second decoding action according to the second action code; Through the scheduling unit, the interpretation unit is controlled to stop translating the first decoding action according to the second decoding action, and the execution unit is controlled to stop executing the native operation.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the action control method applied to an intelligent body as described in any one of claims 7 to 13 is implemented.

15. An intelligent agent, characterized in that: It includes a perception component, an intelligent body and an execution component. The intelligent body includes a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the motion control method applied to the intelligent body as described in any one of claims 7 to 13.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the action control method applied to an intelligent body as described in any one of claims 7 to 13 is implemented.

Images