Heterogeneous agent motion remapping expression method and device and electronic equipment

Abstract

The invention provides a heterogeneous agent motion remapping expression method and device and electronic equipment, and the method comprises the steps: constructing the ontology motion features and ontology environment state features of a source agent based on the motion state of the source agent in a preset motion scene; calling an optimal joint matching matrix between the source agent and the target heterogeneous agent; according to the body motion characteristics of the source agent at the adjacent moments and the optimal joint matching matrix, remapping the body motion characteristics of the source agent to obtain the mapped motion characteristics of the target heterogeneous agent; remapping the ontology environment state characteristics of the source agent according to the ontology environment state characteristics of the source agent to obtain mapped environment state characteristics of the target heterogeneous agent; and based on the mapped motion features and the mapped environment state features of the target heterogeneous agent, obtaining a motion remapping expression of the target heterogeneous agent about the source agent. The action of the source agent can be accurately remapped and expressed.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, and electronic device for representing motion remapping of heterogeneous intelligent agents. Background Art

[0002] The ability of heterogeneous intelligent agents, such as robots / humanoid robots, to accurately recreate the actions of source intelligent agents has significant practical implications for fields such as robotics, animation, and healthcare.

[0003] According to relevant technologies, in the current process of remapping the motion of heterogeneous intelligent agents, only the remapping of the agent's motion is considered, without considering the impact of motion on the environmental state, thus failing to accurately remap the actions of the source intelligent agent.

[0004] Therefore, finding a heterogeneous agent motion remapping representation method that can accurately remap the actions of the source agent has become a research hotspot. Summary of the Invention

[0005] This invention provides a method, apparatus, and electronic device for remapping motion representation of heterogeneous intelligent agents, which enables accurate remapping representation of the actions of the source intelligent agent.

[0006] This invention provides a method for remapping motion representation of heterogeneous intelligent agents. The method includes: constructing ontological motion features and ontological environment state features of the source intelligent agent based on the motion state of the source intelligent agent in a preset motion scenario; calling a pre-determined optimal joint matching matrix between the source intelligent agent and the target heterogeneous intelligent agent; remapping the ontological motion features of the source intelligent agent according to the ontological motion features of the source intelligent agent at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous intelligent agent; remapping the ontological environment state features of the source intelligent agent according to the ontological environment state features of the source intelligent agent to obtain the mapped environment state features of the target heterogeneous intelligent agent; and obtaining the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent based on the mapped motion features and the mapped environment state features of the target heterogeneous intelligent agent.

[0007] According to a method for remapping motion representation of heterogeneous intelligent agents provided by the present invention, the adjacent time points include a previous time point and a subsequent time point; the step of remapping the ontology motion features of the source intelligent agent based on the ontology motion features of the source intelligent agent at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous intelligent agent specifically includes: determining the amount of motion feature change based on the ontology motion features of the source intelligent agent at the subsequent time point and the ontology motion features of the source intelligent agent at the previous time point; and remapping the ontology motion features of the source intelligent agent based on the amount of motion feature change and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous intelligent agent.

[0008] According to a method for remapping motion representation of heterogeneous intelligent agents provided by the present invention, the remapping of the ontological environmental state features of the source intelligent agent based on the ontological environmental state features of the source intelligent agent to obtain the mapped environmental state features of the target heterogeneous intelligent agent specifically includes: determining a first environmental feature based on the ontological environmental state features of the source intelligent agent, wherein the first environmental feature is used to characterize the environmental features generated when the target heterogeneous intelligent agent moves according to the ontological environmental state features of the source intelligent agent and uses the end effector joints of the target heterogeneous intelligent agent to control the target object; determining a second environmental feature based on the difference between the first environmental feature and the ontological environmental state features of the source intelligent agent; and remapping the ontological environmental state features of the source intelligent agent according to the eigenvector norm of the second environmental feature to obtain the mapped environmental state features of the target heterogeneous intelligent agent.

[0009] According to a heterogeneous agent motion remapping representation method provided by the present invention, the optimal joint matching matrix between the source agent and the target heterogeneous agent is constructed in the following manner: obtaining a first kinematic graph model representation of the source agent and a second kinematic graph model representation of the target heterogeneous agent; constructing constraints, wherein the constraints include the element value range of any element in the optimal joint matching matrix being between 0 and 1; based on the first kinematic graph model representation and the second kinematic graph model representation, the optimal joint matching matrix between the source agent and the target heterogeneous agent is constructed through graph matching and the constraints, wherein the optimal joint matching matrix is ​​used to characterize the optimal matching degree between each joint of the source agent and each joint of the target heterogeneous agent.

[0010] According to a heterogeneous intelligent agent motion remapping representation method provided by the present invention, the first kinematic structure graph model representation of the source intelligent agent is constructed in the following manner: obtaining the first joint set in the first kinematic chain of the source intelligent agent, the connection relationship of each first joint of the source intelligent agent, and the first end-effector set in the first kinematic chain of the source intelligent agent; and constructing the first kinematic structure graph model representation of the source intelligent agent based on the first joint set, the connection relationship of each first joint, and the first end-effector set.

[0011] According to a method for remapping motion representation of heterogeneous intelligent agents provided by the present invention, the second kinematic structure graph model representation of the target heterogeneous intelligent agent is constructed in the following manner: obtaining the set of second joints in the second kinematic chain of the target heterogeneous intelligent agent, the connection relationship of each second joint of the target heterogeneous intelligent agent, and the set of second end-effector joints in the second kinematic chain of the target heterogeneous intelligent agent; and constructing the second kinematic structure graph model representation of the target heterogeneous intelligent agent based on the set of second joints, the connection relationship of each second joint, and the set of second end-effector joints.

[0012] This invention also provides a heterogeneous intelligent agent motion remapping representation device, the device comprising: a construction module, used to construct the ontological motion features and ontological environmental state features of the source intelligent agent based on the motion state of the source intelligent agent in a preset motion scenario; a calling module, used to call a pre-determined optimal joint matching matrix between the source intelligent agent and the target heterogeneous intelligent agent; a motion feature remapping module, used to remap the ontological motion features of the source intelligent agent according to the ontological motion features of the source intelligent agent at adjacent time points and the optimal joint matching matrix, to obtain the mapped motion features of the target heterogeneous intelligent agent; an environmental state feature remapping module, used to remap the ontological environmental state features of the source intelligent agent according to the ontological environmental state features of the source intelligent agent, to obtain the mapped environmental state features of the target heterogeneous intelligent agent; and a generation module, used to obtain the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent based on the mapped motion features and the mapped environmental state features of the target heterogeneous intelligent agent.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the heterogeneous intelligent agent motion remapping representation method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the heterogeneous intelligent agent motion remapping representation method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the heterogeneous intelligent agent motion remapping representation method as described above.

[0016] The present invention provides a method, apparatus, and electronic device for heterogeneous intelligent agent motion remapping representation. Based on the motion state of the source intelligent agent in a preset motion scenario, it constructs the source intelligent agent's ontological motion features and ontological environmental state features. It then invokes a pre-determined optimal joint matching matrix between the source intelligent agent and the target heterogeneous intelligent agent. Based on the source intelligent agent's ontological motion features at adjacent time points and the optimal joint matching matrix, it remaps the source intelligent agent's ontological motion features to obtain the mapped motion features of the target heterogeneous intelligent agent. Based on the source intelligent agent's ontological environmental state features, it remaps the source intelligent agent's ontological environmental state features to obtain the mapped environmental state features of the target heterogeneous intelligent agent. Finally, based on the mapped motion features and the mapped environmental state features of the target heterogeneous intelligent agent, it obtains the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent. In this invention, the remapping of ontological motion and the influence of motion on the environmental state are comprehensively considered, achieving accurate remapping representation of the source intelligent agent's actions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the heterogeneous intelligent agent motion remapping representation method provided by the present invention;

[0019] Figure 2 This is a flowchart illustrating the process of remapping the ontological environmental state features of the source intelligent agent based on the ontological environmental state features of the source intelligent agent to obtain the mapped environmental state features of the target heterogeneous intelligent agent, as provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the process for constructing the optimal joint matching matrix between the source intelligent agent and the target heterogeneous intelligent agent provided by the present invention;

[0021] Figure 4 This is a schematic diagram of the heterogeneous intelligent agent motion remapping expression device provided by the present invention;

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Figure 1 This is a flowchart illustrating the heterogeneous intelligent agent motion remapping representation method provided by the present invention.

[0025] The following will combine Figure 1 The process of the heterogeneous intelligent agent motion remapping representation method provided by the present invention will be described.

[0026] In an exemplary embodiment of the present invention, combined with Figure 1 As can be seen, the heterogeneous intelligent agent motion remapping representation method can include steps 110 to 150, and each step will be described below.

[0027] In step 110, based on the motion state of the source agent in the preset motion scenario, the ontological motion features and ontological environmental state features of the source agent are constructed.

[0028] In one embodiment, the preset motion scenario can be adjusted according to the actual situation. For example, it can be a source agent, such as a human using their hands or feet to change the state of objects in the environment, like kicking a ball. The source agent can be considered as the agent that originally generated the motion state.

[0029] In another embodiment, ontological motion features and ontological environment state features of the source agent can be constructed based on the motion state of the source agent in a preset motion scenario. The ontological motion features can be used to characterize the ontological joint features formed by the source agent's own motion. In one example, the ontological motion features... It can include the rotational representation of each joint (represented by quaternions) and the pose information of the root joint (position represented by three-dimensional coordinates and pose represented by quaternions).

[0030] Ontological environment state features can be used to characterize the ontological joint features and the state change features of the target object formed after the source agent comes into contact with and moves with the target object. Among these, ontological environment state features... It can include the pose information of the root joint, the pose information of the key object (corresponding to the target object), and the contact information between the key object and the end effector joint of the agent.

[0031] In step 120, the pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent is invoked.

[0032] In another embodiment, a pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent can be invoked. The target heterogeneous agent can be used to characterize the heterogeneous agent of the source agent.

[0033] In yet another example, the optimal joint matching matrix X can be used to characterize the optimal matching degree between each joint of the source agent and each joint of the target heterogeneous agent.

[0034] In step 130, the ontological motion features of the source agent are remapped based on the ontological motion features of the source agent at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous agent.

[0035] In yet another embodiment, the proprioceptive motion characteristics of the source agent at adjacent time points can be used. And the optimal joint matching matrix X, for the ontological motion features of the source agent. Perform remapping to obtain the mapped motion features of the target heterogeneous intelligent agent.

[0036] In one example, at each time step, changes in the ontology motion features of the source agent can be detected, and the ontology motion features can be remapped using a pre-established optimal joint matching matrix, thereby obtaining the mapped motion features of the target heterogeneous agent. The specific mapping method can be as follows: For each joint of the target heterogeneous intelligent agent, calculate the influence of the motion of each joint of the source intelligent agent on it, and perform a weighted summation based on the optimal joint matching matrix X to obtain the final motion of each joint, that is, to obtain the mapped motion characteristics of the target heterogeneous intelligent agent.

[0037] In step 140, the ontological environment state characteristics of the source agent are remapped according to the ontological environment state characteristics of the source agent to obtain the mapped environment state characteristics of the target heterogeneous agent.

[0038] In step 150, based on the mapped motion features of the target heterogeneous intelligent agent and the mapped environmental state features of the target heterogeneous intelligent agent, the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent is obtained.

[0039] In yet another embodiment, the ontological environment state characteristics of the source agent can be used. Ontological environmental state characteristics of the source intelligent agent Perform remapping to obtain the mapped environmental state features of the target heterogeneous intelligent agent. The calculation yields the motion required for the heterogeneous intelligent agent to achieve the same environmental characteristic changes.

[0040] Furthermore, based on the mapped motion characteristics of the target heterogeneous intelligent agents... Environmental state features after mapping with target heterogeneous intelligent agents Obtain the motion remapping representation M of the target heterogeneous agent with respect to the source agent. t In this embodiment, the motion remapping representation M of the target heterogeneous agent with respect to the source agent is calculated. t In the process, the remapping of the subject's motion and the impact of motion on the environmental state were comprehensively considered, thus achieving the ability to accurately remap and express the actions of the source agent.

[0041] In another embodiment, the mapped environmental state features of the target heterogeneous intelligent agent It can be calculated using the following formula (1):

[0042]

[0043] Where α is a hyperparameter, α∈(0,1).

[0044] Through this embodiment, the intention expressed by the source agent through movement can be transmitted to the target heterogeneous agent in an appropriate manner for accurate expression.

[0045] The heterogeneous agent motion remapping representation method provided by this invention constructs the source agent's ontological motion features and ontological environment state features based on the motion state of the source agent in a preset motion scenario; it calls a pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent; based on the source agent's ontological motion features at adjacent time points and the optimal joint matching matrix, it remaps the source agent's ontological motion features to obtain the mapped motion features of the target heterogeneous agent; based on the source agent's ontological environment state features, it remaps the source agent's ontological environment state features to obtain the mapped environment state features of the target heterogeneous agent; based on the mapped motion features and the mapped environment state features of the target heterogeneous agent, it obtains the motion remapping representation of the target heterogeneous agent with respect to the source agent. This invention comprehensively considers the remapping of ontological motion and the influence of motion on the environment state, achieving an accurate remapping representation of the source agent's actions.

[0046] In an exemplary embodiment of the present invention, adjacent moments can include a previous moment and a subsequent moment. Continuing with the example described above, based on the ontology motion features of the source agent at adjacent moments and the optimal joint matching matrix, the ontology motion features of the source agent are remapped to obtain the mapped motion features of the target heterogeneous agent (corresponding to step 130). This can be achieved in the following way:

[0047] Based on the ontological motion characteristics of the source agent in the later time step and the ontological motion characteristics of the source agent in the earlier time step, the change in motion characteristics is determined.

[0048] Based on the changes in motion features and the optimal joint matching matrix, the ontological motion features of the source agent are remapped to obtain the mapped motion features of the target heterogeneous agent.

[0049] In one embodiment, the change in motion features can be determined based on the ontological motion features of the source agent at a later time step and the ontological motion features of the source agent at an earlier time step. The ontological motion features of the source agent at a later time step can be expressed as follows: The ontological motion characteristics of the source agent in the previous moment can be represented as: Then the change in motion characteristics can be expressed as: in, This can be expressed as finding the changes between the two in the feature space, for example, finding the relative rotation of a quaternion; or finding the coordinate changes of a position.

[0050] Furthermore, the ontological motion features of the source agent can be remapped based on the changes in motion features and the optimal joint matching matrix, thereby obtaining the mapped motion features of the target heterogeneous agent. The mapped motion features of the target heterogeneous agent can be obtained using formula (2):

[0051]

[0052] in, This represents the mapped motion characteristics of the target heterogeneous intelligent agent.

[0053] Figure 2 This is a flowchart illustrating the process of remapping the ontological environmental state features of the source intelligent agent based on the ontological environmental state features of the source intelligent agent to obtain the mapped environmental state features of the target heterogeneous intelligent agent, as provided by the present invention.

[0054] The following will combine Figure 2 The process of remapping the ontological environmental state characteristics of the source intelligent agent based on the ontological environmental state characteristics of the source intelligent agent to obtain the mapped environmental state characteristics of the target heterogeneous intelligent agent is explained.

[0055] In an exemplary embodiment of the present invention, combined with Figure 2 As can be seen, remapping the source agent's ontological environment state features based on the source agent's ontological environment state features to obtain the target heterogeneous agent's mapped environment state features can include steps 210 to 230, which will be described in detail below.

[0056] In step 210, the first environmental feature is determined based on the ontological environmental state features of the source agent.

[0057] In one embodiment, the ontological environment state characteristics of the source agent can be used. Determine the first environmental characteristics Among them, the first environmental feature can be used to characterize the target heterogeneous intelligent agent in accordance with the ontological environmental state features of the source intelligent agent. To perform movement, and to use the end effector of the target heterogeneous intelligent agent to manipulate the joint v e The environmental characteristics that arise when the target object is controlled.

[0058] In step 220, the second environmental feature is determined based on the difference between the first environmental feature and the ontological environmental state feature of the source agent.

[0059] In step 230, the ontological environmental state features of the source agent are remapped according to the eigenvector norm of the second environmental feature to obtain the mapped environmental state features of the target heterogeneous agent.

[0060] In yet another embodiment, it can be based on the first environmental characteristic. The ontological environmental state characteristics of the source intelligent agent The difference determines the characteristics of the second environment.

[0061] Furthermore, based on the eigenvector L2 norm of the second environmental feature, the ontological environmental state features of the source agent are remapped, thereby obtaining the mapped environmental state features of the target heterogeneous agent.

[0062] Among them, the environmental state characteristics after mapping of the target heterogeneous intelligent agent It can be expressed using formula (3):

[0063]

[0064] Where, in the formula This can represent the target heterogeneous intelligent agent in the output Motion, using the end-effector key v of the target heterogeneous intelligent agent e Environmental features generated when controlling target objects in the environment, |·| 2Let L represent the eigenvector L2 norm. Add a term to equation (3). The intention is to enable the target heterogeneous intelligent agent to output Energy is lowest during movement, thus more closely approximating reality.

[0065] Figure 3 This is a schematic diagram of the process for constructing the optimal joint matching matrix between the source intelligent agent and the target heterogeneous intelligent agent provided by the present invention.

[0066] The following will combine Figure 3 The process of constructing the optimal joint matching matrix between the source agent and the target heterogeneous agent is explained.

[0067] In an exemplary embodiment of the present invention, combined with Figure 3 As can be seen, constructing the optimal joint matching matrix between the source agent and the target heterogeneous agent can include steps 310 to 330, which will be described in detail below.

[0068] In step 310, the first kinematic structure graph model representation of the source agent and the second kinematic structure graph model representation of the target heterogeneous agent are obtained.

[0069] In step 320, constraints are constructed, wherein the constraints include the fact that the value of any element in the optimal joint matching matrix is ​​between 0 and 1.

[0070] In step 330, based on the first kinematic structure graph model representation and the second kinematic structure graph model representation, the optimal joint matching matrix between the source agent and the target heterogeneous agent is constructed through graph matching and the constraints.

[0071] In one embodiment, a first kinematic structure graph model representation G of the source agent can be obtained. s And the second kinematic structure diagram model representation G of the target heterogeneous intelligent agent. t .

[0072] Furthermore, constraints are constructed, including ensuring that the value of any element in the optimal joint matching matrix is ​​between 0 and 1. In this embodiment, the value of any element in the optimal joint matching matrix X is required to be between [0,1], and the sum of each row and each column is required to be 1. This means that each joint of the source agent is matched with each joint of the target heterogeneous agent according to its weight, in order to solve the problem of one-to-one mapping failure caused by constraints such as different numbers of joints and motion restrictions between heterogeneous agents, and to ensure that the motion of each joint has a corresponding result in the target heterogeneous agent.

[0073] In another embodiment, based on the first and second kinematic structure graph model representations, the optimal joint matching matrix between the source agent and the target heterogeneous agent can be constructed through graph matching and constraints. The optimal joint matching matrix can characterize the optimal matching degree between each joint of the source agent and each joint of the target heterogeneous agent.

[0074] In one example, the optimal joint matching matrix X is calculated based on a graph matching algorithm, and its objective and constraints can be expressed as formulas (4)-(5):

[0075]

[0076]

[0077] Where X represents the optimal joint matching matrix; W is the first kinematic diagram model representing G. s The second kinematic structure diagram model represents G. t The similarity matrix; X e W represents the end-point matching matrix composed of the corresponding elements of the end-operation joints in the optimal joint matching matrix X; e The first kinematic structure diagram model represents G. s The second kinematic structure diagram model represents G. t The subgraph similarity matrix is ​​composed of the end-effectors of each joint; n s n represents the number of elements in the first joint set of the first kinematic chain of the source agent; t X represents the number of elements in the set of second joints in the second kinetic chain of the target heterogeneous intelligent agent; ij This represents the matching degree between the joints of the source agent corresponding to the i-th row and the joints of the target agent corresponding to the j-th column in the optimal joint matching matrix X.

[0078] It should be noted that when determining the optimal joint matching matrix X, an item is added to formula (4). This ensures that the end effector joints can be correctly matched, producing the correct remapping results when manipulating environmental objects.

[0079] In addition, this problem is a continuous optimization problem. For common intelligent agents (humans, robots, etc.), the number of joints is usually no more than 50, the solution space is relatively determined, and it can be solved by algorithms such as Newton's method and random walk.

[0080] In yet another exemplary embodiment of the present invention, the first kinematic structure diagram model representation of the source intelligent agent can be constructed in the following manner:

[0081] Obtain the set of first joints in the first kinematic chain of the source agent, the connection relationship of each first joint of the source agent, and the set of first end-effector joints in the first kinematic chain of the source agent;

[0082] Based on the first set of joints, the connection relationships between each first joint, and the first set of end-effector joints, a first kinematic structure diagram representation of the source agent is constructed.

[0083] In one embodiment, a first kinematic structure graph model representation G of the source agent can be constructed. s The complete first kinematic structure diagram model represents G. s This can be expressed as formula (6):

[0084]

[0085] Among them, V s This represents the set of the first joints in the first kinematic chain of the source agent, with n elements. s E s The connection relationship of each first joint of the source agent is represented by a matrix, and the elements at the position (i,j), 1<=i,j<=n satisfy the relation (7):

[0086]

[0087] It can represent the set of first end-operation joints in the first kinematic chain of the source agent.

[0088] In yet another exemplary embodiment of the present invention, the second kinematic structure diagram model representation of the target heterogeneous intelligent agent can be constructed in the following manner:

[0089] Obtain the set of second joints in the second kinematic chain of the target heterogeneous intelligent agent, the connection relationship of each second joint of the target heterogeneous intelligent agent, and the set of second end-effector joints in the second kinematic chain of the target heterogeneous intelligent agent;

[0090] Based on the set of second joints, the connection relationships between each second joint, and the set of second end-effector joints, a second kinematic structure diagram model representation of the target heterogeneous intelligent agent is constructed.

[0091] In one embodiment, a second kinematic structure graph model representation G of the target heterogeneous intelligent agent can be constructed. t The complete second kinematic structure diagram model represents G. t This can be expressed as formula (8):

[0092]

[0093] Among them, V tThis represents the set of second joints in the second kinetic chain of the target heterogeneous intelligent agent, with n elements. t E t The connection relationship of each second joint of the target heterogeneous intelligent agent is represented by a matrix, and the elements at the position (i,j), 1<=i,j<=n satisfy the relation (9):

[0094]

[0095] It can represent the set of second-end operation joints in the second kinetic chain of the target heterogeneous intelligent agent.

[0096] The heterogeneous intelligent agent motion remapping representation method provided by this invention enables heterogeneous intelligent agents to achieve mutual transmission of intent and motion remapping representation. Taking humans, quadrupeds, and bi-armed robots as examples, when a human uses its arms to express an action such as "clasping hands" without changing environmental features, a quadruped can support its body with its hind limbs and make a similar clasping hand gesture with its forelimbs. A bi-armed robot can directly simulate the corresponding action of a human arm to achieve the clasping hand gesture. When a human uses its hands or feet to change the state of objects in the environment (such as kicking a ball), a quadruped or bi-armed robot can express similar movements and use its appropriate end effector structure to achieve the same intent to change the state of the ball. The figure below shows the effect of intent transmission between quadrupeds and humanoid robots.

[0097] As described above, the heterogeneous agent motion remapping representation method provided by this invention constructs the source agent's ontology motion features and ontology environment state features based on the motion state of the source agent in a preset motion scenario; it calls a pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent; based on the source agent's ontology motion features at adjacent time points and the optimal joint matching matrix, it remaps the source agent's ontology motion features to obtain the mapped motion features of the target heterogeneous agent; based on the source agent's ontology environment state features, it remaps the source agent's ontology environment state features to obtain the mapped environment state features of the target heterogeneous agent; based on the mapped motion features and the mapped environment state features of the target heterogeneous agent, it obtains the motion remapping representation of the target heterogeneous agent with respect to the source agent. In this invention, the remapping of ontology motion and the influence of motion on the environment state are comprehensively considered, achieving accurate remapping representation of the source agent's actions.

[0098] Based on the same concept, the present invention also provides a heterogeneous intelligent agent motion remapping representation device.

[0099] The heterogeneous intelligent agent motion remapping representation device provided by the present invention is described below. The heterogeneous intelligent agent motion remapping representation device described below can be referred to in correspondence with the heterogeneous intelligent agent motion remapping representation method described above.

[0100] Figure 4 This is a schematic diagram of the heterogeneous intelligent agent motion remapping expression device provided by the present invention.

[0101] In an exemplary embodiment of the present invention, combined with Figure 4 As can be seen, the heterogeneous intelligent agent motion remapping expression device may include a construction module 410, a calling module 420, a motion feature remapping module 430, an environmental state feature remapping module 440, and a generation module 450. Each module will be described below.

[0102] The construction module 410 can be configured to construct the ontological motion features and ontological environmental state features of the source agent based on the motion state of the source agent in a preset motion scenario.

[0103] The calling module 420 can be configured to call a pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent;

[0104] The motion feature remapping module 430 can be configured to remap the ontology motion features of the source agent based on the ontology motion features of the source agent at adjacent time points and the optimal joint matching matrix, so as to obtain the mapped motion features of the target heterogeneous agent.

[0105] The environment state feature remapping module 440 can be configured to remap the ontological environment state features of the source agent based on the ontological environment state features of the source agent, so as to obtain the mapped environment state features of the target heterogeneous agent.

[0106] The generation module 450 can be configured to obtain the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent based on the mapped motion features of the target heterogeneous intelligent agent and the mapped environmental state features of the target heterogeneous intelligent agent.

[0107] In an exemplary embodiment of the present invention, the adjacent time moments include the previous time moment and the subsequent time moment; the motion feature remapping module 430 can remap the ontological motion features of the source agent based on the ontological motion features of the source agent at adjacent time moments and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous agent:

[0108] Based on the ontological motion characteristics of the source agent at the later time step and the ontological motion characteristics of the source agent at the earlier time step, the amount of change in motion characteristics is determined.

[0109] Based on the changes in motion features and the optimal joint matching matrix, the ontological motion features of the source agent are remapped to obtain the mapped motion features of the target heterogeneous agent.

[0110] In an exemplary embodiment of the present invention, the environment state feature remapping module 440 can remap the ontological environment state features of the source agent based on the ontological environment state features of the source agent to obtain the mapped environment state features of the target heterogeneous agent in the following manner:

[0111] Based on the ontological environmental state characteristics of the source agent, a first environmental characteristic is determined, wherein the first environmental characteristic is used to characterize the environmental characteristics generated when the target heterogeneous agent moves according to the ontological environmental state characteristics of the source agent and controls the target object using the end effector joint of the target heterogeneous agent.

[0112] The second environmental feature is determined based on the difference between the first environmental feature and the ontological environmental state feature of the source agent;

[0113] Based on the second norm of the feature vector of the second environmental feature, the ontological environmental state features of the source agent are remapped to obtain the mapped environmental state features of the target heterogeneous agent.

[0114] In an exemplary embodiment of the present invention, the calling module 420 may construct the optimal joint matching matrix between the source agent and the target heterogeneous agent in the following manner:

[0115] Obtain the first kinematic structure graph model representation of the source agent and the second kinematic structure graph model representation of the target heterogeneous agent;

[0116] Construct constraints, wherein the constraints include the fact that any element in the optimal joint matching matrix has a value range between 0 and 1;

[0117] Based on the first kinematic structure graph model representation and the second kinematic structure graph model representation, the optimal joint matching matrix between the source agent and the target heterogeneous agent is constructed through graph matching and the constraints. The optimal joint matching matrix is ​​used to characterize the optimal matching degree between each joint of the source agent and each joint of the target heterogeneous agent.

[0118] In an exemplary embodiment of the present invention, the calling module 420 may construct the first kinematic structure graph model representation of the source agent in the following manner:

[0119] Obtain the first joint set in the first kinematic chain of the source agent, the connection relationship of each first joint of the source agent, and the first end-effector set in the first kinematic chain of the source agent;

[0120] Based on the first set of joints, the connection relationship of each of the first joints, and the first set of end-effector joints, a first kinematic structure diagram representation of the source agent is constructed.

[0121] In an exemplary embodiment of the present invention, the calling module 420 may construct the second kinematic structure diagram model representation of the target heterogeneous intelligent agent in the following manner:

[0122] Obtain the set of second joints in the second kinematic chain of the target heterogeneous intelligent agent, the connection relationship of each second joint of the target heterogeneous intelligent agent, and the set of second end-effector joints in the second kinematic chain of the target heterogeneous intelligent agent;

[0123] Based on the second joint set, the connection relationship of each second joint, and the second end-effector joint set, a second kinematic structure diagram model representation of the target heterogeneous intelligent agent is constructed.

[0124] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a heterogeneous agent motion remapping representation method. This method includes: constructing the source agent's ontological motion features and ontological environment state features based on the motion state of the source agent in a preset motion scenario; calling a pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent; remapping the source agent's ontological motion features according to the source agent's ontological motion features at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous agent; remapping the source agent's ontological environment state features according to the source agent's ontological environment state features to obtain the mapped environment state features of the target heterogeneous agent; and obtaining the motion remapping representation of the target heterogeneous agent with respect to the source agent based on the mapped motion features and the mapped environment state features of the target heterogeneous agent.

[0125] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the heterogeneous intelligent agent motion remapping representation method provided by the above methods. The method includes: constructing the ontological motion features and ontological environment state features of the source intelligent agent based on the motion state of the source intelligent agent in a preset motion scenario; calling a pre-determined optimal joint matching matrix between the source intelligent agent and the target heterogeneous intelligent agent; remapping the ontological motion features of the source intelligent agent according to the ontological motion features of the source intelligent agent at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous intelligent agent; remapping the ontological environment state features of the source intelligent agent according to the ontological environment state features of the source intelligent agent to obtain the mapped environment state features of the target heterogeneous intelligent agent; and obtaining the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent based on the mapped motion features and the mapped environment state features of the target heterogeneous intelligent agent.

[0127] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the heterogeneous agent motion remapping representation method provided by the above methods. The method includes: constructing ontological motion features and ontological environment state features of the source agent based on the motion state of the source agent in a preset motion scenario; invoking a pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent; remapping the ontological motion features of the source agent according to the ontological motion features of the source agent at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous agent; remapping the ontological environment state features of the source agent according to the ontological environment state features of the source agent to obtain the mapped environment state features of the target heterogeneous agent; and obtaining the motion remapping representation of the target heterogeneous agent with respect to the source agent based on the mapped motion features and the mapped environment state features of the target heterogeneous agent.

[0128] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0129] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0130] It is further understood that although the operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for remapping motion representation of heterogeneous intelligent agents, characterized in that, The method includes: Based on the motion state of the source agent in a preset motion scenario, the ontological motion features and ontological environmental state features of the source agent are constructed. Invoke the pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent; Based on the ontological motion features of the source agent at adjacent time points and the optimal joint matching matrix, the ontological motion features of the source agent are remapped to obtain the mapped motion features of the target heterogeneous agent. Based on the ontological environment state characteristics of the source agent, the ontological environment state characteristics of the source agent are remapped to obtain the mapped environment state characteristics of the target heterogeneous agent. Based on the mapped motion features of the target heterogeneous intelligent agent and the mapped environmental state features of the target heterogeneous intelligent agent, the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent is obtained.

2. The heterogeneous intelligent agent motion remapping representation method according to claim 1, characterized in that, The adjacent moments include the preceding moment and the following moment; The step of remapping the ontology motion features of the source agent based on the ontology motion features of the source agent at adjacent time points and the optimal joint matching matrix to obtain the mapped motion features of the target heterogeneous agent specifically includes: Based on the ontological motion characteristics of the source agent at the later time step and the ontological motion characteristics of the source agent at the earlier time step, the amount of change in motion characteristics is determined. Based on the changes in motion features and the optimal joint matching matrix, the ontological motion features of the source agent are remapped to obtain the mapped motion features of the target heterogeneous agent.

3. The heterogeneous intelligent agent motion remapping representation method according to claim 1 or 2, characterized in that, The step of remapping the ontological environment state features of the source agent based on the ontological environment state features of the source agent to obtain the mapped environment state features of the target heterogeneous agent specifically includes: Based on the ontological environmental state characteristics of the source agent, a first environmental characteristic is determined, wherein the first environmental characteristic is used to characterize the environmental characteristics generated when the target heterogeneous agent moves according to the ontological environmental state characteristics of the source agent and controls the target object using the end effector joint of the target heterogeneous agent. The second environmental feature is determined based on the difference between the first environmental feature and the ontological environmental state feature of the source agent; Based on the second norm of the feature vector of the second environmental feature, the ontological environmental state features of the source agent are remapped to obtain the mapped environmental state features of the target heterogeneous agent.

4. The heterogeneous intelligent agent motion remapping representation method according to claim 1, characterized in that, The optimal joint matching matrix between the source agent and the target heterogeneous agent is constructed in the following manner: Obtain the first kinematic structure graph model representation of the source agent and the second kinematic structure graph model representation of the target heterogeneous agent; Construct constraints, wherein the constraints include the fact that any element in the optimal joint matching matrix has a value range between 0 and 1; Based on the first kinematic structure graph model representation and the second kinematic structure graph model representation, the optimal joint matching matrix between the source agent and the target heterogeneous agent is constructed through graph matching and the constraints. The optimal joint matching matrix is ​​used to characterize the optimal matching degree between each joint of the source agent and each joint of the target heterogeneous agent.

5. The heterogeneous intelligent agent motion remapping representation method according to claim 4, characterized in that, The first kinematic structure graph model representation of the source agent is constructed in the following manner: Obtain the first joint set in the first kinematic chain of the source agent, the connection relationship of each first joint of the source agent, and the first end-effector set in the first kinematic chain of the source agent; Based on the first set of joints, the connection relationship of each of the first joints, and the first set of end-effector joints, a first kinematic structure diagram representation of the source agent is constructed.

6. The heterogeneous intelligent agent motion remapping representation method according to claim 4, characterized in that, The second kinematic structure diagram model representation of the target heterogeneous intelligent agent is constructed in the following manner: Obtain the set of second joints in the second kinematic chain of the target heterogeneous intelligent agent, the connection relationship of each second joint of the target heterogeneous intelligent agent, and the set of second end-effector joints in the second kinematic chain of the target heterogeneous intelligent agent; Based on the second joint set, the connection relationship of each second joint, and the second end-effector joint set, a second kinematic structure diagram model representation of the target heterogeneous intelligent agent is constructed.

7. A heterogeneous intelligent agent motion remapping representation device, characterized in that, The device includes: The construction module is used to construct the ontological motion features and ontological environmental state features of the source agent based on the motion state of the source agent in a preset motion scenario. The calling module is used to call the pre-determined optimal joint matching matrix between the source agent and the target heterogeneous agent; The motion feature remapping module is used to remap the ontology motion features of the source agent based on the ontology motion features of the source agent at adjacent time points and the optimal joint matching matrix, so as to obtain the mapped motion features of the target heterogeneous agent. The environment state feature remapping module is used to remap the ontological environment state features of the source agent based on the ontological environment state features of the source agent, so as to obtain the mapped environment state features of the target heterogeneous agent. The generation module is used to obtain the motion remapping representation of the target heterogeneous intelligent agent with respect to the source intelligent agent based on the mapped motion features of the target heterogeneous intelligent agent and the mapped environmental state features of the target heterogeneous intelligent agent.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the heterogeneous intelligent agent motion remapping representation method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the heterogeneous intelligent agent motion remapping representation method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the heterogeneous intelligent agent motion remapping representation method as described in any one of claims 1 to 6.

Images