Intelligent agent control method and device and intelligent agent

Abstract

The invention discloses an intelligent agent control method and device and an intelligent agent, and belongs to the field of intelligent agents. The intelligent agent control method comprises the steps of generating a motion path of an intelligent agent in a task scene based on a target task corresponding to the intelligent agent; determining a target constraint condition based on at least one of an execution result constraint, an obstacle constraint and a pose constraint between the intelligent agent and the target to-be-interacted object; based on the target constraint condition, the motion path and a target kinematics model, controlling the intelligent agent; the target kinematics model is constructed in advance based on the kinematics information of the executing mechanism of the intelligent body, the kinematics information of the object to be interacted and the kinematics information of the moving mechanism corresponding to the intelligent body. According to the intelligent agent control method, under the condition that the dynamic obstacle is detected, the obstacle can be avoided in time, the universality is high, and the use experience of a user is improved.

Description

Technical Field

[0001] The present application belongs to the field of intelligent agents, and in particular, relates to a control method, device and intelligent agent. Background Art

[0002] With the development of science and technology, intelligent agents are widely used in people's daily work and life. In related technologies, sensor data is mainly collected by sensors set on the intelligent agent, so as to control the intelligent agent to move or grasp in static scenes based on the sensor data. However, in actual application scenarios, there may also be dynamic obstacles. The existing control method of the intelligent agent in static scenes cannot be applied to dynamic scenes, has poor versatility, and affects the user experience. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a control method, device and intelligent agent, which can avoid obstacles in time when dynamic obstacles are detected, and is widely used in static and dynamic scenes, has high versatility, and improves the user experience.

[0004] In a first aspect, the present application provides a method for controlling an intelligent agent, the method comprising:

[0005] Based on the target task corresponding to the agent, generating a motion path of the agent in the task scene;

[0006] Determining a target constraint condition based on at least one of an execution result constraint, an obstacle constraint, and a position constraint between the agent and a target object to be interacted with;

[0007] The intelligent agent is controlled based on the target constraint conditions, the motion path and the target kinematic model; the target kinematic model is pre-constructed based on the kinematic information of the actuator of the intelligent agent, the kinematic information of the object to be interacted with and the kinematic information of the mobile mechanism corresponding to the intelligent agent.

[0008] According to the control method of the intelligent agent of the present application, the motion path of the intelligent agent in the task scene corresponding to the target task is generated through the target task that the intelligent agent needs to perform, and path instructions are provided for the intelligent agent to perform the target task, so that the intelligent agent can accurately perform the target task based on the motion path under ideal conditions; in the process of the intelligent agent performing the task based on the motion path, the process of controlling the intelligent agent based on the target kinematic model is constrained by executing one or more determined target constraints among the result constraints, obstacle constraints and posture constraints, and when a dynamic obstacle is detected, the obstacle can be avoided in time. It is widely used in static and dynamic scenes, has high versatility, and improves the user experience.

[0009] According to the control method of the intelligent agent of the present application, the execution result constraint is determined based on the following steps:

[0010] Acquire a path error between an actual path of the agent and the motion path, an input smoothness item, and a final posture of the target object to be interacted with;

[0011] The execution result constraint is determined based on the path error, the input smoothness item, and a final posture of the target object to be interacted with.

[0012] According to the control method of the intelligent body of the present application, the posture constraint is determined based on the following steps:

[0013] The posture constraint is determined based on the distance between the mobile mechanism and the target object to be interacted with, the target maximum distance, and the target minimum distance.

[0014] According to the control method of the intelligent body of the present application, the obstacle constraints include: columnar structure obstacle constraints and plane obstacle constraints.

[0015] According to the control method of the intelligent agent of the present application, the columnar structure obstacle constraint is determined based on the following steps:

[0016] Based on the size parameters of the obstacle, the obstacle is modeled to obtain a columnar structure;

[0017] Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure;

[0018] The columnar structure obstacle constraint is determined based on the distance between the center of each of the spherical structures and the columnar structure.

[0019] According to the control method of the intelligent agent of the present application, the planar obstacle constraint is determined based on the following steps:

[0020] Extracting target points on the planar obstacle;

[0021] Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure;

[0022] The planar obstacle constraint is determined based on the distance between the target point and each of the spherical structures and the number of the planar obstacles.

[0023] According to the control method of the intelligent agent of the present application, the controlling of the intelligent agent based on the target constraint condition, the motion path and the target kinematic model includes:

[0024] The target kinematics model is reactively controlled based on the motion path and the target constraint conditions to control the intelligent agent.

[0025] According to the control method of the intelligent agent of the present application, the reactive control of the target kinematic model based on the motion path and the target constraint condition to control the intelligent agent includes:

[0026] Reactively controlling the target kinematic model based on the motion path and the target constraint condition to obtain the degree of freedom corresponding to the actuator and the speed corresponding to the degree of freedom in the process of the agent performing the target task;

[0027] The intelligent agent is controlled based on the degrees of freedom corresponding to the actuators and the speeds corresponding to the degrees of freedom.

[0028] In a second aspect, the present application provides a control device for an intelligent body, the device comprising:

[0029] A first processing module, configured to generate a motion path of the agent in a task scene based on a target task corresponding to the agent;

[0030] A second processing module, configured to determine a target constraint condition based on at least one of an execution result constraint, an obstacle constraint, and a position constraint between the agent and a target object to be interacted with;

[0031] The third processing module is used to control the intelligent agent based on the target constraint condition, the motion path and the target kinematic model; the target kinematic model is pre-constructed based on the kinematic information of the actuator of the intelligent agent, the kinematic information of the object to be interacted with and the kinematic information of the mobile mechanism corresponding to the intelligent agent.

[0032] According to the control device of the intelligent agent of the present application, the motion path of the intelligent agent in the task scene corresponding to the target task is generated through the target task that the intelligent agent needs to perform, and path instructions are provided for the intelligent agent to perform the target task, so that the intelligent agent can accurately perform the target task based on the motion path under ideal conditions; in the process of the intelligent agent performing the task based on the motion path, the process of controlling the intelligent agent based on the target kinematic model is constrained by executing one or more determined target constraints among the result constraints, obstacle constraints and posture constraints. When a dynamic obstacle is detected, the obstacle can be avoided in time. It is widely used in static and dynamic scenes, has high versatility, and improves the user experience.

[0033] In a third aspect, the present application provides an intelligent agent, comprising:

[0034] sensor;

[0035] Executive body;

[0036] A moving mechanism, wherein the actuator is arranged on the moving mechanism;

[0037] As described in the second aspect, the control device of the intelligent body is electrically connected to the sensor, the actuator and the moving mechanism respectively.

[0038] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the control method of the intelligent agent as described in the first aspect above is implemented.

[0039] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the control method of the intelligent agent as described in the first aspect above.

[0040] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0041] Through the target task that the intelligent agent needs to perform, the motion path of the intelligent agent in the task scene corresponding to the target task is generated, and path instructions are provided for the intelligent agent to perform the target task, so that the intelligent agent can accurately perform the target task based on the motion path under ideal conditions; in the process of the intelligent agent performing the task based on the motion path, through the execution result constraints, obstacle constraints and one or more determined target constraints in the posture constraints, the process of controlling the intelligent agent based on the target kinematic model is constrained. When a dynamic obstacle is detected, the obstacle can be avoided in time. It is widely used in static and dynamic scenes, has high versatility, and improves the user experience.

[0042] Furthermore, the agent is constrained by the path error between the actual path of the agent and the motion path, so that the agent can perform the target task according to the motion path; the agent is constrained by the input smoothness term, so that the agent can move smoothly and perform corresponding actions. In addition, the agent is constrained by the final posture of the target object to be interacted with, ensuring that the target object to be interacted with is in the required position and posture when the agent completes the task. The path error, the input smoothness term and the final posture of the target object to be interacted with are used to jointly constrain the agent, and the results of the agent's execution of the motion path are effectively tracked, so that the agent can smoothly and complete the target task, thereby improving the user experience.

[0043] Furthermore, the posture constraints are determined by the distance between the mobile mechanism and the target object to be interacted with, the maximum distance that can be reached between the mobile mechanism and the target object to be interacted with, and the minimum distance that can be reached between the mobile mechanism and the target object to be interacted with, so that the mobile mechanism and the target object to be interacted with can find a feasible solution that satisfies the posture of the actuator and the target object to be interacted with in the working space of the actuator, thereby determining the speed of the actuator, so that the intelligent agent can accurately avoid obstacles when dynamic obstacles are detected.

[0044] Furthermore, by setting different constraints according to the different types of obstacles that may exist in the task scene, it is possible to flexibly select constraints based on the actual task scene to improve the accuracy of avoiding obstacles.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 It is one of the flow charts of the control method of the intelligent body provided in the embodiment of the present application;

[0048] Figure 2 It is one of the principle schematic diagrams of the control method of the intelligent body provided in the embodiment of the present application;

[0049] Figure 3 This is the second principle schematic diagram of the control method of the intelligent body provided in the embodiment of the present application;

[0050] Figure 4 This is the third principle schematic diagram of the control method of the intelligent body provided in the embodiment of the present application;

[0051] Figure 5 is a schematic diagram of the structure of a control device for an intelligent body provided in an embodiment of the present application;

[0052] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0054] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0055] In conjunction with the accompanying drawings, the control method of the intelligent body, the control device of the intelligent body, the intelligent body and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0056] The control method of the intelligent agent may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0057] The control method of an intelligent body provided in the embodiment of the present application may be executed by an intelligent body or an electronic device connected to the intelligent body for communication, or a functional module or functional entity in the electronic device that can implement the control method of the intelligent body. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The control method of an intelligent body provided in the embodiment of the present application is described below using an intelligent body as an example of an execution subject.

[0058] like Figure 1 As shown, the control method of the intelligent agent includes: step 110, step 120 and step 130.

[0059] It should be noted that the control method of the intelligent agent can be applied to the control scenario of the intelligent agent, wherein the intelligent agent can be a robot or other structure that can achieve the same function. The robot can include but is not limited to household intelligent agents, production intelligent agents, service intelligent agents, mobile robotic arms, aerial work intelligent agents and other intelligent agents.

[0060] like Figure 3 As shown, the intelligent body may include an actuator 310 and a moving mechanism 320 . The actuator 310 is disposed on the moving mechanism 320 , and the moving mechanism 320 drives the actuator 310 to move.

[0061] The actuator 310 is used to perform operations such as grabbing, pushing, pulling, and placing, and may be a robotic arm, for example.

[0062] The moving mechanism 320 may be a roller or a track, etc.

[0063] Step 110: Generate a motion path of the agent in the task scene based on the target task corresponding to the agent;

[0064] In this step, the target task is the task that the agent needs to perform.

[0065] The task scenario can be any scenario, including but not limited to: home intelligent agent scenarios, factory scenarios, shopping malls and other scenarios.

[0066] For example, it can be used for food delivery scenarios in restaurants, household cleaning scenarios, and material transportation scenarios in factories.

[0067] The motion path is the motion path of the agent when performing the entire target task.

[0068] In the actual execution process, the target task can be determined by the instructions received by the agent, thereby generating a movement path of the agent in the task scene during the execution of the target task.

[0069] For example, taking the target task of "taking out the cup from the refrigerator and putting it on the dining table" as an example, when the task scene is a residence, the motion path can be that the agent moves from its initial position to the refrigerator in the kitchen, opens the refrigerator to take out the cup, and continues to move to the location of the dining table, placing the cup on the dining table. The corresponding movement path and operation path.

[0070] Step 120: determining a target constraint condition based on at least one of an execution result constraint, an obstacle constraint, and a position constraint between the agent and a target object to be interacted with;

[0071] In this step, the execution result constraint is the constraint on the result of tracking the agent's execution of the target task. The execution result constraint can be expressed as J t .

[0072] The obstacle constraint is the constraint to prevent the agent from colliding with obstacles. The obstacle constraint can be expressed as J obst .

[0073] The pose constraint is the distance constraint between the agent and the target object to be interacted with. The pose constraint can be expressed as J a .

[0074] The target constraints are the constraints used to control the agent to perform the target task.

[0075] It is understandable that in different task scenarios, the target constraints may be the same or different.

[0076] The target object to be interacted with is the object to be operated by the agent. For example, in a grasping scenario, the target object to be interacted with can be the object to be grasped by the agent, such as a cup or bottle; in a door opening and closing scenario, the target object to be interacted with can be a door that needs to be opened or closed.

[0077] In some embodiments, the target object to be interacted with may be one or more, and the categories of the multiple target objects to be interacted with may be different.

[0078] In the actual execution process, based on the actual situation of the task scenario, one or more of the execution result constraints, obstacle constraints and posture constraints can be selected to determine the target constraint conditions for controlling the intelligent agent.

[0079] When the target constraint condition includes the execution result constraint, obstacle constraint and posture constraint, the target constraint condition can be expressed as: J = J t +J obst +J a .

[0080] Step 130: Control the intelligent agent based on the target constraint conditions, the motion path and the target kinematic model.

[0081] In this step, the target kinematic model is pre-constructed based on the kinematic information of the actuator of the agent, the kinematic information of the object to be interacted with, and the kinematic information of the mobile mechanism corresponding to the agent.

[0082] The actuator kinematic information is used to characterize the relevant information of the actuator, including but not limited to: the structure, size, properties and motion characteristics of the actuator.

[0083] The kinematic information of the object to be interacted with is used to represent relevant information of the object to be interacted with, including but not limited to: information such as the structure, size, shape, and attributes of the object to be interacted with.

[0084] The kinematic information of the mobile mechanism is used to characterize the relevant information of the mobile mechanism, including but not limited to: the structure, dimensional attributes, relative relationship with the external environment, and movement characteristics of the mobile mechanism.

[0085] The kinematic information of the mobile mechanism can reflect the movement possibility of the mobile mechanism.

[0086] In actual implementation, the planar motion of the mobile mechanism can be simulated through a three-degree-of-freedom kinematic chain.

[0087] The target kinematics model is a model used to characterize the serial motion relationship between the mobile mechanism, the actuator and the object to be interacted.

[0088] In some embodiments, there may be one or more objects to be interacted with, and the categories of the multiple objects to be interacted with may be different.

[0089] In some embodiments, when there are multiple objects to be interacted with, step 110 may also include: constructing a target kinematic model corresponding to the target first object to be interacted with based on the kinematic information of the object to be interacted with, the kinematic information of the actuator, and the kinematic information of the moving mechanism corresponding to the target first object to be interacted with among the multiple objects to be interacted with; and then fusing the target kinematic models corresponding to each object to be interacted with to obtain the final target kinematic model.

[0090] The target first object to be interacted with is any object among the multiple objects to be interacted with.

[0091] In the actual execution process, the motion path is used as a reference path for the control agent to perform the target task. In the control process, based on the target constraint conditions and the target kinematic model, the control agent performs the corresponding task.

[0092] According to the control method of the intelligent agent provided in the embodiment of the present application, the motion path of the intelligent agent in the task scene corresponding to the target task is generated through the target task that the intelligent agent needs to perform, and path instructions are provided for the intelligent agent to perform the target task, so that the intelligent agent can accurately perform the target task based on the motion path under ideal conditions; in the process of the intelligent agent performing the task based on the motion path, the process of controlling the intelligent agent based on the target kinematic model is constrained by executing one or more determined target constraints among the result constraints, obstacle constraints and posture constraints. When a dynamic obstacle is detected, the obstacle can be avoided in time. The intelligent agent is widely used in static and dynamic scenes, has high versatility, and improves the user experience.

[0093] In some embodiments, the execution result constraints may be determined based on the following steps:

[0094] Obtain the path error between the actual path of the agent and the motion path, the input smoothness item, and the final posture of the target object to be interacted with;

[0095] The execution result constraints are determined based on the path error, the input smoothness item, and the final posture of the target object to be interacted with.

[0096] In this embodiment, the actual path is the path that the agent executes in the actual task scenario using the motion path as a reference path.

[0097] The path error is the error between the path executed by the agent in the actual task scenario and the generated motion path. The path error can be expressed as Trackingcost.

[0098] The input smoothness term is a metric that ensures that the agent does not make sharp turns or unnecessary jitters during movement, so that the agent can be stable. The input smoothness term can be expressed as Inputsmoothnesscost.

[0099] The final posture of the target object to be interacted with is the posture that the target object to be interacted with needs to reach when the agent completes the target task. The final posture can be expressed as Terminalcost.

[0100] The execution result constraint can be expressed as:

[0101]

[0102] Where z is the degree of freedom of the actuator; u is the speed of the control input of each degree of freedom; k is the kth degree of freedom from 1 to N; Q z is the weighted matrix corresponding to the path error; Q Δu is the weighted matrix corresponding to the input smooth term; Q N is the weighted matrix corresponding to the final posture.

[0103] According to the control method of the intelligent agent provided in the embodiment of the present application, the intelligent agent is constrained by the path error between the actual path of the intelligent agent and the motion path, so that the intelligent agent can perform the target task according to the motion path; the intelligent agent is constrained by the input smoothness item, so that the intelligent agent can move smoothly and perform corresponding actions. In addition, the intelligent agent is constrained by the final posture of the target object to be interacted with, ensuring that the target object to be interacted with can be in the required posture when the intelligent agent completes the task. The path error, the input smoothness item and the final posture of the target object to be interacted with are used to jointly constrain the intelligent agent, and the results of the intelligent agent executing the motion path are effectively tracked, so that the intelligent agent can smoothly and complete the target task, thereby improving the user experience.

[0104] In some embodiments, the pose constraints may be determined based on the following steps:

[0105] The pose constraint is determined based on the distance between the mobile mechanism and the target object to be interacted, the target maximum distance, and the target minimum distance.

[0106] In this embodiment, the distance between the moving mechanism and the target object to be interacted with is the distance between the attachment position points of the moving mechanism and the target object to be interacted with.

[0107] The target maximum distance is the maximum distance between the preset moving mechanism and the target object to be interacted with.

[0108] The target minimum distance is the minimum distance between the preset moving mechanism and the target object to be interacted with.

[0109] The specific values ​​of the target minimum distance and the target maximum distance can be determined when the mobile mechanism and the target object to be interacted with are kept in the working space of the actuator.

[0110] The pose constraint can be determined based on the following formula:

[0111] J a =(f d (d up -d a )+f d (d a -d low )) Qa

[0112] Among them, J a is the pose constraint, d a is the distance between the mobile mechanism and the target object to be interacted with; d up is the maximum distance to the target; d low is the minimum distance to the target; Q a is the weight matrix.

[0113] In some embodiments, when the actuator performs a picking action, there is no fixed relative posture between the actuator and the target object to be interacted with, and the posture constraint is 0.

[0114] In some embodiments, when the actuator performs a placement action, there is a fixed relative posture between the actuator and the target object to be interacted with, and the posture constraint is not zero.

[0115] In the actual execution process, taking the target task of opening the cabinet door as an example, the trajectory of the cabinet door can be obtained through the virtual kinematic chain, and the reactive control of the nonlinear model predictive control can obtain all the postures of the mobile mechanism in the prediction time domain. Then, according to the virtual kinematic chain, the posture of the cabinet door is obtained, and the posture of the actuator at the previous moment is used as the initial value. The first-order difference of the inverse kinematics result is solved to obtain the speed of each joint of the actuator.

[0116] According to the control method of the intelligent system provided in the embodiment of the present application, the posture constraint is determined by the distance between the mobile mechanism and the target object to be interacted with, the maximum distance that can be reached between the mobile mechanism and the target object to be interacted with, and the minimum distance that can be reached between the mobile mechanism and the target object to be interacted with, so that the mobile mechanism and the target object to be interacted with can find a feasible solution that satisfies the posture of the actuator and the target object to be interacted with in the working space of the actuator, thereby determining the speed of the actuator, so that the intelligent agent can accurately avoid obstacles when a dynamic obstacle is detected.

[0117] In some embodiments, obstacle constraints may include: columnar structure obstacle constraints and plane obstacle constraints.

[0118] In this embodiment, the columnar structure obstacle constraint is a constraint corresponding to the agent avoiding the columnar structure obstacle. The columnar structure obstacle constraint can be expressed as J obs .

[0119] Columnar obstacles include, but are not limited to, people, sliding chairs, toy cars and other obstacles.

[0120] The plane obstacle constraint is the constraint corresponding to the agent avoiding the plane obstacle. The plane obstacle constraint can be expressed as J Π .

[0121] Plane obstacles include but are not limited to: doors, baffles and other obstacles.

[0122] In the actual implementation process, the corresponding constraint conditions can be flexibly selected based on the actual situation of the obstacles.

[0123] According to the control method of the intelligent body provided in the embodiment of the present application, different constraints are set by different types of obstacles that may exist in the task scene. It is possible to flexibly select constraints based on the actual task scene to improve the accuracy of avoiding obstacles.

[0124] In some embodiments, the columnar structure obstacle constraint may be determined based on the following steps:

[0125] Based on the size parameters of the obstacles, the obstacles are modeled to obtain a columnar structure;

[0126] Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure;

[0127] Based on the distance between the center of each spherical structure and the columnar structure, the columnar structure obstacle constraint is determined.

[0128] In this embodiment, the size parameters are parameters such as the length, height, and width of the columnar structure obstacle that can characterize the size of the obstacle.

[0129] The structural information is the composition of the actuator, such as the number of joints, joint sizes, arm length, etc.

[0130] When the obstacle is a columnar structure, the obstacle can be modeled as follows based on the specific values ​​of the obstacle's size parameters: Figure 2 The columnar structure shown.

[0131] The structural information of the actuator is as follows Figure 3Taking the robotic arm shown as an example, the actuator includes: a shoulder, an elbow, a wrist and a gripper. The actuator can be modeled based on the actual size of the shoulder, elbow, wrist and gripper, and the shoulder, elbow, wrist and gripper can be modeled as corresponding spherical structures respectively.

[0132] It will be appreciated that the size of the modeled spherical structure will be different depending on the size of the shoulder, elbow, wrist and grip.

[0133] In the actual execution process, after obtaining the columnar structure and the spherical structure, the intelligent agent can be constrained to avoid obstacles based on the columnar structure and the spherical structure.

[0134] The columnar structure obstacle constraint can be determined based on the following formula:

[0135]

[0136] Among them, J obs is the columnar structure obstacle constraint; p i is the center of the i-th spherical structure; is the position of the columnar structure; n o is the number of spherical structures; f d is the penalty function; Q p is the weighting matrix.

[0137] like Figure 4 As shown, the penalty function may be a non-segmented penalty function, and the penalty function is used to penalize the distance that the columnar obstacle moves to the spherical structure.

[0138] According to the control method of the intelligent agent provided in the embodiment of the present application, by modeling the obstacle as a columnar structure and the actuator of the intelligent agent as one or more spherical structures, the obstacle and the actuator are effectively abstracted as regular objects, and the predicted trajectory of the dynamic obstacle is introduced to ensure that the dynamic obstacle and the intelligent agent will not collide within the predicted time domain. It not only focuses on the current moment, but also constrains the distance between the columnar structure and each spherical structure to enable the intelligent agent to avoid the columnar obstacles in the dynamic environment.

[0139] In some embodiments, the planar obstacle constraint may be determined based on the following steps:

[0140] Extract target points on planar obstacles;

[0141] Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure;

[0142] The planar obstacle constraints are determined based on the distances between the target point and each spherical structure and the number of planar obstacles.

[0143] In this embodiment, the target point is any point on the planar obstacle.

[0144] During the actual execution process, the target points on the planar obstacles are extracted, and the actuator is modeled based on the actual size and setting method of the actuator. The joint positions of the actuator are modeled as spherical structures. Then, the planar obstacle constraints are determined based on the distance between the target points and each spherical structure and the number of planar obstacles in the task scenario.

[0145] The planar obstacle constraint can be determined based on the following formula:

[0146]

[0147] Among them, J π is the planar obstacle constraint; p i is the center of the i-th spherical structure; is the position of the target point; (π) is the distance from the target point to the center of the spherical structure, n o is the number of spherical structures; n π is the number of planar obstacles; f d is the penalty function; Q π is the weighting matrix.

[0148] According to the control method of the intelligent agent provided in the embodiment of the present application, the actuator is modeled as one or more spherical structures, and the intelligent agent is constrained by the distance between the target point on the planar obstacle and each spherical structure to avoid the planar obstacles, thereby effectively avoiding the planar obstacles in the dynamic environment.

[0149] In some embodiments, step 130 may further include:

[0150] The target kinematic model is reactively controlled based on the motion path and target constraints to control the agent.

[0151] In this embodiment, reactive control is a control method that can respond to changes in the mission scenario environment in real time.

[0152] Reactive control can effectively avoid dynamic obstacles when the agent moves rigid articulated objects.

[0153] In the actual execution process, the process of controlling the intelligent agent based on the target kinematic model is constrained by determining the target constraint conditions and the obtained motion path.

[0154] In the actual implementation process, the intelligent agent can be controlled based on the reactive control method of nonlinear model predictive control.

[0155] The reactive control of nonlinear model predictive control can be expressed as:

[0156]

[0157] Where z represents the degree of freedom of the actuator, and u is the speed of each degree of freedom of the control input.

[0158] It is used to represent the relationship between the derivative of z and the control input, which is a nonlinear relationship.

[0159] In some embodiments, the reactive control of the nonlinear model predictive control can be constrained by target constraints so that when the intelligent agent detects a dynamic obstacle during the execution of the target task, the obstacle can be avoided in time.

[0160] According to the control method of the intelligent body provided in the embodiment of the present application, by constraining reactive control through target constraints during the process of controlling the intelligent body, the intelligent body can accurately avoid obstacles when dynamic obstacles are detected, thereby improving the user experience.

[0161] In some embodiments, step 130 may further include:

[0162] Based on the motion path and target constraints, the target kinematic model is reactively controlled to obtain the corresponding degrees of freedom of the actuator and the speed of the corresponding degrees of freedom in the process of the intelligent agent performing the target task;

[0163] The intelligent agent is controlled based on the degrees of freedom corresponding to the actuators and the speeds corresponding to the degrees of freedom.

[0164] In this embodiment, during the actual execution process, the target constraint condition is used to constrain the agent to perform a reactive control process of the target task based on the motion path and the target kinematic model.

[0165] In some embodiments, the target constraint condition may be constructed based on a soft constraint.

[0166] In the actual execution process, for example, taking the target constraints including: execution result constraints, cylindrical obstacle constraints and posture constraints as an example, in the process of solving the reactive control optimization, one of the constraints may not be met. However, for the part that does not meet the constraint, the area where the penalty function gradient of the target constraint is increased will be subject to a large penalty, thereby playing a constraint role, so as to solve the position of the degree of freedom corresponding to the actuator and the speed corresponding to the degree of freedom.

[0167] When the intelligent agent detects the presence of a dynamic obstacle, the intelligent agent is controlled to avoid the obstacle based on the degrees of freedom corresponding to the actuator and the speed corresponding to the degrees of freedom.

[0168] According to the control method of the intelligent agent provided in the embodiment of the present application, a reactive control process is adopted in which the intelligent agent is constrained to execute the target task based on the motion path and the target kinematic model through target constraint conditions. By solving the reactive control process, the degrees of freedom corresponding to the actuator and the speed corresponding to the degrees of freedom are obtained, so that the intelligent agent can avoid obstacles accurately. This obstacle avoidance method can be applied to both static and dynamic scenes, thereby improving the versatility of the intelligent agent and enhancing the user experience.

[0169] The control method of the intelligent body provided in the embodiment of the present application can be executed by the control device of the intelligent body. In the embodiment of the present application, the control device of the intelligent body executing the control method of the intelligent body is taken as an example to illustrate the control device of the intelligent body provided in the embodiment of the present application.

[0170] An embodiment of the present application also provides a control device for an intelligent body.

[0171] like Figure 5 As shown, the control device of the intelligent agent includes: a first processing module 510, a second processing module 520 and a third processing module 530.

[0172] The first processing module 510 is used to generate a motion path of the agent in the task scene based on the target task corresponding to the agent;

[0173] A second processing module 520, configured to determine a target constraint condition based on at least one of an execution result constraint, an obstacle constraint, and a position constraint between the agent and a target object to be interacted with;

[0174] The third processing module 530 is used to control the intelligent agent based on the target constraint conditions, motion path and target kinematic model; the target kinematic model is pre-constructed based on the kinematic information of the actuator of the intelligent agent, the kinematic information of the object to be interacted with and the kinematic information of the mobile mechanism corresponding to the intelligent agent.

[0175] According to the control device of the intelligent agent provided in the embodiment of the present application, the motion path of the intelligent agent in the task scene corresponding to the target task is generated through the target task that the intelligent agent needs to perform, and path instructions are provided for the intelligent agent to perform the target task, so that the intelligent agent can accurately perform the target task based on the motion path under ideal conditions; in the process of the intelligent agent performing the task based on the motion path, the process of controlling the intelligent agent based on the target kinematic model is constrained by executing one or more determined target constraints among the result constraints, obstacle constraints and posture constraints. When a dynamic obstacle is detected, the obstacle can be avoided in time. It is widely used in static scenes and dynamic scenes, has high versatility, and improves the user experience.

[0176] In some embodiments, the second processing module 520 may also be used to:

[0177] Obtain the path error between the actual path of the agent and the motion path, the input smoothness item, and the final posture of the target object to be interacted with;

[0178] The execution result constraints are determined based on the path error, the input smoothness item, and the final posture of the target object to be interacted with.

[0179] In some embodiments, the second processing module 520 may also be used to:

[0180] The pose constraint is determined based on the distance between the mobile mechanism and the target object to be interacted, the target maximum distance, and the target minimum distance.

[0181] In some embodiments, the second processing module 520 may also be used to:

[0182] Based on the size parameters of the obstacles, the obstacles are modeled to obtain a columnar structure;

[0183] Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure;

[0184] Based on the distance between the center of each spherical structure and the columnar structure, the columnar structure obstacle constraint is determined.

[0185] In some embodiments, the second processing module 520 may also be used to:

[0186] Extract target points on planar obstacles;

[0187] Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure;

[0188] The planar obstacle constraints are determined based on the distances between the target point and each spherical structure and the number of planar obstacles.

[0189] In some embodiments, the third processing module 530 may also be used to:

[0190] The target kinematic model is reactively controlled based on the motion path and target constraints to control the agent.

[0191] In some embodiments, the third processing module 530 may also be used to:

[0192] Based on the motion path and target constraints, the target kinematic model is reactively controlled to obtain the corresponding degrees of freedom of the actuator and the speed of the corresponding degrees of freedom in the process of the intelligent agent performing the target task;

[0193] The intelligent agent is controlled based on the degrees of freedom corresponding to the actuators and the speeds corresponding to the degrees of freedom.

[0194] The control device of the intelligent body in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0195] The control device of the intelligent body in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0196] The control device of the intelligent body provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0197] like Figure 3 As shown, an embodiment of the present application further provides an intelligent body, comprising: a sensor, an actuator 310, a moving mechanism 320 and a control device of the intelligent body as described in any of the above embodiments.

[0198] The actuator 310 is used to perform operations such as grabbing, pushing, pulling, and placing.

[0199] The actuator 310 is disposed on the moving mechanism 320 , and the moving mechanism 320 drives the actuator 310 to move.

[0200] The control device of the intelligent body is electrically connected to the sensor, the actuator 310 and the moving mechanism 320 respectively.

[0201] The device for controlling an intelligent agent is used to execute the method for controlling an intelligent agent as described in any of the above embodiments.

[0202] According to the intelligent agent provided in the embodiment of the present application, a motion path of the intelligent agent in the task scene corresponding to the target task is generated through the target task that the intelligent agent needs to perform, and path instructions are provided for the intelligent agent to perform the target task, so that the intelligent agent can accurately perform the target task based on the motion path under ideal conditions; in the process of the intelligent agent performing the task based on the motion path, the process of controlling the intelligent agent based on the target kinematic model is constrained by executing one or more determined target constraints among the result constraints, obstacle constraints and posture constraints. When a dynamic obstacle is detected, the obstacle can be avoided in time. The intelligent agent is widely used in static and dynamic scenes, has high versatility, and improves the user experience.

[0203] In some embodiments, Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the above-mentioned intelligent body control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0204] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0205] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned intelligent body control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0206] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0207] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned control method of the intelligent agent when executed by a processor.

[0208] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0209] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned intelligent body control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0210] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0211] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0212] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0213] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0214] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0215] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A control method for an intelligent agent, characterized in that: include: Based on the target task corresponding to the agent, generating a motion path of the agent in the task scene; Determining a target constraint condition based on at least one of an execution result constraint, an obstacle constraint, and a position constraint between the agent and a target object to be interacted with; The intelligent agent is controlled based on the target constraint conditions, the motion path and the target kinematic model; the target kinematic model is pre-constructed based on the kinematic information of the actuator of the intelligent agent, the kinematic information of the object to be interacted with and the kinematic information of the mobile mechanism corresponding to the intelligent agent.

2. The control method of an intelligent agent according to claim 1, characterized in that: The execution result constraint is determined based on the following steps: Acquire a path error between an actual path of the agent and the motion path, an input smoothness item, and a final posture of the target object to be interacted with; The execution result constraint is determined based on the path error, the input smoothness item, and a final posture of the target object to be interacted with.

3. The control method of an intelligent agent according to claim 1, characterized in that: The pose constraints are determined based on the following steps: The posture constraint is determined based on the distance between the mobile mechanism and the target object to be interacted with, the target maximum distance, and the target minimum distance.

4. The control method of an intelligent agent according to claim 1, characterized in that: The obstacle constraints include: columnar structure obstacle constraints and plane obstacle constraints.

5. The control method of an intelligent agent according to claim 4, characterized in that: The columnar structure obstacle constraint is determined based on the following steps: Based on the size parameters of the obstacle, the obstacle is modeled to obtain a columnar structure; Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure; The columnar structure obstacle constraint is determined based on the distance between the center of each of the spherical structures and the columnar structure.

6. The control method of an intelligent agent according to claim 4, characterized in that: The planar obstacle constraint is determined based on the following steps: Extracting target points on the planar obstacle; Modeling the actuator based on the structural information of the actuator to obtain at least one spherical structure; The planar obstacle constraint is determined based on the distance between the target point and each of the spherical structures and the number of the planar obstacles.

7. The control method of an intelligent agent according to any one of claims 1 to 6, characterized in that: The controlling the intelligent agent based on the target constraint condition, the motion path and the target kinematic model comprises: The target kinematics model is reactively controlled based on the motion path and the target constraint conditions to control the intelligent agent.

8. The method for controlling an intelligent agent according to claim 7, characterized in that: The reactively controlling the target kinematic model based on the motion path and the target constraint condition to control the intelligent agent comprises: Reactively controlling the target kinematic model based on the motion path and the target constraint condition to obtain the degree of freedom corresponding to the actuator and the speed corresponding to the degree of freedom in the process of the agent performing the target task; The intelligent agent is controlled based on the degrees of freedom corresponding to the actuators and the speeds corresponding to the degrees of freedom.

9. A control device for an intelligent agent, characterized in that: include: A first processing module, configured to generate a motion path of the agent in a task scene based on a target task corresponding to the agent; A second processing module, configured to determine a target constraint condition based on at least one of an execution result constraint, an obstacle constraint, and a position constraint between the agent and a target object to be interacted with; A third processing module, configured to control the agent based on the target constraint, the motion path and the target kinematic model; The target kinematic model is constructed in advance based on the kinematic information of the actuator of the agent, the kinematic information of the object to be interacted with, and the kinematic information of the mobile mechanism corresponding to the agent.

10. An intelligent agent, characterized in that: include: sensor; Executive body; A moving mechanism, wherein the actuator is arranged on the moving mechanism; According to the control device of the intelligent body as described in claim 9, the control device of the intelligent body is electrically connected to the sensor, the actuator and the moving mechanism respectively.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the control method of the intelligent agent as described in any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the control method of the intelligent agent as described in any one of claims 1-8 is implemented.

Images