Intelligent body grasping control method and device

Abstract

The present application discloses a grasping control method and device for an intelligent body, which belongs to the field of intelligent bodies. The grasping control method for an intelligent body includes: controlling the joint movement of the actuator of the intelligent body based on a target speed; the target speed includes the speed magnitude and the rotation direction; when it is detected that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame, reducing the speed magnitude and adjusting the rotation direction to the opposite direction of the rotation direction corresponding to the previous image frame; controlling the joint movement of the actuator based on the adjusted target speed; when the number of adjustments is not less than the iteration threshold, determining the skeletal parameters corresponding to the joint of the actuator after the last adjustment as the grasping parameters of the intelligent body. The grasping control method for an intelligent body of the present application can improve the accuracy of the grasping gesture and improve the control precision of the intelligent body.

Description

Technical Field

[0001] The present application belongs to the field of intelligent entities, and in particular, relates to a grasping control method and device for an intelligent entity. Background Art

[0002] Intelligent agents are widely used in daily life and entertainment. In related technologies, taking virtual intelligent agents as an example, most of the intelligent agent actions are realized by pre-setting actions. When performing actions such as grasping, problems such as inaccurate grasping gestures are prone to occur. For embodied intelligent agents, accurate grasping is often impossible, and the control precision and accuracy are low. In addition, corresponding actions need to be set for different objects to be grasped, which consumes a lot of manpower, material resources and time resources, and is inefficient. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a grasping control method and device for an intelligent body, which improves the accuracy of grasping gestures and the control precision of the intelligent body; and there is no need to configure different grasping parameters for different objects to be interacted with, which has high configuration efficiency and saves a lot of manpower, material resources and time costs.

[0004] In a first aspect, the present application provides a grasping control method of an intelligent body, the method comprising:

[0005] Controlling the joint movement of the actuator of the intelligent body based on a target speed; the target speed includes speed magnitude and rotation direction;

[0006] When it is detected that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame; the joint movement of the actuator is controlled based on the adjusted target speed; the collision state includes collision or non-collision;

[0007] When the number of adjustments is not less than the iteration threshold, the skeletal parameters corresponding to the joints of the actuator after the last adjustment are determined as the grasping parameters of the agent.

[0008] According to the grasping control method of the intelligent body of the present application, when it is detected that the collision state between the actuator of the intelligent body and the target object changes, the speed is reduced and the rotation direction is adjusted to the opposite direction, and then the actuator joint movement is controlled based on the adjusted target speed, and the cycle is repeated until convergence is reached. The skeletal parameters of the actuator joints finally obtained are determined as the grasping parameters of the intelligent body, which can make the actuator joints and the object to be interacted reach a state of just contact as much as possible, improve the accuracy of the grasping gesture, and improve the control precision of the intelligent body; and there is no need to configure different grasping parameters for different objects to be interacted, the configuration efficiency is high, and a lot of manpower, material resources and time costs are saved.

[0009] According to one embodiment of the present application, before controlling the joint movement of the actuator of the intelligent body based on the target speed, the method further includes:

[0010] Binding physical bones to respective actuator joints included in the actuator of the agent;

[0011] Based on the association relationship between the joints of the actuators, determining the rotation speed ratio between the associated joints of the actuators and the rotation angle range threshold corresponding to the joints of the actuators;

[0012] Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bone are limited.

[0013] According to an embodiment of the present application, based on the association relationship between the joints of the actuators, determining the rotation speed ratio between the associated joints of the actuators and the rotation angle range threshold corresponding to the joints of the actuators includes:

[0014] Based on the association between the posture category of the intelligent body and the joints of each actuator, the rotation speed ratio between the associated joints of each actuator and the rotation angle range threshold corresponding to each joint of the actuator are determined.

[0015] According to one embodiment of the present application, the controlling the joint movement of the actuator of the intelligent body based on the target speed includes:

[0016] Based on the target speed, determining the rotation speed and rotation direction of the bone parameters corresponding to each of the actuator joints;

[0017] Based on the rotation speed and the rotation direction, the bone parameters corresponding to the actuator joint are adjusted to control the movement of the actuator joint.

[0018] According to an embodiment of the present application, when detecting that the collision state between the actuator of the agent and the target object has changed compared with the previous image frame, reducing the speed and adjusting the rotation direction to the opposite direction of the rotation direction corresponding to the previous image frame include:

[0019] When the actuator of the intelligent body and the target object change from non-collision to collision, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame;

[0020] When the actuator of the intelligent body changes from collision to non-collision with the target object, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame.

[0021] According to an embodiment of the present application, the collision state is detected and determined based on the following steps:

[0022] Abstracting each of the actuator joints into a regular polyhedron;

[0023] During the movement of the joint of the actuator, obtaining the range interval swept by the regular polyhedron in the virtual space;

[0024] The collision state is determined based on the intersection between the contour of the range interval and the target object.

[0025] In a second aspect, the present application provides a grasping control device of an intelligent body, the device comprising:

[0026] A first processing module is used to control the joint movement of the actuator of the intelligent body based on a target speed; the target speed includes a speed magnitude and a rotation direction;

[0027] The second processing module is used to reduce the speed and adjust the rotation direction to the opposite direction of the rotation direction corresponding to the previous image frame when detecting that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame; control the joint movement of the actuator based on the adjusted target speed; the collision state includes collision or non-collision;

[0028] The third processing module is used to determine the skeletal parameters corresponding to the joints of the actuator after the last adjustment as the grasping parameters of the intelligent body when the number of adjustments is not less than the iteration threshold.

[0029] According to the grasping control device of the intelligent body of the present application, when it is detected that the collision state between the actuator of the intelligent body and the target object changes, the speed is reduced and the rotation direction is adjusted to the opposite direction, and then the movement of the actuator joint is controlled based on the adjusted target speed, and the cycle is repeated until convergence is reached. The skeletal parameters of the actuator joints finally obtained are determined as the grasping parameters of the intelligent body, which can make the actuator joints and the object to be interacted reach a state of just contact as much as possible, improve the accuracy of the grasping gesture, and improve the control precision of the intelligent body; and there is no need to configure different grasping parameters for different objects to be interacted, the configuration efficiency is high, and a lot of manpower, material resources and time costs are saved.

[0030] In a third aspect, the present application provides an intelligent agent that performs grasping control based on the grasping control method of the intelligent agent described in the first aspect.

[0031] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the grasping control method of the intelligent body as described in the first aspect above is implemented.

[0032] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the grasping control method of an intelligent body as described in the first aspect above.

[0033] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the grasping control method of an intelligent body as described in the first aspect above.

[0034] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0035] By reducing the speed and adjusting the rotation direction to the opposite direction when detecting a change in the collision state between the actuator of the intelligent body and the target object, and then controlling the movement of the actuator joint based on the adjusted target speed, the cycle is repeated until convergence is reached, and the skeleton parameters of the actuator joints are finally determined as the grasping parameters of the intelligent body. This can maximize the contact state between the actuator joints and the object to be interacted with, improve the accuracy of the grasping gesture, and improve the control precision of the intelligent body. There is no need to configure different grasping parameters for different objects to be interacted with, so the configuration efficiency is high, saving a lot of manpower, material resources and time costs.

[0036] Furthermore, by binding physical bones to each actuator joint included in the actuator of the intelligent body, the posture of the actuator joint can be adjusted by adjusting the bone parameters, thereby achieving precise control of each joint. The operation is simple and convenient, and it is convenient for subsequent simulated collision detection. Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bones are limited, so that the posture of the actuator joint can be controlled within a reasonable range to improve the control accuracy and authenticity of the intelligent body.

[0037] Furthermore, the rotation speed ratio and the rotation angle range threshold are determined by the body posture category of the intelligent agent to limit the movement of the actuator joints. This can be flexibly set based on different intelligent agents to achieve personalized control of the intelligent agent, and the operation is simple, convenient and universal.

[0038] Furthermore, by abstracting the joints of the actuator into regular polyhedrons, collision detection is performed based on whether the range area swept by the regular polyhedron during movement intersects with the object to be interacted with. The operation is simple and fast, and has high detection precision and accuracy.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0041] Figure 1 is a flow chart of a grasping control method of an intelligent body provided in an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of the result of the grasping control method of the intelligent body provided by the embodiment of the present application;

[0043] Figure 3 is a structural schematic diagram of a grasping control device of an intelligent body provided in an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0046] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0047] In conjunction with the accompanying drawings, the grasping control method of an intelligent body, the grasping control device of an intelligent body, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0048] Among them, the grasping control method of the intelligent body can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

[0049] The grasping control method of an intelligent entity provided in an embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the grasping control method of the intelligent entity. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The grasping control method of an intelligent entity provided in an embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0050] It should be noted that the grasping control method of the intelligent body of the present application can be applied to the field of virtual reality or the field of embodied intelligence.

[0051] like Figure 1 As shown, the grasping control method of the intelligent agent includes: step 110, step 120 and step 130.

[0052] Step 110, controlling the joint movement of the actuator of the intelligent body based on the target speed; the target speed includes the speed magnitude and the rotation direction;

[0053] In this step, the agent may be a physical object, such as embodied intelligence or a robot, etc.; or, the agent may be a virtual object.

[0054] The virtual objects may include, but are not limited to, virtual characters, virtual animals, and other virtual objects with motion characteristics.

[0055] It is understandable that in a virtual scene, the agent can interact with other virtual objects in the virtual scene.

[0056] Take the virtual scene including multiple virtual objects such as a table and a cup placed on the table as an example. Figure 2 As shown, the virtual character can interact with the water cup by performing actions such as grabbing the water cup and putting down the water cup.

[0057] The actuator joints are joints corresponding to the actuators when the intelligent agent interacts with other virtual objects. Taking the intelligent agent as a virtual character as an example, the actuator joints may be hand joints of the virtual character.

[0058] It can be understood that an actuator may include one or more actuator joints.

[0059] For example, continue to refer to Figure 2 In the case where the actuator is a virtual hand, the virtual hand may include multiple virtual fingers, each finger may include multiple finger joints, and these finger joints are all actuator joints.

[0060] In some embodiments, an actuator joint may also include a parent joint associated with a child joint.

[0061] Continuing with the finger joint as an example, the wrist joint corresponding to the virtual wrist can serve as the parent joint corresponding to the finger joint, that is, the actuator joint can include: a wrist joint and a plurality of finger joints associated with the wrist joint.

[0062] It is understandable that, based on the motion characteristics of the virtual character, there may be a relative motion relationship between the joints of the actuators, such as a linkage motion, for example, there is a linkage motion relationship between multiple finger joints on the thumb.

[0063] The target velocity is a vector value used to control the motion of the actuator joint, and can be expressed as the displacement of the actuator joint relative to the previous image frame.

[0064] The target speed includes: speed magnitude and rotation direction.

[0065] The initial value of the speed may be set to a fixed value, such as 10° or 11°, for example, in each image frame, the finger joint is controlled to rotate at a speed of 10°.

[0066] The direction of rotation includes: inward rotation or outward rotation.

[0067] The inward rotation refers to the rotation of the actuator joint toward the object to be interacted with, and the outward rotation refers to the rotation of the actuator joint toward a direction away from the object to be interacted with.

[0068] In some embodiments, before step 110, the method may further include:

[0069] Bind physical bones to the actuator joints of the actuator of the intelligent body respectively;

[0070] Based on the association relationship between the joints of the actuators, determining the rotation speed ratio between the joints of the actuators having the association relationship and the rotation angle range threshold corresponding to the joints of the actuators;

[0071] Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bone are limited.

[0072] In this embodiment, each physical bone corresponds to a bone parameter, and the position and posture of the actuator joint can be adjusted by adjusting the bone parameter.

[0073] Different actuator joints may have the same or different corresponding movement speeds.

[0074] The rotation speed ratio is the ratio of the movement speeds between multiple actuator joints that have an associated relationship.

[0075] The rotation speed ratio can be user-defined or determined based on simulation experimental data.

[0076] Let's continue to use the actuator as a virtual hand, and each virtual finger includes three finger joints as an example:

[0077] The rotation speed ratios among the root bone joint, middle bone joint and terminal bone joint of the thumb are 1:1:2 or 1:1:1.8, etc., which can be configured based on specific engineering requirements and are not limited in this application.

[0078] The rotation speed ratios of the other fingers including the root bone joints, middle bone joints and terminal bone joints are 2.5:2:1 respectively.

[0079] Of course, in other embodiments, different rotation speed ratios may be set for other fingers separately, and this application does not limit this.

[0080] The rotation angle range threshold is the maximum angle range that the actuator joint can rotate.

[0081] Different actuator joint mechanisms may have different corresponding rotation angle range thresholds.

[0082] The rotation angle range threshold may be user-defined or may be determined based on simulation experiment data.

[0083] For example, for the multiple finger joints included in the thumb, the rotation angle range thresholds can be set to: root bone joint: 20°~50°, middle bone joint: -10°~30°, end bone: -10°~20°.

[0084] For the multiple finger joints included in the index finger, the rotation angle range thresholds can be set to: root bone joint: -15°~30°, middle bone joint: -15°~20°, and end bone: -30°~20°.

[0085] Of course, in other embodiments, the rotation angle range threshold may also be set to other values, which is not limited in this application.

[0086] Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bone are restricted to control the position and posture of the actuator joint within a reasonable range.

[0087] According to the grasping control method of the intelligent body provided in the embodiment of the present application, by binding physical bones to each actuator joint included in the actuator of the intelligent body, the posture of the actuator joint can be adjusted by adjusting the bone parameters, thereby achieving precise control of each joint. The operation is simple and convenient, and it is convenient for subsequent simulated collision detection. Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bone are limited, so that the posture of the actuator joint can be controlled within a reasonable range to improve the control accuracy and authenticity of the intelligent body.

[0088] In some embodiments, based on the association relationship between the actuator joints, determining the rotation speed ratio between the actuator joints having the association relationship and the rotation angle range threshold corresponding to each actuator joint may include:

[0089] Based on the body posture category of the intelligent body and the association relationship between the joints of each actuator, the rotation speed ratio between the associated actuator joints and the rotation angle range threshold corresponding to each actuator joint are determined.

[0090] In this embodiment, the body type categories include: human body or animal.

[0091] Taking the human body as an example, body shape categories may also include: adult or child.

[0092] It should be noted that the number of actuator joints, the association relationship between the actuator joints, and the motion characteristics of the actuator joints may be different for different body types.

[0093] In the actual execution process, the rotation speed ratio and the rotation angle range threshold corresponding to each actuator joint can be set based on the specific body type of the intelligent agent.

[0094] For example, the rotation angle range thresholds corresponding to the finger joints of the thumbs of adults and children are different. For the finger joints of the thumbs of children, the rotation angle range thresholds can be set to: root bone joint: 20°~50°, middle bone joint: -10°~30°, terminal bone: -10°~20°.

[0095] According to the grasping control method of the intelligent body provided in the embodiment of the present application, the rotation speed ratio and the rotation angle range threshold are determined by the body posture category of the intelligent body to limit the movement of the actuator joints. It can be flexibly set based on different intelligent bodies to achieve personalized control of the intelligent body, and the operation is simple and convenient, and has universal applicability.

[0096] In some embodiments, step 110 may include:

[0097] Based on the target speed, determine the rotation speed and rotation direction of the bone parameters corresponding to each actuator joint;

[0098] Based on the rotation speed and rotation direction, the bone parameters corresponding to the actuator joints are adjusted to control the actuator joint movement.

[0099] In this embodiment, the target speed includes: speed magnitude and rotation direction.

[0100] The rotation speed can be determined based on the speed magnitude.

[0101] Taking the rotation speed ratio of 2.5:2:1 among the root bone joint, middle bone joint and terminal bone joint as an example, when the speed is 1° per frame, the rotation speed among the bone joint, middle bone joint and terminal bone joint can be set to rotate 2.5°, 2° and 1° per frame respectively.

[0102] The direction of rotation can be determined based on the direction of rotation, ie the direction of rotation is consistent with the direction of rotation.

[0103] Step 120: when it is detected that the collision state between the actuator of the agent and the target object has changed compared with the previous image frame, the speed is reduced, and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame; the joint movement of the actuator is controlled based on the adjusted target speed;

[0104] In this step, the target object is the object to be interacted with.

[0105] The target object may be any virtual object included in the virtual environment.

[0106] The collision status includes: collision or no collision.

[0107] In the event of a collision, it indicates that the agent has penetrated the mold.

[0108] If no collision occurs, it indicates that the agent is not in contact with the object to be interacted with or just touches it.

[0109] The change in collision state may include: changing from no collision to collision, or changing from collision to no collision.

[0110] In some embodiments, collision detection may be performed using ray detection, or any other feasible method may be selected for collision detection. The present application will specifically explain collision detection in the following embodiments, which will not be elaborated here.

[0111] In the actual execution process, collision detection is performed for each image frame. When no collision state is detected in the current image frame compared with the previous image frame, the current target speed can be maintained to control the joint movement of the actuator.

[0112] When it is detected in the current image frame that the collision state has changed compared to the previous image frame, the speed may be reduced and the rotation direction may be adjusted to the opposite direction, so as to control the actuator joint to move in the opposite direction in the next image frame.

[0113] In some embodiments, when it is detected that the collision state between the actuator of the agent and the target object has changed compared to the previous image frame, reducing the speed may include:

[0114] Adjust the speed to the target ratio of the speed corresponding to the previous image frame.

[0115] In this embodiment, the target ratio is a value between 0 and 1.

[0116] For example, the target ratio may be set to 0.5, 0.7, 0.8 or other values, which are not limited in this application.

[0117] It should be noted that the target ratio selected for adjusting the target speed each time may be the same value or may be different values.

[0118] For example, taking the rotation speeds corresponding to the joints of the thumb and fingers as: skeletal joints, middle skeletal joints, terminal skeletal joints: 2.5°, 2° and 1°, and the target ratio is 0.5, the new rotation speeds after one adjustment become: skeletal joints, middle skeletal joints, terminal skeletal joints: 1.25°, 1° and 0.5°.

[0119] In some embodiments, step 120 may include:

[0120] When the actuator of the intelligent body and the target object change from non-collision to collision, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame;

[0121] When the actuator of the intelligent body changes from collision to non-collision with the target object, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame.

[0122] In this embodiment, taking the scene of grabbing a water cup as an example, in the initial state, the intelligent agent spreads out all fingers, and then controls the skeletal joints, middle skeletal joints and terminal skeletal joints of the intelligent agent's thumb finger to rotate in the direction of grasping the water cup at a rotation speed of 2.5°, 2° and 1° per frame respectively. In the current image frame, it is detected that the thumb finger and the water cup have changed from no collision to collision, and the collision state change is determined. Taking the target ratio of 0.5 as an example, the rotation speed can be adjusted to: the skeletal joints, middle skeletal joints and terminal skeletal joints rotate at 1.25°, 1° and 0.5° per frame respectively, and the rotation direction is adjusted to the direction away from the cup, to obtain a new rotation direction and rotation speed, and at the beginning of the next image frame, the movement of each joint of the thumb is controlled based on the new rotation direction and rotation speed, that is, the thumb is controlled to spread out based on the rotation speed, and continue to detect whether a collision occurs.

[0123] If the finger collides with the water cup for multiple consecutive frames, indicating that the collision state has not changed, the current target speed is maintained unchanged and the finger continues to be controlled to unfold.

[0124] When it is detected that no collision occurs between the finger and the cup, indicating that the collision state has changed, the rotation speed can be adjusted to 0.625°, 0.5° and 0.25° per frame for the skeletal joint, the middle skeletal joint and the terminal skeletal joint respectively, and the rotation direction is adjusted to the direction toward the cup to obtain a new rotation direction and rotation speed. At the beginning of the next image frame, the movement of each joint of the thumb is controlled based on the new rotation direction and rotation speed, that is, the thumb is controlled to be clenched based on the rotation speed, and detection of whether a collision occurs continues.

[0125] It can be understood that as the target speed decreases, convergence can be achieved after multiple iterations, so that the actuator joints and the objects to be interacted can be closer to a state of just contacting without collision, thereby reducing the risk of penetration.

[0126] In some embodiments, when a parent joint exists for an actuator joint, the parent joint can also be controlled synchronously to move.

[0127] For example, the rotation direction of the parent joint is synchronously adjusted to the opposite direction, and the rotation speed of the parent joint is adjusted to the previous target ratio.

[0128] Step 130: When the number of adjustments is not less than the iteration threshold, the skeletal parameters corresponding to the actuator joints adjusted last time are determined as the grasping parameters of the intelligent body.

[0129] In this step, the grasping parameters are the parameters corresponding to when the agent performs the grasping action.

[0130] After obtaining the grasping parameters, the movement of the agent can be controlled based on the grasping parameters to perform grasping actions and interact with other virtual objects in the virtual environment.

[0131] The iteration threshold may be user-defined, or the iteration threshold may be set accordingly based on the setting of the target ratio.

[0132] It can be understood that the larger the value of the target ratio is, the larger the value of the iteration threshold is.

[0133] Taking the target ratio of 0.5 as an example, the iteration threshold can be set to 10 or 15, etc., which is not limited in this application.

[0134] When the number of adjustments to the target speed reaches the iteration threshold, it can be approximately considered that in the current image frame, the actuator joints and the objects to be interacted with have just reached a state of contact and no penetration has occurred. The bone parameters corresponding to the actuator joints in the current image frame are determined as the grasping parameters used by the intelligent agent to perform the grasping action with each object to be interacted with.

[0135] According to the method of the present application, the same intelligent agent can use the same grasping parameters to configure any object to be interacted with, without having to configure different grasping parameters for different objects to be interacted with. The operation is simple and convenient, with universal applicability, saving a lot of manpower, material resources and time costs.

[0136] In addition, by adjusting the contact state between the actuator joints and the object to be interacted with in a convergent iterative manner with decreasing target speed, the actuator joints and the object to be interacted with can be made to be in a state of just-in-contact with each other as much as possible. For the control of virtual objects, the probability of penetration can be further reduced, the accuracy of the grasping gesture can be improved, and the control accuracy of the intelligent body can be improved.

[0137] According to the grasping control method of the intelligent body provided in the embodiment of the present application, when it is detected that the collision state between the actuator of the intelligent body and the target object changes, the speed is reduced and the rotation direction is adjusted to the opposite direction, and then the actuator joint movement is controlled based on the adjusted target speed, and the cycle is repeated until convergence is reached. The skeletal parameters of the actuator joints finally obtained are determined as the grasping parameters of the intelligent body. This can make the actuator joints and the object to be interacted with reach a state of just contact as much as possible, improve the accuracy of the grasping gesture, and improve the control precision of the intelligent body. There is no need to configure different grasping parameters for different objects to be interacted with, and the configuration efficiency is high, saving a lot of manpower, material resources and time costs.

[0138] The following is an explanation of how to implement collision detection.

[0139] In some embodiments, the collision state may be determined based on the following steps:

[0140] Abstract each actuator joint into a regular polyhedron;

[0141] During the movement of the joints of the actuator, the range interval swept by the regular polyhedron in the virtual space is obtained;

[0142] The collision state is determined based on the intersection between the contour of the range interval and the target object.

[0143] In this embodiment, the regular polyhedron may include a cylinder or a cuboid, etc.

[0144] The joints of the actuator correspond one to one with the regular polyhedron, and the regular polyhedron moves correspondingly with the movement of the joints of the actuator.

[0145] The regular polyhedron can be set accordingly based on the length and width of the actuator.

[0146] The target object is the object to be interacted with.

[0147] The target object may be any virtual object included in the virtual environment.

[0148] During the movement process, based on the movement status, position information and size information of the regular polyhedron in each image frame, the range interval that the regular polyhedron passes through in the virtual space is determined, the outline of the range interval and the outline of the object to be interacted with are extracted, and it is detected whether the outline of the range interval intersects with the outline of the object to be interacted with.

[0149] When the contour of the range interval does not intersect the contour of the object to be interacted with, it can be approximately considered that the joint of the actuator does not collide with the object to be interacted with.

[0150] When the contour of the range interval intersects the contour of the object to be interacted with, it is determined that the joint of the actuator collides with the object to be interacted with.

[0151] According to the grasping control method of the intelligent body provided in the embodiment of the present application, by abstracting the joints of the actuator into regular polyhedrons, collision detection is performed based on whether the range area swept by the regular polyhedron during the movement intersects with the object to be interacted. The operation is simple and fast, and has high detection precision and accuracy.

[0152] The grasping control method of the intelligent body provided in the embodiment of the present application can be executed by the grasping control device of the intelligent body. In the embodiment of the present application, the grasping control device of the intelligent body executing the grasping control method of the intelligent body is taken as an example to illustrate the grasping control device of the intelligent body provided in the embodiment of the present application.

[0153] An embodiment of the present application also provides a grasping control device for an intelligent body.

[0154] like Figure 3 As shown, the grasping control device of the intelligent body includes: a first processing module 310, a second processing module 320 and a third processing module 330.

[0155] The first processing module 310 is used to control the joint movement of the actuator of the intelligent body based on the target speed; the target speed includes the speed magnitude and the rotation direction;

[0156] The second processing module 320 is used to reduce the speed and adjust the rotation direction to the opposite direction of the rotation direction corresponding to the previous image frame when detecting that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame; control the joint movement of the actuator based on the adjusted target speed; the collision state includes collision or non-collision;

[0157] The third processing module 330 is used to determine the skeletal parameters corresponding to the actuator joints adjusted last time as the grasping parameters of the intelligent body when the number of adjustments is not less than the iteration threshold.

[0158] According to the grasping control device of the intelligent body provided in the embodiment of the present application, when it is detected that the collision state between the actuator of the intelligent body and the target object changes, the speed is reduced and the rotation direction is adjusted to the opposite direction, and then the movement of the actuator joint is controlled based on the adjusted target speed, and the cycle is repeated until convergence is reached. The skeletal parameters of the actuator joints finally obtained are determined as the grasping parameters of the intelligent body. This can make the actuator joints and the object to be interacted with reach a state of just-in-contact as much as possible, improve the accuracy of the grasping gesture, and improve the control precision of the intelligent body. There is no need to configure different grasping parameters for different objects to be interacted with, and the configuration efficiency is high, saving a lot of manpower, material resources and time costs.

[0159] In some embodiments, the apparatus may further include a fourth processing module, configured to:

[0160] Before controlling the movement of the actuator joint of the intelligent body based on the target speed, physical bones are respectively bound to each actuator joint included in the actuator of the intelligent body;

[0161] Based on the association relationship between the joints of the actuators, determining the rotation speed ratio between the joints of the actuators having the association relationship and the rotation angle range threshold corresponding to the joints of the actuators;

[0162] Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bone are limited.

[0163] In some embodiments, the fourth processing module may also be used to:

[0164] Based on the body posture category of the intelligent body and the association relationship between the joints of each actuator, the rotation speed ratio between the associated actuator joints and the rotation angle range threshold corresponding to each actuator joint are determined.

[0165] In some embodiments, the first processing module 310 may also be used to:

[0166] Based on the target speed, determine the rotation speed and rotation direction of the bone parameters corresponding to each actuator joint;

[0167] Based on the rotation speed and rotation direction, the bone parameters corresponding to the actuator joints are adjusted to control the actuator joint movement.

[0168] In some embodiments, the second processing module 320 may also be used to:

[0169] When the actuator of the intelligent body and the target object change from non-collision to collision, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame;

[0170] When the actuator of the intelligent body changes from collision to non-collision with the target object, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame.

[0171] In some embodiments, the apparatus may further include a fifth processing module, configured to:

[0172] Abstract each actuator joint into a regular polyhedron;

[0173] During the movement of the joints of the actuator, the range interval swept by the regular polyhedron in the virtual space is obtained;

[0174] The collision state is determined based on the intersection between the contour of the range interval and the target object.

[0175] The grasping control device of the intelligent body in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0176] The grasping control device of the intelligent body in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0177] The grasping control device of the intelligent body provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0178] The embodiment of the present application also provides an intelligent agent.

[0179] The agent can be a physical object, such as embodied intelligence or a robot; or, the agent can be a virtual object.

[0180] The virtual objects may include, but are not limited to, virtual characters, virtual animals, and other virtual objects with motion characteristics.

[0181] It is understandable that in a virtual scene, the agent can interact with other virtual objects in the virtual scene.

[0182] Taking a virtual scene including a table and a cup placed on the table as an example, the virtual character can interact with the cup by performing actions such as grabbing the cup and putting down the cup.

[0183] An agent may include one or more actuator joints.

[0184] The actuator joints are joints corresponding to the actuators when the intelligent agent interacts with other virtual objects. Taking the intelligent agent as a virtual character as an example, the actuator joints may be hand joints of the virtual character.

[0185] It can be understood that an actuator may include one or more actuator joints.

[0186] For example, when the actuator is a virtual hand, the virtual hand may include multiple virtual fingers, each finger may include multiple finger joints, and these finger joints are all actuator joints.

[0187] In some embodiments, an actuator joint may also include a parent joint associated with a child joint.

[0188] Continuing with the finger joint as an example, the wrist joint corresponding to the virtual wrist can serve as the parent joint corresponding to the finger joint, that is, the actuator joint can include: a wrist joint and a plurality of finger joints associated with the wrist joint.

[0189] The intelligent agent performs grasping control based on the grasping control method of the intelligent agent as described in any of the above embodiments.

[0190] According to the intelligent body provided in the embodiment of the present application, when it is detected that the collision state between the actuator of the intelligent body and the target object changes, the speed is reduced and the rotation direction is adjusted to the opposite direction, and then the actuator joint movement is controlled based on the adjusted target speed, and the cycle is repeated until convergence is reached. The skeletal parameters of the actuator joints finally obtained are determined as the grasping parameters of the intelligent body, which can make the actuator joints and the object to be interacted with reach a state of just contact as much as possible, improve the accuracy of the grasping gesture, and improve the control precision of the intelligent body; and there is no need to configure different grasping parameters for different objects to be interacted with, the configuration efficiency is high, and a lot of manpower, material resources and time costs are saved.

[0191] In some embodiments, Figure 4 As shown, an embodiment of the present application also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, each process of the above-mentioned intelligent body grasping control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0192] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0193] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned intelligent body grasping control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0194] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0195] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned grasping control method of the intelligent body when executed by a processor.

[0196] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0197] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned intelligent body grasping control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0198] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0199] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0200] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0201] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0202] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0203] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A grasping control method of an intelligent agent, characterized in that: include: Controlling the joint movement of the actuator of the intelligent body based on a target speed; the target speed includes speed magnitude and rotation direction; When it is detected that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame; the joint movement of the actuator is controlled based on the adjusted target speed; the collision state includes collision or non-collision; When the number of adjustments is not less than the iteration threshold, the skeletal parameters corresponding to the joints of the actuator after the last adjustment are determined as the grasping parameters of the agent.

2. The grasping control method of an intelligent body according to claim 1, characterized in that: Before controlling the joint movement of the actuator of the intelligent body based on the target speed, the method further includes: Binding physical bones to respective actuator joints included in the actuator of the agent; Based on the association relationship between the joints of the actuators, determining the rotation speed ratio between the associated joints of the actuators and the rotation angle range threshold corresponding to the joints of the actuators; Based on at least one of the rotation speed ratio and the rotation angle range threshold, the bone parameters corresponding to the physical bone are limited.

3. The grasping control method of an intelligent body according to claim 2, characterized in that: The determining of the rotation speed ratio between the associated actuator joints and the rotation angle range threshold corresponding to each actuator joint based on the associated relationship between the actuator joints includes: Based on the association between the posture category of the intelligent body and the joints of each actuator, the rotation speed ratio between the associated joints of each actuator and the rotation angle range threshold corresponding to each joint of the actuator are determined.

4. The grasping control method of an intelligent body according to any one of claims 1 to 3, characterized in that: The method of controlling the joint motion of the actuator of the intelligent body based on the target speed includes: Based on the target speed, determining the rotation speed and rotation direction of the bone parameters corresponding to each of the actuator joints; Based on the rotation speed and the rotation direction, the bone parameters corresponding to the actuator joint are adjusted to control the movement of the actuator joint.

5. The grasping control method of an intelligent body according to any one of claims 1 to 3, characterized in that: When it is detected that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame, reducing the speed and adjusting the rotation direction to the opposite direction of the rotation direction corresponding to the previous image frame include: When the actuator of the intelligent body and the target object change from non-collision to collision, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame; When the actuator of the intelligent body changes from collision to non-collision with the target object, the speed is reduced and the rotation direction is adjusted to the opposite direction of the rotation direction corresponding to the previous image frame.

6. The grasping control method of an intelligent body according to any one of claims 1 to 3, characterized in that: The collision state is determined based on the following steps: Abstracting each of the actuator joints into a regular polyhedron; During the movement of the joint of the actuator, obtaining the range interval swept by the regular polyhedron in the virtual space; The collision state is determined based on the intersection between the contour of the range interval and the target object.

7. A grasping control device for an intelligent agent, characterized in that: include: A first processing module is used to control the joint movement of the actuator of the intelligent body based on a target speed; the target speed includes a speed magnitude and a rotation direction; A second processing module is used to reduce the speed and adjust the rotation direction to the opposite direction of the rotation direction corresponding to the previous image frame when detecting that the collision state between the actuator of the intelligent body and the target object has changed compared with the previous image frame; Controlling the joint movement of the actuator based on the adjusted target speed; the collision state includes collision or non-collision; The third processing module is used to determine the skeletal parameters corresponding to the joints of the actuator after the last adjustment as the grasping parameters of the intelligent body when the number of adjustments is not less than the iteration threshold.

8. An intelligent agent, characterized in that: The intelligent agent performs grasping control based on the grasping control method of the intelligent agent as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the grasping control method of the intelligent agent as described in any one of claims 1-6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the grasping control method of the intelligent body as described in any one of claims 1-6 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the grasping control method of the intelligent agent as described in any one of claims 1-6 is implemented.

Images