Control Method, Device, Robot and Storage Medium of Robot

Abstract

The embodiments of the present application provide a control method, device, robot and storage medium for a robot, relating to the technical fields of robots and automatic control. The method includes: during the process of the robot moving towards a target object, controlling the end effector of the robot to collide with the target object; controlling the relative rotation between the end effector and the target object to adjust the target object to a target pose suitable for being grasped by the end effector; after the target object is adjusted to the target pose, controlling the end effector to grasp the target object; wherein, during the entire stage from the collision between the end effector and the target object to the end effector grasping the target object, the robot is in a non-fixed state. The technical solution provided by the embodiments of the present application can achieve dynamic object grasping by the robot and improve the control efficiency of the robot for grasping objects.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical fields of robots and automatic control, and particularly to a control method, device, robot, and storage medium for a robot. Background Art

[0002] A robot is an automatic control machine that simulates human operations, and the part thereof used to simulate human hand operations can be referred to as an end effector.

[0003] In the related art, after the end effector moves to a position very close to the target object and stops moving, it starts to move again after grasping the target object, that is, the target object is grasped by means of static picking. However, in the related art, the object can be grasped only after the speed of the end effector is reduced to 0, which takes a long time and the control efficiency of the robot for grasping the object is low. Summary of the Invention

[0004] The embodiments of the present application provide a control method, device, robot, and storage medium for a robot, which can enable the robot to dynamically grasp an object and improve the control efficiency of the robot for grasping the object. The technical solutions are as follows:

[0005] According to one aspect of the embodiments of the present application, a control method for a robot is provided. The method includes:

[0006] During the process of the robot moving towards the target object, controlling the end effector of the robot to collide with the target object;

[0007] Controlling the end effector and the target object to rotate relative to each other to adjust the target object to a target pose suitable for the end effector to grasp;

[0008] After the target object is adjusted to the target pose, controlling the end effector to grasp the target object; wherein, during the entire stage from the end effector colliding with the target object to the end effector grasping the target object, the robot is in a non-fixed state.

[0009] According to one aspect of the embodiments of the present application, a control device for a robot is provided. The device includes:

[0010] A collision control module, configured to control the end effector of the robot to collide with the target object during the process of the robot moving towards the target object;

[0011] A rotation control module, configured to control the end effector and the target object to rotate relative to each other to adjust the target object to a target pose suitable for the end effector to grasp;

[0012] The grasping control module is used to control the end effector to grasp the target object after the target object is adjusted to the target pose; wherein, from the moment when the end effector collides with the target object to the complete stage when the end effector grasps the target object, the robot is in a non-fixed state.

[0013] According to one aspect of the embodiments of the present application, a robot is provided. The robot includes a processor and a memory. A computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the above-mentioned control method of the robot.

[0014] According to one aspect of the embodiments of the present application, a computer-readable storage medium is provided. A computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the above-mentioned control method of the robot.

[0015] According to one aspect of the embodiments of the present application, a computer program product is provided. The computer program product includes a computer program, and the computer instruction is stored in a computer-readable storage medium. The processor of the robot reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the robot executes the above-mentioned control method of the robot.

[0016] The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

[0017] By controlling the robot to move towards the target object and collide with the target object during the movement, so that the end effector of the robot rotates relative to the target object until the target object is adjusted to the target pose, then control the end effector to grasp the target object. During the grasping process, the robot is in a non-fixed state. The robot does not need to reduce its speed to 0 and can grasp the target object in a non-fixed state, that is, dynamic grasping of the object is realized, the time required for grasping the object is saved, and the control efficiency of the robot for grasping the object is improved.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of the control method of the robot provided by an embodiment of the present application;

[0020] Figure 2 is a flowchart of the control method of the robot provided by an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the control method of the robot provided by another embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the control method of the robot provided by another embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of the simulation result provided by an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the simulation result provided by another embodiment of the present application;

[0025] Figure 7 It is a schematic diagram of the control method of the robot provided by another embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the control method of the robot provided by another embodiment of the present application;

[0027] Figure 9 It is a block diagram of the control device of the robot provided by an embodiment of the present application;

[0028] Figure 10 It is a block diagram of the control device of the robot provided by another embodiment of the present application;

[0029] Figure 11 It is a block diagram of the robot provided by an embodiment of the present application. Detailed implementation manners

[0030] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with the present application. On the contrary, they are only examples of the methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] As Figure 1 shown, the control method of the robot provided by the embodiment of the present application includes the following parts:

[0032] Approach: As Figure 1 (a) shown, before the end effector 11 of the robot touches the target object 12, it first moves towards the target object 12 to approach the target object 12;

[0033] Collision: As Figure 1 (b) shown, after continuously approaching the target object 12, the end effector 11 will collide with the target object 12;

[0034] Rotation: As Figure 1As shown in (c), after the end effector 11 collides with the target object 12, the two will rotate relative to each other around the contact position, and the angle between the two gradually decreases;

[0035] Grasping: When the angle between the end effector 11 and the target object 12 becomes 0 as it rotates, it means that the end effector 11 grasps the target object 12; among them, the grasping can be a non-grasping type as shown in Figure 1 (d1), that is, only one contact surface is maintained between the end effector 11 and the target object 12 to keep relative static, and the fingers of the end effector 11 do not participate in restricting the displacement of the target object 12; the grasping can also be a grasping type as shown in Figure 1 (d2), that is, the fingers of the end effector 11 are used to restrict the displacement of the target object 12, presenting a form of "holding" the target object 12.

[0036] That is to say, the process of the robot grasping the target object can be Figure 1 "a - b - c - d1" in, that is, "approach - collision - rotation - non-grasping type grasping"; it can also be Figure 1 "a - b - c - d2" in, that is, "approach - collision - rotation - grasping type grasping".

[0037] Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, a theory, method, technology and application system that perceives the environment, acquires knowledge and uses knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is also to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

[0038] Artificial intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0039] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0040] To fully exploit the potential of robots in dexterous manipulation, the embodiments of this application propose a complete dynamic grasping process to achieve smooth grasping of a target object by the end effector of the robot. This process starts from the stage of approaching the target object, goes through the stage of colliding with the target object, makes the target object rotate around the collision point, until the end effector grasps the target object (non - grasping grasp) or uses the fingers of the end effector to grasp the target object (grasping grasp). The embodiments of this application start from the articulated body dynamics based on spatial vectors and derive a unified model of this hybrid dynamic operation process, which is specifically represented as approach - collision - rotation - non - grasping grasp / grasping grasp. Then, the whole process is formulated as a free - terminal - constrained multi - stage optimal control problem (OCP, Optimal Control Problem). We extend the traditional differential dynamic programming (DDP, Differential Dynamic Programming) to solve this free - terminal OCP problem, where the backward pass of DDP involves a constrained quadratic programming (QP, Quadratic Programming) problem and is solved using the primal - dual augmented Lagrangian (PDAL, Primal Dual Augmented Lagrangian) method. The effectiveness of this method in robot dynamic grasping is verified through simulation and experiments.

[0041] For the method provided by the embodiments of this application, the execution subject of each step can be a robot, which refers to an electronic device with data calculation, processing, and storage capabilities. The robot can be a quadruped robot, a biped robot, a wheel - legged robot, a robotic arm (manipulator), etc. The robot provided by the embodiments of this application can be applied to scenarios such as industry (such as industrial robots), service (such as serving robots), entertainment (such as performing robots), healthcare (such as medical robots), etc. The embodiments of this application do not make specific limitations on this.

[0042] The solution provided by the embodiments of this application relates to technologies such as automatic control of artificial intelligence and can achieve the control of robots. Specific illustrations are given through the following embodiments.

[0043] Please refer to Figure 2, which shows a flowchart of a control method for a robot provided by an embodiment of the present application. In this embodiment, it is exemplified that this method is applied to the robot introduced above. This method may include the following steps (210-230):

[0044] Step 210, during the process of the robot moving towards the target object, control the end effector of the robot to collide with the target object.

[0045] In some embodiments, a robot is an intelligent machine capable of performing semi-automatic or full-automatic operations. The robot can perform tasks such as operations and movements through programming and automatic control. The robot can be a humanoid controlled machine (such as a humanoid robot including a head, body, arms, and legs), or a controlled manipulator, or other forms. The embodiments of the present application do not make specific limitations on this.

[0046] In some embodiments, the robot moving towards the target object may only refer to a part of the robot (including the part of the end effector) moving towards the target object, and other parts of the robot can remain stationary or move in other directions. For example, a humanoid robot can have only its arm moving towards the target object, and other parts such as its legs or head can remain stationary; the robot moving towards the target object can also mean that each part of the robot is moving towards the target object, that is, the whole robot is moving towards the target object. For example, a wheeled robot or a wheel-legged robot moves the whole towards the target object, and of course its end effector also moves together. For the robot, regardless of whether other components move, the end effector is always in a non-fixed state (such as keeping moving).

[0047] In some embodiments, the end effector is located at the end part of the robot. The end effector of the robot is used to perform tasks such as installation, handling, grinding, spraying, painting, and detection. According to different tasks, different end effectors can be configured for the robot. Exemplarily, the end effector can be configured as an end two-finger gripper, and the end two-finger gripper can be used to perform repetitive movement tasks such as object grasping and handling. Controlling the end effector to move towards the target object, the end effector of the robot will come into contact and collide with the target object. Specifically, reference can be made to the above Figure 1 (a) and (b).

[0048] Step 220, control the end effector and the target object to generate relative rotation to adjust the target object to a target pose suitable for the end effector to grasp.

[0049] In some embodiments, after the end effector of the robot collides with the target object, the relative rotation between the end effector and the target object can be generated by controlling the motion parameters (such as displacement, velocity, acceleration, etc.) and posture of the end effector. In some embodiments, during the relative rotation between the end effector and the target object, the angle between the target object and the end effector becomes smaller and smaller (as shown in Figure 1 (c) above), so that the target object rotates relative to the end effector and finally lands on the end effector. The target pose may refer to that the target object and the end effector maintain surface contact, and the contact surface is horizontal or inclined downward (obviously, the inclination angle should not be too large), so that the end effector can lift the target object through the contact surface, facilitating the end effector to grasp the target object.

[0050] In some embodiments, in order for the end effector and the target object to generate relative rotation, rather than the target object being pushed by the end effector without rotation, obviously, the position where the end effector initially contacts and collides with the target object should be below the center of mass of the target object.

[0051] Step 230, after the target object is adjusted to the target pose, control the end effector to grasp the target object.

[0052] Among them, from the moment when the end effector collides with the target object to the moment when the end effector grasps the target object in the complete stage, the robot is in a non-fixed state. For example, the robot moves during the complete stage. The complete stage includes a collision stage, a rotation stage, and a grasping stage, and the robot grasps the target object in a non-fixed state. Optionally, the complete stage further includes an approaching stage before the collision stage.

[0053] In some embodiments, the end effector grasping the target object means that by controlling the end effector, the target object and the end effector remain relatively stationary and do not move away from the end effector. Among them, the ways for the end effector to grasp the target object include non-grasping grasping and grasping grasping.

[0054] In some embodiments, as described above Figure 1As shown in (d1), the grasping method is non - grasping. Non - grasping means that the palm of the end - effector bears the target object, and based on the static friction of the bearing surface, the target object and the end - effector remain relatively stationary. In some embodiments, when the bearing surface is parallel to the horizontal plane, and both the end - effector and the target object are stationary or moving at the same constant speed, the static friction between the target object and the end - effector can be 0. For non - grasping, after the target object is adjusted to the target pose, the end - effector is controlled to keep the target object in the target pose, that is, the non - grasping of the target object is completed. This grasping method is relatively simple and has low requirements for the shape of the end - effector (the end - effector only needs to have an upward - facing plane to bear the target object), thus saving the manufacturing cost of the robot. Moreover, since this grasping method has low restrictions on the shape of the end - effector (non - grasping can be performed with or without fingers), the application range of the grasping method is expanded.

[0055] In some embodiments, as described above Figure 1 As shown in (d2), the grasping method is grasping. Grasping means that the fingers of the end - effector close towards the palm to grasp the target object between the palm and the fingers. That is, the fingers of the end - effector hold the target object to keep the target object relatively stationary with respect to the end - effector. For grasping, after the target object is adjusted to the target pose, the fingers of the end - effector are closed to hold the target object. After the grasping is completed, the possible postures of the target object can be relatively rich. For example, the posture of the target object can be vertical or even downward, and it will not move out of the end - effector, so that the pose of the end - effector and the target object after the grasping operation is more flexible and free.

[0056] In summary, in the technical solution provided by the embodiments of the present application, the robot is controlled to move towards the target object and collide with the target object during the movement, so that the end - effector of the robot rotates relative to the target object until the target object is adjusted to the target pose, and then the end - effector is controlled to grasp the target object. During the grasping process, the robot is in a non - fixed state. The robot does not need to reduce its speed to 0 and can grasp the target object while maintaining the non - fixed state, that is, dynamic object grasping is achieved, saving the time required for object grasping and improving the control efficiency of the robot for object grasping.

[0057] In addition, the method provided by the embodiments of the present application adopts two dynamic grasping strategies (i.e., grasping and non - grasping) to achieve dynamic grasping of the target object.

[0058] In some possible implementation manners, the control method of the robot further includes: for a hybrid system including an end effector and a target object, based on the unified state equation corresponding to the hybrid system, obtaining the pose information and force information of the hybrid system at each time step in the complete stage; and controlling the end effector at each time step in the complete stage according to the pose information and force information of the hybrid system at each time step in the complete stage. Wherein, the unified state equation refers to an equation used to describe the state of the hybrid system; the time step refers to the time span of each segment obtained by dividing the complete stage; the pose information refers to the position information and attitude information of the hybrid system, the end effector in the hybrid system, and the target object; and the force information refers to the information of various forces in the hybrid system.

[0059] In some embodiments, the pose information and force information at each time step in the complete stage satisfy the following constraint conditions: an inequality constraint condition for constraining the frictional force at the contact position between the end effector and the target object; and an equality link constraint condition for constraining the motion trajectory and acting force between the end effector and the target object.

[0060] In some embodiments, the complete stage is divided into N time steps, N being a positive integer, and the attitude of the hybrid system is defined as x ro =[x r x o , where the attitude of the end effector is defined as The attitude of the target object is defined as represents the centroid vector of the end effector in the world coordinate system, represents the attitude of the end effector in the world coordinate system, represents the centroid vector of the target object in the world coordinate system, represents the attitude of the target object in the world coordinate system, In addition, and are used to represent the general state and input of the hybrid system, x ro is used to represent the pose information of the hybrid system, is used to represent the velocity information of the hybrid system, represents the driving force received by the end effector, represents the force exerted by the end effector on the target object. Wherein, the vector table with the shape of 'l' is a spatial vector, N s is used to represent that the hybrid system corresponds to N s stages, t m is used to represent N of the hybrid system sA phase time vector, m can be either 0 or f. The subscripts "0" and "f" represent the start time and end time of each phase respectively. Therefore, the following unified state equation in discrete form can be derived:

[0061]

[0062] Among them, N [i] is the discrete value of the i-th phase. x n+1 = f(x n , |u n , Δt) is the discrete form of the state space equation. x n is the pose information of the hybrid system at each time step in the complete phase. u n is the force information of the hybrid system at each time step in the complete phase. And Δt is the time step (i.e., the "time step" mentioned above). h n (x n , u n ) is the inequality constraint condition, that is, the friction cone constraint at the adhesion point of the end effector and the target object. g n (x n , u n ) is the equality link constraint condition.

[0063] In some embodiments, the motion trajectory ζ κ (t) of the hybrid system can be described by the following method:

[0064]

[0065] Among them, the superscripts P and Ω represent position and attitude respectively. κ is Figure 3 (a) to (d) the B r , C r , B o or C o point. g n (x n , u n ) represents the equality link constraint, which consists of a motion connection condition and a force connection condition. The motion connection condition is the relationship between and , and the force connection condition is the relationship between and .

[0066] In some embodiments, based on DDP, the hybrid system is unifiedly modeled, and the complete phase of the hybrid system is described by the following unified state equation:

[0067]

[0068] Among them, J is the objective function, is the relative termination time of the i-th segment, is the absolute termination time of the i-th segment, is the absolute start time of the i-th segment, and are the trajectory cost function (trajectorycost) and the terminal cost function (terminalcost), respectively. And, represents a state error, is the expected value.

[0069] In some embodiments, based on the unified state equation corresponding to the hybrid system, the pose information and force information of each time step in the complete stage of the hybrid system are obtained, including: constructing an objective function based on the Bellman optimal equation, the unified state equation, and the constraint conditions; using the primal-dual Lagrangian multiplier method to solve the objective function to obtain the pose information and force information of each time step in the complete stage of the hybrid system.

[0070] In some embodiments, Q, R, and Q f are weight matrices. Based on DDP, u k is derived sequentially backward. Since the above multi-constraints are solved by the primal-dual incremental Lagrangian (PDAL) method, we can rewrite the Bellman recurrence equation as:

[0071]

[0072] where n = 1, 2,.., N. λ E and λ I are Lagrange multipliers. V(x n ) and Q(x n , u u ) represent the value function and the action-state value function. The subscripts E and I represent the equality part and the inequality part, respectively. and are the modified values. In addition, we denote and as the trajectory cost related to the constraint. Then can be expressed as:

[0073]

[0074]

[0075]

[0076] where μ E , μ I are positive scalars. λ Eand λ I denote Lagrange multipliers. The superscript ′ denotes the updated value; [.] + denotes the orthogonal projection operator; [.] denotes the active set inequality constraint; the optimal descent direction of quantum b can be adopted as:

[0077]

[0078] and

[0079]

[0080] wherein, is the Hessian matrix of ζ with respect to and ζ. is the Jacobian matrix of ξ with respect to . K is the KKT (Karush-Kuhn-Tucker, necessary conditions for the optimal solution of nonlinear programming) matrix. Then the optimal increment values δu, δx, δλ, δμ are obtained, and the value function is estimated using the second-order Taylor expansion.

[0081]

[0082] The first-order optimality condition can be used to derive the

[0083]

[0084] In the above implementation, each stage in the complete stage can be uniformly represented by the above state equation (1) and state equation (2), realizing the unified representation of the state equations in different stages. The same input variables and control variables can be adopted in each stage, that is, a unified model applicable to different stages is realized, eliminating the need for stage-by-stage modeling, reducing the modeling cost and operation cost, and improving the modeling efficiency and the control efficiency of the robot. An algorithm based on differential dynamic programming is used to solve the multi-stage task with a fixed order. The traditional differential dynamic programming is extended to be compatible with free terminals, multiple constraints, and multi-stage tasks at the same time.

[0085] In some possible implementation manners, controlling the end effector of the robot to collide with the target object includes the following steps:

[0086] 1. Obtain the contact speed between the end effector and the target object and the change amount before and after the end effector collides with the target object; wherein, the contact speed refers to the relative speed of the contact point between the end effector and the target object;

[0087] 2. Determine the velocity of the target object after the collision based on the change amount and the velocity of the target object before the collision;

[0088] 3. Determine the velocity that the end effector needs to reach before the collision based on the velocity of the target object after the collision, the maximum acceleration of the end effector, and the constraint relationship between the velocity of the target object after the collision and the velocities of the end effector before and after the collision;

[0089] 4. Control the end effector to collide with the target object based on the velocity that the end effector needs to reach before the collision.

[0090] In some embodiments, when the end effector collides with the target object, the velocity of the end effector will change abruptly, and the target object will also obtain an acceleration due to the collision, that is, both the end effector and the target object will have a velocity change due to the collision. Therefore, it is necessary to control the end effector so that after the end effector collides with the target object, the target object and the end effector remain in contact and do not separate.

[0091] In some embodiments, as Figure 3 (a) to (d) show, B j is the centroid of object j, where j = r or o, that is, it represents the centroid of the end effector or the target object. is the spatial inertia of the end effector - target object (simplified as joint body A). m j , σ j are the mass and inertia of target object j respectively. are the spatial inertia and spatial force vector of target object j respectively. rB j is the position vector of point B j . S is the joint axis. In Figure 3 (b) to (d), C r,i and C o,i are contact point pairs, where i = 1, 2, 3, 4. p j is the equivalent contact point of the joint body and is also the action point. r o is the vector from B o to P o . is the contact force acting on point C o,i . {w} is the inertial frame. {b} is the body - fixed coordinate system of the object (i.e., the coordinate system of the hybrid system composed of the end effector and the target object).

[0092] As Figure 3 (b) shows, during the collision phase, the contact points of the end effector and the target object can be considered as C o,1 (C r,1 ) and C o,2 (Cr,2 ), the mass and inertia of the robot are both assumed to be ∞. Therefore, the speed of the robot will not change in a very short time. We use ρ to represent the actual exchange momentum of energy loss, because a general collision consists of a compression phase and a restitution phase. When W is called the inverse inertia matrix, the change in the contact velocity can be defined as:

[0093]

[0094] Therefore, in the post-collision stage:

[0095]

[0096] where M = [1 / m o ; r o × / σ o is the matrix related to dynamic parameters. The superscripts ′+′ and ′-′ represent after-collision and before-collision respectively. We use to represent the maximum acceleration of the end effector, then If the end effector and the target object will stick together all the time; if the end effector needs to accelerate to stick to the target object all the time. If the end effector and the target object will separate. The selection of the collision point can be roughly divided into A o , B o , C o several types. The palm edge of the end effector collides with point C o so that the object can leave the current object (such as a table) in time, reducing the uncertainty of friction. The selection of an appropriate grasping point makes the dynamic grasping task more deterministic.

[0097] In the above implementation, by controlling the end effector, the target object and the end effector remain in contact and do not separate after the collision, thus providing guarantee for the subsequent grasping stage.

[0098] In some embodiments, as shown in Figure 3 (c), in the rotation stage, the number N c of the contact points between the end effector and the target object is 2, C r = C r,1 ∪C r,2 . The contact of these two points cannot completely constrain the object (i.e., the target object), so the target object can rotate. Therefore, we can describe the equality constraints of the motion spectrum and force spectrum conditions as:

[0099]

[0100]

[0101] wherein, is to transformation matrix, j = r or o; and E is the identity matrix. Generally, this equation can be expressed as an equality constraint of g(x,u). In addition, for the target object, the force screw (such as the resultant force and resultant moment) at its centroid net force vector and the contact force have the following relationship:

[0102]

[0103] wherein, T is the spatial transformation matrix, and G r is the grasping transformation matrix. In addition, the friction cone constraint h(x,u) can ensure the relative rotation between the end effector and the target object, and the friction cone constraint can be expressed as follows:

[0104]

[0105] wherein, μ i is the Coulomb friction coefficient, n i is the unit normal vector, and o i , are two orthogonal tangent vectors describing the hybrid system. are respectively the non - negative lower bound and upper bound of the normal contact force. The dynamic unactuated rotational operation is subject to a one - way constraint, making the energy of the end effector continuously increase. Therefore, the boundary conditions of the hybrid system need to meet this requirement.

[0106] For the grasping phase, the constraint conditions are almost the same as those in the previous rotational phase, except that the number of contact points becomes N c = 4, C r = C r,1 ∪C r,2 ∪C r,3 ∪C r,4This four-point surface contact completely restricts the movement of the target object, causing the target object to remain in the palm of the end effector. Therefore, non-grasping and grasping share the same kinematic and force spectra of the equality constraint g(x,u). The main difference between non-grasping and grasping is that non-grasping also requires friction constraints, while grasping does not. During the grasping process, grasping the target object requires the robot's fingers to close and the palm to form multi-directional displacement constraints. After the fingers are closed, the linear and angular velocities of the end effector and the target object are the same, so the velocity of the target object needs to be eliminated. After that, the target object is moved to a specific position and orientation. The non-grasping configuration is as shown in Figure 1 (d1). Different from grasping, non-grasping needs to utilize dynamic balance during the movement of the target object to make the target object adhere to the palm and maintain contact with the palm. The contact force between the palm and the target object must always comply with the friction cone constraint conditions, so the inequality constraint h(x,u) needs to be considered.

[0107] In some embodiments, as shown, the DDP-based optimization framework includes two main parts: forward pass and backward pass. The backward channel is formulated as a constrained QP problem and solved using the PDAL method. The modified constrained DDP is as follows:

[0108] Given

[0109] x0,xg,xn+1=f(xn,un),g(xn,un),h(xn,un) / / Given x0,xg,xn+1, g(xn,un),h(xn,un)

[0110] Initialization

[0111] U={u0,u1,···,uN-1},X={x0,x1,···,xN} / / Initialize U and X and n n

[0112] n n =0, / / Iteration counter

[0113] while(|Jn-Jn-1|≥∈)do / / When |Jn-Jn-1|≥∈

[0114]

[0115] for k=N-1to 0do / / Start the loop from k=N-1 / / until k=0

[0116] Calculation of Jacobian and Hessian matrix Then the optimal incremental value, δu, δx, δλ, δμ / / Calculation of the optimal increments δu, δx, δλ, δμ for the Jacobian matrix and the Hessian matrix

[0117] end for

[0118] for n = 0 to N - 1 do / / Loop from k = 0 / / until k = N - 1

[0119] u′ n = u n + δu n

[0120] x n+1 = f(x n , u′ n )

[0121] end for

[0122] δt N ← Eq.17 / / Update δt using the calculation result of the above formula (17) N

[0123] t N = min(max(t N + δt, t min ), t max )

[0124] N′ = N + δt N / Δt

[0125] J ← J′, X ← X′, U ← U′, n n ← n n + 1, N ← N′ / / Update J, X, U, n n , N

[0126] end while = 0

[0127] Where ε is a given decimal, the superscript ′ represents the updated value, X is the state sequence, and U is the control input sequence.

[0128] Assume that two objects (as Figure 1 shown) are in contact, and the dynamics based on spatial vectors can be derived as:

[0129]

[0130]

[0131]

[0132] Among them, is the acceleration and offset force of object j. is the offset force of the articulated object A, is the relative acceleration.

[0133] According to the contact state and motion state of the two-body system, it can be divided into the following three cases:

[0134] Case 1: The end effector r and the target object o are separated. and are independent of each other,

[0135] Case 2: The end effector r and the target object o are contact-coupled and have relative motion. and are interdependent and are the coupling force vectors. and are also interdependent,

[0136] Case 3: The end effector r and the target object o are contact-coupled and have no relative motion (no adhesive contact). In this case, the kinematics and dynamics of the two objects are roughly the same as those in the second case, except that

[0137] Considering We can obtain the unified differential equation:

[0138]

[0139] Among them, A, B, and C are system matrices, A = [0, E; 0, 0],

[0140] The approach phase in this paper belongs to Case 1, the rotation phase belongs to Case 2, and Case 3 can be applied to grasping or non-grasping. Compared with the general three-dimensional vector, the hybrid system model based on the six-dimensional vector is more concise.

[0141] The control method of the robot provided by the embodiments of this application is jointly simulated in Recurdyn and Matlab. The experiment depends on the dual UR16e robot. All test results are also shown in the attached video. The simulation and experiment are carried out in the aircraft gravitational field. Now, we define x r = [x r y r γ r ], x o = [x o y o γ o , x ro = [x r x o and Since the x o , y o of the object is either fixed or constrained by the motion spectrum in this task, only the simplified general coordinates are simply listed as follows.

[0142] As Figure 4 shown, the weight of the target object 12 is 0.3 kg, and the size of the target object is 12 cm × 2.5 cm × 5 cm. The actual friction coefficient between the end effector and the target object is 0.55. In the simulation, Figure 4 (a) and (b) correspond to the cases of lower and higher collision speeds respectively, Figure 4 and the curves of the required postures and energies for (a) and (b) are respectively as Figure 5 and Figure 6 shown. Since the time of the collision process is very short (denoted as ), the posture of the target object 12 has not changed yet, but the target object has obtained an instantaneous velocity.

[0143] Among them, the expected boundary conditions for different stages are as shown in Table 1 below:

[0144] Table 1

[0145]

[0146] Among them, the above symbol "◇" indicates no constraint, that is, a free end.

[0147] In some embodiments, the complete stage corresponding to non - grasping is called Strategy One, and the complete stage corresponding to grasping is called Strategy Two.

[0148] Simulation of Strategy One at different speeds: The end effector adopts a flat plate structure and can dynamically grasp the object without closing the fingers. This part gives the simulation results of dynamic non - grasping at low - speed and high - speed collision cases (distinguished by the contact speed v). The corresponding switching time vectors are respectively t f = [0.2, 0.2, 0.32, 0.6] seconds and t f= [0.1, 0.1, 0.22, 0.57] s. The constraints in the rotation stage and the grasping stage ensure stable contact between the palm and the target object. According to the simulation results, it can be seen that the method provided by the embodiment of the present application can successfully grasp the target object at different collision speeds.

[0149] Simulation of Strategy 2 at different speeds: Use AllegroHand to implement a hybrid operation of grasping and non-grasping. This part presents the simulation results of dynamic grasping in the cases of low-speed and high-speed collisions. The corresponding switching time vectors are t f = [0.2, 0.2, 0.33, 0.55] s and t f = [0.08, 0.08, 0.16, 0.39] s. According to the simulation results, it can be seen that by using the method provided by the embodiment of the present application, the robot can successfully grasp the target object and the end effector will not stop moving.

[0150] Robot experiment on Strategy 1: In this experiment, the target object to be grasped is a cuboid. The weight of the cuboid is 0.3 kg, and the dimensions of the cuboid are 12 cm × 2.5 cm × 5 cm. The centroid of the target object is set at the geometric center. The actual friction coefficient between the palm and the cuboid is 0.55. The corresponding switching time vector t f = [0.15, 0.15, 0.3, 0.5] s. The end effector is flat and does not decelerate when grasping the target object. By effectively unifying the dynamic operation principles of the three grasping methods, as Figure 7 shown, it is finally possible to dynamically grasp an object with a palm without finger closing. The success rate of this experiment is 90% (9 / 10).

[0151] Robot experiment on Strategy 2: This part uses two UR16es and AllegroHand to implement a hybrid operation of grasping and non-grasping for dynamic grasping of a large vibrating screen and a bottle. The weights of the large vibrating screen and the bottle are 0.5 kg and 0.3 kg respectively. The actual friction coefficients between the end effector and these two objects are 0.45 and 0.5 respectively. Assuming that the centroid of the object is located at their geometric center, the motion trajectory of the robot can be calculated, and the corresponding switching time vector t f = [0.2, 0.2, 0.35, 0.52] s. As Figure 8 shown, the fingers of the end effector can grasp the objects (the large vibrating screen and the bottle) to complete tasks such as pouring water. The success rate of this experiment is 100% (10 / 10).

[0152] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0153] Please refer to Figure 9 , which shows a block diagram of a control device for a robot provided in an embodiment of the present application. The device has the function of implementing the method example for controlling the above-mentioned robot, and the function can be implemented by hardware or by software executed by the hardware. The device can be the robot introduced above or can be provided on the robot. The device 900 may include: a collision control module 910, a rotation control module 920, and a grasping control module 930.

[0154] The collision control module 910 is configured to control the end effector of the robot to collide with the target object during the process of the robot moving towards the target object.

[0155] The rotation control module 920 is configured to control the relative rotation between the end effector and the target object to adjust the target object to a target pose suitable for being grasped by the end effector.

[0156] The grasping control module 930 is configured to control the end effector to grasp the target object after the target object is adjusted to the target pose; wherein, during the entire stage from the collision between the end effector and the target object to the end effector grasping the target object, the robot is in a non-fixed state.

[0157] In some embodiments, the grasping method is non-grasping, and the non-grasping means that the target object is carried by the palm of the end effector, and the target object and the end effector remain relatively stationary based on the static friction force of the bearing surface.

[0158] In some embodiments, the grasping method is grasping, and the grasping means that the fingers of the end effector close towards the palm to grasp the target object between the palm and the fingers.

[0159] In some embodiments, as Figure 10 shown, the device 900 further includes: an information acquisition module 940.

[0160] The information acquisition module 940 is configured to obtain the pose information and force information of each time step of the hybrid system during the entire stage for a hybrid system including the end effector and the target object based on the unified state equation corresponding to the hybrid system.

[0161] The grasping control module 930 is further configured to control the end effector at each time step during the entire stage according to the pose information and force information of each time step of the hybrid system during the entire stage.

[0162] In some embodiments, the pose information and force information at each time step of the complete stage satisfy the following constraint conditions:

[0163] Inequality constraint conditions for constraining the frictional force at the contact position between the end effector and the target object;

[0164] Equality link constraint conditions for constraining the motion trajectory and acting force between the end effector and the target object.

[0165] In some embodiments, as Figure 10 shown, the information acquisition module 940 is configured to:

[0166] Construct an objective function based on the Bellman optimal equation, the unified state equation, and the constraint conditions;

[0167] Solve the objective function by using the primal-dual Lagrange multiplier method to obtain the pose information and force information at each time step of the hybrid system in the complete stage.

[0168] In some embodiments, the collision control module 910 is configured to:

[0169] Obtain the change amount of the contact velocity between the end effector and the target object before and after the collision occurs between the end effector and the target object; wherein, the contact velocity refers to the relative velocity of the contact point between the end effector and the target object;

[0170] Determine the velocity of the target object after the collision according to the change amount and the velocity of the target object before the collision;

[0171] Determine the velocity that the end effector needs to reach before the collision according to the velocity of the target object after the collision, the maximum acceleration of the end effector, and the constraint relationship between the velocity of the target object after the collision and the velocity of the end effector before and after the collision;

[0172] Control the end effector to collide with the target object according to the velocity that the end effector needs to reach before the collision.

[0173] In summary, in the technical solution provided by the embodiment of the present application, by controlling the robot to move towards the target object and collide with the target object during the movement, so that the end effector of the robot rotates relative to the target object until the target object is adjusted to the target pose, then the end effector is controlled to grasp the target object. During the grasping process, the robot is in a non-fixed state. The robot does not need to reduce its speed to 0 and can grasp the target object in a non-fixed state, that is, dynamic grasping of the object is realized, the time required for grasping the object is saved, and the control efficiency of the robot for grasping the object is improved.

[0174] It should be noted that for the device provided in the above embodiment, when implementing its functions, only the division of the above function modules is used for illustration. In actual applications, the above functions can be allocated to different function modules according to needs, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be elaborated here.

[0175] Figure 11 The block diagram of the structure of a robot provided by an exemplary embodiment of the present application is shown. The robot 1100 may be the robot introduced above.

[0176] Generally, the robot 1100 includes a processor 1101 and a memory 1102.

[0177] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 11-core processor, etc. The processor 1101 may be implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0178] The memory 1102 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1102 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 stores a computer program, which is loaded and executed by the processor 1101 to implement the control method of the robot provided by the above method embodiments.

[0179] In an exemplary embodiment, a computer-readable storage medium is further provided, in which a computer program is stored, and when the computer program is executed by a processor, the control method of the above robot is implemented.

[0180] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical discs, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0181] In an exemplary embodiment, a computer program product is further provided, which includes a computer program stored in a computer-readable storage medium. The processor of the robot reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the robot executes the control method of the above robot.

[0182] It should be understood that the "plurality" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0183] The above are only exemplary embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A control method for a robot, characterized in that, The method includes: During the process of the robot moving towards the target object, controlling the end effector of the robot to collide with the target object; Controlling the relative rotation between the end effector and the target object to adjust the target object to a target pose suitable for being grasped by the end effector; After the target object is adjusted to the target pose, controlling the end effector to grasp the target object; wherein, from the moment when the end effector collides with the target object to the moment when the end effector grasps the target object in the complete stage, the robot is in a non-fixed state. At each time step in the complete stage, the end effector is controlled according to the pose information and force information of the hybrid system at each time step. The pose information and force information of the hybrid system at each time step are obtained based on the unified state equation corresponding to the hybrid system. The hybrid system refers to a system composed of the end effector and the target object, and the unified state equation refers to an equation used to describe the state of the hybrid system.

2. The method according to claim 1, characterized in that, The grasping method is non-grasping, and the non-grasping is a grasping method in which the target object is carried by the palm of the end effector and the target object and the end effector remain relatively stationary based on the static friction force of the bearing surface.

3. The method according to claim 1, wherein The grasping method is grasping, and the grasping is a grasping method in which the fingers of the end effector close towards the palm to grasp the target object between the palm and the fingers.

4. The method according to claim 1, wherein The pose information and force information at each time step in the complete stage satisfy the following constraint conditions: Inequality constraint conditions, which are used to constrain the friction force at the contact position between the end effector and the target object. The inequality constraint conditions include the friction cone constraint at the adhesion point between the end effector and the target object; Equality link constraint conditions, which are used to constrain the motion trajectory and acting force between the end effector and the target object. The equality link constraint conditions are constraint conditions constructed with the pose information and force information at each time step in the complete stage as parameters.

5. The method according to claim 4, characterized in that, The method further includes: Based on the state value function of the unified state equation, constructing a Bellman optimal equation. The state value function is constructed with the pose information and force information at each time step in the complete stage, and the Lagrange multiplier as parameters; Based on the Bellman optimal equation and the constraint conditions, constructing an objective function. The objective function is constructed based on the trajectory cost related to the constraint conditions on the basis of the objective function that has not been transformed in the Bellman optimal equation. The trajectory cost is obtained based on the Lagrange multiplier; Using the primal-dual Lagrange multiplier method to construct the necessary condition KKT matrix of the optimal solution of the nonlinear programming based on the objective function; Solving the KKT matrix to obtain the pose information and force information of the hybrid system at each time step in the complete stage.

6. The method according to claim 1, wherein The controlling the end effector of the robot to collide with the target object includes: Obtain the change amount of the contact velocity between the end effector and the target object before and after the collision between the end effector and the target object; wherein, the contact velocity refers to the relative velocity of the contact point between the end effector and the target object, and the change amount is used to indicate the velocity change of the contact velocity caused by the collision; Determine the velocity of the target object after the collision according to the change amount and the velocity of the target object before the collision; Determine the velocity that the end effector needs to reach before the collision according to the velocity of the target object after the collision, the maximum acceleration of the end effector, and the constraint relationship between the velocity of the target object after the collision and the velocities of the end effector before and after the collision, wherein the constraint relationship is used to constrain the magnitude relationship between the velocity of the target object after the collision, the velocity of the end effector after the collision, and the velocity of the end effector before the collision, so that the end effector and the target object remain in contact after the collision; Control the end effector to collide with the target object according to the velocity that the end effector needs to reach before the collision.

7. A control device for a robot, characterized in that, The device includes: A collision control module, configured to control the end effector of the robot to collide with the target object during the process of the robot moving towards the target object; A rotation control module, configured to control the relative rotation between the end effector and the target object to adjust the target object to a target pose suitable for the end effector to grasp; A grasping control module, configured to control the end effector to grasp the target object after the target object is adjusted to the target pose; wherein, during the complete stage from the collision between the end effector and the target object to the end effector grasping the target object, the robot is in a non-fixed state, and at each time step of the complete stage, the end effector is controlled according to the pose information and force information of the hybrid system at each time step. The pose information and force information of the hybrid system at each time step are obtained based on the unified state equation corresponding to the hybrid system. The hybrid system refers to a system composed of the end effector and the target object, and the unified state equation refers to an equation used to describe the state of the hybrid system.

8. The device according to claim 7, characterized in that, The grasping method is non-grasping, and the non-grasping means a grasping method in which the palm of the end effector bears the target object, and the target object and the end effector remain relatively stationary based on the static friction force of the bearing surface.

9. The device according to claim 7, characterized in that, The grasping method is grasping, and the grasping means a grasping method in which the fingers of the end effector close towards the palm to grasp the target object between the palm and the fingers.

10. The device according to claim 7, characterized in that, The pose information and force information at each time step of the complete stage satisfy the following constraint conditions: Inequality constraint conditions are used to constrain the frictional force at the contact position between the end effector and the target object, and the inequality constraint conditions include friction cone constraints at the adhesion points of the end effector and the target object; Equality link constraint conditions are used to constrain the motion trajectory and acting force between the end effector and the target object, and the equality link constraint conditions are constraint conditions constructed with the pose information and force information of each time step in the complete stage as parameters.

11. The device according to claim 10, characterized in that, The device further includes: an information acquisition module; the information acquisition module is used for: Construct a Bellman optimal equation based on the state value function of the unified state equation, where the state value function is constructed with the pose information and force information of each time step in the complete stage, and the Lagrange multiplier as parameters; Construct an objective function based on the Bellman optimal equation and the constraint conditions. The objective function is constructed based on the trajectory cost related to the constraint conditions on the basis of the objective function that is not transformed in the Bellman optimal equation, and the trajectory cost is obtained based on the Lagrange multiplier; Use the primal-dual Lagrange multiplier method to construct the necessary condition KKT matrix of the optimal solution of the nonlinear programming based on the objective function; Solve the KKT matrix to obtain the pose information and force information of each time step in the complete stage of the hybrid system.

12. The device according to claim 7, characterized in that The collision control module is used for: Obtain the change amount of the contact velocity between the end effector and the target object before and after the collision between the end effector and the target object; wherein, the contact velocity refers to the relative velocity of the contact point between the end effector and the target object, and the change amount is used to indicate the velocity change of the contact velocity caused by the collision; Determine the velocity of the target object after the collision according to the change amount and the velocity of the target object before the collision; Determine the velocity that the end effector needs to reach before the collision according to the velocity of the target object after the collision, the maximum acceleration of the end effector, and the constraint relationship between the velocity of the target object after the collision and the velocities of the end effector before and after the collision, where the constraint relationship is used to constrain the magnitude relationship between the velocity of the target object after the collision, the velocity of the end effector after the collision, and the velocity of the end effector before the collision, so that the end effector and the target object remain in contact after the collision; Control the end effector to collide with the target object according to the velocity that the end effector needs to reach before the collision.

13. A robot, characterized in that, The robot includes a processor and a memory, and a computer program is stored in the memory. The computer program is loaded and executed by the processor to implement the control method of the robot according to any one of claims 1 to 6 above.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by the processor to implement the control method of the robot according to any one of claims 1 to 6 above.

15. A computer program product, characterized in that, The computer program product includes a computer program which is stored in a computer-readable storage medium. The processor reads and executes the computer program to implement the control method of the robot according to any one of claims 1 to 6.

Images