Control method and device for dual-arm robot, carrying method, and dual-arm robot

Abstract

Provided are a control method and device of a dual-arm robot, a carrying method, and a dual-arm robot. Each mechanical arm of the dual-arm robot has an operating end that non-fixedly contacts an object and carries the object from a starting position to a target position, and there is no relative motion between the operating end and the object. The method includes: obtaining an optimal motion trajectory of the operating end carrying the object; obtaining a reference pose of the object corresponding to each time and a reference force applied by the operating end to the object based on the optimal motion trajectory; determining a correction force applied by the operating end to the object based on an actual pose of the object corresponding to a current time and the reference force, the reference pose of the object corresponding to at least one time after the current time, and the reference force; and determining driving information of a joint of the dual-arm robot based on the correction force.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of robots, and more particularly, to a method and device for controlling a dual-arm robot, a method for a dual-arm robot to carry an object, a dual-arm robot capable of performing the above-mentioned method, and a control device. BACKGROUND

[0002] With the continuous development of technology in the field of robots, various types of robots such as industrial robots, service robots, etc. have been increasingly used in various technical fields, such as smart agriculture, smart factories, and smart warehousing, etc. In the above-mentioned fields, the robot can be a dual-arm robot in many cases, i.e. including two mechanical arms, the end of each mechanical arm can be referred to as an operating end (for example, a mechanical arm with a mechanical gripper at the end of each mechanical arm), with the help of a suitable operating end of the robot, it is possible to achieve the desired operation on various objects in the above-mentioned fields (for example, workpieces to be carried in a smart factory; goods in the field of smart logistics, smart warehousing, etc.), such as acquisition, movement, placement, etc. of the operating object, so as to significantly improve the degree of automation and reduce the cost of human resources.

[0003] Generally, the operation of the dual-arm robot on the object is divided into fixed operation and non-fixed operation. Here, the expression "fixed" means that the two operating ends of the dual-arm robot establish a firm fixed connection with the object to be carried. The operation in which the operating end establishes a firm fixed connection with the object is fixed operation, and vice versa, which is non-fixed operation, such as pushing, pulling, pulling, sliding, and hitting the object without establishing a fixed connection. For example, the operation of acquiring the object by means of an operating end with an acquisition component such as a mechanical gripper, a controllable suction component, etc. that can establish a fixed connection with the object belongs to fixed operation. The operation of a humanoid robot carrying a plurality of stacked high-foot cups by means of a tray belongs to non-fixed operation.

[0004] How to quickly and smoothly carry objects and carry larger weight objects is a technical problem to be solved in the art, therefore the present disclosure proposes a scheme for controlling a dual-arm robot to quickly and smoothly carry large weight objects, especially for the case where the operating end of the above-mentioned dual-arm robot is in non-fixed contact with the object to be operated. SUMMARY

[0005] According to a first aspect of the present disclosure, a method for controlling a dual-arm robot is provided, each arm of the dual-arm robot having an operating end, the operating end being in non-fixed contact with an object and carrying the object from a starting position to a target position, and there being no relative motion between the operating end and the object, the method comprising: obtaining an optimal motion trajectory of the operating end carrying the object; obtaining, based on the optimal motion trajectory, a reference pose of the object corresponding to each time instant and a reference force applied by the operating end to the object; determining a modified force applied by the operating end to the object based on an actual pose of the object corresponding to a current time instant and the reference force, and the reference pose and the reference force of the object corresponding to at least one time instant after the current time instant; and determining driving information of a joint of the dual-arm robot based on the modified force.

[0006] According to a second aspect of the present disclosure, a method for carrying an object by a dual-arm robot is provided, each arm of the dual-arm robot having an operating end, the operating end being in non-fixed contact with the object, the method comprising: obtaining driving information and determining driving force of a joint of the dual-arm robot based on the driving information; and driving the joint of each arm of the dual-arm robot based on the driving force, so that the operating end carries the object from a starting position to a target position, and there is no relative motion between the operating end and the object, wherein the driving information is generated based on the method as described above.

[0007] According to a third aspect of the present disclosure, an apparatus for controlling a dual-arm robot is provided, each arm of the dual-arm robot having an operating end, the operating end being in non-fixed contact with an object and carrying the object from a starting position to a target position, and there being no relative motion between the operating end and the object, the apparatus comprising: a trajectory obtaining module configured to obtain an optimal motion trajectory of the operating end carrying the object; a reference information determining module configured to obtain, based on the optimal motion trajectory, a reference pose of the object corresponding to each time instant and a reference force applied by the operating end to the object; a modifying module configured to determine a modified force applied by the operating end to the object based on an actual pose of the object corresponding to a current time instant and the reference force, and the reference pose and the reference force of the object corresponding to at least one time instant after the current time instant; and a driving information determining module configured to determine driving information of a joint of the dual-arm robot based on the modified force.

[0008] According to a fourth aspect of the present disclosure, a dual-arm robot is provided, comprising: a robot body comprising: two operating ends corresponding to two mechanical arms of the dual-arm robot respectively, each operating end being in non-fixed contact with an object, the operating end applying a force to the object to carry the object from a starting position to a target position; and a controller configured to perform the method according to the second aspect of the present disclosure.

[0009] According to a fifth aspect of the present disclosure, a control device for controlling a dual-arm robot is provided, each mechanical arm of the dual-arm robot having an operating end in non-fixed contact with an object and carrying the object from a starting position to a target position without relative motion between the operating end and the object, the control device comprising: a processor; a memory having a computer program stored thereon, the computer program causing the processor to perform the method according to the first aspect of the present disclosure when executed.

[0010] According to a sixth aspect of the present disclosure, a computer-readable storage medium having a computer program stored thereon is also provided, the computer program causing the processor to perform the method according to the first aspect or the second aspect when executed.

[0011] According to a seventh aspect of the present disclosure, a computer program product comprising a computer program is also provided, the computer program implementing the method according to the first aspect or the second aspect when executed.

[0012] The present disclosure provides a scheme for controlling the carrying of an object when an operating end of a dual-arm robot is in non-fixed contact with the object, after determining the optimal motion trajectory of the operating end, the force correction of the object by the operating end of the two mechanical arms is locally performed for each time, so that even in the case that the two mechanical arms cannot be accurately controlled, the operating end can quickly carry the object of large mass and ensure that there is no relative motion between the object and the operating end. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some exemplary embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0014] Herein, in the drawings:

[0015] Figure 1A A schematic diagram of carrying an object by a fixed operating dual-arm robot is shown;

[0016] Figure 1BA schematic diagram showing a fixed operating dual-arm robot carrying an object is shown;

[0017] Figures 2A-2C A schematic diagram showing an example process of carrying an object is shown;

[0018] Figure 3 A flowchart showing a method of controlling a dual-arm robot according to an embodiment of the present disclosure is shown;

[0019] Figure 4 A flowchart showing a method of generating an optimal motion trajectory according to an embodiment of the present disclosure is shown;

[0020] Figures 5A-5E Several manifestations of the contact between the operating ends of two robot arms and an object are shown;

[0021] Figure 6 A flowchart showing a method of modifying the operation of the operating end of a dual-arm robot according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A flowchart showing a process of determining the driving information of the joints of a dual-arm robot based on the modified force is shown;

[0023] Figure 8 A schematic process diagram of a control method for a dual-arm robot according to an embodiment of the present disclosure is shown;

[0024] Figure 9 A method of carrying an object by a dual-arm robot according to an embodiment of the present disclosure is shown;

[0025] Figure 10 A structural block diagram of an apparatus for controlling a dual-arm robot according to an embodiment of the present disclosure is shown; and

[0026] Figure 11 A schematic diagram of a dual-arm robot according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions, and advantages of the present disclosure more apparent, the following will describe example embodiments according to the present disclosure in detail with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the example embodiments described herein.

[0028] In addition, in the present specification and the accompanying drawings, steps and elements having substantially the same or similar functions are denoted by the same or similar reference signs, and repeated descriptions of these steps and elements will be omitted.

[0029] In addition, in the present specification and drawings, the terms "first\second" involved are merely to distinguish similar objects, and do not represent a specific order of the objects. Understandably, the "first\second" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0030] In addition, in the present specification and drawings, the terms "up", "down", "vertical", "horizontal", and the like related to the orientation or positional relationship are used only for the convenience of describing the embodiments according to the present disclosure, and are not intended to limit the present disclosure thereto. Therefore, it should not be understood as a limitation on the present disclosure.

[0031] In addition, in the present specification and drawings, unless otherwise explicitly stated, "connection" or "contact" does not mean "direct connection" or "direct contact". Here, "connection" can mean both a fixed action and an electrical connection.

[0032] In addition, in the present specification and drawings, unless otherwise explicitly stated, "motion trajectory" does not only mean the displacement or trajectory of an object. Here, as is customary in robot dynamics, "motion trajectory" means not only the position, displacement, or angle formed in space of an object, but also the speed, acceleration, angular velocity, and angular acceleration of the object.

[0033] In addition, in the present specification and drawings, unless otherwise explicitly stated, "driving force" should be understood in a broad sense. That is, "driving force" does not necessarily mean "force". For a specific driving object of the "driving force", the "driving force" here can mean a narrow sense of force, or a "driving torque".

[0034] In addition, in the present specification and drawings, unless otherwise explicitly stated, "action force" includes the force applied or provided by the operating end to the object being carried, or also referred to as the contact force or gripping force, and "action force" does not necessarily mean "force". For a specific action object of the "action force", the "action force" here can mean a narrow sense of force, or an "action torque". For example, the "action force" can be expressed as a three-dimensional force and / or a three-dimensional torque.

[0035] As an example, the present disclosure can be applied to the field of intelligent sensors combined with artificial intelligence (AI). Among them, artificial intelligence is a theory, method, technology and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, so that machines have perception, reasoning and decision-making functions.

[0036] Currently, with the research and progress of artificial intelligence technology, artificial intelligence technology is researched and applied in many fields, such as common smart home, smart wearable device, virtual assistant, smart speaker, smart marketing, unmanned vehicle, autonomous vehicle, unmanned aerial vehicle, robot, smart medical treatment, smart customer service, etc. At present, by using the perception, reasoning and decision-making functions of artificial intelligence, artificial intelligence has been combined with various types of robots and applied to various fields such as intelligent agriculture, intelligent factory and intelligent warehouse, in order to realize the purpose of replacing manual work to perform desired operations (such as moving objects to a target location) on various objects in the above fields and significantly improve the automation level of the above fields and reduce human resource costs.

[0037] For fixed operation, since there is no relative motion between the object and the operation end after establishing a fixed connection, in the prior art, a dynamic model can be conveniently established between the operation end and the object, and the dynamic model can be analyzed by using Lagrange function balance method, etc. to facilitate subsequent accurate control of the motion trajectory of the operation end and the object.

[0038] Figure 1A A schematic diagram of a dual-arm robot based on fixed operation for object carrying is shown. Figure 1B A schematic diagram of a dual-arm robot based on fixed operation for object carrying is shown.

[0039] As shown in Figure 1A The dual-arm robot 100 includes two mechanical arms, each of which is schematically shown as having three degrees of freedom. Each mechanical arm has an operation end, and the object 200 to be carried is squeezed between the operation ends of the two mechanical arms, and the object 200 can be carried by clamping.

[0040] That is, Figure 1AThe shown carrying manner is based on the cooperative operation of the two arms, and an internal force is generated between the two arms and the object, for example, the force exerted by the operating end of the left mechanical arm on the object 200 is not equal to the force exerted by the operating end of the right mechanical arm on the object 200, which belongs to quasi-static operation.

[0041] In addition, since the mechanical arms of the robot correspond to a limited load range capable of carrying objects in a mass range, large-mass objects cannot be effectively carried, and the action is slow.

[0042] Therefore, it can be considered to adopt non-fixed operation, such as Figure 1B As shown, for example, unlike the traditional double-arm clamping of the same object, the non-fixed operation adopts two mechanical arms to "lift" the same object, so that the generation of internal force during the cooperative operation of the two arms can be effectively avoided.

[0043] However, for non-fixed operation, since the operating ends of the two mechanical arms of the double-arm robot are not fixedly connected with the object, relative motion between the operating end and the object or even the object falling off the operating end can occur, and it is difficult to control the motion trajectory of the non-fixedly connected operating end and the object. For example, when the operating end carries a plurality of stacked high cups at a relatively fast speed, the stacked high cups are easy to fall off the operating end once the carrying speed is too fast.

[0044] In addition, for the double-arm robot, even if the optimal motion trajectory of the carried object is determined based on time optimization and force optimization, during the cooperative operation, the two mechanical arms of the robot cannot necessarily be accurately controlled, for example, the contact mode of the operating ends of the two mechanical arms with the object can change, and further control of the non-fixed operation is needed for this case, so as to better ensure that the object can be quickly carried without falling off.

[0045] The present disclosure provides a method for controlling a carried object when the operating end of a double-arm robot is in non-fixed contact with the object, wherein after determining the optimal motion trajectory of the operating end, the force of the operating end of the two mechanical arms on the object is locally corrected for each time, so that even in the case that the two mechanical arms cannot be accurately controlled, the operating end can quickly carry a large-mass object and ensure that there is no relative motion between the object and the operating end.

[0046] In the context of the present disclosure, the object can include only one sub-object or a conjoined object including a plurality of sub-objects, wherein the operating end exerts a force on one sub-object in non-fixed contact with it, and there is a contact force at each of the equivalent contact points of each sub-object.

[0047] Figures 2A-2C A schematic diagram showing an example process of carrying an object is shown.

[0048] In Figures 2A-2C , the object is a connected body (the connected body is denoted as CRB) including a plurality of sub-objects, wherein the sub-object CRB0 is located at the lowermost non-fixed contact with the operating end of the dual-arm robot. In a specific example, the sub-object CRB0 can be a tray for carrying other sub-objects (for example, a goblet). Of course, the object can also include only one sub-object, for example, only carrying one cargo.

[0049] During the process of carrying the object, there can be acceleration, constant speed or deceleration.

[0050] As Figure 2A shown, when accelerating the object, the two operating ends are tilted at equal angles to the right with respect to the horizontal plane on which the dual-arm robot is placed, thereby providing the object with a rightward force for achieving acceleration; as Figure 2B shown, when carrying the object at constant speed, the two operating ends are not tilted, and there is no need to provide the object with a force for achieving acceleration; as Figure 2C shown, when accelerating the object, the two operating ends are tilted at equal angles to the left, thereby providing the object with a leftward force for achieving reverse acceleration.

[0051] In addition, Figures 2A-2C , the equivalent contact points (also referred to as gripping positions) are also marked on each contact surface, each equivalent contact point shows a friction cone, and the acting force (also referred to as contact force or gripping force) at each equivalent contact point is shown with an arrow. During the process of carrying the object, the object will not slide, so the acting force on each sub-object at each gripping position should be within the corresponding friction cone.

[0052] That is, the acting force to be applied to the object by the operating end can be determined according to relevant information such as mass information or inertia information of the operating end or the object, motion trajectory information, etc., so that the object can be quickly carried to the target position without sliding.

[0053] The method of controlling the dual-arm robot according to the embodiments of the present disclosure will be described below in conjunction with Figures 3-7 .

[0054] Figure 3A flowchart of a method of controlling a dual-arm robot according to an embodiment of the present disclosure is shown. In some embodiments, the method can be performed by the control or processing / computing device of the dual-arm robot itself; in other embodiments, considering the computing power of the robot itself or being limited, or being productized, the method can be performed by another control or processing / computing device arranged outside the robot, and the resulting result is sent to the robot (body) as driving information, and then the robot can generate driving force for each joint of the robot according to the received driving information. Thus, even a robot with limited computing power can also be controlled according to the method of the present embodiment.

[0055] In step S310, an optimal motion trajectory of the operating end carrying the object is obtained.

[0056] Optionally, the motion trajectory is an optimal motion trajectory of the operating end carrying the object. For example, the optimal motion trajectory of the dual-arm robot can be planned for the Cartesian displacement and / or Euler angle of the operating end of the dual-arm robot relative to the time variable to achieve control of its position and / or Euler angle, so as to obtain the optimal motion trajectory of the dual-arm robot, based on which the motion of the robot is controlled to carry the object. Of course, other parameters relative to the time variable can also be planned to achieve control.

[0057] For example, the optimal motion trajectory can be determined by a Time-Optimal Path Parameterization (TOPP) method. For example, the optimal motion trajectory is planned offline, involving the entire carrying process of the operating end carrying the object from the starting position to the target position.

[0058] For example, the optimal motion trajectory can be generated by considering both time and force on the object. More details of the generation of the specific motion trajectory will be described later with reference to Figure 4 in detail.

[0059] In step S320, the reference pose of the object corresponding to each time and the reference force of the force exerted by the operating end on the object are determined based on the optimal motion trajectory.

[0060] In the context of the present disclosure, the mentioned pose includes but is not limited to position and / or Euler angle, and the mentioned motion information can include but is not limited to velocity, acceleration, angular velocity and / or angular acceleration.

[0061] Optionally, the optimal motion trajectory can include an optimal motion trajectory of the pose, which can be regarded as a function of time, according to which a plurality of related information can be obtained, for example, the reference pose (for example, the position and Euler angle of the world coordinate system) and the reference motion information of the operating end at each time can be determined based on the optimal motion trajectory, and the reference force exerted by the operating end on the object can also be determined. In addition, since there is no sliding between the operating end and the object, the reference pose of the object can be determined based on the reference pose of the operating end, for example, based on the distance between the centroid of a sub-object (for example, Figures 2A-2C CRB0) in contact with the operating end and the centroid of the operating end.

[0062] In step S330, a corrected force exerted by the operating end on the object is determined based on the actual pose of the object corresponding to the current time and the reference force, the reference pose and the reference force corresponding to at least one time after the current time.

[0063] As mentioned earlier, the two robot arms are not necessarily accurately controllable, so the contact mode of the operating ends of the two arms with the object can change, and the force exerted on the object is not necessarily accurate, so the actual pose of the object can be different from the reference pose determined based on the optimal motion trajectory, and therefore the force exerted by the control end on the object needs to be corrected accordingly so as to still keep the object from sliding relative to the operating end.

[0064] Optionally, the correction process applies a predictive method, that is, the reference pose and the reference force corresponding to at least one time in the future are considered to comprehensively correct the force exerted on the object.

[0065] In addition, as will be described later, the friction cone of each sub-object of the object is also considered in the process of determining the corrected force, so that only the force exerted by the control end on the object needs to be determined, and the sub-objects will not slide relative to each other.

[0066] Specific details will be described later with reference to Figures 5A-6 .

[0067] In step S340, the driving information of the joints of the dual-arm robot is determined based on the corrected force.

[0068] For example, in step S330, it is determined that the force exerted by the control end on the object needs to be corrected, and the force is provided by the operating end, and further provided by driving each joint.

[0069] Therefore, by determining the driving information, the driving information can be used to finally generate the driving force of driving each joint of the two mechanical arms, i.e., for the actual pose of the object, by appropriately driving each joint of the two mechanical arms, the operating end can provide a suitable acting force to the object, so that the object (including one or more sub-objects) will not slide relative to the operating end.

[0070] By referring to Figure 3 The method described, by locally performing the acting force correction of the operating end of the two mechanical arms to the object for each time after determining the optimal motion trajectory of the operating end, makes the operating end able to quickly carry the object of large mass and ensure that the object always maintains no relative motion with the operating end even in the case that the two mechanical arms cannot accurately control; it can also carry multiple objects at a time; and it does not need a special operating end that can provide a clamping or squeezing operation, thus improving the flexibility of carrying and the accuracy of control, and improving the performance of the collaborative operation of the dual-arm robot.

[0071] Figure 4 A flowchart of a method for generating an optimal motion trajectory according to an embodiment of the present disclosure is shown.

[0072] For example, the optimal motion trajectory can be generated by constructing a first objective function based on time and the acting force of the operating end to the object using a time optimal algorithm, and optimizing the first objective function based on a corresponding dynamic model of the dual-arm robot, a corresponding dynamic model of the object, and a stable constraint equation set.

[0073] Therefore, the generation process of the optimal motion trajectory can first construct the corresponding dynamic model of the dual-arm robot and the corresponding dynamic model of the object as shown in step S410.

[0074] Alternatively, the corresponding dynamic model of the dual-arm robot indicates the relationship between the inertia, Coriolis force, centrifugal force, gravity, external force of the dual-arm robot, and the joint angle, joint angular velocity and joint angular acceleration of the dual-arm robot and the joint driving force of the dual-arm robot.

[0075] For example, the corresponding dynamic model of the dual-arm robot can be as shown in the following formula (1), where the following formula is based on the world coordinate system:

[0076]

[0077] In formula (1), M(q) = diag(M L ,M R ) represents the inertia matrix of the dual-arm robot, and the subscripts L and R represent the left mechanical arm and the right mechanical arm, respectively; q = [qL q R ] T is a vector of joint angles; is a matrix of Coriolis and centrifugal force terms; is a matrix of gravitational force terms. are joint angles, joint angular velocities and joint angular accelerations of the dual-arm robot, respectively. is a matrix of joint driving forces, In some embodiments, the effect of frictional Coulomb forces can also be considered, in which case it is placed to the left of the bracket.

[0078] Optionally, the dynamic model corresponding to the object can be used to represent a relationship between a motion trajectory of the object and a force condition of the object, for example, the dynamic model of the object indicates a relationship between an action force applied to the object by the operating end, a gravity of the object, an inertial force of the object, and Coriolis and centrifugal forces of the object and a pose, a linear velocity, and a linear acceleration of the object.

[0079] In the present disclosure, the action force can be represented by a spatial force vector or a contact force, both of which are involved in different parts of the present disclosure, can be used interchangeably, replaced by each other, or refer to the same meaning in many places. In the present disclosure, the spatial force vector information is represented by w, each vector in w (for each sub-object) includes three-dimensional action force information and three-dimensional action torque information, and the contact force is represented by f C , each vector in f C (for each sub-object) only includes three-dimensional action force information. w and f C can also be converted into each other, for example, w = G r f C , where G r is a preset conversion matrix.

[0080] In addition, the dynamic model corresponding to the object can be as shown in the following formula (2), where the following formula is based on a connected coordinate system (with subscript b), that is, the center of mass or the center of mass of the lowest end sub-object is taken as the origin of the connected coordinate system, and based on the conversion between coordinates, the conversion between the connected coordinate system and the world coordinate system, the inertial coordinate system, etc. can be realized.

[0081]

[0082] where M b (x b ) is a matrix of inertial force terms corresponding to all sub-objects, is a matrix of Coriolis and centrifugal force terms corresponding to all sub-objects, g b (x b ​) is the gravity term matrix corresponding to all sub-objects. Force N k is the number of sub-objects; Force and the force f C j = [f C,1 j ,…,f C,K j ] T where K is the number of equivalent contact points between two sub-objects.

[0083] As mentioned above, in order to ensure that each sub-object in the connected object including multiple sub-objects does not slide, the contact force of each equivalent contact point needs to be within the friction cone of the equivalent contact point.

[0084] In the present disclosure, the equivalent contact point can be an actual contact point or a contact point determined according to a common method, for example, the boundary center of a contact surface, and the present disclosure does not limit this.

[0085] As shown in step S420, a first objective function can be constructed based on time and the force by using a time optimal algorithm.

[0086] For example, considering the normalized path coordinate s(t) of the operation end, the starting time is t=0, the ending time is t=T, and s(0)=0≤s(t)≤1=s(T). The first order differential term and the second order differential term of s(t) are and Considering that the movement time of the object from the starting position to the target position needs to be minimized, the following can be designed

[0087] Therefore, the joint speed and acceleration of the dual-arm robot can be expressed by formula (3) as follows:

[0088]

[0089] In formula (3),

[0090] Therefore, the following dynamics model corresponding to the dual-arm robot is generated as shown in formula (4).

[0091]

[0092] Specifically, in formula (4), m(s)=M(q(s))q′(s), c(s)=M(q(s))q″(s)+C(q(s),q′(s))q′(s), g(s)=G(q(s)) (if there is a friction force term, add F s(q(s))sgn(q'(s))) respectively. Accordingly, m(s) is regarded as an inertia matrix, c(s) is regarded as a Coriolis force and centrifugal force matrix, and g(s) is regarded as a gravity matrix.

[0093] Based on the above, a first objective function can be constructed as follows, which is a function of and is associated with the force provided by the operating end to the object, as shown in the following formula (5):

[0094]

[0095] wherein ζ is a weight coefficient matrix of w,

[0096] That is, the first objective function is associated with the force provided by the operating end to the object and has the to-be-optimized variable s.

[0097] As shown in step S430, the optimal motion trajectory can be generated by taking the first objective function to be minimum under the condition of conforming to the stable constraint equation set.

[0098] The stable constraint equation set comprises: a limit equation of the motion information of the operating end; a limit equation of the motion information of the joint of the dual-arm robot; and a dynamics equivalent equation and a slip suppression equation, wherein the dynamics equivalent equation is an equivalent equation corresponding to a dynamics model corresponding to the dual-arm robot and takes the to-be-optimized variable of the first objective function as a variable, and the slip suppression equation is configured to constrain the contact force of each equivalent contact point between the operating end and the object within the friction cone range corresponding to the equivalent contact point.

[0099] For example, the limit equation limits the velocity and acceleration of the operating end / joint within a reasonable range, and the limit equation is shown in the following formula (6):

[0100]

[0101]

[0102]

[0103]

[0104] In addition, the dynamics equivalent equation is formula (4), and the slip suppression equation can be Nf C ≤b, N is a Coulomb friction coefficient matrix of all sub-objects, and b is a force matrix of all sub-objects corresponding to the friction cone range.

[0105] By taking the minimum value of the first objective function while satisfying the stability constraint equation, the optimal path coordination variable s(t) and the optimal motion trajectory (position and / or Euler angle, or pose as a function of time) of the operating end of the dual-arm robot can be obtained, and based on the optimal motion trajectory, the reference force on the object being carried at each time can also be inferred.

[0106] In summary, based on Figure 4 The method for generating an optimal motion trajectory can obtain an optimal path for carrying an object from a starting position to a target position, which consumes less time, and thus the object can be carried faster.

[0107] As mentioned earlier, it is desirable for the object to move with the operating end of the dual-arm robot throughout the entire carrying process, so as to be able to track the optimal motion trajectory obtained above. Ideally, both operating ends of the dual-arm robot are in complete contact with the object, for example, there are three desired contact points with the same left-middle-right spacing on the contact surface, i.e., the operating end completely tracks the optimal motion trajectory, and the object completely tracks the trajectory based on the optimal motion trajectory of the operating end, as shown in 5A below. However, due to the fact that the control of the operating end of the dual-arm robot is not necessarily completely accurate, there may be motion trajectory tracking errors, or the force given at a certain time is too large, causing the object to move faster. Without any improvement, the coordinated operation performance of the operating end of the dual-arm robot may not be good enough, and the object may slip relative to the operating end. In the case of a connected object, even if the lowermost object in contact with the operating end does not slip, some objects will still slip, for example, a highball on a tray may tip over during the carrying process.

[0108] Therefore, in order to further improve the stability of the object during the carrying process, further embodiments of the present disclosure also provide a scheme for correcting the force of the operating end on the object based on model predictive control during the carrying process.

[0109] Figures 5A-5E Several manifestations of the contact between the operating ends of the two robot arms and the object are shown.

[0110] Figures 5A-5EIn order to facilitate the description of the manifestations of the contact mode of the operating end with the object due to the motion trajectory tracking error, only one sub-object CRB0 is shown, but if the object is a plurality of objects connected together, the one sub-object CRB0 shown can be the one directly contacted by the operating end (for example, a tray), and the size of the force can be reasonably designed so that the contact force at each sub-object is within its friction cone. Here, it is assumed that the object (sub-object CRB0) does not slip with the left operating end, even if the contact mode changes, that is, even if both the left and right operating ends have errors, but the relative pose of the right operating end is determined based on the left operating end.

[0111] Figure 5A The ideal contact mode is shown, each operating end is at the expected position and has the expected angle, and each operating end has three equivalent contact points with the sub-object CRB0 with the same left-middle-right spacing.

[0112] As shown in the left side of the figure, the right operating end is translated upward relative to the ideal position based on the left operating end, and each operating end has only one equivalent contact point with the sub-object CRB0 on the left side; in the right side of the figure, the right operating end is translated upward, and each operating end has only one equivalent contact point with the sub-object CRB0 on the right side. Figure 5B The state of the operating end based on the displacement tracking error can be considered. Figure 5B

[0113] As shown in the left side of the figure, the right operating end is tilted downward relative to the expected position based on the left operating end (the center of mass of the right operating end does not change, and the left boundary is higher than the right boundary), and each operating end has only one equivalent contact point with the sub-object CRB0 on the left side; in the right side of the figure, the right operating end is tilted upward (the center of mass of the right operating end does not change, and the left boundary is higher than the right boundary), and each operating end has only one equivalent contact point with the sub-object CRB0 on the right side. Figure 5C The state of the operating end based on the angle tracking error can be considered. Figure 5C

[0114] As shown in the left side of the figure, the right operating end is slid to the right relative to the expected position based on the left operating end, and each operating end has three contact points with the sub-object CRB0 with the same left-middle-right spacing; in the right side of the figure, the right operating end is slid to the left, and each operating end has three equivalent contact points with the sub-object CRB0 with the same left-middle-right spacing. Figure 5D The state of the operating end based on the sliding error can be considered. Figure 5D

[0115] Figure 5E The state of the operating end based on the sliding error can be considered. Figures 5B-5D ​​​The contact method is obtained by combining the contact methods, wherein the position of the right operating end is offset to the upper right relative to the desired position and tilted downward.

[0116] Of course, there may be other ways to make contact, but they can all be based on the above references. Figures 5B-5D It is obtained by combining the basic methods.

[0117] Therefore, in actual control, it is necessary to modify the force exerted by the end effector on the object according to the actual contact method between the end effector and the object, rather than directly providing the force obtained based on the optimal motion trajectory.

[0118] Determining the corrective force requires knowledge of the object's actual pose in order to obtain an appropriate corrective force for that pose. The object's actual pose can be detected by sensors. Alternatively, it can be determined through estimation methods to save costs.

[0119] For the estimation method, for example, since the real-time pose and motion information of the end effectors are known to the controller, the desired equivalent contact point position and actual equivalent contact point position of the end effectors of the two robotic arms and the object can be determined based on the configuration of the robotic arms; then, the actual pose (position and angle) and / or actual motion information (velocity and acceleration) of the object can be estimated based on the difference between the actual equivalent contact point position and the desired equivalent contact point position, and according to the real-time actual pose and / or actual motion information of the end effectors. For example, the configuration includes at least the original relative position and real-time relative position of the two robotic arms, and the ideal position of the right end effector (when the left end effector is used as a reference).

[0120] Optionally, the difference between the actual equivalent contact point location and the expected equivalent contact point location can be reflected in various angles, various line distances, etc.

[0121] For example, refer to again Figures 5A-5E A schematic diagram showing the actual equivalent contact point location for each contact method is provided. Figure 5A The three equivalent contact points at the left and right operating ends are the desired equivalent contact point locations, which can be determined by the relative positions of the two robotic arms. Figures 5A-5E In the middle, P L1 P L and P L2 P represents the three desired equivalent contact point locations on the left operating end. R1 P R and P R2 Let P represent the three desired equivalent contact points on the right-hand operating end, and one actual equivalent contact point on each of the left and right sides. LC and PRC express.

[0122] As an example rather than a limitation, the actual pose (position and angle) and / or actual motion information (velocity and acceleration) of an object can be determined by the following formula (7):

[0123]

[0124]

[0125]

[0126] Among them, T 0 The trajectory storage matrix for the sub-object CRB0 of the object can include pose information at each past moment, as well as motion information, and is updated in real time. To store the trajectory from the end of the operation to T 0 heterogeneous transformation matrix; The current trajectory storage matrix of the operation end; ρ is the relative tilt angle of the first line connecting the contact point positions at each operation end with respect to the second line (length l0) connecting the center contact points when the operation end is in the ideal position (e.g., ...). Figure 5E As shown), it is related to the difference between the actual equivalent contact point location and the expected equivalent contact point location; R x (·) represents information related to the actual pose and motion of the end effector; ρ0 is the angle between the line connecting the actual equivalent contact point on the left and the centroid of the sub-object CRB0 and the second line (e.g., ...). Figure 5E (as shown); ε L and ε R It is also determined by the actual equivalent contact point location, and is 1 or -1 (when P RC =P R1 At that time, ε R =-1; when P RC =P R2 At that time, ε R =1; P LC =P L1 At that time, ε L =-1; when P LC =P L2 At that time, ε L =1); The actual contact point P on the left LC The vector to the centroid of CRB0; dμ R dv R and dδ R This refers to the relative movement between the desired and actual positions of the right-hand operating end.

[0127] The trajectory storage matrix T of the updated sub-object CRB0 is obtained.0 In this way, the estimated actual pose and actual motion information of the object at the current time can be obtained.

[0128] In this way, based on the dynamic model corresponding to the object, the actual force acting on the object at the current time can be estimated by the estimated pose of the object at the current time and the estimated motion information and at the current time. The 0 in the bracket represents the current time.

[0129] Generally, when correcting the actual force at the current time, the pose and force of the object at the next multiple times can be considered comprehensively so as to be corrected at each time smoothly and evenly. Therefore, the pose of the object at the next multiple times can be associated with the pose and force of the object at the current time, and then the correction force at the current time is determined by optimizing the force through the objective function (second objective function).

[0130] Figure 6 The flowchart of the operation of correcting the force of the object by the end effector is shown, as a further refinement of step S230 in Figure 3 .

[0131] As shown in Figure 6 , in step S230-1, based on the actual pose of the object at the current time and the unknown force variable applied to the object, the predicted pose of the object at each time in the first continuous N times after the current time is determined, where N is an integer greater than or equal to 1.

[0132] For example, based on the actual pose of the object (which can be one object or one sub-object) at the current time (i time, if the current time is represented by 0, then i = 0) and the unknown force variable (f C and w can be converted to each other), the system matrix Ad and the control matrix Bd are used to predict the predicted pose of the object at the next time (i+1 time). For example, The derivation of the system matrix and the control matrix will be given later, so that the predicted pose of the object at the (i+1) time is associated with the actual pose of the object at the i time and the unknown force variable, and then for the (i+2) time, the predicted pose of the object is associated with the actual pose of the object at the i time and the unknown force variable and the unknown force variable of the object at the (i+1) time, and so on.

[0133] In step S230-2, a second objective function is constructed based on the difference between the predicted pose of the object at each of the first continuous N time instants and the reference pose, and the unknown force variable of the object at each of the second continuous N time instants including the current time instant as the starting point and the difference between the unknown force variable of the object and the reference force.

[0134] For example, the second objective function can be as shown in formula (8):

[0135]

[0136] wherein N k is the number of sub-objects in the object, and are weight coefficients, and w j,ref (k) are the reference pose of the jthsub-object at the i+k+1time instant and the reference force of the jthsub-object at the i+ktime instant (i is the current time instant and is represented as 0 here) respectively, which can be obtained by the optimal motion trajectory; and w j (k) are the predicted pose of the jthsub-object at the i+k+1time instant and the unknown force variable of the jthsub-object at the i+ktime instant respectively, which is also equivalent to the unknown force variable of the jthsub-object at the i+ktime instant.

[0137] As can be seen from formula (7), the term is for the difference between the predicted pose and the reference pose for the N time instants (i+1to i+N) after the current time instant i, the term and the term is for the unknown actual force variable for the continuous N time instants (i to i+N-1) including the current time instant i.

[0138] In step S230-3, the second objective function is optimized, and the contact force of each equivalent contact point included in the object is constrained within the friction cone range corresponding to the equivalent contact point, to obtain the modified force to be applied to the object by the operating end at the current time instant.

[0139] For example, the optimization of the second objective function is performed for each unknown actual force variable, and the optimal solution of the force (denoted as u, which can be f C or w) at the current time instant is obtained as the modified force at the current time instant.

[0140] For the current time, the optimal solution of the action force (denoted as u, which can be f C or w) on the sub-object at the current time can be obtained by minimizing the second objective function while the action force of each sub-object is within the friction cone range of the corresponding equivalent contact point, and the action force on the sub-object in contact with the operation end is taken as the correction action force for the current time.

[0141] That is, the above process of obtaining the correction action force at the current time is actually derived iteratively for N periods in turn, and in order to facilitate solving operation, the above process can be expressed by matrix derivation, and then the second objective function in matrix form is used accordingly. The specific derivation process is as follows.

[0142] First, the pose and action force in N periods are derived in the inertial coordinate system. The following derivation process is only for general description, so the variables with different notations are used as in the previous description.

[0143] First, the dynamics of the object in the inertial coordinate system can be shown as formula (9):

[0144]

[0145]

[0146] where P, is the position, velocity and acceleration of the object, ω and are the angular velocity and angular acceleration, m is the mass, I' and I are the inertia matrices in the inertial coordinate system and the connected coordinate system respectively, r i is the distance from the i-th contact point to the center of mass of the connected body, is the skew-symmetric matrix of r i .

[0147] The pose of the object at the current time is constructed: Then formula (10) can be obtained:

[0148]

[0149]

[0150] where A q and B q are the system matrix and control matrix in the state equation of the object respectively, f C is the action force on the object.

[0151] Therefore, further discretization of the state equation can obtain formula (11):

[0152]

[0153] where k indicates each of the current time (0) and the N-1 times after that, A d = A q dt + E, B d = B q dt. E is a unit matrix.

[0154] The state equation of N periods can then be derived as shown in equation (12):

[0155]

[0156] X = [x q (1) x q (2) x q (3)... x q (N)]

[0157] U f = [f C (0) f C (1) f C (2)... f C (N-1)]

[0158]

[0159]

[0160] where A qp and B qp are the system matrix and the control matrix in the state equation of the object in N periods, is the k-th power of A d .

[0161] In this way, returning to the dual-arm robot of the present disclosure, the state equation of the jth sub-object can be similarly shown as equation (13):

[0162]

[0163] The system matrix and the control matrix of the jth sub-object and correspond to A qp and B qp in the above equation (12), X j and are the state variable and the control variable, respectively, corresponding to x and U f in the above equation (12).

[0164] Further, combining equation (7) and equation (13), the matrix expression of the second objective function as shown in equation (14) can be obtained:

[0165]

[0166] where variables with superscript Λ represent the state variables and control variables of all the sub-objects in the connected body.

[0167] The formula (13) can be further transformed and optimized, and considering that the force of each sub-object needs to be within the range of its friction cone, the transformed second objective function can be shown as formula (15):

[0168]

[0169] The formula (15) can be regarded as a quadratic programming (QP) problem, and the optimal force of each time in N cycles is obtained after solving it However, since the only optimal force available for controlling the dual-arm robot at the current time, and the operating end can only act on one sub-object such as CRB0 in contact with it, only the optimal force acting on the sub-object CRB0 at the current time needs to be selected from the multiple results obtained after solving, that is, the modified force (both f and w can be modified): f 0* =f 0 (0) 求解 ∈R 6×1 and w 0* =w 0 (0) 求解 ∈R 12×1 .

[0170] The modified force obtained in step S230 needs to be applied to the object by driving each joint of the robot arm, therefore, how to determine the driving force of each joint of the robot arm based on the modified force will be described below in combination with Figure 7 And, Figure 7 The object described in the process actually refers to the sub-object in contact with the operating end, because it can only be modified by changing the force applied to it by the operating end.

[0171] Figure 7 A flowchart of a process for determining the driving information of the joints of the dual-arm robot based on the modified force is shown. The process is a further refinement of step S340 in Figure 3 .

[0172] As shown in Figure 7 , in step S340-1, the force applied to the object by the operating end is estimated based on the actual motion information and the actual pose of the object corresponding to the current time.

[0173] For example, as mentioned above, based on the position change of the equivalent contact point of the operating end and the object, the actual motion information and the actual pose of the object are estimated, and then the actual force can be estimated based on the estimated actual motion information and the actual pose of the object at the current time by using the corresponding dynamic equation of the object. As shown in formula (16):

[0174]

[0175] where w r is the estimated actual force, and the meanings of the remaining terms are the same as the previous definitions, and the superscript 0 of each term indicates that it is calculated for the sub-object CRB0, and represent the estimated actual acceleration, velocity and pose of the sub-object CRB0at the current time corresponding to the object.

[0176] In step S340-2, the driving information of the joints of the dual-arm robot is determined based on the determined correction force at the current time and the estimated force.

[0177] Optionally, the driving information of each joint of the dual-arm robot can be determined by using the admittance control equation.

[0178] For example, the admittance control equation is shown in formula (17):

[0179]

[0180] where M d , B d and K d are the expected inertia matrix, the expected impedance matrix and the expected stiffness matrix, respectively.

[0181] Optionally, the error between the expected pose (unknown) of the object at the current time and the reference pose can be taken as the first error; the error between the expected velocity (unknown) of the object and the reference velocity can be taken as the second error; and the error between the expected acceleration of the object and the reference acceleration can be taken as the third error, and then the first to third errors can be associated with the correction force and the error (Δw 0 = w 0* -w r ) of the estimated actual force by using the admittance control equation, and then the expected velocity of the object is obtained as the driving information.

[0182] For example, let and assume that the control loop period of the admittance control is Δt, and at time t, the acceleration of the object can be written as formula (18):

[0183]

[0184] The speed at time t can be calculated as driving information, such as:

[0185]

[0186] Accordingly, the driving information that can be used to generate the driving force of each joint of the dual-arm robot can be obtained.

[0187] In addition, in some other embodiments, after obtaining the driving information, different operations can be performed according to the computing power level of the local controller of the dual-arm robot.

[0188] For example, when the dual-arm robot includes a local controller with sufficient computing power, the process of generating driving information as described above and the process of determining the driving force corresponding to each joint of the robot based on the driving information can be performed in the dual-arm robot.

[0189] For example, when the dual-arm robot does not include a local controller with sufficient computing power, the driving information can be generated by an external control, computing / processing device, and then provided to the controller in the robot, which can determine the driving force corresponding to each joint of the robot body based on the driving information. Even the final driving force can be obtained directly at the external control, computing / processing device, and the obtained driving force can be provided to the dual-arm robot for direct driving.

[0190] Optionally, based on the first mapping relationship between the speed of the object and the speed of the operating end and the second mapping relationship between the speed of the operating end and the driving force of each joint, the driving force of each joint can be obtained.

[0191] Since the first mapping relationship and the second mapping relationship can be respectively represented as:

[0192]

[0193]

[0194] Therefore, the joint speed corresponding to each joint can be obtained as shown in equation (21):

[0195]

[0196] Further integration can obtain the angle of each joint, and the corresponding driving force can be obtained accordingly.

[0197] That is, so far, through a series of relevant information at the current moment, such as object reference pose, object actual pose, position of the equivalent contact point of the object and the operating end, reference force of the object, etc., through local control for the current moment, the current control can obtain the driving information of each joint of the dual-arm robot, and can better achieve trajectory tracking without additional sensor settings, and has the optimal force (also known as contact force or grasping force) on the object.

[0198] Figure 8 The control method for the dual-arm robot is shown. Figures 3-7 The schematic process of the control method for the dual-arm robot of the embodiment of the present disclosure is shown.

[0199] Figure 8 The two parts are shown in the middle, one part is an external controller (TOPP-MPC) independent of the robot, and the other part is the robot (including the local controller). However, it should be understood that the external controller shown can also be in the dual-arm robot, that is, if the computing power of the local controller of the dual-arm robot is sufficient, the logical functions of the external controller can be merged into the local controller, and the present disclosure does not limit this.

[0200] As shown in Figure 8 First, the external controller can receive operator input. The operator input can include carrying commands, various information needed for modeling, and various parameters, etc.

[0201] Then, the external controller can generate the optimal motion trajectory of the operating end of the dual-arm robot based on the time optimal algorithm, and accordingly the optimal motion trajectory of the object to be carried can also be obtained. The optimal motion trajectory can provide the reference pose and reference motion information of the operating end / object at each moment.

[0202] Next, the external controller can obtain the estimated (or sensor detected) actual pose and actual motion information of the object based on the position change of the equivalent contact point of the object and the operating end (through the object pose recognition process combined with the pose of the left and right operating ends at the current moment), and determine the correction force based on the estimated (or sensor detected) actual pose of the object at the current moment, the reference force, and the reference pose and reference force of the object at the subsequent moment, and then obtain the actual force of the object based on the actual pose and actual motion information. Further, the driving information is determined based on the actual force and the correction force, wherein the driving information is provided to the robot, and the driving force for driving each joint of the dual-arm robot is obtained through the joint angle generation process and the driving force generation process.

[0203] The control based on the optimal motion trajectory can be understood as global control, and the further model predictive control can be understood as local control, so that at each time, the optimal force control mode for the current time can be determined, and through the two-level control, the force of the operating end (i.e., the robot arm) on the object can be adjusted in real time, so that even if the control of the operating end of the dual-arm robot is not necessarily completely accurate, the object (including multiple sub-objects) can be guaranteed not to slide relative to the operating end.

[0204] By referring to Figures 3-8 The method of controlling a dual-arm robot according to the embodiments of the present disclosure described above, by further locally correcting the force based on the contact point position of the operating end and the object for each time after the optimal motion trajectory is determined, makes the operating end able to quickly carry large-mass objects and ensure that the object always maintains no relative motion with the operating end even if the two robot arms are not necessarily able to be accurately controlled; it is also able to carry multiple objects at a time; and it does not need a special operating end that can provide a clamping or squeezing operation, thus improving the flexibility of carrying and the accuracy of control.

[0205] As described above, the dual-arm robot can be independent of the control device that generates the driving information, which has a local controller but limited computing power and can only perform simple calculation functions. In this case, according to another aspect of the present disclosure, a method of carrying an object by such a dual-arm robot is also provided, wherein each robot arm of the dual-arm robot has an operating end that is in non-fixed contact with the object.

[0206] Figure 9 A method 900 of carrying an object by a dual-arm robot according to an embodiment of the present disclosure is shown.

[0207] In step S910, driving information is obtained, and driving forces of respective joints of the dual-arm robot are determined based on the driving information.

[0208] For example, the driving information can be generated based on the method as described above with reference to Figures 3-8 .

[0209] For example, the driving information can be the expected speed of the object, which can be obtained based on a first mapping relationship between the speed of the object and the speed of the operating end, and a second mapping relationship between the speed of the operating end and the angular speed of the respective joints, and then the angle and the driving force of the respective joints can be obtained. As described above with reference to formulas (19) and (20).

[0210] In step S920, the joints of each mechanical arm of the dual-arm robot are driven based on the driving force, so that the operating end carries the object from the starting position to the target position, and there is no relative motion between the operating end and the object.

[0211] Optionally, the dual-arm robot can be provided with a driving motor. After obtaining the angle, angular velocity and angular acceleration of each joint in the last step, the motor can be controlled by the local controller to drive each joint, so that the operating end can apply a suitable force to the object to carry the object from the starting position to the target position, and there is no relative motion between the operating end and the object.

[0212] In this way, the dual-arm robot can obtain driving information or even the final driving force from the outside. Only simple processing is required locally, so that the dual-arm robot can quickly carry large-mass objects and ensure that there is no relative motion between the operating end and the object even if the two mechanical arms cannot be accurately controlled; it can also carry multiple objects at a time; and it does not require a special operating end that can provide gripping or squeezing operations, thereby improving the flexibility of the carrying and the accuracy of the control.

[0213] According to another aspect of the present disclosure, a device for controlling a dual-arm robot is also disclosed, wherein each mechanical arm of the dual-arm robot has an operating end that non-fixedly contacts an object and carries the object from a starting position to a target position, and there is no relative motion between the operating end and the object.

[0214] Figure 10 A structural block diagram of the device 1000 for controlling a dual-arm robot according to an embodiment of the present disclosure is shown.

[0215] As Figure 10 shown, the device 1000 includes a trajectory acquisition module 1010, a reference information determination module 1020, a correction module 1030 and a driving information determination module 1040.

[0216] The trajectory acquisition module 1010 can be configured to refer to Figure 3 The operation in step S310 described above, i.e., obtaining the optimal motion trajectory of the operating end carrying the object.

[0217] The reference information determination module 1020 can be configured to refer to Figure 3 The operation in step S320 described above, i.e., obtaining the reference pose of the object corresponding to each time and the reference force applied by the operating end to the object based on the optimal motion trajectory.

[0218] The correction module 1030 can be configured to refer to Figure 3The operation in step S330 described is that, based on the actual pose of the object corresponding to the current moment and the reference force, a correction force exerted on the object by the operation end is determined.

[0219] The driving information determination module 1040 can be configured to refer to Figure 3 The operation in step S330 described is that, based on the correction force, driving information of each joint of the dual-arm robot is determined.

[0220] More details of the operations performed by each module can be referred to the foregoing description of the method according to Figures 3-8 The content described is therefore not repeated here.

[0221] Each module according to the function implemented thereby can also be divided into more sub-modules, or different modules can be combined into a new module, which is not limited by the present disclosure.

[0222] According to another aspect of the present disclosure, a dual-arm robot is provided. Figure 11 A schematic diagram of a dual-arm robot 1100 according to an embodiment of the present disclosure is shown.

[0223] The dual-arm robot 1100 can include a robot body 1110 and a controller 1120.

[0224] Optionally, the robot body 1110 includes two mechanical arms, each of which can have the same number of degrees of freedom according to actual needs, and each of which has an operation end, which can adopt a common arrangement.

[0225] When the computing power of the controller 1120 is sufficient, the controller 1120 can be configured to perform the method according to Figures 3-8 The method of controlling the dual-arm robot described and the dual-arm robot according to Figure 9 The method of the robot carrying the object described.

[0226] When the computing power of the controller 1120 is limited, the controller 1120 can be configured to perform the method of the robot carrying the object according to Figure 9 The method of the robot carrying the object described.

[0227] According to a more detailed embodiment of the dual-arm robot of the present disclosure, the controller 1120 can be implemented as any device that can perform the method as described in aspects of the present disclosure, including but not limited to FPGA, DSP, ARM single-chip microcomputer, processor, microprocessor, etc.

[0228] More details of the functions of the controller 1120 can be referred to the foregoing description of the method according to aspects of the present disclosure, which is not repeated here.

[0229] Although not shown in the figure, robot 1100 may also include an input / output component to support input / output data flow between robot 1100 and other components. Robot 1100 may also send and receive information and data from a network via a communication port.

[0230] According to another aspect of this disclosure, a control device is proposed. The control device is used to control a dual-arm robot, each of the robotic arms having an end effector that is in non-fixed contact with an object and moves the object from a starting position to a target position, with no relative motion between the end effector and the object. The control device includes: a processor; and a memory storing a computer program, which, when executed, causes the processor to perform actions according to the foregoing reference. Figures 3-8 A method for controlling a dual-arm robot is described.

[0231] The processor is used to execute program instructions. The memory can exist in different forms of program storage units and data storage units, such as hard disks, read-only memory (ROM), and random access memory (RAM), which can be used to store various data files processed and / or executed by the processor, as well as possible program instructions executed by the processor.

[0232] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed, implements the method described above. Figures 3-8 The method for controlling a dual-arm robot described or referenced Figure 9 The method described is for a dual-arm robot to move objects.

[0233] According to the seventh aspect of this disclosure, a computer program product is also provided, comprising a computer program that, when executed, implements the above-mentioned references. Figures 3-8 The method for controlling a dual-arm robot described or referenced Figure 9 The method described is for a dual-arm robot to move objects.

[0234] Furthermore, those skilled in the art will understand that aspects of this disclosure can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Accordingly, aspects of this disclosure can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The aforementioned hardware or software may be referred to as a “data block,” “module,” “engine,” “unit,” “component,” or “system.” Furthermore, aspects of this disclosure may be embodied as a computer product located on one or more computer-readable media, the product including computer-readable program code.

[0235] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure.

Claims

1. A method for controlling a dual-arm robot, wherein each robotic arm of the dual-arm robot has an end effector, the end effector being in non-fixed contact with an object and lifting the object from a starting position to a target position, and there is no relative motion between the end effector and the object, the method comprising: Obtain the optimal motion trajectory of the object being transported by the end effector; Based on the optimal motion trajectory, the reference pose of the object at each moment and the reference force applied to the object by the end effector are obtained; Based on the actual pose and reference force of the object at the current moment, and the reference pose and reference force of the object at at least one moment after the current moment, the corrective force applied to the object by the operation end is determined. as well as The drive information of the joints of the dual-arm robot is determined based on the corrective force.

2. The method according to claim 1, wherein, The optimal motion trajectory is generated as follows: a first objective function is constructed based on time and the force applied to the object by the end of the operation using a time-optimal algorithm, and the first objective function is optimized based on the dynamic model corresponding to the dual-arm robot, the dynamic model corresponding to the object, and the stability constraint equations.

3. The method according to claim 1, wherein, The object may include a single sub-object or a conjoined body comprising multiple sub-objects, wherein the operational end effector applies a force to a sub-object that is not in fixed contact with it, such that a contact force exists at each of the equivalent contact points of each sub-object. During the process of transporting the object at the end of the operation, the contact force of each equivalent contact point of each sub-object is constrained within the range of the friction cone corresponding to the equivalent contact point.

4. The method according to claim 3, further comprising: The actual pose and motion information of the object at the current moment are detected by sensors; or Based on the equivalent contact point positions between the operating ends of the two robotic arms of the dual-arm robot and the sub-object, the actual pose and motion information of the object at the current moment are estimated.

5. The method according to claim 4, wherein, Based on the equivalent contact point positions between the operating ends of the two robotic arms of the dual-arm robot and the sub-object, the actual pose and motion information of the object at the current moment are estimated, including: Based on the configuration of the robotic arms, the desired equivalent contact point position and the actual equivalent contact point position of the object for each of the two robotic arms' respective end effectors are determined; and Based on the difference between the actual equivalent contact point position and the expected equivalent contact point position, and based on the actual pose and motion information of the operating ends of the two robotic arms, the actual pose and motion information of the object are estimated.

6. The method according to claim 4 or 5, wherein, Based on the actual pose and reference force of the object at the current moment, and the reference pose and reference force of the object at at least one moment after the current moment, the correction force applied by the end effector to the object is determined, including: Based on the actual pose of the object at the current moment and the unknown force variable applied to the object, determine the predicted pose of the object at each of the first consecutive N moments after the current moment; Based on the difference between the predicted pose and the reference pose of the object at each of the first N consecutive time steps, and the unknown force variables of the object at each of the second N consecutive time steps starting from the current time step, as well as the difference between the unknown force variables of the object and the reference force, a second objective function is constructed; and By optimizing the second objective function and constraining the contact force of each equivalent contact point of the object within the range of the friction cone corresponding to the equivalent contact point, a corrective force to be applied to the object by the end of the operation at the current moment is obtained.

7. The method according to claim 6, wherein, Determining the joint drive information of the dual-arm robot based on the corrective force includes: The force exerted by the end effector on the object is estimated based on the object's actual motion information and actual pose at the current moment; and Based on the determined corrective force and the estimated force at the current moment, the drive information of the joints of the dual-arm robot is determined.

8. The method according to claim 7, further comprising: The driving force corresponding to the joints of the robot body of the dual-arm robot is determined based on the driving information. or The driving information is provided to the dual-arm robot, and the driving information is used by the dual-arm robot to determine the driving force corresponding to the joint.

9. The method according to claim 7 or 8, wherein, Based on the determined corrected force and the estimated force at the current moment, the drive information of the joints of the dual-arm robot is determined, including: Based on the determined corrected force and the estimated force at the current moment, the drive information of the joints of the dual-arm robot is determined using the admittance control equation.

10. The method according to claim 2, wherein, The dynamic model of the object indicates the relationship between the force exerted on the object by the operating end, the object's gravity, the object's inertial force, and the object's Cossian force and centrifugal force, and the object's pose, linear velocity, and linear acceleration.

11. The method according to claim 2, wherein, The dynamic model corresponding to the dual-arm robot indicates the relationship between the robot's inertia, Coriolis force, centrifugal force, gravity, external forces, joint angles, joint angular velocities, and joint angular accelerations and the joint driving forces of the dual-arm robot.

12. The method according to claim 2, wherein, The first objective function is associated with the force exerted by the end of the operation on the object and has variables to be optimized. The stability constraint equations include: constraint equations for the motion information of the end effector; constraint equations for the motion information of the joints of the dual-arm robot; and dynamic equivalent equations and sliding inhibition equations. The dynamic equivalent equation is the equivalent equation corresponding to the dynamic model of the dual-arm robot, with the variable to be optimized of the first objective function as the variable, and the sliding inhibition equation is configured to constrain the contact force of each equivalent contact point between the end effector and the object within the range of the friction cone corresponding to the equivalent contact point.

13. A method for a dual-arm robot to handle an object, each robotic arm of the dual-arm robot having an end effector that is in non-fixed contact with the object, the method comprising: Acquire driving information and determine the driving force of the joints of the dual-arm robot based on the driving information; The driving force drives the joints of each arm of the dual-arm robot, causing the end effector to lift the object from its starting position to its destination position, with no relative motion between the end effector and the object. The driving information is generated based on the method described in any one of claims 1-12.

14. An apparatus for controlling a dual-arm robot, each robotic arm of the dual-arm robot having an end effector, the end effector being in non-fixed contact with an object and lifting the object from a starting position to a target position, and there being no relative motion between the end effector and the object, the apparatus comprising: The trajectory acquisition module is used to acquire the optimal motion trajectory of the end effector transporting the object; The reference information determination module is used to obtain the reference pose of the object at each moment and the reference force applied to the object by the operation end based on the optimal motion trajectory; The correction module is used to determine the correction force applied to the object by the operation end based on the actual pose and reference force of the object at the current time and the reference pose and reference force at at least one time after the current time. as well as The drive information determination module determines the drive information of the joints of the dual-arm robot based on the corrective force.

15. A dual-arm robot, comprising: The robot body includes: two end effectors, each corresponding to one of the two robotic arms of the dual-arm robot; each end effector is in non-fixed contact with an object, and the end effector applies a force to the object to lift it from a starting position to a target position; and The controller is configured to perform the method according to claims 1-13.

16. A control device for controlling a dual-arm robot, each robotic arm of the dual-arm robot having an end effector, the end effector being in non-fixed contact with an object and lifting the object from a starting position to a target position, and there being no relative motion between the end effector and the object, the control device comprising: processor; A memory having a computer program stored thereon, which, when executed, causes the processor to perform the method according to any one of claims 1-12.

17. A computer-readable storage medium having a computer program stored thereon, the computer program, when executed, implementing the method according to any one of claims 1-13.

18. A computer program product comprising a computer program that, when executed, implements the method according to any one of claims 1-13.

Images