Control method, device, robot and storage medium of dual-arm robot

Abstract

The present application discloses a control method, device, robot and storage medium for a dual-arm robot, and relates to the field of robot control technology of artificial intelligence technology. The method comprises: obtaining a first operation task to be performed by the dual-arm robot and the corresponding environmental information; based on the first operation task and the environmental information, obtaining the planned motion trajectory of the object and the expected contact force of the object corresponding to the first operation task; based on the planned motion trajectory of the object and the expected contact force of the object, determining the expected joint torque that satisfies the kinematic constraints and dynamic constraints; based on the expected joint torque, controlling the movement of each joint of the dual-arm robot to perform the first operation task. The present application plans the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level based on the operation task, so that the dual-arm robot can perform operation tasks that focus on both kinematic constraints and dynamic constraints, thereby expanding the scope of application of the dual-arm robot.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robot control technology using artificial intelligence technology, and in particular to a dual-arm robot control method, device, robot, and storage medium. Background Art

[0002] In recent years, with the development of robot control technology, dual-arm robots have a wider range of applications, such as home, service, industry, laboratory and other applications.

[0003] Related technologies classify tasks based on kinematic constraints and then plan the motion of dual-arm collaborative robots based on the resulting task types. However, these technologies are only suitable for tasks that require kinematic constraints, resulting in a narrow range of applicable tasks and limited versatility. Summary of the Invention

[0004] The embodiments of the present application provide a control method, device, robot, and storage medium for a dual-arm robot, which can expand the range of operational tasks that the dual-arm robot can perform and improve the versatility of the dual-arm robot. The technical solution is as follows:

[0005] According to one aspect of an embodiment of the present application, a control method for a dual-arm robot is provided, the method comprising:

[0006] Acquire a first operation task to be performed by the dual-arm robot and environmental information corresponding to the first operation task;

[0007] Based on the first operation task and the environmental information, obtaining a planned motion trajectory of an object and an expected contact force of the object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object;

[0008] Determining a desired joint torque that satisfies kinematic constraints and dynamic constraints based on the planned motion trajectory of the object and the desired contact force of the object;

[0009] Based on the expected joint torque, the movement of each joint of the dual-arm robot is controlled to perform the first operation task.

[0010] According to one aspect of an embodiment of the present application, a control method for a dual-arm robot is provided, the method comprising:

[0011] Obtaining a first operation task to be performed by the dual-arm robot;

[0012] determining a type of the first operation task;

[0013] If the first operation task is a loose collaborative task, obtaining expected joint torques that satisfy kinematic constraints, and controlling the motion of each joint of the dual-arm robot based on the expected joint torques that satisfy the kinematic constraints to perform the first operation task; wherein the loose collaborative task refers to a collaborative task with the kinematic constraints between the dual arms of the dual-arm robot and the target operation object in the first operation task;

[0014] Alternatively, if the first manipulation task is a tightly coordinated task, obtaining desired joint torques that satisfy kinematic and dynamic constraints, and controlling the motion of each joint of the dual-arm robot to perform the first manipulation task based on the desired joint torques that satisfy the kinematic and dynamic constraints; wherein the tightly coordinated task refers to a collaborative task between the dual arms of the dual-arm robot and the target manipulation object in the first manipulation task having the kinematic and dynamic constraints;

[0015] The collaborative task refers to an operation task that requires the coordinated movement of both arms of the dual-arm robot.

[0016] According to one aspect of an embodiment of the present application, a control device for a dual-arm robot is provided, the device comprising:

[0017] An operation task acquisition module, configured to acquire a first operation task to be performed by the dual-arm robot and environmental information corresponding to the first operation task;

[0018] a planning information acquisition module, configured to acquire, based on the first operation task and the environmental information, a planned motion trajectory of an object and an expected contact force of the object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object;

[0019] an expected torque acquisition module, configured to determine an expected joint torque satisfying kinematic constraints and dynamic constraints based on the planned motion trajectory of the object and the expected contact force of the object;

[0020] An operation task execution module is used to control the movement of each joint of the dual-arm robot to execute the first operation task based on the expected joint torque.

[0021] According to one aspect of an embodiment of the present application, a control device for a dual-arm robot is provided, the device comprising:

[0022] An operation task acquisition module, configured to acquire a first operation task to be performed by the dual-arm robot;

[0023] A task type determination module, configured to determine the type of the first operation task;

[0024] an operation task execution module, configured to, when the first operation task is a loose collaborative task, obtain expected joint torques that satisfy kinematic constraints, and control the motions of the joints of the dual-arm robot based on the expected joint torques that satisfy the kinematic constraints to perform the first operation task; wherein the loose collaborative task refers to a collaborative task with the kinematic constraints between the dual arms of the dual-arm robot and the target operation object in the first operation task;

[0025] Alternatively, the operation task execution module is further configured to, when the type of the first operation task is a tight collaborative task, obtain expected joint torques that satisfy kinematic constraints and dynamic constraints, and control the motions of each joint of the dual-arm robot to perform the first operation task based on the expected joint torques that satisfy the kinematic constraints and dynamic constraints; wherein the tight collaborative task refers to a collaborative task between the dual arms of the dual-arm robot and the target operation object in the first operation task having the kinematic constraints and the dynamic constraints;

[0026] The collaborative task refers to an operation task that requires the coordinated movement of both arms of the dual-arm robot.

[0027] According to one aspect of an embodiment of the present application, a dual-arm robot is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned dual-arm robot control method.

[0028] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above-mentioned control method of the dual-arm robot.

[0029] According to one aspect of an embodiment of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a dual-arm robot reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the dual-arm robot to perform the aforementioned dual-arm robot control method.

[0030] The technical solutions provided by the embodiments of the present application include at least the following beneficial effects:

[0031] By planning the motion trajectory of the object and the expected contact force of the object based on the first operation task, the joint torque of the dual-arm robot is planned, and the joint torque of the dual-arm robot is planned at the kinematic constraint level and the dynamic constraint level. This allows the dual-arm robot to perform operation tasks that require both kinematic constraints and dynamic constraints on the dual-arm robot, expanding the scope of operation tasks that can be performed by the dual-arm robot and improving the versatility of the dual-arm robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 This is a schematic structural diagram of a dual-arm robot with 7 degrees of freedom provided by one embodiment of the present application;

[0034] Figure 2 This is a schematic diagram of the classification of operation tasks provided by an embodiment of the present application;

[0035] Figure 3 This is a schematic diagram of the classification of operation tasks provided by an embodiment of the present application;

[0036] Figure 4 This is a flow chart of a control method for a dual-arm robot provided by one embodiment of the present application;

[0037] Figure 5 is a flow chart of a control method for a dual-arm robot provided by another embodiment of the present application;

[0038] Figure 6 is a flow chart of a method for adjusting a desired joint torque provided by one embodiment of the present application;

[0039] Figure 7 This is a flow chart of a position tracking method for a dual-arm robot provided by one embodiment of the present application;

[0040] Figure 8 This is a flowchart of a method for transitioning an operation task provided by an embodiment of the present application;

[0041] Figure 9 This is a schematic diagram of the architecture of a control system for a dual-arm robot provided by one embodiment of the present application;

[0042] Figure 10 This is a block diagram of a control device for a dual-arm robot provided by one embodiment of the present application;

[0043] Figure 11 is a block diagram of a control device for a dual-arm robot provided by another embodiment of the present application;

[0044] Figure 12 is a block diagram of a control device for a dual-arm robot provided by another embodiment of the present application;

[0045] Figure 13 This is a simplified structural block diagram of a dual-arm robot provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0047] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0048] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0049] The technical solution of this application mainly relates to robotics technology in artificial intelligence technology, mainly to intelligent control of robots. A robot is a mechanical and electronic device that can imitate certain human skills by combining mechanical transmission and modern microelectronics technology. Robots are developed on the basis of electronic, mechanical and information technology. A robot does not necessarily have to look like a human. As long as it can independently complete the tasks and commands assigned to it by humans, it belongs to the robot family. A robot is an automated machine that has some intelligent capabilities similar to those of humans or biological organisms, such as perception, planning, movement and coordination. It is an automated machine with high flexibility. With the development of computer technology and artificial intelligence technology, robots have been greatly improved in terms of function and technical level. Mobile robots and robot vision and touch technologies are typical representatives.

[0050] The technical solution of this application mainly relates to a dual-arm robot. Compared with a single-arm robot, a dual-arm robot is equipped with dual robotic arms that are more similar to the arms of the human body. The dual arms of a dual-arm robot are not just two independent single robotic arms, but an inseparable whole with information exchange. The dual-arm robot can coordinate operations through its two arms and can complete complex operational tasks. For example, a dual-arm robot can provide people with basic life services such as carrying objects and doing housework. The dual-arm robot can provide people with simple entertainment activities or interactive activities. The dual-arm robot can also replace people to complete some work in dangerous and harsh environments, etc.

[0051] The control method for a dual-arm robot provided in an embodiment of the present application can expand the scope of operational tasks that can be performed by the dual-arm robot and improve the versatility of the dual-arm robot. For example, when the operational task focuses on kinematic constraints, the joint torque of the dual-arm robot is planned to satisfy the kinematic constraints, allowing the dual-arm robot to perform the operational task. When the operational task focuses on both kinematic and dynamic constraints, the joint torque of the dual-arm robot is planned to satisfy both kinematic and dynamic constraints, allowing the dual-arm robot to perform the operational task.

[0052] Alternatively, dual-arm robots can be categorized based on degrees of freedom. For example, there are dual-arm robots with 6 degrees of freedom (i.e., each robotic arm has 6 degrees of freedom), dual-arm robots with 7 degrees of freedom (i.e., each robotic arm has 7 degrees of freedom), dual-arm robots with 8 degrees of freedom (i.e., each robotic arm has 8 degrees of freedom), and so on. Dual-arm robots can also be redundant with respect to the dimensions (i.e., degrees of freedom) required for the task. This means that there are infinitely many feasible solutions to satisfy a given task, both from a kinematic and a dynamic perspective. For example, for a task requiring 6 degrees of freedom, a dual-arm robot with 7 degrees of freedom is used, and this 7-degree-of-freedom dual-arm robot is redundant.

[0053] like Figure 1 As shown, it exemplarily shows a structural diagram of a dual-arm robot with 7 degrees of freedom. The dual-arm robot 100 includes a robotic arm 101, a robotic arm 102, a two-finger gripper 103 at the end, a visual workstation 104, and a dual-arm control workstation 105. The robotic arm 101 and the robotic arm 102 can be symmetrical structures. The robotic arm 101 is taken as an example for introduction: the robotic arm 101 is a 7-degree-of-freedom redundant robotic arm (i.e., one more degree of freedom than the spatial degree of freedom), and the end of the robotic arm 101 is equipped with a two-finger gripper 103 at the end, which can be used to perform operational tasks such as grasping and moving objects. The wrist of the robotic arm 101 is equipped with a hand-eye camera and a six-dimensional torque sensor, which can be used to obtain local environmental information and the end driving force (i.e., the true value of the six-dimensional force) in the operation task space. The robotic arm 101 adopts a modular joint design, and each joint is equipped with a torque sensor, a Hall current sensor and a position encoder, which can be used to obtain real-time status information of the joints of the robotic arm 101, such as joint angle, joint angular velocity, joint torque and joint current, etc. (which is beneficial to the active compliance, object impedance control, internal force impedance control, etc. of the robotic arm). The body of the robotic arm 101 is also covered with artificial tactile skin, which can be used to obtain information acting on the body of the robotic arm 101 (which is beneficial to the whole-body compliance control of the robotic arm). Optionally, a binocular vision camera is also installed above the robotic arm 101 and the robotic arm 102, which can be used to obtain global visual information in the operation task space.

[0054] Optionally, the visual workstation 104 is used to process local environment information and global visual information. The dual-arm control workstation 105 is used to control the movement of the robotic arms 101 and 102. The dual-arm control workstation 105 can be a NUC (Next Unit of Computing) small computer.

[0055] In one exemplary embodiment, reference Figure 2 The types of manipulation tasks 201 can be divided into collaborative tasks 203 and non-collaborative tasks 202. Collaborative tasks 203 refer to manipulation tasks that require the coordinated movement of the two arms of a dual-arm robot; for example, the two arms of a dual-arm robot coordinate to carry an object. Non-collaborative tasks 202 refer to manipulation tasks that do not require the coordinated movement of the two arms of a dual-arm robot; for example, one arm of a dual-arm robot moves object 1 from position A to position B, while the other arm moves object 2 from position C to position D.

[0056] Optionally, the collaborative task 203 can be further divided into a symmetric collaborative task 204 and an asymmetric collaborative task 205. Symmetric collaborative task 204 refers to an operation task in which, during the collaborative movement of the dual arms of a dual-arm robot, the motion trajectories of the end effectors of the two arms have a fixed relative position relationship; for example, the end effectors of the two arms collaborate to rotate the steering wheel from position A to position B. Asymmetric collaborative task 205 refers to an operation task in which, during the collaborative movement of the dual arms of a dual-arm robot, the motion trajectories of the end effectors of the two arms have a variable relative position relationship; for example, one robotic arm is used to fix a bottle body, and the other robotic arm is used to screw the bottle cap corresponding to the bottle body.

[0057] Optionally, the symmetrical collaborative task 204 can be further divided into a symmetrical loose collaborative task 206 and a symmetrical tight collaborative task 207. The symmetrical loose collaborative task 206 refers to a symmetrical collaborative task in which kinematic constraints exist between the dual arms of the dual-arm robot and the target object; for example, the dual arms of the dual-arm robot collaborate to lift the target object. The symmetrical tight collaborative task 207 refers to a symmetrical collaborative task in which kinematic constraints and dynamic constraints exist between the dual arms of the dual-arm robot and the target object; for example, the dual arms of the dual-arm robot collaborate to grasp the target object.

[0058] Optionally, the asymmetric collaborative task 205 can be further divided into an asymmetric loose collaborative task 208 and an asymmetric tight collaborative task 209. The asymmetric loose collaborative task 208 refers to an asymmetric collaborative task in which kinematic constraints exist between the dual arms of the dual-arm robot and the target object. For example, the dual arms of the dual-arm robot collaborate to place object A into object B. The dual arms of the dual-arm robot collaborate to lift the target object. The asymmetric tight collaborative task 209 refers to an asymmetric collaborative task in which kinematic constraints and dynamic constraints exist between the dual arms of the dual-arm robot and the target object. For example, one robotic arm is used to fix a bottle, and the other robotic arm is used to unscrew the bottle cap corresponding to the bottle.

[0059] Optionally, the non-collaborative task 202 can be divided into a loose non-collaborative task 210 and a tight non-collaborative task 211. A loose non-collaborative task 210 refers to a non-collaborative task in which a single arm of a dual-arm robot has a kinematic constraint with the target object; for example, one robotic arm picks up object A, and the other robotic arm picks up object B. A tight non-collaborative task 211 refers to a non-collaborative task in which a single arm of a dual-arm robot has both a kinematic constraint and a dynamic constraint with the target object; for example, one robotic arm picks up object A, and the other robotic arm picks up object B.

[0060] The classification of operation tasks described above is for illustrative and explanatory purposes only. The classification of operation tasks can be adjusted based on actual circumstances. For example, the classification criteria can be appropriately adjusted (e.g., further subdivided based on the weight or size of the target operation object). This application does not limit the classification criteria for operation tasks. Any type of operation task applicable to this application shall be within the scope of protection of this application.

[0061] In one example, reference Figure 3 , which shows a relationship diagram of the type of operation task, the first structure matrix and the constraints provided by an embodiment of the present application. In Table 301, the first structure matrix of the symmetric loose collaborative task is the absolute Jacobian matrix, and the constraints corresponding to the symmetric loose collaborative task are kinematic constraints; the first structure matrix of the symmetric tight collaborative task is the absolute Jacobian matrix, and the constraints corresponding to the symmetric tight collaborative task are kinematic constraints and dynamic constraints; the first structure matrix of the asymmetric loose collaborative task is the weighted sum of the absolute Jacobian matrix and the relative Jacobian matrix, and the constraints corresponding to the asymmetric loose collaborative task are kinematic constraints; the first structure matrix of the asymmetric tight collaborative task is the weighted sum of the absolute Jacobian matrix and the relative Jacobian matrix, and the constraints corresponding to the asymmetric tight collaborative task are kinematic constraints and dynamic constraints; the loose non-collaborative task has no first structure matrix, and the constraints corresponding to the loose non-collaborative task are kinematic constraints; the tight non-collaborative task has no first structure matrix, and the constraints corresponding to the tight non-collaborative task are kinematic constraints and dynamic constraints.

[0062] Please refer to Figure 4 , which shows a flow chart of a control method for a dual-arm robot provided by an embodiment of the present application. The execution entity of each step of the method can be the dual-arm robot 100 described above. The method can include the following steps (401-404):

[0063] Step 401: Acquire a first operation task to be performed by a dual-arm robot and environmental information corresponding to the first operation task.

[0064] In the embodiments of the present application, the first operation task refers to a task that is performed by using the dual arms of a dual-arm robot. For example, in a service scenario, the dual arms of a dual-arm robot can be used to perform operations such as pouring water for a user, opening a bottle cap, and delivering food. In an industrial scenario, the dual arms of a dual-arm robot can be used to perform operations such as assembling parts and moving parts. In an autonomous driving scenario, the dual arms of a dual-arm robot can be used to perform operations such as steering wheel control, which is not limited in the embodiments of the present application.

[0065] Environmental information is used to describe the environment in the task space corresponding to the operation task. For example, the environmental information corresponding to the first operation task is used to describe the environment in the task space corresponding to the first operation task. This environmental information may include information such as objects, people, object locations, and the robotic arms of a dual-arm robot in the task space.

[0066] Step 402: Based on the first operation task and environmental information, obtain the planned motion trajectory of the object and the expected contact force of the object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object.

[0067] The target object refers to the object that the dual-arm robot is to manipulate when performing an operation task, such as the food, parts, steering wheel, etc. in the first operation task mentioned above. In the embodiment of the present application, the end effector refers to the actuator used by the dual-arm robot to manipulate the object. The end effector can be a clamping type or a suction type, which is not limited in the embodiment of the present application.

[0068] In one example, by sampling images of objects in the environmental information and then analyzing the acquired images, characteristic data of different objects, such as color, shape, size, position, etc., can be obtained. Then, based on the object database pre-stored in the dual-arm robot (for example, feature data comparison), the types of each object in the task space can be determined. Finally, based on the object to be operated by the dual-arm robot in the first operation task, the target operation object in the task space and the initial position of the target operation object can be determined.

[0069] Optionally, the process for determining the planned object trajectory can be as follows: based on the content of the first operation task and environmental information, the target operation object's final position is determined; and the target operation object's trajectory is planned based on the target operation object's initial and final positions to obtain the planned object trajectory. For example, the first operation task involves handing object 1 from position A to user B. Object 1's initial position A is obtained, user B's position is determined, and user B's position is determined as object 1's final position. Based on object 1's initial and final positions, object 1's trajectory is planned to obtain the corresponding planned object trajectory.

[0070] In one example, the expected contact force of the object is determined based on the availability information pre-stored in the dual-arm robot. The availability information can be used to provide safe force information, safe driving force information, optimal operating position (such as grasping position), etc. of the object. The expected contact force of the object includes the expected clamping internal force and the expected driving force of the object. The expected driving force of the object is used to drive the target operating object. The expected clamping internal force refers to the expected internal force formed by the end effector of the dual arms of the dual-arm robot on the target object, which can be used to clamp the target operating object without affecting the motion trajectory of the end effector. The expected clamping internal force can be determined based on the safe force information of the target operating object, and the expected driving force of the object can be determined based on the safe driving force information of the target operating object.

[0071] Optionally, the motion trajectory of the dual-arm robot's end effectors to the target object can be customized based on the end effectors' initial position, optimal operating position, and desired contact force with the object. For example, the shortest end effector trajectory can be planned based on the end effectors' initial position and optimal operating position.

[0072] Step 403 : Determine the desired joint torque that satisfies the kinematic constraints and the dynamic constraints based on the planned motion trajectory of the object and the desired contact force of the object.

[0073] In the embodiment of the present application, the kinematic constraint refers to the motion constraint existing between the end effectors of the dual arms of the dual-arm robot and the target operation object. For example, when the end effectors of the dual arms of the dual-arm robot are in contact with the target operation object, the end effectors of the dual arms of the dual-arm robot and the target operation object move synchronously, and the relative speed between the two is 0. The dynamic constraint refers to the interaction force constraint existing between the end effectors of the dual arms of the dual-arm robot and the target operation object. For example, when the end effectors of the dual arms of the dual-arm robot are in contact with the target operation object, the end effectors of the dual arms of the dual-arm robot need to apply a fixed force to the target operation object to ensure that the target operation object does not fall or moves along the desired trajectory.

[0074] The desired joint torque is used to control the motion of each joint of the dual-arm robot so that the end effectors of the dual arms can move along the desired trajectory. In one example, the process of obtaining the desired joint torque can be as follows:

[0075] Obtain a first structure matrix corresponding to the type of the first operation task, where the first structure matrix is used to represent a mapping relationship between the planned motion trajectory of the object and the expected end motion trajectory, where the expected end motion trajectory refers to the motion trajectory of the end effector while satisfying the kinematic constraints; convert the planned motion trajectory of the object according to the first structure matrix to obtain the expected end motion trajectory that satisfies the kinematic constraints; determine a first joint torque based on the expected end motion trajectory, where the first joint torque refers to the joint torque that satisfies the inverse kinematics of the dual-arm robot; determine a second joint torque based on the expected contact force of the object, where the second joint torque refers to the joint torque that satisfies the inverse dynamics of the dual-arm robot; superimpose and fuse the first joint torque and the second joint torque to obtain the expected joint torque that satisfies the kinematic constraints and dynamic constraints.

[0076] Optionally, the transpose of the first structure matrix is multiplied by the planned motion trajectory of the object to obtain the desired end motion trajectory. The elements in the first structure matrix can be calculated based on the real-time position of the end effectors of the two arms of the dual-arm robot and the real-time position of the target operation object. For example, the first structure matrix is determined based on the center position of the target operation object and the center position of the end effectors of the two arms of the dual-arm robot. The style of the first structure matrix is associated with the type of operation task, and the specific process of determining the style of the first structure matrix can be as follows:

[0077] When the type of the first operation task is a symmetric collaborative task, the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix. The absolute Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in the absolute motion space. When the type of the first operation task is an asymmetric collaborative task, the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix. The relative Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in the relative motion space.

[0078] In one example, the process of obtaining the first joint torque may be as follows:

[0079] Obtain the single-arm Jacobian matrices corresponding to each of the two arms of the dual-arm robot; perform centralized fusion processing on the single-arm Jacobian matrices corresponding to each of the two arms to obtain a second structure matrix, which is used to represent the mapping relationship between the joints of each of the two arms of the dual-arm robot and the end effector; based on the second structure matrix, transform the expected end motion trajectory to obtain expected joint motion data of the dual-arm robot; and based on the expected joint motion data, obtain a first joint torque. The single-arm Jacobian matrix is used to represent the mapping relationship between the velocities of each joint of the single arm of the dual-arm robot and the velocity of the end effector. The second structure matrix can be a diagonal matrix formed by the single-arm Jacobian matrices of each of the two arms of the dual-arm robot.

[0080] Exemplarily, the specific content of the centralized fusion processing can be as follows: the single-arm Jacobian matrix corresponding to the left arm and the single-arm Jacobian matrix corresponding to the right arm are used as elements on the diagonal of the diagonal matrix (such as the diagonal from the upper left corner to the lower right corner of the diagonal matrix), and the remaining elements of the diagonal matrix are set to 0. The diagonal matrix obtained in this way is the second structure matrix. The transpose of the second structure matrix and the expected end motion trajectory are multiplied together to obtain the expected joint motion data corresponding to each of the two arms of the dual-arm robot. The expected joint motion data may include the expected joint angle, the expected joint angular velocity, etc. Based on the expected joint motion data, the first joint torque corresponding to each of the two arms of the dual-arm robot can be calculated. For example, based on the joint angle increment or the joint angular velocity, the required joint torque is determined, that is, the joint torque required from joint angle A to joint angle B. Optionally, the joint torque and the joint angle increment or the joint angular velocity are positively correlated. In this way, the first joint torque corresponding to each of the two arms can be obtained simultaneously through a Jacobian matrix, thereby ensuring the synergy of the two-arm motion.

[0081] In one example, the process of obtaining the second joint torque may be as follows:

[0082] The desired contact force of the object is converted according to the force Jacobian matrix of the dual-arm robot to obtain the second joint torque; wherein the force Jacobian matrix is used to represent the mapping relationship between the joint torques of the two arms of the dual-arm robot and the desired contact force of the object.

[0083] Optionally, the second joint torque of each of the two arms of the dual-arm robot can be obtained by multiplying the transpose of the force Jacobian matrix of each of the two arms of the dual-arm robot and the expected contact force of the object corresponding to each of the two arms of the dual-arm robot.

[0084] Step 404 : Based on the desired joint torque, control the movement of each joint of the dual-arm robot to perform the first operation task.

[0085] Optionally, the first operation task can be broken down into multiple subtasks (simple operation tasks do not require subdivision). Then, based on the expected end trajectory corresponding to each subtask, the motion primitives of each subtask are estimated to achieve motion control of the dual-arm robot. The specific content can be as follows:

[0086] The first operation task is decomposed and processed based on the environmental information and the operation diagram of the dual-arm robot to obtain a subtask sequence corresponding to the first operation task; wherein the operation diagram is used to guide the dual-arm robot to plan its own motion; the expected end motion trajectory corresponding to the target subtask in the subtask sequence is obtained; the starting position and the ending position of the end effector in the expected end trajectory corresponding to the target subtask are obtained; based on the starting position and the ending position, and the expected trajectory characteristics corresponding to the target subtask, the estimated motion primitives of the target subtask are determined, the expected trajectory characteristics are used to adjust the motion trajectory of the end effector, and the estimated motion primitives are used to estimate the motion process of the end effector corresponding to the target subtask; based on the estimated motion primitives of the target subtask, the movement of the dual-arm robot is controlled to perform the first operation task.

[0087] An operation diagram refers to an operational flow chart that divides an operation task into a series of subtasks with specific rules (such as execution order and degree of correlation) based on human operational knowledge. The first operation task is decomposed according to the operation diagram, and the decomposition process is dynamically adjusted based on environmental information to obtain the final subtask sequence. For example, taking the task of removing an item from a cabinet can be divided into subtasks such as approaching the cabinet, opening the cabinet, taking out the item, and closing the cabinet. If there are obstacles during the approach, a subtask for avoiding the obstacle is required. Optionally, the process of the end effector acting on the target object also needs to be further divided. For example, the motion process of the end effector's two-finger gripper 103 grasping the target object can include subtasks such as retracting the two-finger gripper 103, grasping the target object, and moving the target object. Based on the subtask sequence, the desired end trajectory can be refined into a corresponding number of desired end trajectory segments. The target subtask can be any subtask in the subtask sequence. Motion primitives can refer to the basic motion units into which motion can be divided, such as linear motion, curved motion (curved motion can also be divided into multiple linear motions), etc.

[0088] Optionally, the motion trajectory of the end effector in the task space can be estimated based on the expected end motion trajectory using a dynamic system method, thereby obtaining the corresponding estimated motion primitives. For example, the expected end motion trajectory corresponding to the target subtask is used as a learning example (i.e., the starting position and the ending position are obtained), and the expected end motion trajectory corresponding to the target subtask is fitted using the planning equation corresponding to the dynamic system method. The fitted planning equation can be used to control the end effector of the dual-arm robot to move according to a motion trajectory that is infinitely close to the expected end motion trajectory (which can be divided into corresponding estimated motion primitives), and the ending position of the motion trajectory is consistent with the ending position corresponding to the expected end motion trajectory. At the same time, a normalized weighted mixed Gaussian function used to describe the motion characteristics of humanoid arms or hands (i.e., the expected trajectory characteristics) can be superimposed in the planning equation corresponding to the dynamic system method. This normalized weighted mixed Gaussian function can achieve an approximate description of any motion characteristics. Furthermore, the motion characteristics of humanoid arms and hands can be integrated into the estimated motion primitives to obtain the estimated motion primitives with humanoid motion characteristics corresponding to the target subtask. The motion of the dual-arm robot can then be controlled according to the estimated motion primitives, thereby realizing the bionic motion of the dual-arm robot and making the movement of the robot's arms or the end effectors of the arms more natural and smooth.

[0089] To sum up, the technical solution provided in the embodiment of the present application plans the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level by planning the motion trajectory of the object and the expected contact force of the object corresponding to the first operation task, so as to realize the planning of the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level, so that the dual-arm robot can perform operation tasks that have both kinematic constraints and dynamic constraints requirements for the dual-arm robot, expand the coverage of the operation tasks that can be performed by the dual-arm robot, and improve the versatility of the dual-arm robot.

[0090] In addition, the types of operation tasks are further divided according to the different constraints in the operation tasks (such as kinematic constraints, dynamic constraints, etc.). Based on the different emphasis on constraints in different types of operation tasks, the motion of the dual-arm robot is planned, so that the dual-arm robot can perform more diverse operation tasks, further improving the versatility of the dual-arm robot.

[0091] Furthermore, a dynamic systems approach is used to predict the motion primitives of the manipulation task. This predicted motion primitives are then used to control the dual-arm robot's motion, resulting in more natural and smooth movements of both arms. This approach also provides an interface for incorporating humanoid motion characteristics into the robot's motion, achieving biomimetic movement of the dual-arm robot.

[0092] refer to Figure 5, which shows a flow chart of a control method for a dual-arm robot provided by another embodiment of the present application. The execution entity of each step of the method can be the dual-arm robot 100 described above. The method can include the following steps (501-505):

[0093] Step 501: Obtain a first operation task to be performed by a dual-arm robot.

[0094] In the embodiment of the present application, the first operation task refers to a task of completing the operation by using the dual arms of the dual-arm robot.

[0095] Step 502: Determine the type of the first operation task.

[0096] Optionally, the type of the first operation task may include a non-cooperative task, a symmetrical tightly coordinated task, a symmetrical loosely coordinated task, an asymmetrical tightly coordinated task, and an asymmetrical loosely coordinated task, etc., which is not limited in the present embodiment. Based on the classification criteria of the operation type in the above embodiment, the type of the first operation task is determined.

[0097] Step 503: If the type of the first operation task is a loosely coordinated task, obtain the expected joint torque that satisfies the kinematic constraints, and control the movement of each joint of the dual-arm robot to perform the first operation task based on the expected joint torque that satisfies the kinematic constraints.

[0098] The loose collaborative task refers to a collaborative task with kinematic constraints between the dual arms of the dual-arm robot and the target object in the first manipulation task. The loose collaborative task can include symmetrical loose collaborative tasks and symmetrical tight collaborative tasks.

[0099] In one example, the process of obtaining the expected joint torque corresponding to the loose collaborative task can be as follows: based on the first operation task and the environmental information corresponding to the first operation task, obtain the planned motion trajectory of the object corresponding to the first operation task; obtain the first structure matrix corresponding to the type of the first operation task; convert the planned motion trajectory of the object according to the first structure matrix to obtain the expected end motion trajectory that meets the kinematic constraints; determine the first joint torque based on the expected end motion trajectory, and use the first joint torque as the expected joint torque.

[0100] Among them, the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task; the first structure matrix is used to represent the mapping relationship between the planned motion trajectory of the object and the expected end motion trajectory; the expected end motion trajectory refers to the motion trajectory of the end effector under kinematic constraints; the first joint torque refers to the joint torque that satisfies the inverse kinematics of the dual-arm robot.

[0101] Optionally, the specific determination process of the first joint torque can be as follows: obtain the single-arm Jacobian matrix corresponding to each arm of the dual-arm robot; perform centralized fusion processing on the single-arm Jacobian matrix corresponding to each arm to obtain a second structure matrix, and the second structure matrix is used to represent the mapping relationship between the joints and end effectors of each arm of the dual-arm robot; based on the second structure matrix, the expected end motion trajectory is converted to obtain the expected joint motion data of the dual-arm robot; based on the expected joint motion data, the first joint torque is obtained.

[0102] Optionally, the specific determination process of the first structural matrix can be as follows: when the type of the first operation task is a symmetric loose collaboration task or a symmetric tight collaboration task, the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix, and the absolute Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in the absolute motion space; when the type of the first operation task is an asymmetric loose collaboration task or an asymmetric tight collaboration task, the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix, and the relative Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in the relative motion space.

[0103] Step 504: If the type of the first operation task is a tight coordination task, obtain the expected joint torque that satisfies the kinematic constraints and dynamic constraints, and control the movement of each joint of the dual-arm robot to perform the first operation task based on the expected joint torque that satisfies the kinematic constraints and dynamic constraints.

[0104] Among them, a tight collaborative task refers to a collaborative task with kinematic constraints and dynamic constraints between the dual arms of the dual-arm robot and the target operation object in the first operation task; a collaborative task refers to an operation task that requires the coordinated movement of the dual arms of the dual-arm robot.

[0105] In one example, the process of obtaining the expected joint torque corresponding to the tight collaboration task can be as follows: based on the first operation task and the environmental information corresponding to the first operation task, obtain the planned motion trajectory of the object and the expected contact force of the object corresponding to the first operation task; obtain the first structure matrix corresponding to the type of the first operation task; convert the planned motion trajectory of the object according to the first structure matrix to obtain the expected end motion trajectory that satisfies the kinematic constraints; determine the first joint torque based on the expected end motion trajectory; determine the second joint torque based on the expected contact force of the object; superimpose and fuse the first joint torque and the second joint torque to obtain the expected joint torque that satisfies the kinematic constraints and dynamic constraints.

[0106] Among them, the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task; the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object; the first structure matrix is used to represent the mapping relationship between the planned motion trajectory of the object and the expected end motion trajectory; the expected end motion trajectory refers to the motion trajectory of the end effector under kinematic constraints; the first joint torque refers to the joint torque that satisfies the inverse kinematics of the dual-arm robot; the second joint torque satisfies the joint torque of the inverse dynamics of the dual-arm robot.

[0107] Optionally, the desired contact force of the object can be converted according to the force Jacobian matrix of the dual-arm robot to obtain a second joint torque; wherein the force Jacobian matrix is used to represent the mapping relationship between the joint torques of the two arms of the dual-arm robot and the desired contact force of the object.

[0108] In one example, when the type of the first operation task is a non-cooperative task, the motion planning process of the dual-arm robot can be as follows: if the type of the first operation task is a non-cooperative task, then the target sub-task corresponding to the single arm of the dual-arm robot is obtained, and the target sub-task refers to a task that requires the single arm of the dual-arm robot to complete independently; determine the type of the target sub-task; if the type of the target sub-task is a loose non-cooperative task, then the expected joint torque of the single arm that meets the kinematic constraints is obtained, and based on the expected joint torque of the single arm that meets the kinematic constraints, the single arm movement of the dual-arm robot is controlled to perform the target sub-task; In the present invention, a loose non-collaborative task refers to a non-collaborative task with kinematic constraints between the single arm of the dual-arm robot and the target operation object in the target sub-task; or, if the type of the target sub-task is a tight non-collaborative task, the expected joint torque of the single arm that satisfies the kinematic constraints and the dynamic constraints is obtained, and based on the expected joint torque of the single arm that satisfies the kinematic constraints and the dynamic constraints, the single arm movement of the dual-arm robot is controlled to perform the target sub-task; wherein, a tight non-collaborative task refers to a non-collaborative task with kinematic constraints and dynamic constraints between the single arm of the dual-arm robot and the target operation object in the target sub-task.

[0109] Exemplarily, when the target sub-task is a loose non-collaborative task, based on the target sub-task and the environmental information corresponding to the target sub-task, the expected end motion trajectory corresponding to the target sub-task is obtained, the first joint torque is determined based on the expected end motion trajectory, and the single-arm motion of the dual-arm robot is controlled based on the first joint torque to perform the target sub-task.

[0110] In the case where the target subtask is a tight non-cooperative task, based on the target subtask and the environmental information corresponding to the target subtask, the expected terminal motion trajectory and the expected driving force of the object corresponding to the target subtask are obtained, and the first joint torque is determined based on the expected terminal motion trajectory; the second joint torque is determined based on the expected contact force of the object; and the single-arm motion of the dual-arm robot is controlled based on the first joint torque and the second joint torque to perform the target subtask.

[0111] In summary, the technical solutions provided by the embodiments of the present application enable the dual-arm robot to perform the task in question by determining the constraints that must be satisfied by the dual-arm robot's motion based on the type of task. Furthermore, the robot's joint torques are planned based on these constraints, enabling the robot to perform the task in question. This allows the robot to perform a variety of tasks, improving its versatility.

[0112] refer to Figure 6 , which shows a flow chart of a method for adjusting the desired joint torque provided by an embodiment of the present application. The execution subject of each step of the method can be the dual-arm robot 100 described above. The method can include the following steps (601-604):

[0113] Step 601: Determine the expected joint configuration of the dual-arm robot in null space based on the redundant degrees of freedom of the dual-arm robot.

[0114] Wherein, redundant degrees of freedom refer to the extra degrees of freedom possessed by the dual-arm robot when performing an operation task. For example, in the case where the dual-arm robot is a redundant dual-arm robot, since the degrees of freedom of the redundant dual-arm robot are greater than the spatial degrees of freedom, the redundant dual-arm robot always has a null space. Wherein, null space refers to the space formed by all the rotatable configurations of the joints of the dual-arm robot when the end effector of the dual-arm robot remains stationary, that is, the expected joint configuration can refer to the joint configuration belonging to the null space of the dual-arm robot required for the secondary task (or obstacle avoidance task) without affecting the movement of the end effector of the dual-arm robot, or it can refer to the joint configuration belonging to the null space of the dual-arm robot under the local optimization objective. Optionally, the expected joint configuration can be the optimal joint configuration among the multiple joint configurations corresponding to the null space of the dual-arm robot.

[0115] Step 602 : transforming the expected joint configuration according to the null space structure matrix of the dual-arm robot to obtain the expected null space joint motion data of the dual-arm robot.

[0116] The null-space structure matrix represents the mapping between the desired joint configuration and the desired null-space joint motion data. Multiplying the transpose of the null-space structure matrix by the desired joint configuration yields the desired null-space joint motion data. The desired null-space joint motion data refers to the desired joint motion data without affecting the end effector's motion. The null-space structure matrix is the Jacobian matrix of the dual-arm robot in the null space.

[0117] Step 603 : adjusting the expected joint torque based on the expected null-space joint torque corresponding to the expected null-space joint motion data to obtain an adjusted expected joint torque.

[0118] Exemplarily, the expected zero-space joint torque and the expected joint torque are superimposed to obtain an adjusted expected joint torque.

[0119] Step 604 : Based on the adjusted desired joint torque, control the movement of each joint of the dual-arm robot to perform the first operation task.

[0120] To sum up, the technical solution provided in the embodiment of the present application adjusts the desired joint torque based on the zero space of the dual-arm robot to achieve local target optimization (or completion of the secondary task) without affecting the execution of the first operation task.

[0121] refer to Figure 7 , which shows a flow chart of a position tracking method for a dual-arm robot provided by an embodiment of the present application. The execution subject of each step of the method can be the dual-arm robot 100 described above. The method can include the following steps (701-705):

[0122] Step 701: Obtain the real position of the object, the real position of the end, and the real position of the joint at the target moment.

[0123] Among them, the true position of the object, the true position of the end, and the true position of the joint refer to the position of the target operation object, the position of the end effector, and the position of each joint of the dual-arm robot at the target moment, respectively.

[0124] Step 702 : Determine first following error information based on the expected object position corresponding to the expected joint torque at the target moment and the actual position of the object. The first following error information is used to adjust the position of the target operation object.

[0125] Step 703 : Determine second following error information based on the expected end position corresponding to the expected joint torque at the target moment and the actual end position. The second following error information is used to adjust the position of the end effector.

[0126] Step 704 : Determine third following error information based on the expected joint position corresponding to the expected joint torque at the target moment and the actual joint position. The third following error information is used to adjust the position of the joint of the dual-arm robot.

[0127] Step 705 : Based on the first following error information, the second following error information, the third following error information, and the expected joint torque, the dual-arm robot is controlled to move so as to perform a first operation task.

[0128] Exemplarily, the difference between the actual position of the object at the target moment and the desired position (which may also include broader data calculations) is used as the first tracking error information. The deviation in the target object's motion is then corrected based on this first tracking error information. This deviation is then mapped to the joint layer, and the desired joint torque at the next target moment is adjusted to return the target object's position at the next target moment to the desired object position at the next target moment. Optionally, real-time tracking of the end effector and joint positions employs the same method as described above and will not be further elaborated here.

[0129] Optionally, during the dual-arm motion of the dual-arm robot, the desired gripping internal force and the desired object driving force can be tracked in real time to ensure operational safety of the dual-arm robot and controllable interaction between the end effector and the target object. The specific contents can be as follows:

[0130] The desired object contact forces of the two arms of the dual-arm robot are synchronously controlled, that is, the relative error between the desired object contact forces of the two arms of the dual-arm robot is controlled within a set range, so that the internal clamping force generated by the two arms of the dual-arm robot on the target operation object is not too large, and the driving force generated by the two arms of the dual-arm robot on the target operation object is ensured to be within a safe range, thereby avoiding damage to the target operation object. For example, the contact force between the end effectors of the two arms of the dual-arm robot and the target operation object is obtained in real time through force sensors. The contact force can be decomposed into the driving force on the target operation object and the inward clamping force on the target operation object (the clamping force is composed of the corresponding component forces of the two arms), ensuring that the driving forces of the two arms on the target operation object are synchronized and the clamping forces of the two arms on the target operation object are within a safe range.

[0131] In one example, the true value of the joint current at the target moment is obtained, and a closed-loop control system is formed based on the true value of the joint current and the expected joint current corresponding to the expected joint torque. That is, the true value of the joint current at the next moment is adjusted in real time by the true value of the joint current at the target moment and the expected joint current, as well as the expected joint current at the next moment, to ensure that the output joint torque of the dual-arm robot can track the command torque generated by the joint controller, thereby achieving precise control of the output joint torque.

[0132] In summary, the technical solution provided in the embodiment of the present application ensures the accuracy of the dual-arm movement of the dual-arm robot by tracking the positions of the target operating object, end effector and joints in real time during the dual-arm movement of the dual-arm robot.

[0133] In addition, by tracking the desired internal clamping force, the desired object driving force and the joint current, the motion safety constraints of the dual-arm robot are formed, which improves the precise control of the force of the dual-arm robot while ensuring the safety of the dual-arm robot and the target operated object.

[0134] refer to Figure 8 , which shows a flow chart of a method for transitioning an operation task provided by an embodiment of the present application. The execution subject of each step of the method can be the dual-arm robot 100 described above. The method can include the following steps (801-802):

[0135] Step 801 : When it is necessary to switch from a first operation task to a second operation task, an intermediate variable is calculated based on the difference between the actual terminal position corresponding to the first operation task and the expected terminal position of the first operation task.

[0136] The intermediate variable is used to represent the completion progress of the first operation task. Optionally, the intermediate variable can be determined as follows: if the difference is greater than or equal to a threshold, the intermediate variable is determined to be 0; if the difference is greater than 0 and less than the threshold, the intermediate variable is determined to be n, where n is a positive number between 0 and 1; if the difference is equal to 0, the intermediate variable is determined to be 1; wherein the set value is 1. The threshold can be adjusted by the designer based on needs. For example, the threshold can be adjusted based on the planning time and planning progress required by the task planner.

[0137] Step 802: If the intermediate variable is equal to the first set value, the dual-arm robot is controlled to perform the first operation task; if the intermediate variable is greater than the first set value and less than the second set value, the dual-arm robot is controlled to transition from performing the first operation task to performing the second operation task; if the intermediate variable is equal to the second set value, the dual-arm robot is controlled to perform the second operation task.

[0138] The first setting value is smaller than the second setting value. The first setting value and the second setting value can be defined and adjusted by the designer, and are not limited in this embodiment of the present application.

[0139] Exemplarily, the transition of the operation task can be achieved by the error-driven coupling method, and the specific process can be as follows: the intermediate variable is calculated by setting a multi-order continuous function with a value range of [0, 1]. The multi-order continuous function can be a Barrier Lyapunov function, a Sigmoid function, etc. By adjusting the function parameters or form, the output of the multi-order continuous function is a value range of 0 to 1 (including 0 and 1). The input of the multi-order continuous function is the difference between the actual position of the end effector at the target moment and the expected position corresponding to the expected end motion trajectory. If the above difference is 0, the input of the multi-order continuous function is 1. If the above difference is greater than the threshold, the output of the multi-order continuous function is 0. If the above difference is greater than 0 and less than the threshold, the output of the multi-order continuous function is a positive number between 0 and 1.

[0140] The transition process between operation tasks can be expressed by the following formula:

[0141] V=[1-f(x)]*v l +f(x)*v2;

[0142] Where x represents the difference between the actual position of the end effector at the target moment and the expected position corresponding to the expected end motion trajectory, f is a multi-order continuous function, v1 represents the planner for the first target task, and v2 represents the planner for the second target task. For example, if the first set value is 0 and the second set value is 1, and the difference is greater than the threshold, the output of f is 0 (i.e., equal to the first set value), V = v1, indicating that the dual-arm robot is performing the first target task. If the difference is greater than 0 but less than the set threshold, the output of f begins to change continuously and smoothly toward 1, indicating that the dual-arm robot is smoothly transitioning from performing the first target task to performing the second target task. If the difference is 0, the output of f is 1 (i.e., equal to the second set value), V = v2, indicating that the dual-arm robot has begun performing the second operation task.

[0143] Optionally, the transition between subtasks adopts the same method as the transition between operation tasks, which will not be described here.

[0144] In summary, the technical solution provided in the embodiment of the present application realizes the transition between operation tasks (or subtasks) through the error-driven coupling method, making the dual-arm robot switching tasks more continuous and natural.

[0145] refer to Figure 9 , which shows a schematic diagram of the architecture of a control system for a dual-arm robot provided by one embodiment of the present application. The dual-arm robot control system 900 includes a task layer 901, a perception layer 902, a priori knowledge layer 903, a planning layer 904, and a control layer 905.

[0146] 1. The task layer 901 is used to obtain the first operation task and determine the type of the first operation task. Then, based on the type of the first operation task, it determines the first structural matrix (e.g., the absolute Jacobian matrix, the weighted sum of the absolute Jacobian matrix and the relative Jacobian matrix), as well as the constraints emphasized by the first operation task (e.g., kinematic constraints, kinetic constraints, and dynamic constraints, etc.).

[0147] 2. The perception layer 902 is used to obtain environmental information in the task space corresponding to the first operation task. The perception layer 302 includes visual perception, torque perception, and tactile perception. The environmental information includes the position of the target operation object, the joint position of the dual-arm robot, the end effector position, the end driving force, the joint torque, etc.

[0148] 3. Prior knowledge layer 903 is used to provide object database, operation map and affordance information. Affordance information can be used for intent recognition.

[0149] 4. The planning layer 904 is used to plan the motion of the dual-arm robot based on the first structural matrix and the constraints of the first operational task, obtaining the desired end-user motion trajectory and the desired joint torque / current. The specific planning process is the same as that described in the above embodiment and will not be repeated here.

[0150] 5. The control layer 905 is used to control the motion of each joint of the dual-arm robot based on the desired joint torque / current to perform the first manipulation task. The specific control process is the same as that described in the above embodiment and will not be repeated here. The control layer 905 also performs task refinement, motion primitive estimation, manipulation task transition, desired force tracking, position tracking, etc.

[0151] To sum up, the technical solution provided in the embodiment of the present application plans the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level by planning the motion trajectory of the object and the expected contact force of the object corresponding to the first operation task, so as to realize the planning of the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level, so that the dual-arm robot can perform operation tasks that have both kinematic constraints and dynamic constraints requirements for the dual-arm robot, expand the coverage of the operation tasks that can be performed by the dual-arm robot, and improve the versatility of the dual-arm robot.

[0152] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0153] Please refer to Figure 10, which shows a block diagram of a control device for a dual-arm robot provided by an embodiment of the present application. The device has the function of implementing the control method of the dual-arm robot described above, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be the dual-arm robot 100 described above, or it can be set in the dual-arm robot 100. The device 1000 may include: an operation task acquisition module 1001, a planning information acquisition module 1002, a desired torque acquisition module 1003, and an operation task execution module 1004.

[0154] The operation task acquisition module 1001 is used to acquire a first operation task to be performed by the dual-arm robot and environmental information corresponding to the first operation task.

[0155] The planning information acquisition module 1002 is used to obtain the planned motion trajectory of the object and the expected contact force of the object corresponding to the first operation task based on the first operation task and the environmental information; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object.

[0156] The expected torque acquisition module 1003 is used to determine the expected joint torque that meets the kinematic constraints and the dynamic constraints based on the planned motion trajectory of the object and the expected contact force of the object.

[0157] The operation task execution module 1004 is used to control the movement of each joint of the dual-arm robot to perform the first operation task based on the expected joint torque.

[0158] In an exemplary embodiment, Figure 11 As shown, the desired torque acquisition module 1003 includes: a first matrix acquisition submodule 1003a, a desired trajectory acquisition submodule 1003b, a first torque determination submodule 1003c, a second torque determination submodule 1003d and a desired torque acquisition submodule 1003e.

[0159] The first matrix acquisition submodule 1003a is used to obtain a first structure matrix corresponding to the type of the first operation task. The first structure matrix is used to represent the mapping relationship between the planned motion trajectory of the object and the expected end motion trajectory. The expected end motion trajectory refers to the motion trajectory of the end effector under the kinematic constraints.

[0160] The expected trajectory acquisition submodule 1003b is configured to convert the planned motion trajectory of the object according to the first structure matrix to obtain the expected terminal motion trajectory that satisfies the kinematic constraints.

[0161] The first torque determination submodule 1003c is configured to determine a first joint torque based on the desired terminal motion trajectory, where the first joint torque refers to a joint torque that satisfies the inverse kinematics of the dual-arm robot.

[0162] The second torque determination submodule 1003d is configured to determine a second joint torque based on the expected contact force of the object, where the second joint torque refers to a joint torque that satisfies the inverse dynamics of the dual-arm robot.

[0163] The expected torque acquisition submodule 1003e is used to perform superposition and fusion processing on the first joint torque and the second joint torque to obtain the expected joint torque that meets the kinematic constraints and dynamic constraints of the dual-arm robot.

[0164] In an exemplary embodiment, the first torque determination submodule 1003c is configured to:

[0165] Obtaining the single-arm Jacobian matrix corresponding to each of the two arms of the dual-arm robot;

[0166] Performing a centralized fusion process on the single-arm Jacobian matrices corresponding to the two arms to obtain a second structure matrix, where the second structure matrix is used to represent a mapping relationship between the joints of the two arms and the end effector of the two arms of the dual-arm robot;

[0167] performing conversion processing on the desired end motion trajectory based on the second structure matrix to obtain desired joint motion data of the dual-arm robot;

[0168] The first joint torque is obtained based on the expected joint motion data.

[0169] In an exemplary embodiment, the second torque determination submodule 1003d is configured to:

[0170] The desired contact force of the object is converted according to the force Jacobian matrix of the dual-arm robot to obtain the second joint torque; wherein the force Jacobian matrix is used to represent the mapping relationship between the joint torques of the two arms of the dual-arm robot and the desired contact force of the object.

[0171] In an exemplary embodiment, the first matrix acquisition submodule 1003a is configured to:

[0172] When the first operation task is a symmetric collaborative task, an absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix, where the absolute Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in an absolute motion space; wherein the symmetric collaborative task refers to an operation task in which, during the collaborative motion of the dual arms of the dual-arm robot, the motion trajectories of the end effectors of the dual arms have a fixed relative positional relationship;

[0173] In the case where the type of the first operation task is an asymmetric collaborative task, the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix, and the relative Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in the relative motion space; wherein, the asymmetric collaborative task refers to an operation task in which the motion trajectories of the end effectors of the dual arms have a variable relative position relationship during the collaborative motion of the dual arms of the dual-arm robot.

[0174] In an exemplary embodiment, the symmetrical collaborative task includes a symmetrical loose collaborative task and a symmetrical tight collaborative task; wherein the symmetrical loose collaborative task refers to a symmetrical collaborative task in which the two arms of the dual-arm robot perform collaborative motion under the kinematic constraints; the symmetrical tight collaborative task refers to a symmetrical collaborative task in which the two arms of the dual-arm robot perform collaborative motion under the kinematic constraints and the dynamic constraints;

[0175] The asymmetric collaborative tasks include asymmetric loose collaborative tasks and asymmetric tight collaborative tasks; wherein, the asymmetric loose collaborative tasks refer to asymmetric collaborative tasks in which the two arms of the dual-arm robot perform collaborative movements under the kinematic constraints; the asymmetric tight collaborative tasks refer to asymmetric collaborative tasks in which the two arms of the dual-arm robot perform collaborative movements under the kinematic constraints and the dynamic constraints.

[0176] In an exemplary embodiment, the operation task execution module 1004 is configured to:

[0177] When the type of the first operation task is the symmetrical tight coordination task or the asymmetrical tight coordination task, controlling the dual-arm robot to move based on the first joint torque and the second joint torque to perform the first operation task;

[0178] In a case where the type of the first operation task is the symmetrical loose collaborative task or the asymmetrical loose collaborative task, the dual-arm robot is controlled to move based on the first joint torque to perform the first operation task.

[0179] In an exemplary embodiment, the operation task execution module 1004 is further configured to:

[0180] Determining an expected joint configuration of the dual-arm robot in null space based on the redundant degrees of freedom of the dual-arm robot;

[0181] The desired joint configuration is converted according to a null space structure matrix of the dual-arm robot to obtain desired null space joint motion data of the dual-arm robot, wherein the null space structure matrix is used to represent a mapping relationship between the desired joint configuration and the desired null space joint motion data, wherein the desired null space joint motion data refers to desired joint motion data without affecting the motion of the end effector;

[0182] adjusting the expected joint torque based on the expected null-space joint torque corresponding to the expected null-space joint motion data to obtain an adjusted expected joint torque;

[0183] Based on the adjusted expected joint torque, the movement of each joint of the dual-arm robot is controlled to perform the first operation task.

[0184] In an exemplary embodiment, the operation task execution module 1004 is further configured to:

[0185] Decomposing the first operation task based on the environmental information and an operation diagram of the dual-arm robot to obtain a subtask sequence corresponding to the first operation task; wherein the operation diagram is used to guide the dual-arm robot to perform motion planning for itself;

[0186] Obtaining the expected terminal motion trajectory corresponding to the target subtask in the subtask sequence;

[0187] Obtaining the starting position and ending position of the end effector in the desired end trajectory corresponding to the target subtask;

[0188] Determining an estimated motion primitive for the target subtask based on the starting position and the ending position, and expected trajectory features corresponding to the target subtask, wherein the expected trajectory features are used to adjust the motion trajectory of the end effector, and the estimated motion primitive is used to estimate the motion process of the end effector corresponding to the target subtask;

[0189] Based on the estimated motion primitives of the target subtask, the dual-arm robot is controlled to move to perform the first operation task.

[0190] In an exemplary embodiment, the operation task execution module 1004 is further configured to:

[0191] Obtaining the object's true position, the end effector's true position, and the joint's true position at the target moment, where the object's true position, the end effector's true position, and the joint's true position refer to the position of the target object, the end effector's position, and the joints of the dual-arm robot at the target moment, respectively;

[0192] determining first following error information based on an expected object position corresponding to the expected joint torque at the target moment and the actual position of the object, wherein the first following error information is used to adjust the position of the target operation object;

[0193] determining second following error information based on an expected end position corresponding to the expected joint torque at the target moment and the actual end position, wherein the second following error information is used to adjust the position of the end effector;

[0194] determining third following error information based on an expected joint position corresponding to the expected joint torque at the target moment and the actual joint position, wherein the third following error information is used to adjust the position of the joint of the dual-arm robot;

[0195] Based on the first following error information, the second following error information, the third following error information, and the expected joint torque, the dual-arm robot is controlled to move so as to perform the first operation task.

[0196] In an exemplary embodiment, Figure 11 As shown, the device 1000 further includes: an intermediate variable acquisition module 1005.

[0197] The intermediate variable acquisition module 1005 is used to calculate the intermediate variable based on the difference between the actual end position corresponding to the first operation task and the expected end position of the first operation task when it is necessary to switch from the first operation task to the second operation task; wherein, the intermediate variable is used to represent the completion progress of the first operation task.

[0198] The operation task execution module 1004 is also used to control the dual-arm robot to perform the first operation task if the intermediate variable is equal to a first set value; if the intermediate variable is greater than the first set value and less than a second set value, control the dual-arm robot to transition from performing the first operation task to performing the second operation task; if the intermediate variable is equal to the second set value, control the dual-arm robot to perform the second operation task.

[0199] In an exemplary embodiment, the intermediate variable acquisition module 1005 is configured to:

[0200] If the difference is greater than or equal to a threshold, determining that the intermediate variable is 0;

[0201] If the difference is greater than 0 and less than the threshold, the intermediate variable is determined to be n, where n is a positive number between 0 and 1;

[0202] If the difference is equal to 0, the intermediate variable is determined to be 1;

[0203] Wherein, the set value is 1.

[0204] To sum up, the technical solution provided in the embodiment of the present application plans the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level by planning the motion trajectory of the object and the expected contact force of the object corresponding to the first operation task, so as to realize the planning of the joint torque of the dual-arm robot at the kinematic constraint level and the dynamic constraint level, so that the dual-arm robot can perform operation tasks that have both kinematic constraints and dynamic constraints requirements for the dual-arm robot, expand the coverage of the operation tasks that can be performed by the dual-arm robot, and improve the versatility of the dual-arm robot.

[0205] Please refer to Figure 12 , which shows a block diagram of a control device for a dual-arm robot provided by another embodiment of the present application. The device has the function of implementing the control method of the dual-arm robot described above, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be the dual-arm robot 100 described above, or it can be set in the dual-arm robot 100. The device 1200 may include: an operation task acquisition module 1201, a task type determination module 1202, and an operation task execution module 1203.

[0206] The operation task acquisition module 1201 is used to acquire a first operation task to be performed by the dual-arm robot.

[0207] The task type determination module 1202 is configured to determine the type of the first operation task.

[0208] The operation task execution module 1203 is used to obtain the expected joint torque that satisfies the kinematic constraints when the type of the first operation task is a loose collaborative task, and to control the movement of each joint of the dual-arm robot to perform the first operation task based on the expected joint torque that satisfies the kinematic constraints; wherein the loose collaborative task refers to a collaborative task with the kinematic constraints between the two arms of the dual-arm robot and the target operation object in the first operation task.

[0209] The operation task execution module 1203 is also used to obtain the expected joint torque that satisfies the kinematic constraints and the dynamic constraints when the type of the first operation task is a tight collaboration task, and to control the movement of each joint of the dual-arm robot to perform the first operation task based on the expected joint torque that satisfies the kinematic constraints and the dynamic constraints; wherein, the tight collaboration task refers to a collaborative task with the kinematic constraints and the dynamic constraints between the two arms of the dual-arm robot and the target operation object in the first operation task; wherein, the collaborative task refers to an operation task that requires the collaborative movement of the two arms of the dual-arm robot.

[0210] In an exemplary embodiment, the operation task execution module 1203 is configured to:

[0211] Based on the first operation task and the environment information corresponding to the first operation task, obtaining a planned motion trajectory of an object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task;

[0212] Obtaining a first structure matrix corresponding to the type of the first operation task, where the first structure matrix is used to represent a mapping relationship between the planned motion trajectory of the object and an expected end motion trajectory, where the expected end motion trajectory refers to the motion trajectory of the end effector under the kinematic constraints;

[0213] Converting the planned motion trajectory of the object according to the first structure matrix to obtain the desired end motion trajectory that satisfies the kinematic constraints;

[0214] A first joint torque is determined based on the expected terminal motion trajectory, and the first joint torque is used as the expected joint torque. The first joint torque refers to a joint torque that satisfies the inverse kinematics of the dual-arm robot.

[0215] In an exemplary embodiment, the operation task execution module 1203 is further configured to:

[0216] Based on the first operation task and the environment information corresponding to the first operation task, obtaining a planned motion trajectory of an object and an expected contact force of the object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object;

[0217] Obtaining a first structure matrix corresponding to the type of the first operation task, where the first structure matrix is used to represent a mapping relationship between the planned motion trajectory of the object and an expected end motion trajectory, where the expected end motion trajectory refers to the motion trajectory of the end effector under the kinematic constraints;

[0218] Converting the planned motion trajectory of the object according to the first structure matrix to obtain the desired end motion trajectory that satisfies the kinematic constraints;

[0219] Determining a first joint torque based on the expected terminal motion trajectory, where the first joint torque refers to a joint torque that satisfies inverse kinematics of the dual-arm robot;

[0220] determining a second joint torque based on the expected contact force of the object, where the second joint torque refers to a joint torque that satisfies inverse dynamics of the dual-arm robot;

[0221] The first joint torque and the second joint torque are superimposed and fused to obtain a desired joint torque that satisfies the kinematic constraint and the dynamic constraint.

[0222] In an exemplary embodiment, the operation task execution module 1203 is also used to: convert the expected contact force of the object according to the force Jacobian matrix of the dual-arm robot to obtain the second joint torque; wherein the force Jacobian matrix is used to represent the mapping relationship between the joint torques of the two arms of the dual-arm robot and the expected contact force of the object.

[0223] In an exemplary embodiment, the operation task execution module 1203 is further configured to:

[0224] Obtaining the single-arm Jacobian matrix corresponding to each of the two arms of the dual-arm robot;

[0225] Performing a centralized fusion process on the single-arm Jacobian matrices corresponding to the two arms to obtain a second structure matrix, where the second structure matrix is used to represent a mapping relationship between the joints of the two arms and the end effector of the two arms of the dual-arm robot;

[0226] performing conversion processing on the desired end motion trajectory based on the second structure matrix to obtain desired joint motion data of the dual-arm robot;

[0227] The first joint torque is obtained based on the expected joint motion data.

[0228] In an exemplary embodiment, the operation task execution module 1203 is further configured to:

[0229] When the type of the first operation task is a symmetric loosely coordinated task or a symmetric tightly coordinated task, an absolute Jacobian matrix of the dual-arm robot is determined as the first structure matrix, where the absolute Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in an absolute motion space;

[0230] When the type of the first operation task is an asymmetric loose collaboration or an asymmetric tight collaboration task, the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix, and the relative Jacobian matrix refers to the Jacobian matrix of the dual arms of the dual-arm robot in the relative motion space.

[0231] In an exemplary embodiment, the operation task execution module 1203 is further configured to:

[0232] If the type of the first operation task is a non-cooperative task, obtaining a target subtask corresponding to a single arm of the dual-arm robot, where the target subtask refers to a task that needs to be completed independently by a single arm of the dual-arm robot;

[0233] Determining the type of the target sub-task;

[0234] If the type of the target subtask is a loose non-cooperative task, obtaining a desired joint torque of a single arm that satisfies the kinematic constraints, and controlling the motion of the single arm of the dual-arm robot based on the desired joint torque of the single arm that satisfies the kinematic constraints to perform the target subtask; wherein the loose non-cooperative task refers to a non-cooperative task with the kinematic constraints between the single arm of the dual-arm robot and the target operation object in the target subtask;

[0235] Alternatively, if the type of the target sub-task is a tight non-collaborative task, the expected joint torque of the single arm that satisfies the kinematic constraint and the dynamic constraint is obtained, and based on the expected joint torque of the single arm that satisfies the kinematic constraint and the dynamic constraint, the single-arm movement of the dual-arm robot is controlled to perform the target sub-task; wherein, the tight non-collaborative task refers to a non-collaborative task with the kinematic constraint and the dynamic constraint between the single arm of the dual-arm robot and the target operating object in the target sub-task.

[0236] In summary, the technical solutions provided by the embodiments of the present application enable the dual-arm robot to perform the task in question by determining the constraints that must be satisfied by the dual-arm robot's motion based on the type of task. Furthermore, the robot's joint torques are planned based on these constraints, enabling the robot to perform the task in question. This allows the robot to perform a variety of tasks, improving its versatility.

[0237] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0238] Please refer to Figure 13 , which shows a simplified structural block diagram of a dual-arm robot provided in one embodiment of the present application. The dual-arm robot may be the dual-arm robot 100 described above, etc., and the present embodiment of the application does not limit this.

[0239] Alternatively, as Figure 13 As shown, the robot includes a processor 1301 and a memory 1302. The processor 1301 includes, but is not limited to, any of the following: a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The memory 1302 may include storage devices such as RAM (Random-Access Memory) and ROM (Read-Only Memory). The processor 1301 and the memory 1302 may be connected via a system bus.

[0240] In an exemplary embodiment, the memory 1302 stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor 1301 to implement the above-mentioned control method of the dual-arm robot.

[0241] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. When the at least one instruction, the at least one program, the code set or the instruction set is executed by a processor of a computer device, the control method of the dual-arm robot is implemented.

[0242] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0243] In an exemplary embodiment, a computer program product or computer program is further provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a dual-arm robot reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the dual-arm robot to perform the above-described dual-arm robot control method.

[0244] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.

[0245] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A control method for a dual-arm robot, characterized in that: The method comprises: Acquire a first operation task to be performed by the dual-arm robot and environmental information corresponding to the first operation task; Based on the first operation task and the environmental information, obtaining a planned motion trajectory of an object and an expected contact force of the object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object; In a case where the type of the first operation task is a symmetrical collaborative task, determining the absolute Jacobian matrix of the dual-arm robot as a first structural matrix; In a case where the type of the first operation task is an asymmetric collaborative task, a weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot is determined as the first structural matrix; Determining a desired joint torque that satisfies kinematic constraints and dynamic constraints based on the first structural matrix, the planned motion trajectory of the object, and the desired contact force of the object; Based on the expected joint torque, the movement of each joint of the dual-arm robot is controlled to perform the first operation task.

2. The method according to claim 1, characterized in that The determining of the desired joint torque that satisfies kinematic constraints and dynamic constraints based on the first structural matrix, the planned motion trajectory of the object, and the desired contact force of the object includes: converting the planned motion trajectory of the object according to the first structure matrix to obtain a desired end motion trajectory that satisfies the kinematic constraints; Determining a first joint torque based on the expected terminal motion trajectory, where the first joint torque refers to a joint torque that satisfies inverse kinematics of the dual-arm robot; determining a second joint torque based on the expected contact force of the object, where the second joint torque refers to a joint torque that satisfies inverse dynamics of the dual-arm robot; The first joint torque and the second joint torque are superimposed and fused to obtain a desired joint torque that satisfies the kinematic constraint and the dynamic constraint.

3. The method according to claim 2, characterized in that The determining of the first joint torque based on the expected end motion trajectory includes: Obtaining the single-arm Jacobian matrix corresponding to each of the two arms of the dual-arm robot; Using the Jacobian matrices of the single arms corresponding to the two arms as the diagonal elements of the diagonal matrix, and setting the remaining elements of the diagonal matrix to 0, to obtain a second structure matrix, wherein the second structure matrix is used to represent the mapping relationship between the joints of the two arms and the end effector; performing conversion processing on the desired end motion trajectory based on the second structure matrix to obtain desired joint motion data of the dual-arm robot; The first joint torque is obtained based on the expected joint motion data.

4. The method according to claim 2, characterized in that The determining of the second joint torque based on the expected contact force of the object comprises: converting the expected contact force of the object according to the force Jacobian matrix of the dual-arm robot to obtain the second joint torque; The force Jacobian matrix is used to represent the mapping relationship between the joint torques of the two arms of the dual-arm robot and the expected contact force of the object.

5. The method according to claim 1, wherein The symmetrical collaborative task includes a symmetrical loose collaborative task and a symmetrical tight collaborative task; wherein the symmetrical loose collaborative task refers to a symmetrical collaborative task in which the two arms of the dual-arm robot perform collaborative motion under the kinematic constraints; the symmetrical tight collaborative task refers to a symmetrical collaborative task in which the two arms of the dual-arm robot perform collaborative motion under the kinematic constraints and the dynamic constraints; The asymmetric collaborative tasks include asymmetric loose collaborative tasks and asymmetric tight collaborative tasks; wherein, the asymmetric loose collaborative tasks refer to asymmetric collaborative tasks in which the two arms of the dual-arm robot perform collaborative movements under the kinematic constraints; the asymmetric tight collaborative tasks refer to asymmetric collaborative tasks in which the two arms of the dual-arm robot perform collaborative movements under the kinematic constraints and the dynamic constraints.

6. The method according to claim 5, characterized in that The method further comprises: When the type of the first operation task is the symmetrical tight coordination task or the asymmetrical tight coordination task, controlling the dual-arm robot to move to perform the first operation task based on the first joint torque and the second joint torque corresponding to the dual-arm robot; In a case where the type of the first operation task is the symmetrical loose collaborative task or the asymmetrical loose collaborative task, the dual-arm robot is controlled to move based on the first joint torque to perform the first operation task.

7. The method according to claim 1, characterized in that The method further comprises: Determining an expected joint configuration of the dual-arm robot in a null space based on redundant degrees of freedom of the dual-arm robot, wherein the expected joint configuration is implemented as at least one of the following: a joint configuration belonging to the null space required for a secondary task without affecting motion of the end effector, a joint configuration belonging to the null space under a local optimization objective, or an optimal joint configuration among multiple joint configurations corresponding to the null space; The desired joint configuration is converted according to a null space structure matrix of the dual-arm robot to obtain desired null space joint motion data of the dual-arm robot, wherein the null space structure matrix is used to represent a mapping relationship between the desired joint configuration and the desired null space joint motion data, wherein the desired null space joint motion data refers to desired joint motion data without affecting the motion of the end effector; adjusting the expected joint torque based on the expected null-space joint torque corresponding to the expected null-space joint motion data to obtain an adjusted expected joint torque; Based on the adjusted expected joint torque, the movement of each joint of the dual-arm robot is controlled to perform the first operation task.

8. The method according to claim 1, characterized in that The method further comprises: Decomposing the first operation task based on the environmental information and an operation diagram of the dual-arm robot to obtain a subtask sequence corresponding to the first operation task; wherein the operation diagram is used to guide the dual-arm robot to perform motion planning for itself; Obtaining the expected terminal motion trajectory corresponding to the target subtask in the subtask sequence; Obtaining the starting position and ending position of the end effector in the desired end trajectory corresponding to the target subtask; Determining an estimated motion primitive for the target subtask based on the starting position and the ending position, and expected trajectory features corresponding to the target subtask, wherein the expected trajectory features are used to adjust the motion trajectory of the end effector, and the estimated motion primitive is used to estimate the motion process of the end effector corresponding to the target subtask; Based on the estimated motion primitives of the target subtask, the dual-arm robot is controlled to move to perform the first operation task.

9. The method according to claim 1, characterized in that The method further comprises: Obtaining the object's true position, the end effector's true position, and the joint's true position at the target moment, where the object's true position, the end effector's true position, and the joint's true position refer to the position of the target object, the end effector's position, and the joints of the dual-arm robot at the target moment, respectively; determining first following error information based on an expected object position corresponding to the expected joint torque at the target moment and the actual position of the object, wherein the first following error information is used to adjust the position of the target operation object; determining second following error information based on an expected end position corresponding to the expected joint torque at the target moment and the actual end position, wherein the second following error information is used to adjust the position of the end effector; determining third following error information based on an expected joint position corresponding to the expected joint torque at the target moment and the actual joint position, wherein the third following error information is used to adjust the position of the joint of the dual-arm robot; Based on the first following error information, the second following error information, the third following error information, and the expected joint torque, the dual-arm robot is controlled to move so as to perform the first operation task.

10. The method according to claim 1, characterized in that The method further comprises: When it is necessary to switch from the first operation task to the second operation task, an intermediate variable is calculated based on the difference between the actual end position corresponding to the first operation task and the expected end position of the first operation task; wherein the intermediate variable is used to represent the completion progress of the first operation task; If the intermediate variable is equal to a first set value, controlling the dual-arm robot to perform the first operation task; If the intermediate variable is greater than the first set value and less than the second set value, controlling the dual-arm robot to transition from executing the first operation task to executing the second operation task; If the intermediate variable is equal to the second set value, the dual-arm robot is controlled to perform the second operation task.

11. The method according to claim 10, characterized in that The calculating of the intermediate variable based on the difference between the actual position of the end point corresponding to the first operation task and the expected end position of the first operation task includes: If the difference is greater than or equal to a threshold, determining that the intermediate variable is 0; If the difference is greater than 0 and less than the threshold, the intermediate variable is determined to be n, where n is a positive number between 0 and 1; If the difference is equal to 0, the intermediate variable is determined to be 1; The threshold is 1.

12. A control method for a dual-arm robot, characterized in that: The method comprises: Obtaining a first operation task to be performed by the dual-arm robot; determining a type of the first operation task; If the type of the first operation task is a loose collaborative task, the expected joint torque that satisfies the kinematic constraints is obtained according to the first structure matrix, and based on the expected joint torque that satisfies the kinematic constraints, the movement of each joint of the dual-arm robot is controlled to perform the first operation task; wherein, the loose collaborative task refers to a collaborative task with the kinematic constraints between the two arms of the dual-arm robot and the target operation object in the first operation task, and the loose collaborative task includes a symmetric loose collaborative task and an asymmetric loose collaborative task. When the type of the first operation task is the symmetric loose collaborative task, the first structure matrix is the absolute Jacobian matrix of the dual-arm robot. When the type of the first operation task is the asymmetric loose collaborative task, the first structure matrix is the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot; Alternatively, if the type of the first operation task is a tight collaboration task, the expected joint torque that satisfies the kinematic constraints and the dynamic constraints is obtained according to the first structure matrix, and based on the expected joint torque that satisfies the kinematic constraints and the dynamic constraints, the movement of each joint of the dual-arm robot is controlled to perform the first operation task; wherein, the tight collaboration task refers to a collaborative task with the kinematic constraints and the dynamic constraints between the dual arms of the dual-arm robot and the target operation object in the first operation task, and the tight collaboration task includes a symmetric tight collaboration task and an asymmetric tight collaboration task. When the type of the first operation task is the symmetric tight collaboration task, the first structure matrix is the absolute Jacobian matrix of the dual-arm robot. When the type of the first operation task is the asymmetric tight collaboration task, the first structure matrix is the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot; The collaborative task refers to an operation task that requires the coordinated movement of both arms of the dual-arm robot.

13. The method according to claim 12, characterized in that The step of obtaining the desired joint torque satisfying the kinematic constraints according to the first structure matrix includes: Based on the first operation task and the environment information corresponding to the first operation task, obtaining a planned motion trajectory of an object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of a target operation object in the first operation task, and the first structure matrix is used to represent a mapping relationship between the planned motion trajectory of the object and an expected end motion trajectory, wherein the expected end motion trajectory refers to the motion trajectory of an end effector under the kinematic constraints; Converting the planned motion trajectory of the object according to the first structure matrix to obtain the desired end motion trajectory that satisfies the kinematic constraints; A first joint torque is determined based on the expected terminal motion trajectory, and the first joint torque is used as the expected joint torque. The first joint torque refers to a joint torque that satisfies the inverse kinematics of the dual-arm robot.

14. The method according to claim 12, characterized in that The step of obtaining the desired joint torque satisfying the kinematic constraints and the dynamic constraints according to the first structure matrix includes: Based on the first operation task and the environmental information corresponding to the first operation task, a planned motion trajectory of an object and an expected contact force of the object corresponding to the first operation task are obtained; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object. The first structure matrix is used to represent the mapping relationship between the planned motion trajectory of the object and the expected end motion trajectory, and the expected end motion trajectory refers to the motion trajectory of the end effector under the kinematic constraints. Converting the planned motion trajectory of the object according to the first structure matrix to obtain the desired end motion trajectory that satisfies the kinematic constraints; Determining a first joint torque based on the expected terminal motion trajectory, where the first joint torque refers to a joint torque that satisfies inverse kinematics of the dual-arm robot; determining a second joint torque based on the expected contact force of the object, where the second joint torque refers to a joint torque that satisfies inverse dynamics of the dual-arm robot; The first joint torque and the second joint torque are superimposed and fused to obtain a desired joint torque that satisfies the kinematic constraint and the dynamic constraint.

15. The method according to claim 14, characterized in that The determining of the second joint torque based on the expected contact force of the object comprises: converting the expected contact force of the object according to the force Jacobian matrix of the dual-arm robot to obtain the second joint torque; The force Jacobian matrix is used to represent the mapping relationship between the joint torques of the two arms of the dual-arm robot and the expected contact force of the object.

16. The method according to claim 13 or 14, characterized in that The determining of the first joint torque based on the expected end motion trajectory includes: Obtaining the single-arm Jacobian matrix corresponding to each of the two arms of the dual-arm robot; Using the single-arm Jacobian matrices corresponding to the two arms as the diagonal elements of a diagonal matrix, and setting the remaining elements of the diagonal matrix to 0, to obtain a second structure matrix, wherein the second structure matrix is used to represent the mapping relationship between the joints of the two arms of the dual-arm robot and the end effector; performing conversion processing on the desired end motion trajectory based on the second structure matrix to obtain desired joint motion data of the dual-arm robot; The first joint torque is obtained based on the expected joint motion data.

17. The method according to claim 12, wherein: After determining the type of the first operation task, the method further includes: If the type of the first operation task is a non-cooperative task, obtaining a target subtask corresponding to a single arm of the dual-arm robot, where the target subtask refers to a task that needs to be completed independently by a single arm of the dual-arm robot; Determining the type of the target sub-task; If the type of the target subtask is a loose non-cooperative task, obtaining a desired joint torque of a single arm that satisfies the kinematic constraints, and controlling the motion of the single arm of the dual-arm robot based on the desired joint torque of the single arm that satisfies the kinematic constraints to perform the target subtask; wherein the loose non-cooperative task refers to a non-cooperative task with the kinematic constraints between the single arm of the dual-arm robot and the target operation object in the target subtask; Alternatively, if the type of the target sub-task is a tight non-collaborative task, the expected joint torque of the single arm that satisfies the kinematic constraint and the dynamic constraint is obtained, and based on the expected joint torque of the single arm that satisfies the kinematic constraint and the dynamic constraint, the single-arm movement of the dual-arm robot is controlled to perform the target sub-task; wherein, the tight non-collaborative task refers to a non-collaborative task with the kinematic constraint and the dynamic constraint between the single arm of the dual-arm robot and the target operating object in the target sub-task.

18. A control device for a dual-arm robot, characterized in that: The device comprises: An operation task acquisition module, configured to acquire a first operation task to be performed by the dual-arm robot and environmental information corresponding to the first operation task; a planning information acquisition module, configured to acquire, based on the first operation task and the environmental information, a planned motion trajectory of an object and an expected contact force of the object corresponding to the first operation task; wherein the planned motion trajectory of the object refers to the planned motion trajectory of the target operation object in the first operation task, and the expected contact force of the object refers to the expected contact force between the end effector of the dual-arm robot and the target operation object; an expected torque acquisition module, configured to, when the first operation task is a symmetric collaborative task, determine the absolute Jacobian matrix of the dual-arm robot as a first structural matrix; and, when the first operation task is an asymmetric collaborative task, determine the weighted sum of the relative Jacobian matrix of the dual-arm robot and the absolute Jacobian matrix as the first structural matrix; and determine the expected joint torque that satisfies kinematic constraints and dynamic constraints based on the first structural matrix, the planned motion trajectory of the object, and the expected contact force of the object; An operation task execution module is used to control the movement of each joint of the dual-arm robot to execute the first operation task based on the expected joint torque.

19. A control device for a dual-arm robot, characterized in that: The device comprises: An operation task acquisition module, configured to acquire a first operation task to be performed by the dual-arm robot; A task type determination module, configured to determine the type of the first operation task; an operation task execution module, configured to obtain, if the type of the first operation task is a loose collaborative task, an expected joint torque satisfying the kinematic constraints according to a first structural matrix, and control the movement of each joint of the dual-arm robot based on the expected joint torque satisfying the kinematic constraints to perform the first operation task; wherein, the loose collaborative task refers to a collaborative task with the kinematic constraints between the dual arms of the dual-arm robot and the target operation object in the first operation task, and the loose collaborative task includes a symmetric loose collaborative task and an asymmetric loose collaborative task. When the type of the first operation task is the symmetric loose collaborative task, the first structural matrix is the absolute Jacobian matrix of the dual-arm robot. When the type of the first operation task is the asymmetric loose collaborative task, the first structural matrix is the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot; Alternatively, the operation task execution module is further used to obtain the expected joint torque that satisfies the kinematic constraints and the dynamic constraints according to the first structural matrix if the type of the first operation task is a tight collaboration task, and control the movement of each joint of the dual-arm robot to perform the first operation task based on the expected joint torque that satisfies the kinematic constraints and the dynamic constraints based on the expected joint torque that satisfies the kinematic constraints and the dynamic constraints; wherein the tight collaboration task refers to a collaborative task with the kinematic constraints and the dynamic constraints between the dual arms of the dual-arm robot and the target operation object in the first operation task, and the tight collaboration task includes a symmetric tight collaboration task and an asymmetric tight collaboration task. When the type of the first operation task is the symmetric tight collaboration task, the first structural matrix is the absolute Jacobian matrix of the dual-arm robot; when the type of the first operation task is the asymmetric tight collaboration task, the first structural matrix is the weighted sum of the relative Jacobian matrix and the absolute Jacobian matrix of the dual-arm robot; The collaborative task refers to an operation task that requires the coordinated movement of both arms of the dual-arm robot.

20. A dual-arm robot, characterized in that: The dual-arm robot includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the control method of the dual-arm robot as described in any one of claims 1 to 11, or to implement the control method of the dual-arm robot as described in any one of claims 12 to 17.

21. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction, which is loaded and executed by the processor to implement the control method of the dual-arm robot as described in any one of claims 1 to 11, or to implement the control method of the dual-arm robot as described in any one of claims 12 to 17.

22. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. The processor reads and executes the computer instructions from the computer-readable storage medium to implement the control method of the dual-arm robot as described in any one of claims 1 to 11, or implements the control method of the dual-arm robot as described in any one of claims 12 to 17.

Images