Robot, operation method and device thereof, storage medium and program product

Abstract

The embodiment of the invention provides a robot, an operation method and device of the robot, a storage medium and a program product. The method comprises the steps that sensing force data of a mechanical arm is received; the external force and the internal force are distributed based on the sensing force data of each mechanical arm in the at least two mechanical arms, so that the corresponding sensing external force and sensing internal force of each mechanical arm are determined; based on the sensing external force and the sensing internal force corresponding to each mechanical arm, the expected external force and the expected internal force of the target object and the expected position of the target object, calculating the force and the position so as to determine the expected position of each mechanical arm; and according to the expected position of each mechanical arm, inverse kinematics solution based on whole-body control is carried out on the robot, so that the joint positions of each mechanical arm and each joint in the waist are determined, and each mechanical arm is controlled to operate the target object based on the joint positions. According to the embodiment of the invention, a multi-arm force control framework based on whole-body control is constructed, the force control effect is good, and the operation stability of the robot can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of robots, and in particular to a robot and an operating method, device, storage medium and program product thereof. BACKGROUND

[0002] With the continuous expansion of the application scenarios of robots, robots gradually develop from single-arm operation to dual-arm or even more mechanical arm collaborative operation to meet the task requirements of complex assembly, collaborative handling and fine processing. However, for robots with two or more mechanical arms, the force control effect of the related single-arm force control framework is not good. In addition, with the increase of the complexity of the structure of the robot, the operation stability of the robot based on the related single-arm force control framework is also insufficient.

[0003] Based on this, embodiments of the present application provide a robot and an operating method, device, storage medium and program product thereof to improve related technologies. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a robot and an operating method, device and storage medium thereof, which have good force control effect for robots with at least two mechanical arms and can improve the stability of robot operation.

[0005] The purpose of the embodiments of the present application is achieved by adopting the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a robot operating method, the robot comprising a plurality of mechanical arms and a waist, the method comprising: in the case that the robot operates a target object using a plurality of mechanical arms, receiving sensing force data of a corresponding mechanical arm; respectively distributing external force and internal force based on the sensing force data of each mechanical arm in at least two mechanical arms to determine the sensing external force and the sensing internal force of each mechanical arm applied to the target object; based on the corresponding sensing external force and sensing internal force of each mechanical arm, the expected external force and internal force of the target object, and the expected position of the target object, solving between force and position to determine the expected position of each mechanical arm; according to the expected position of each mechanical arm, performing inverse kinematics solving based on whole body control on the robot to determine the joint position of each joint in each mechanical arm and the waist, and controlling each mechanical arm to operate the target object based on the joint position.

[0007] In some embodiments, the distribution of the external force and the internal force based on the sensing force data of each of the at least two robotic arms respectively, to determine the sensing external force and the sensing internal force applied to the target object by each of the robotic arms, comprises: for each of the robotic arms, performing the following processing: based on a force conversion relationship between the force applied by the robotic arm and the force borne by the target object, distributing the sensing force data of the robotic arm to obtain the sensing external force and the sensing internal force applied to the target object by the robotic arm.

[0008] In some embodiments, the force conversion relationship is represented by a grasp matrix.

[0009] In some embodiments, the solving of the force and the position based on the corresponding sensing external force and the sensing internal force of each of the robotic arms, the desired external force and the desired internal force of the target object, and the desired position of the target object, to determine the desired position of each of the robotic arms, comprises: based on the corresponding sensing external force of each of the robotic arms, the desired external force of the target object, and the desired position of the target object, performing external force impedance control on the target object and the external environment to generate an object position of the target object; performing closed-chain constraint based on the object position to generate a robotic arm position of each of the robotic arms; based on the corresponding sensing internal force of each of the robotic arms, the desired internal force of the target object, and the robotic arm position of each of the robotic arms, performing internal force impedance control on each of the robotic arms and the target object respectively to generate the desired position of each of the robotic arms.

[0010] In some embodiments, the inverse kinematics solving of the robot based on the whole body control according to the desired position of each of the robotic arms, to determine the joint position of each of the joints in each of the robotic arms and the waist, comprises: based on the desired position of each of the robotic arms, performing minimum solving on a first target function to generate a first joint position increment of each of the joints; updating the corresponding joint position according to the first joint position increment of each of the joints; wherein the first target function is constructed in a quadratic programming manner, and a gradient term of the first target function is established based on a whole body Jacobian matrix, which is used to represent a mapping relationship between the joint velocity of each of the joints and the end effector velocity of each of the robotic arms.

[0011] In some embodiments, the at least part of the robot arm has redundant degrees of freedom; the method further comprises: based on the reference configuration information of the robot arm with redundant degrees of freedom, performing a minimization solution on the second objective function to generate a second joint position increment of each joint; and the updating of the joint position of each joint according to the first joint position increment comprises: updating the joint position of each joint according to the first joint position increment and the second joint position increment of each joint in the robot arm with redundant degrees of freedom; and the second objective function is constructed in a quadratic programming manner, and a gradient term of the second objective function is established based on a null space of the whole-body Jacobian matrix.

[0012] In some embodiments, the first objective function and / or the second objective function correspond to a quadratic programming with specified physical constraints, and the physical constraints include at least one of a joint torque constraint, a joint position constraint, a joint velocity constraint, and a joint acceleration constraint.

[0013] In a second aspect, the embodiments of the present application provide a robot operating device, the robot comprising a plurality of robot arms and a waist, the device comprising: a data receiving module configured to receive sensing force data of each robot arm when the robot operates a target object using the plurality of robot arms; a force distribution module configured to distribute external force and internal force based on the sensing force data of each robot arm in the at least two robot arms, respectively, to determine sensing external force and sensing internal force applied to the target object by each robot arm; a position solving module configured to solve between force and position based on the sensing external force and the sensing internal force of each robot arm, expected external force and expected internal force of the target object, and expected position of the target object, to determine expected position of each robot arm; and an inverse kinematics solving module configured to perform inverse kinematics solving based on whole-body control of the robot according to the expected position of each robot arm, to determine joint position of each joint in each robot arm and the waist, to control the robot arms to operate the target object based on the joint position.

[0014] In a third aspect, the embodiments of the present application provide a robot, the robot comprising a control module configured to perform any of the above methods.

[0015] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of any of the above methods.

[0016] In a fifth aspect, the embodiments of the present application provide a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the steps of any of the above methods.

[0017] The embodiment of the application provides a robot and an operating method, device, storage medium and program product thereof, the robot comprises a plurality of mechanical arms and a waist, and the method comprises the following steps: in the case that the robot operates a target object by using the plurality of mechanical arms, receiving sensing force data of corresponding mechanical arms; respectively distributing external force and internal force based on the sensing force data of each mechanical arm in the at least two mechanical arms, so as to determine sensing external force and sensing internal force applied to the target object by each mechanical arm; based on the corresponding sensing external force and sensing internal force of each mechanical arm, expected external force and expected internal force of the target object, and an expected position of the target object, performing calculation between force and position, so as to determine an expected position of each mechanical arm; and based on the expected position of each mechanical arm, performing inverse kinematics solving based on whole body control, so as to determine joint positions of each joint in each mechanical arm and the waist, and controlling the plurality of mechanical arms to operate the target object based on the joint positions.

[0018] The above embodiment can receive and distribute sensing force data, obtain sensing external force and sensing internal force, and perform calculation between force and position in combination with expected external force, expected internal force and an expected position, so as to generate an expected position of each mechanical arm, and then obtain joint positions by inverse kinematics solving based on whole body control, and control the mechanical arms to perform operation. Thus, a multi-arm force control framework based on whole body control is constructed, and force control of each mechanical arm can be realized based on load distribution and internal and external force control. For a robot with at least two mechanical arms, the force control effect is good, and the stability of robot operation task execution can be improved. The inverse kinematics solving method is universal, and ensures that the force control framework has good expansibility and can be applied to robots of any configuration and any degree of freedom. Secondly, the sensing force data is distributed into external force and internal force, the sensing internal force between multi-arm operations and the environmental sensing external force of the target object are completely decoupled, and the two do not affect each other. The target object and the external environment can interact with force sensing, the stability of the target object gripped by the plurality of mechanical arms can be ensured by internal force control between the plurality of mechanical arms, and the gripping stability and environmental interaction do not interfere with each other. In addition, the plurality of degrees of freedom are coordinated by whole body control, and the stability and efficiency of robot operation are improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] The embodiment of the application will be further described below in combination with the accompanying drawings and specific embodiments.

[0020] Figure 1 FIG. 1 is a flow diagram of a robot operation method provided by the embodiment of the application.

[0021] Figure 2 FIG. 4 is a schematic diagram of a load distribution model provided by the embodiment of the application.

[0022] Figure 3 FIG. 6 is a schematic diagram of a dual-arm force control framework provided by the embodiment of the application.

[0023] Figure 4 is a structural block diagram of a robot operation device provided by an embodiment of the present application.

[0024] Figure 5 is a structural block diagram of a robot provided by an embodiment of the present application.

[0025] Figure 6 is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present application.

[0027] In the description of the embodiments of the present application, it should be understood that the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly and specifically limited.

[0028] Most related robot force control frameworks are based on single-arm systems, but in the context of dual-arm or even multi-arm operation, it is difficult to meet the needs with only single-arm ideas. Specifically, the difficulty of dual-arm and multi-arm operation lies in both maintaining stable grasping of the object and reducing the risk of object deformation or falling due to improper clamping, and ensuring the force interaction between the object and the environment and reducing the force imbalance or rigid collision during operation. In addition, when the robot includes a waist or lower limb joint, the operation is essentially a whole-body coordinated action, and the related single-arm force control framework lacks the ability to uniformly schedule the degrees of freedom of the whole body.

[0029] Referring to Figure 1 , Figure 1 is a flowchart of a robot operation method provided by an embodiment of the present application.

[0030] In order to improve the related art, the embodiments of the present application provide a robot operation method, the robot includes a plurality of mechanical arms and a waist, and the method includes steps S101-S104.

[0031] Step S101: In the case where the robot uses a plurality of mechanical arms to operate a target object, receiving sensing force data of the corresponding mechanical arm.

[0032] Step S102: distributing the external force and the internal force based on the sensing force data of each of the at least two mechanical arms respectively, to determine the sensing external force and the sensing internal force of each mechanical arm applied to the target object.

[0033] Step S103: performing calculation between force and position based on the corresponding sensing external force and sensing internal force of each mechanical arm, the expected external force and the expected internal force of the target object, and the expected position of the target object, to determine the expected position of each mechanical arm.

[0034] Step S104: performing inverse kinematics solving of the whole body control of the robot according to the expected position of each mechanical arm, to determine the joint position of each joint in each mechanical arm and the waist, to control the operation of each mechanical arm on the target object based on the joint position.

[0035] In the above embodiments, the robot can be equipped with two or more mechanical arms, capable of achieving collaborative operation. Compared with single-arm robots, such robots have higher operation freedom and stronger collaboration ability when performing tasks, and can complete complex operations that single-arm robots cannot achieve. In terms of structure, the robot can include a plurality of mechanical arms and a waist, each mechanical arm can include one or more joints (the joints can adopt a serial configuration, a parallel configuration or a serial-parallel configuration), which can be rotary joints or movement joints, and are equipped with corresponding driving devices and control units. In addition to the arm and the waist, the robot can also include one or more of a head, a neck, a lower limb and a foot, which are not limited by the above embodiments. The end effector of each mechanical arm can be a gripper, a suction cup, a tool interface or a force-controlled gripper, etc., for direct contact with the target object and exerting operating force. In terms of function, the robot can achieve object grasping, carrying, assembly, docking and other operations through collaborative control of multiple mechanical arms. The robot can interact with the target object from multiple directions at the same time, improving the stability and flexibility of the operation; can collaboratively carry or adjust the posture of large, heavy or complex-shaped objects; can share the force exerted by multiple arms during operation, reducing the overload of a single arm. In terms of application scenarios, the robot can be widely used in industrial robots, service robots and other fields.

[0036] The above embodiments do not limit the number of mechanical arms, which can be two or more. Some embodiments herein take a two-arm robot with two mechanical arms as an example, however, the embodiments of the present application are not limited thereto, and in possible implementations, the description of the two-arm robot is applicable to robots with more mechanical arms.

[0037] It should be noted that the position in the embodiments of the present application refers to a general position. For the target object and the end effector of the robot arm, the relevant position can be pose information, such as position information and attitude information in a three-dimensional space, which can be represented as six-dimensional information (x, y, z, and rotation angle around three coordinate axes or in the form of a quaternion). For the robot joint, the relevant position can be a general joint position, such as a joint angle, a linear displacement of the joint, or other parameters capable of representing the state of the joint. Accordingly, for the robot joint, the relevant velocity can be a general velocity, such as a joint angular velocity, a joint linear velocity, etc.; and the relevant acceleration can be a general acceleration, such as a joint angular acceleration, a joint linear acceleration, etc.

[0038] It can be seen that the robot can operate the same target object through multiple robot arms in cooperation, and the waist part can serve as a floating base to provide an additional degree of freedom, so that the robot has the ability of whole body coordination. By introducing the waist degree of freedom, the robot can more flexibly allocate the joint motion in the operation task, and improve the overall operability and stability.

[0039] The above embodiments do not limit the acquisition method of the sensing force data. In some embodiments, the sensing force data can be, for example, force data from a force / torque sensor (for example, a six-dimensional force / torque sensor) or an estimation module. The force / torque sensor can be installed on the end effector of the robot arm, for example, to sense the force / torque applied by the end effector to the target object. The sensing force data is the input of the force control framework, which is used to determine whether the actual operation force meets the expected requirements.

[0040] The distribution of external force and internal force is a force distribution method. The external force reflects the interaction between the target object and the external environment, and the internal force reflects the internal force generated by the multi-arm clamping of the target object. By performing distribution calculation on the sensing force data, the “external force conducive to the movement of the target object” and the “internal force generated by clamping” can be obtained.

[0041] The joint position is used to control the operation of the target object by the robot arms, for example, the joint position can be converted into robot joint control instructions (or the joint position or joint velocity is used as the robot joint control instruction) and sent to the robot drive system to realize actual operation. The robot drive system can convert the robot joint control instruction into a motor control signal, execute in a closed loop, and complete the operation task.

[0042] The above embodiment can receive and distribute the perceived force data when the robot uses multiple mechanical arms to operate the target object, obtain the perceived external force and the perceived internal force, combine the expected external force, the expected internal force and the expected position to solve the force and the position, thereby generating the expected position of the mechanical arm, and then obtaining the joint position through inverse kinematics based on the whole body control to control the mechanical arm to perform the operation. Thus, a multi-arm force control framework based on the whole body control is constructed, which can realize force control of each mechanical arm based on load distribution and internal and external force control. This control framework can be expanded to a robot containing more parts. For a robot with at least two mechanical arms, the force control effect is good, and the stability of the robot operation task execution can be improved. The inverse kinematics solving method is universal, which ensures that the force control framework has good expansibility and can be applied to robots of any configuration and any degree of freedom. Secondly, the perceived force data is distributed to the external force and the internal force, which completely decouples the perceived internal force between the multi-arm (i.e., two or more mechanical arms) operation and the environmental perception of the target object, and the two do not affect each other. The target object and the external environment can interact with force perception, the stability of the multi-arm grasping the target object can be ensured through internal force control between the multiple mechanical arms, and the grasping stability and environmental interaction do not interfere with each other. In addition, the multiple degrees of freedom are coordinated through the whole body control, which improves the stability and efficiency of the robot operation.

[0043] The above embodiment does not limit the configuration of the robot, which can be, for example, a series configuration, a parallel configuration, a series-parallel configuration, etc.

[0044] In some embodiments, the robot can be a series configuration robot (which can be referred to as a series robot). When the method in the above embodiment is applied to a series robot, the solving method is universal for series robots of any configuration and any degree of freedom.

[0045] In the cooperative operation of multiple mechanical arms on the same object, in order to realize stable cooperation of multiple arms, the force data perceived by the end effector of the mechanical arm can be used as the total force, which is accurately distributed to the external force (related to the interaction between the object and the environment) and the internal force (related to the cooperation between the multiple arms), and the force and position are regulated based on this. To this end, the force can be orthogonally distributed through the grasp matrix and pseudo-inverse decomposition, and the internal force and the external force can be distinguished from a mathematical point of view. Alternatively, hierarchical regulation of the force can also be attempted based on a virtual flexible control model, or impedance / admittance hybrid control. Alternatively, a sensor fusion method can also be used to combine visual, tactile and force sensing information to infer the force state of the object, thereby improving the robustness of force distribution. In addition, a learning control method can also be introduced, and a deep learning model can be used to learn the distribution rule of internal and external forces in a large number of operation samples to provide auxiliary judgment for real-time force control.

[0046] In some embodiments, the distribution of the external force and the internal force based on the sensing force data of each of the at least two robot arms respectively, to determine the sensing external force and the sensing internal force applied to the target object by each of the robot arms, can comprise: for each of the robot arms, performing the following processing: based on a force conversion relationship between the force applied by the robot arm and the force borne by the target object, distributing the sensing force data of the robot arm to obtain the sensing external force and the sensing internal force applied to the target object by the robot arm.

[0047] The force conversion relationship, for example, refers to a mathematical mapping relationship between the force applied by the robot arm (for example, the force applied by the end effector of the robot arm) and the force borne by the target object. The above embodiments do not limit the force conversion relationship, and in some embodiments, the force conversion relationship can be represented by a grasp matrix. Each robot arm corresponds to a grasp matrix, and the overall grasp matrix can be constructed based on the grasp matrices of the plurality of robot arms. The overall grasp matrix can be used to convert the generalized force applied by the plurality of robot arms into the force and moment borne by the target object at the center of mass, and to provide a mathematical basis for the distribution of the external force and the internal force.

[0048] Referring to Figure 2 , Figure 2 is a schematic diagram of a load distribution model provided by an embodiment of the present application. The robot can include, for example, a body, a waist, and two robot arms (i.e., a left arm and a right arm), and the left arm, the right arm, and the waist are rotatable relative to the body. The purpose of load distribution is, for example, to distribute based on the sensing force data to obtain the sensing external force and the sensing internal force.

[0049] For example, assuming that the robot is a dual-arm robot, the relationship between the dual-arm force and the force borne by the center of mass of the target object is described by the Newton-Euler equation as follows.

[0050]

[0051] where f i , n i are the force and the moment of the end effector of the i th robot arm, r i is the position vector of the center of mass relative to the end effector, f m , n m are the resultant force and the resultant moment acting on the center of mass of the target object, and m represents the center of mass of the target object.

[0052] Next, a grasp matrix W is constructed to map the generalized force of the end effectors of the dual arms to the overall generalized force of the center of mass of the target object.

[0053] h m = Wh r

[0054]

[0055] where h m is the overall generalized force on the object's center of mass, h r is the generalized force on the dual-arm end-effector, I is the identity matrix, and S(r1), S(r2) are the skew-symmetric matrices of position vectors r1, r2, respectively.

[0056] By pseudo-inverse operation W + W, the following external force component (i.e., perceived external force) and internal force component (i.e., perceived internal force) can be obtained.

[0057] h Er = W + Wh r

[0058] h Ir = (I - W + W)h r

[0059] In practical applications, the generalized force applied by the dual-arm end-effector to the target object can be perceived by force / torque sensors on the end-effector. In order to let the distribution of internal and external forces not interfere with each other, when selecting the pseudo-inverse of the grasp matrix W, the following relationship can be satisfied.

[0060]

[0061] The above embodiments distribute the perceived force data into perceived external force and perceived internal force based on the force conversion relationship between the force applied by the robot's arm and the force on the target object. Through accurate distribution of external and internal forces, complete decoupling of internal and external forces in dual-arm or multi-arm operation is achieved, so that the stability of object grasping and the compliance of environmental interaction do not interfere with each other.

[0062] To achieve the calculation between force and position, the embodiment of the application can combine the force data perceived by the multiple robot arms with the expected external force, the expected internal force and the expected position to perform the calculation between force and position, thereby generating a reasonable expected position of the robot arm. For example, based on external force guidance control, the interaction position of the target object and the external environment can be adjusted by comparing the difference between the perceived external force and the expected external force, so that the target object maintains flexibility when subjected to external interference. Based on closed-chain constraint modeling, the motion relationship between the multiple robot arms and the same object is ensured to be coordinated, and imbalance in force and displacement transmission is avoided. On this basis, internal force guidance control is introduced, the end posture of each robot arm is dynamically adjusted by comparing the perceived internal force and the expected internal force, the appropriate force distribution is ensured, and the target object is prevented from being crushed or falling. Alternatively, the embodiment of the application can improve the accuracy of external force estimation by sensor fusion combining vision, touch and force sense. Alternatively, the embodiment of the application can use historical operation data to train a neural network to predict a suitable internal and external force distribution strategy by using a learning type prediction model, thereby improving the adaptability and robustness of control.

[0063] In some embodiments, the calculation between force and position based on the respective perceived external force and the perceived internal force of each robot arm, the expected external force and the expected internal force of the target object, and the expected position of the target object can include: performing external force guidance control on the target object and the external environment based on the respective perceived external force of each robot arm, the expected external force of the target object and the expected position of the target object to generate an object position of the target object; performing closed-chain constraint based on the object position to generate a robot arm position of each robot arm; and performing internal force guidance control on each robot arm and the target object based on the respective perceived internal force of each robot arm, the expected internal force of the target object and the robot arm position of each robot arm, respectively, to generate an expected position of each robot arm.

[0064] Referring to Figure 3 , Figure 3 is a schematic diagram of a dual-arm force control framework provided by the embodiment of the application. The end effector of the left arm can collect perceived force data (for example, force information and / or torque information) through a torque sensor, perform force / torque distribution, and obtain the respective perceived external force h Er1 and the perceived internal force h Ir1 of the left arm. The end effector of the right arm can collect perceived force data through a torque sensor, perform force / torque distribution, and obtain the respective perceived external force h Er2 and the perceived internal force h Ir2 of the right arm. Based on the respective perceived external force h Er1 of the left arm, the respective perceived external force h Er2 of the right arm and the expected external force h EdThe object position modification ΔT is obtained by performing the external force admittance control. The desired object position T d and the object position modification ΔT to determine the object position T r (i.e., the reference position of the target object). The object position T r is subjected to the closed-chain constraint to obtain the left-arm position T r1 (i.e., the reference position of the left-arm end effector) and the right-arm position T r2 (i.e., the reference position of the right-arm end effector). The corresponding sensed internal force h Ir1 and the desired internal force h Id1 The left-arm position modification ΔT1 is obtained by performing the internal force admittance control. The desired left-arm position is determined based on the left-arm position T r1 and the left-arm position modification ΔT1 and provided to the dual-arm system. The corresponding sensed internal force h Ir1 and the desired internal force h Id2 The right-arm position modification ΔT2 is obtained by performing the internal force admittance control. The desired right-arm position is determined based on the right-arm position T r2 and the right-arm position modification ΔT2 and provided to the dual-arm system. As an example, the dual-arm system can perform inverse kinematics based on the desired positions of the left and right arms to determine joint positions of joints in the left and right arms and the waist to control the left and right arms to operate the target object based on the joint positions, to complete a control cycle. In the next control cycle, the torque sensor is used to collect sensed force data again, and the steps of force / torque distribution, external force admittance control, closed-chain constraint, internal force admittance control, etc. are performed, and so on, which will not be repeated here.

[0065] In the above framework, the desired position of the target object is subjected to closed-chain constraint distribution to obtain the robot arm position, so that the dual-arm system satisfies the required pose transformation relationship, and then the internal and external force admittance control processes are superimposed, the position is adjusted according to the sensed force data, and finally the whole body control is performed according to the configuration of the dual-arm system.

[0066] For example, the admittance control can be expressed as follows.

[0067] e = F ext ― F d

[0068]

[0069] where F ext , F drespectively, e is the difference between the perceived force and the desired force (e.g., the reference force set by the task planner), M is the mass matrix, B is the damping matrix, K is the stiffness matrix, X is the displacement of the end-effector (the desired position adjustment of the admittance control output), K p is the proportional gain matrix (also referred to as the proportional stiffness parameter), K i is the integral gain matrix. In practical applications, K is, for example, 0 to eliminate the coupling of the end position and the stiffness and simplify to a pure mass-damping model. M is, for example, 1, i.e., a unit mass model, for simplifying the control calculation and improving the real-time performance.

[0070] As shown in Figure 3 , the dual-arm force control framework includes an outer force admittance control loop, which mainly affects the position tracking of the target object being manipulated and the process of environmental force interaction. Two inner force admittance control loops (one for each robotic arm) mainly affect the inner force tracking of the dual-arm end-effector on the target object, ensuring that the target object is not squeezed and deformed or at risk of falling off.

[0071] For the outer force admittance control loop, the expression of e can be modified as: e = h Er1 +h Er2 ―h Ed . Wherein, h Er1 , h Er2 are the respective perceived external forces of the left arm and the right arm, and h Ed is the desired external force of the target object.

[0072] For the inner force admittance control loop, the expression of e can be modified as: e = h Iri ―h Idi . Wherein, h Iri is the respective perceived internal force of the i-th robotic arm; h Idi is the respective desired internal force of the i-th robotic arm, which can be determined based on the desired internal force of the target object.

[0073] In the above embodiments, first, based on the perceived external force, the desired external force and the desired position of each robot arm, the external force steering control is performed to generate the object position of the target object. Then, based on the object position, the closed-chain constraint is applied, and the corresponding robot arm position of each robot arm is calculated. Then, combined with the perceived internal force, the desired internal force and the robot arm position of each robot arm, the internal force steering control is performed to obtain the desired position of each robot arm. Through this hierarchical processing mode, independent control of the external force and the internal force is realized, and it is ensured that the object can be stably grasped and compliantly interacted. Through the closed-chain constraint, it is ensured that multiple robot arms remain coordinated at the kinematic level, reducing force and pose conflicts. In addition, combined with the external force steering control and the internal force steering control, the robot can adaptively adjust the grasping and operation strategy in a complex task scenario, thereby significantly improving the stability, safety and universality of the operation, and having good scalability.

[0074] When a dual-arm or multi-arm robot needs to perform coordinated operation in the full-body range, the related inverse kinematics solving method is often low in calculation efficiency, resulting in unstable solving results or difficulty in meeting complex operation requirements. In particular, in a full-body robot including waist or lower limb joints, there is a high coupling between the desired position of the robot arm and the full-body joint movement. If only the related single-arm inverse kinematics method is used, problems such as movement unattainability, solution divergence or unnatural action are likely to occur, which seriously affects the safety and robustness of the operation. Therefore, the embodiments of the present application establish an inverse kinematics optimization method based on full-body control, which can consider other control targets (such as joint constraints and redundant degrees of freedom, etc.) while ensuring that the end effector completes the task, thereby realizing efficient and stable joint calculation. In specific implementation, for example, a pseudo-inverse Jacobian method can be used for fast calculation, or a weighted least squares optimization can be tried to improve the priority of part of the joints by increasing the constraint weight, or a hierarchical optimization method can be used to first ensure the pose accuracy of the main task, and then optimize the secondary index on the redundant degrees of freedom, such as energy consumption, obstacle avoidance or joint approach to the reference configuration. In addition, learning-based inverse kinematics solving can be performed to realize faster response in a specific task scenario by using a neural network to approximate the inverse Jacobian mapping.

[0075] In some embodiments, the inverse kinematics solving of the robot based on full-body control according to the desired position of each robot arm to determine the joint position of each joint in each robot arm and the waist can include: performing minimization solving on a first target function based on the desired position of each robot arm to generate a first joint position increment of each joint; and updating the corresponding joint position according to the first joint position increment of each joint. The first target function is constructed in a quadratic programming manner, and the gradient term of the first target function is established based on a full-body Jacobian matrix, which is used to represent the mapping relationship between the joint velocity of each joint in the full body and the end effector velocity of each robot arm.

[0076] Inverse kinematics solving refers to, for example, a process of solving each joint position from a desired position of a robotic arm (may also be referred to as a target pose of an end effector of the robotic arm). Unlike solving each arm independently, the above embodiments adopt whole-body control, that is, modeling and solving the whole body degrees of freedom including the arms and the waist as a whole. By establishing a whole-body Jacobian matrix, end-effector velocity constraints are converted into joint velocity constraints, and then an optimization method is used to solve the coordinated motion of the whole-body joints.

[0077] A first objective function is constructed to quantify the pros and cons of joint velocities, and by minimizing the first objective function, a first joint position increment is obtained.

[0078] Quadratic Programming (QP) is a class of optimization methods with a quadratic objective function and linear constraint conditions, which has the advantages of fast calculation speed and stable solution, and is suitable for real-time control. In practical applications, joint position limits, velocity constraints, torque limits and other conditions can be converted into linear constraint conditions and input into a QP solver. When the first objective function is quadratic and the constraints are linear, QP can efficiently solve the joint position increment under multiple constraints.

[0079] The whole-body Jacobian matrix is used to represent the mapping relationship between the joint velocities of the whole body and the end effector velocities of each robotic arm. Here, the joints of the whole body can include the joints of each robotic arm and the joints of the waist. In the case where the robot includes other parts, the whole-body joints can also include the joints of the other parts. In the above embodiments, the whole-body Jacobian matrix not only includes the mapping relationship between the joint velocities of each robotic arm and the end effector velocities, but also includes the mapping relationship between the joint velocities of the waist and the end effector velocities. The gradient term of the first objective function can be constructed based on the whole-body Jacobian matrix, so as to project the end task demand into the joint space and ensure that the arms and the waist coordinate to complete the target action.

[0080] The first objective function is minimized to output a solution of the first joint position increment. The solution ensures the optimality of the end effector of the robotic arm tracking the target pose.

[0081] For example, the first objective function can be expressed as follows.

[0082] min: dx T g + 0.5dx T Hdx

[0083] s.t. lb―x < dx < ub―x

[0084] Wherein, the gradient term g and the Hessian matrix H (i.e., the Hessian matrix) of the first objective function are as follows.

[0085] g = J T f(x)

[0086] H = J T J

[0087] In the above formula, x is the joint position, which can be represented by a vector, for example. dx is the first joint position increment, which is used to represent the amount of change in the joint position that needs to be adjusted in one iteration. lb and ub are the lower limit value and the upper limit value of the joint position, respectively, which are used to limit the value range of each joint. f(x) is a task function or a residual function, which is used to represent the deviation between the current pose of the end effector and the target pose (i.e., the desired position of the robot arm). g is the gradient term of the first objective function, which is used to characterize the sensitivity of the joint position update to the task error. H is the Hessian matrix, which is used to reflect the quadratic approximation characteristics of the function in the joint space. J is the full-body Jacobian matrix, which is used to represent the mapping relationship between the joint velocity of the robot and the end effector velocity. T and J T are the transpose matrices of x and J, respectively.

[0088] In the above embodiment, the inverse kinematics solving based on full-body control is performed using the convex optimization iterative method, which has stable solving speed and can superimpose linear constraint conditions such as joint position limits and operational degrees. A QP solver is used to solve the minimization of the first objective function to obtain dx. Then, dx is used to update x, for example, x = x + αdx. Wherein, α is a step factor, which is used to control the amplitude of the joint position update, for example, it can satisfy α ≤ 1.

[0089] In the above embodiment, based on the desired position of each robot arm, a first objective function represented in the form of quadratic programming is first constructed, and the first objective function is minimized to generate the first joint position increment of each joint. The gradient term of the first objective function is established by the full-body Jacobian matrix. Through this modeling method, the end task requirement can be accurately mapped to the joint space. Subsequently, the first joint position increment is input to the control link, and the corresponding joint position is updated according to the first joint position increment, thereby obtaining the full-body joint position including the double arms and the waist, and realizing the coordinated motion of the robot in the full-body range. Thus, the joint position that satisfies the end task constraint can be quickly and stably solved under the condition of any degree of freedom. By introducing quadratic programming optimization, the solving process has the characteristics of convergence stability and high calculation efficiency, and the full-body calculation can be completed within microseconds, which can meet the end accuracy while considering other linear constraint conditions. Through the mapping relationship established by the full-body Jacobian matrix, the coordination between the double arms and the waist is ensured, so that the robot moves more naturally and smoothly when performing the operation task.

[0090] When the robot arm has redundant degrees of freedom, the inverse kinematics method can solve the joint positions that meet the end task, but there is often non-uniqueness of the solution, which can cause unnatural joint movement, deviation from the reference configuration, excessive energy consumption, and even potential interference with obstacles or itself. In this case, although the main task can be completed, the overall motion efficiency, stability and safety of the robot are insufficient. Therefore, after completing the end position tracking of the main task, the redundant degrees of freedom can be optimized, so that the robot arm can ensure the end accuracy while considering secondary goals such as approaching the reference configuration, avoiding joint limits, improving operability or reducing energy consumption. In practical applications, the pseudo-inverse Jacobian method combined with zero space projection can be used to adjust the redundant joints in the zero space of the end task solution. Alternatively, the main task and secondary task can be considered simultaneously based on a multi-objective optimization framework. Alternatively, a hierarchical QP can be used to first solve the main objective function (e.g., the first objective function) to ensure task completion, and then optimize the secondary objective function (e.g., the second objective function below) in the zero space, thereby realizing hierarchical control. In addition, a machine learning model can be used to predict the appropriate redundant degree of freedom configuration, or the optimization objective can be dynamically changed in different task scenarios through online adaptive weight adjustment.

[0091] In some embodiments, at least part of the robot arm can have redundant degrees of freedom. The method can further include: based on the reference configuration information of the robot arm with redundant degrees of freedom, minimizing the second objective function to generate a second joint position increment of the corresponding joint. The updating of the joint position of each joint according to the first joint position increment can include: updating the joint position of each joint according to the first joint position increment and the second joint position increment of each joint in the robot arm with redundant degrees of freedom. The second objective function is constructed in a quadratic programming manner, and the gradient term of the second objective function is established based on the zero space of the full-body Jacobian matrix.

[0092] Redundant degrees of freedom, for example, refer to the additional joint degrees of freedom of the robot arm beyond the minimum degrees of freedom required to complete a given end task.

[0093] The reference configuration information, for example, is a pre-set joint space target state, which can be used to indicate the reference position of each joint. As an example, the reference configuration information can include a desired joint position distribution or a default pose. The reference configuration information can be used as a target for redundant degree of freedom optimization, so that the joint is kept in a safe, natural or energy-optimal position. In the redundant optimization process, the deviation between the actual joint position and the reference position can be used as the optimization variable of the secondary objective function (i.e., the second objective function). As an example, by minimizing the weighted difference between the joint position and the reference position, the final joint position is as close as possible to its corresponding reference position.

[0094] The second objective function, i.e., the sub-objective function, is established on the basis of the completion of the main task, and is used to constrain and optimize the redundant degrees of freedom. The second objective function can adopt a quadratic function, which is convenient for quickly solving by using a quadratic programming method. Solving the second objective function in the null space makes the optimization only affect the redundant degrees of freedom, without destroying the main task.

[0095] For example, for a robotic arm with redundant degrees of freedom, after completing the main optimization objective, there are additional degrees of freedom that can achieve the sub-optimization objective. In actual applications, the deviation from the reference position can be selected to reduce the complexity of subsequent planning tasks. The redundant degrees of freedom can be reflected in the null space term N of the gradient term in the first objective function, i.e., the direction that does not affect the main optimization objective.

[0096] J·N=0

[0097] where the null space term N can be obtained by QR decomposition. N can also be referred to as a null space basis matrix. The joint motion in the null space direction will not affect the main task of the end effector.

[0098] Next, the sub-optimization objective can be solved by using hierarchical QP. For example, after completing the minimization solution of the first objective function, the following second objective function is solved.

[0099] min:0.5(x+Ndx2-x ref ) T W(x+Ndx2-x ref )

[0100] s.t.lb<x+Ndx2<ub

[0101] At this time, the gradient term g2 and the hession matrix H2 of the second objective function (corresponding to the sub-optimization objective) can be represented as follows.

[0102] g2=N T W(x―x ref )

[0103] H2=N T WN

[0104] where x ref is the reference position of the joint. dx2 is the second joint position increment. W is a reference weight matrix, which is used to set the importance of the reference configuration constraint in the sub-optimization objective. N T is the transpose matrix of N.

[0105] The single-arm system corresponds to a single-arm Jacobian matrix. Unlike the single-arm system, the robot in the above embodiment corresponds to a full-body Jacobian matrix. The full-body Jacobian matrix can be obtained by deforming and integrating the single-arm Jacobian matrix.

[0106] The above embodiment uses a convex optimization sequence iteration method to solve inverse kinematics of a robot arm with redundant degrees of freedom, and the solving stability and physical constraint limitation are relatively comprehensive, the method has strong universality, and the expansibility of the force control framework to robots of any configuration and any degree of freedom is ensured.

[0107] In some embodiments, the quadratic programming corresponding to the first objective function and / or the second objective function can have specified physical constraint conditions. As an example, these physical constraint conditions may, for example, be linear constraint conditions.

[0108] The above embodiment does not limit the physical constraint conditions, and in some embodiments, the physical constraint conditions can include at least one of a joint torque constraint condition, a joint position constraint condition, a joint velocity constraint condition, and a joint acceleration constraint condition.

[0109] For a single-arm force control framework, the task target is relatively single, the number of joints is limited, and the physical constraint condition is usually used to prevent a single joint from exceeding the limit, so as to ensure the safety of local operation. However, in a multi-arm force control framework based on whole body control, multiple robot arms and whole body degrees of freedom such as the waist need to be coordinated, and there is higher coupling and redundancy. If no physical constraint is added, some joints may be overburdened in task allocation, joint torque overload, motion trajectory distortion, and even structural interference problems may occur. By introducing the above physical constraint condition in the quadratic programming, the motion range and load level of each joint of the whole body can be effectively limited while ensuring the end task accuracy, so that the joint motion is always in a safe and controllable working interval. In this way, not only the stability and robustness of the multi-arm force control framework based on whole body control under high degrees of freedom are improved, but also the collaborative operation of multiple arms does not cause excessive extrusion or uneven stress on the target object. In addition, the introduction of the physical constraint condition makes the force control framework have stronger universality and expansibility, and can adapt to multi-arm whole body robot systems of different configurations and different task scenarios.

[0110] Reference is made to Figure 4 , Figure 4 is a structural block diagram of a robot operating device provided by an embodiment of the present application.

[0111] The embodiment of the present application further provides a robot operating device, the robot comprising a plurality of robot arms and a waist, and the device comprising a data receiving module, a force distribution module, a position calculation module, and an inverse kinematics solving module.

[0112] The data receiving module is configured to receive sensing force data of a corresponding robot arm when the robot operates a target object using a plurality of robot arms.

[0113] The force distribution module is configured to distribute the external force and the internal force based on the perceived force data of each of the at least two robot arms, respectively, to determine the perceived external force and the perceived internal force applied by each of the robot arms to the target object.

[0114] The position solving module is configured to solve the relationship between the force and the position based on the corresponding perceived external force and the perceived internal force of each of the robot arms, the expected external force and the expected internal force of the target object, and the expected position of the target object, to determine the expected position of each of the robot arms.

[0115] The inverse kinematics solving module is configured to perform inverse kinematics solving based on the whole body control of the robot according to the expected position of each of the robot arms, to determine the joint position of each joint in each of the robot arms and the waist, to control each of the robot arms to operate the target object based on the joint position.

[0116] Referring to Figure 5 , Figure 5 is a structural block diagram of a robot provided by an embodiment of the present application.

[0117] An embodiment of the present application further provides a robot, which comprises a control module configured to execute any of the above methods.

[0118] An embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the above methods.

[0119] An embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of any of the above methods.

[0120] The computer program product can adopt a portable compact disc read-only memory (CD-ROM) and comprise a program code, and can run on a terminal device, for example, a personal computer. However, the computer program product of the embodiment of the present application is not limited to this, and the computer program product can adopt any combination of one or more computer readable media.

[0121] Referring to Figure 6 , Figure 6 is a structural block diagram of a computer device provided by an embodiment of the present application.

[0122] An embodiment of the present application further provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0123] The computer device is not limited in the embodiments of the present application, and for example, can be a local computer device, a cloud computer device, a distributed computer device, etc.

[0124] The computer device can include a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through an internal connection path.

[0125] The memory 110 is configured to store a computer program. In some implementations, the computer program can include codes for implementing the method of the embodiments of the present application.

[0126] The processor 120 is configured to execute the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, output operation results, etc. In some implementations, when the scheme of the embodiments of the present application is implemented by software or firmware, the computer program for implementing the scheme of the embodiments of the present application can be stored in the processor 120 and executed by the processor 120.

[0127] The memory 110 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, these and other suitable types of any memory. As an example, the memory 110 includes a random access memory (RAM), a cache memory, and a read-only memory (ROM). The memory 110 stores a computer program, which can be executed by the processor 120, so that the processor 120 implements the steps of any of the above methods.

[0128] The processor 120 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can also be any conventional processor.

[0129] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 120 or the instruction in the form of software. The method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution completion, or combined with hardware and software modules in the processor 120 to complete execution. The software module can be located in a storage medium mature in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. The storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0130] In some implementations, in addition to the hardware units introduced above, the computer device can also include software modules, where the software modules can be, for example, an operating system, a basic input and output system (BIOS), application software, etc.

[0131] The operating system is used to manage hardware and / or software resources of the computer device, and is the kernel and cornerstone of the computer device. The operating system needs to handle basic transactions such as managing and configuring memory, determining the priority of system resource supply and demand, controlling input and output devices, operating network and managing file system, etc. In order to facilitate user operation, most operating systems will provide an operation interface for user to interact with the system.

[0132] The BIOS is used to run hardware initialization in the power-on boot stage, and provides runtime services for the operating system and application programs. In some implementations, the BIOS can also monitor the processor temperature and perform temperature protection strategies, etc.

[0133] Application software, also known as application program, can be understood as software written for a specific application purpose of a user, and is one of the main classifications of computer software. For example, the application software can be a program for realizing power control, temperature management, etc.

[0134] It can be understood that the specific examples in the present application are only to help those skilled in the art better understand the embodiments of the present application, and do not limit the protection scope of the present application.

[0135] It can be understood that in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the present application.

[0136] It can be understood that the various embodiments described in the present application can be implemented alone or in combination, and the present application does not limit this.

[0137] Unless otherwise specified, all technical and scientific terms used in the present application have the same meaning as understood by those skilled in the art. The terms used in the present application are only for the purpose of describing the specific embodiments and are not intended to limit the scope of the present application. The singular forms "a", "an" and "the" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0138] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of each embodiment described above can refer to the corresponding process in other embodiments, which will not be repeated here.

[0140] In the embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the embodiments of the device described above are merely schematic, and the division of the units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0141] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to actual needs to achieve the purposes of the technical solutions of the present application.

[0142] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit.

[0143] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the related art or the parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0144] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical scope disclosed by the present application, and all the changes or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A robot operation method, characterized in that, The robot includes multiple robotic arms and a waist, and the method includes: When the robot uses multiple robotic arms to manipulate a target object, it receives the sensing force data of the corresponding robotic arms. The external force and internal force are allocated based on the sensing force data of each of the at least two robotic arms, so as to determine the sensing external force and sensing internal force applied by each robotic arm to the target object; Based on the perceived external force and perceived internal force of each robotic arm, the expected external force and expected internal force of the target object, and the expected position of the target object, the relationship between force and position is calculated to determine the expected position of each robotic arm. Based on the desired position of each robotic arm, the robot is subjected to inverse kinematics solution based on whole-body control to determine the joint position of each joint in each robotic arm and waist, so as to control each robotic arm to operate the target object based on the joint position.

2. The robot operation method according to claim 1, characterized in that, The allocation of external and internal forces based on the sensing force data of each of the at least two robotic arms, to determine the sensed external and sensed internal forces applied by each robotic arm to the target object, includes: For each robotic arm, perform the following processing: Based on the force conversion relationship between the force applied by the robotic arm and the force received by the target object, the sensing force data of the robotic arm is divided into external force and internal force to obtain the sensing external force and sensing internal force applied by the robotic arm to the target object.

3. The robot operation method according to claim 1, characterized in that, The process of calculating the relationship between force and position based on the perceived external and internal forces of each robotic arm, the desired external and internal forces of the target object, and the desired position of the target object, to determine the desired position of each robotic arm, includes: Based on the perceived external force of each robotic arm, the desired external force of the target object, and the desired position of the target object, external force admittance control is performed on the target object and the external environment to generate the object position of the target object. Closed-chain constraints are performed based on the object's position to generate the robot arm positions for each robot arm. Based on the perceived internal force of each robotic arm, the desired internal force of the target object, and the position of each robotic arm, internal force admittance control is performed on each robotic arm and the target object to generate the desired position of each robotic arm.

4. The robot operation method according to claim 1, characterized in that, The step of performing inverse kinematics solution based on whole-body control on the robot according to the desired position of each robotic arm, to determine the joint positions of each joint in each robotic arm and the waist, includes: Based on the desired positions of each robotic arm, the first objective function is minimized to generate the first joint position increment for each joint. Update the corresponding joint position based on the incremental first joint position of each joint; The first objective function is constructed using quadratic programming, and the gradient term of the first objective function is established based on the whole-body Jacobian matrix, which is used to characterize the mapping relationship between the joint velocities of each joint in the whole body and the end effector velocities of each robotic arm.

5. The robot operation method according to claim 4, characterized in that, At least some parts of the robotic arm have redundant degrees of freedom; The method further includes: minimizing the second objective function based on the reference configuration information of the robotic arm with redundant degrees of freedom, so as to generate the second joint position increment of the corresponding joint; The step of updating the corresponding joint position based on the first joint position increment of each joint includes: updating the corresponding joint position based on the first joint position increment and the second joint position increment of each joint in the robotic arm with redundant degrees of freedom. The second objective function is constructed using a quadratic programming approach, and the gradient term of the second objective function is established based on the null space of the whole body Jacobian matrix.

6. The robot operation method according to claim 5, characterized in that, The quadratic programming corresponding to the first objective function and / or the second objective function has specified physical constraints, which include at least one of joint torque constraints, joint position constraints, joint velocity constraints, and joint acceleration constraints.

7. A robot operating device, characterized in that, The robot includes multiple robotic arms and a waist section, and the device includes: The data receiving module is used to receive the sensing force data of the corresponding robotic arms when the robot uses multiple robotic arms to operate the target object. The force distribution module is used to distribute external and internal forces based on the perceived force data of each of the at least two robotic arms, so as to determine the perceived external and internal forces applied by each robotic arm to the target object. The position calculation module is used to calculate the relationship between force and position based on the corresponding perceived external force and perceived internal force of each robotic arm, the expected external force and expected internal force of the target object, and the expected position of the target object, so as to determine the expected position of each robotic arm. The inverse kinematics solution module is used to perform inverse kinematics solution based on whole-body control of the robot according to the desired position of each robotic arm, so as to determine the joint position of each joint in each robotic arm and waist, and control each robotic arm to operate the target object based on the joint position.

8. A robot, characterized in that, The robot includes a control module for performing the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images