Robot cooperative control method, robot, and storage medium

Abstract

The embodiment of the application provides a kind of robot collaborative control method, robot and storage medium, belong to robot control technical field;The method comprises: obtaining the pose coordination constraint information of current period, the pose coordination constraint information is characterized in that in current period, with the centroid of the same target object in the coordinate system as reference coordinate system, the pose transformation relationship between the end of slave manipulator and the end of main manipulator;Obtain the first end pose information of the end of main manipulator that is obtained in real time in current period planning;According to the first end pose information and the pose coordination constraint information, obtain the second end pose information of the end of slave manipulator in current period, to control the main manipulator motion according to the first end pose information and control the slave manipulator motion according to the second end pose information;The embodiment of the application can improve the collaborative control efficiency of robot.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot cooperative control method, a robot, and a storage medium. Background Technology

[0002] Robotic cooperative control is commonly used in applications such as object handling and grasping. In practical applications, the robot's master arm typically performs trajectory planning, and then the displacement deviation of the slave arm relative to the master arm within the command issuance interval is calculated. Based on this displacement deviation and the master arm's planned trajectory, the slave arm's planned trajectory is obtained. Then, based on the expected position difference between the master arm's end-effector position and the slave arm's end-effector position, real-time displacement compensation is performed on the slave arm's planned trajectory, thereby achieving synchronous control of the slave arm following the master arm within the same robot. However, this method involves two trajectory planning paths and requires a large amount of real-time computation data, resulting in low efficiency in robot cooperative control. Summary of the Invention

[0003] The main objective of this application is to propose a robot cooperative control method, a robot, and a storage medium that can improve the efficiency of robot cooperative control.

[0004] To achieve the above objectives, a first aspect of this application proposes a robot cooperative control method, wherein the robot includes a master robotic arm and a slave robotic arm, and the method includes: Obtain pose coordination constraint information for the current cycle. The pose coordination constraint information represents the pose transformation relationship between the end effector of the slave robot and the end effector of the master robot within the current cycle, with the coordinate system where the centroid of the same target object is located as the reference coordinate system. Obtain the first end-effector pose information of the main robotic arm obtained in real time during the current cycle; Based on the first end-effector pose information and the pose coordination constraint information, the second end-effector pose information of the slave robot arm in the current cycle is obtained, so as to control the movement of the master robot arm based on the first end-effector pose information and control the movement of the slave robot arm based on the second end-effector pose information.

[0005] To achieve the above objectives, a second aspect of the present application provides a robot including a robot controller, wherein the robot controller stores a computer program, and when the robot controller executes the computer program, it implements the robot cooperative control method as described in any one of the first aspects.

[0006] To achieve the above objectives, a third aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a robot controller, implements the robot cooperative control method described in any of the first aspects.

[0007] The robot cooperative control method, robot, and storage medium proposed in this application directly calculate the second end-effector pose information based on the pose cooperative constraint information and the first end-effector pose information of the current cycle. This eliminates the need for trajectory planning for the slave robot, thereby reducing the computational load of trajectory planning in each cycle. Furthermore, since the pose cooperative constraint information is based on the coordinate system containing the centroid of the same target object as a reference coordinate system, the pose transformation relationship between the end-effectors of the slave and master robots can be maintained within the same cycle, ensuring that the master and slave robots maintain the expected cooperative state towards the same target object. Therefore, compared with the prior art, the embodiments of this application can improve the efficiency of cooperative control. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating an embodiment of the robot cooperative control method provided in this application; Figure 2 This is a schematic diagram of the application process of the collaborative control method provided in this application, representing one embodiment of the collaborative control method for robots. Figure 3 This is a schematic diagram of the coordinate systems of the master and slave robotic arms satisfying tight cooperative constraints in one embodiment of the robot cooperative control method provided in this application. Figure 4 This is a schematic diagram of the coordinate systems of the master and slave robotic arms satisfying the loose collaboration constraint conditions in another embodiment of the robot cooperative control method provided in this application; Figure 5 This is a schematic diagram of the hardware structure of the robot controller corresponding to the robot cooperative control method provided in this application. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0010] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0012] Robot cooperative control is commonly used in applications such as object handling and grasping. In practical applications, to ensure the accuracy of robot cooperative control, the robot's master arm typically performs trajectory planning. Then, the displacement deviation of the slave arm relative to the master arm within the command issuance interval is statistically analyzed. Based on this displacement deviation and the master arm's planned trajectory, the slave arm's planned trajectory is obtained. Then, based on the expected position difference between the master arm's end-effector position and the slave arm's end-effector position, real-time displacement compensation is performed on the slave arm's planned trajectory, thereby achieving synchronous control of the slave arm following the master arm within the same robot. However, this method involves two trajectory planning paths and requires a large amount of real-time computation data, resulting in low robot cooperative control efficiency. Therefore, this application provides a robot cooperative control method, a robot, and a storage medium that can improve the efficiency of robot cooperative control.

[0013] Understandably, referring to Figure 1 As shown, according to the robot cooperative control method provided in the embodiments of this application, the robot includes a master robotic arm and a slave robotic arm, and the method includes: Step S100: Obtain the pose coordination constraint information for the current cycle. The pose coordination constraint information represents the pose transformation relationship between the end effector of the robotic arm and the end effector of the main robotic arm within the current cycle, with the coordinate system where the centroid of the same target object is located as the reference coordinate system. Step S200: Obtain the first end pose information of the end of the main robotic arm, which is planned in real time within the current cycle; Step S300: Based on the first end-effector pose information and pose coordination constraint information, obtain the second end-effector pose information of the end of the robotic arm in the current cycle, so as to control the movement of the main robotic arm based on the first end-effector pose information and control the movement of the slave robotic arm based on the second end-effector pose information.

[0014] By directly calculating the second end-effector pose information based on the pose coordination constraint information and the first end-effector pose information of the current cycle, trajectory planning for the slave robot arm can be eliminated, thereby reducing the computational load of trajectory planning in each cycle. Furthermore, since the pose coordination constraint information is based on the coordinate system containing the centroid of the same target object as the reference coordinate system, the pose transformation relationship between the end-effectors of the slave and master robots can be maintained within the same cycle, ensuring that the master and slave robots maintain the expected coordinated state towards the same target object. Therefore, the embodiments of this application can improve the efficiency of coordinated control.

[0015] The robot cooperative control method of this application embodiment can be applied to a robot controller or a host computer. The robot controller can be a controller integrated on the robot or a main controller in the robot system. In this case, the controller integrated on the robot is equivalent to a slave controller. The main robot arm and the slave robot arm receive motion control parameters sent by the main controller through the slave controller of the robot, so that the main robot arm moves according to the first end pose information and the slave robot arm moves according to the second end pose information.

[0016] The current cycle refers to the cycle in which the corresponding step is executed. In this embodiment, each cycle is used to instruct the robot controller of the robot cooperative control method to complete one instruction operation on the master and slave robotic arms. In some embodiments, the cycle can be distinguished by time, such as a preset duration as one cycle. In other embodiments, the cycle can be distinguished by whether the instruction steps are completed. For example, if a cycle requires the following instruction steps: pose information acquisition -> pose information planning -> issuance of planning result instructions, then re-executing the pose information acquisition step indicates the start of a new cycle. This embodiment does not elaborate further on this, and those skilled in the art can selectively set it according to actual needs. When the cycle is distinguished by duration, this embodiment does not restrict the duration of each cycle or the method of cycle triggering. For example, if the cycle is 1ms, it can be triggered by a 1k interrupt. In other embodiments, other methods can also be used for triggering. For example, if the cycle is 0.125ms, it can be triggered by an 8K interrupt. This embodiment does not elaborate further on this. When the cycle is distinguished by duration, this embodiment does not restrict whether the durations of two adjacent cycles are the same. Those skilled in the art can selectively set it according to actual needs.

[0017] This application does not impose any restrictions on how the pose coordination constraint information is calculated. It can be calculated directly from the pose information of the end effector of the main robotic arm relative to the target object, or it can be calculated directly from the transformation relationship of the main robotic arm relative to the target object in the world coordinate system. This application will not elaborate on these points.

[0018] Steps S100 to S300 are actions executed within the same cycle. This application embodiment does not restrict whether the pose coordination constraint information is calculated in real-time in the current cycle or calculated in real-time after the control of step S300 in the previous cycle, and only the calculation result is obtained in the current cycle. Those skilled in the art can selectively set this according to the actual situation. For example, in some embodiments, step S100 is calculated in real-time in the current cycle. In this case, the operation of one cycle includes: real-time calculation of pose coordination constraint information -> calculation of first end-effector pose information -> calculation of second end-effector pose information -> control for sending first and second end-effector pose information. In other embodiments, step S100 is calculated in real-time in the previous cycle. In this case, the operation of one cycle includes: obtaining pose coordination constraint information -> calculation of first end-effector pose information -> calculation of second end-effector pose information -> control for sending first and second end-effector pose information -> real-time calculation of pose coordination constraint information.

[0019] This application does not limit whether the motion control of the master and slave robotic arms is executed by the robot controller to which the method is applied. In some embodiments, the method is applied to the master controller, in which case the first and second end-effector pose information are directly sent to the corresponding slave controllers of the master and slave robotic arms, and the slave controllers control the movement of the master and slave robotic arms. In other embodiments, the method is applied to a controller integrated on the robot, in which case the controller directly controls the master and slave robotic arms respectively based on the first and second end-effector pose information. This application will not elaborate on these specific implementations.

[0020] The first end-effector pose information is the pose planning information determined directly based on the first planned trajectory of the main manipulator in the current cycle. The second end-effector pose information is the pose planning information obtained directly based on pose cooperative constraint information in the current cycle. Both the first and second end-effector pose information represent the pose that the end-effector of the corresponding manipulator is expected to achieve in the next cycle. The pose includes at least one of orientation and three-dimensional coordinates. The first end-effector pose information indicates the planned position of the end-effector of the main manipulator in the first base coordinate system of the main manipulator. The second end-effector pose information indicates the planned position of the end-effector of the slave manipulator in the second base coordinate system of the slave manipulator. The first base coordinate system represents the coordinate system of the base of the main manipulator (i.e., the fixed mounting base of the manipulator, which is the mounting reference for each joint of the manipulator), and the second base coordinate system represents the coordinate system of the base of the slave manipulator. During the operation of the target object, the end effector of the main robotic arm must always move with its own base, and the end effector of the secondary robotic arm must also move with its own base. Therefore, based on the pose coordination constraint information, it is possible to perform tight coordination operations such as grasping on the target object synchronously, and it is also possible to perform loose coordination operations after release on the target object synchronously.

[0021] This application embodiment does not limit how steps S100 to S300 are triggered. They can be executed periodically after being triggered by a user command, or they can be executed periodically after automatically detecting that the conditions are met. This application embodiment will not elaborate on these points one by one.

[0022] The target object is the object on which the robot's end effector performs an operation, such as a handling operation or a grasping operation. In this case, the target object is the object of the handling operation or the object of the grasping operation. This application will not elaborate on these points in the embodiments.

[0023] The pose of the base of the main robotic arm in the world coordinate system can be measured in real time. Therefore, the pose of the second end effector of the robotic arm in the current cycle can be obtained based on the first end effector pose information and the pose coordination constraint information.

[0024] In some embodiments, taking the second end-effector pose information calculated using a matrix method as an example, the second end-effector pose information satisfies the following pose transformation formula: (Formula 1); in, A matrix representing the pose coordination constraint information of the current time period; A matrix representing the pose information of the first robotic arm; The matrix representing the pose information of the second robotic arm; This represents the matrix corresponding to the first end pose information. The matrix represents the second end-effector pose information. Therefore, the above embodiments of this application do not require obtaining the second end-effector pose information in real time based on the planned trajectory of the robotic arm, but only require obtaining the first end-effector pose information in real time based on the planned trajectory of the main robotic arm, thereby reducing the amount of computation required in a single cycle.

[0025] This application does not limit the number of slave robots following the same master robot arm in a robot, nor does it limit the number of master robots. Taking one master robot arm as an example, in some embodiments, the number of collaborations is set to one, meaning there is a one-to-one correspondence between the master and slave robots. In other embodiments, the number of collaborations is set to multiple, meaning the master robot arm and multiple slave robots collaborate together. Correspondingly, the pose collaboration constraint information and the number of collaborating robots are set in a one-to-one correspondence. In this case, after the first end-effector pose information of the master robot arm is determined, the corresponding second end-effector pose information can be obtained based on the pose collaboration constraint information corresponding to each slave robot arm collaborating with the master robot arm, thereby enabling multiple slave robots to collaborate simultaneously. This application will not elaborate further on this aspect.

[0026] Understandably, obtaining the first end-effector pose information of the main robotic arm, planned in real-time within the current cycle, includes: Obtain the first planned trajectory obtained by path planning with the first end reference coordinate system of the main robotic arm as the path planning object; Determine the first end pose information of the current cycle from the first planned trajectory.

[0027] The first end pose information can be determined by interpolating the discrete trajectory points output by the first planned trajectory in the current cycle. This application embodiment does not limit how the first end pose information is determined from the first planned trajectory.

[0028] This application uses the origin of the first end-effector reference coordinate system as the end-effector planning object. The embodiments of this application do not limit the specific location of this end-effector planning object on the end, and those skilled in the art can selectively set it according to the mechanical structure of the end of the main robotic arm. In some embodiments, the first end-effector reference coordinate system may also coincide with the free coordinate system where the end of the main robotic arm is located. The first planned trajectory is pre-planned.

[0029] Understandably, given that the tight collaborative constraint condition is satisfied, the method also includes: Move the end effector of the main robotic arm to the pose corresponding to the first end effector pose information; Move from the end of the robotic arm to the pose corresponding to the second end pose information.

[0030] Satisfying tight coordination constraints means that the target object is constrained relative to the master and slave robotic arms. For example, in a material handling task, the target object is being held or grasped by the master and slave robotic arms. Satisfying loose coordination constraints means that the target object is free relative to the master and slave robotic arms. For example, in a material handling task, satisfying loose coordination constraints means that the master and slave robotic arms have released the target object.

[0031] Under tight coordination conditions, the end effector's free coordinate system almost coincides with the first end effector reference coordinate system. Therefore, control can be performed directly based on the first end effector's pose information, thus further saving computational resources. Similarly, since the end effector's free coordinate system almost coincides with the second end effector reference coordinate system, control can be performed directly based on the second end effector's pose information, further saving computational resources.

[0032] For example, when the method is applied to the master controller, under the condition of tight cooperative constraints, the first end pose information and the second end pose information are directly sent to the corresponding slave controller. The slave controller controls the end of the master robot arm to move to the pose corresponding to the first end pose information and the end of the robot arm to move to the pose corresponding to the second end pose information.

[0033] Understandably, given that the loosely coordinated constraints are satisfied, the method also includes: The first reference pose information of the end effector of the main robotic arm in the first end effector reference coordinate system is obtained. The first reference pose information of the main robotic arm and the first end effector pose information are processed by coordinate system pose transformation to obtain the third end effector pose information. The end effector of the main robotic arm is then moved to the pose corresponding to the third end effector pose information. Obtain the second robotic arm reference pose information from the end of the robotic arm in the second end-effector reference coordinate system; and perform coordinate system pose transformation processing on the second robotic arm reference pose information and the second end-effector pose information to obtain the fourth end-effector pose information, and move the end of the robotic arm to the pose corresponding to the fourth end-effector pose information.

[0034] The first robotic arm's reference pose information is the pose information directly obtained from trajectory planning based on the first planned trajectory.

[0035] By ensuring that the positional deviation between the end-effector free coordinate system and the first end-effector reference coordinate system is corrected through coordinate system pose transformation, the accuracy of trajectory planning can be further ensured, provided that the slack coordination constraint condition is met.

[0036] By ensuring that the pose deviation between the end-effector free coordinate system and the second reference coordinate system is satisfied under the condition of slack coordination constraint, the accuracy of trajectory planning can be further ensured through the coordinate system pose transformation process.

[0037] This application does not restrict how to determine whether tight or loose cooperative constraints are met. For example, it may be based on the trend of trajectory planning or on instructions; however, this application will not elaborate on these methods.

[0038] Understandably, based on the first end-effector pose information and pose coordination constraint information, the second end-effector pose information of the robotic arm in the current cycle is obtained, including: Obtain the first robot arm pose information of the first base coordinate system in the world coordinate system; Obtain the pose information of the second robotic arm from the second base coordinate system in the world coordinate system; The inverse coordinate pose information of the second robotic arm, the pose information of the first robotic arm, the pose coordination constraint information, and the pose information of the first end effector are subjected to coordinate system pose transformation to obtain the pose information of the second end effector.

[0039] Understandably, obtaining the pose cooperative constraint information determined in real time for the current period includes: Based on the pose information of the third robotic arm in the current cycle, coordinate system pose transformation processing is performed on the pose information of the fourth robotic arm to obtain pose cooperative constraint information; The third robotic arm pose information is used to indicate the pose information of the target object's centroid coordinate system in the end-effector coordinate system of the main robotic arm within the current cycle, and the fourth robotic arm pose information is used to indicate the pose information of the end-effector free coordinate system of the slave robotic arm in the centroid coordinate system of the target object within the current cycle; or, the third robotic arm pose information is used to indicate the pose information of the world coordinate system in the end-effector free coordinate system of the main robotic arm within the current cycle, and the fourth robotic arm pose information indicates the pose information of the end-effector free coordinate system of the slave robotic arm in the world coordinate system.

[0040] For example, as in some embodiments, pose cooperative constraint information ;in, This indicates the pose information of the third robotic arm. This indicates the pose information of the fourth robotic arm.

[0041] Understandably, the pose information of the third robotic arm is obtained through one of the following steps: Parameters are extracted from the motion parameters of the current cycle; The coordinate system pose transformation is performed on the pose information of the fifth and sixth robotic arms in the motion parameters of the current cycle. Among them, the pose information of the fifth robotic arm is the pose information of the first base coordinate system of the main robotic arm in the world coordinate system during the current cycle; the pose information of the sixth robotic arm is the pose information of the end effector of the main robotic arm in the first base coordinate system during the current cycle.

[0042] In some embodiments, the position of the target object is known. In this case, the pose information of the fifth robotic arm can be directly determined using the pose information of the target object in the acquired world coordinate system. In other embodiments, the position of the target object is unknown. In this case, it can be directly calculated using the pose information of the fifth and sixth robotic arms. In some embodiments, regardless of whether the position of the target object is known or not, the pose information of the third robotic arm can be calculated based on the pose information of the fifth and sixth robotic arms.

[0043] In some embodiments of this application, the motion parameters can be Cartesian pose data.

[0044] Understandably, the pose information of the fourth robotic arm is obtained through one of the following steps: The ninth robotic arm pose information of the motion parameters of the current cycle is obtained by inverse pose processing; The coordinate system pose transformation is performed on the pose information of the seventh and eighth robotic arms in the motion parameters of the current cycle. Among them, the pose information of the seventh robotic arm is the pose information of the second base coordinate system of the robotic arm in the world coordinate system during the current cycle; the pose information of the eighth robotic arm is the pose information of the end of the robotic arm in the second base coordinate system during the current cycle.

[0045] In some embodiments, the pose information of the third and fourth robotic arms can be determined based on parameters extracted from motion parameters. In other embodiments, they are obtained through real-time coordinate system calculation. In still other embodiments, both the pose information of the third and fourth robotic arms can be obtained through coordinate system pose transformation. In yet another embodiment, either the pose information of the third or fourth robotic arm can be selected; however, this will not be elaborated further in the embodiments of this application.

[0046] Understandably, coordinate system pose transformation processing means converting each pose information to be processed into a pose matrix and multiplying each pose matrix in the order of multiplication of the pose matrix corresponding to the pose information expected to be output by coordinate system pose transformation processing.

[0047] If the first end pose information and pose coordination constraint information are both matrices, then the second end pose information will first convert each pose information involved in the calculation into a pose matrix, and then multiply each pose matrix based on the coordinate system transformation rules of the second end pose information.

[0048] The pose information expected to be output by the coordinate system pose transformation process represents the processing result of the coordinate system pose transformation process. For example, the second end pose information is the processing result obtained by performing coordinate system pose transformation on the inverse coordinate pose information of the second robot arm pose information, the first robot arm pose information, the pose cooperative constraint information and the first end pose information.

[0049] Understandably, obtaining the pose coordination constraint information for the current period includes: In response to detecting that the main robotic arm is within a preset space range of the target object and / or receiving an object operation command, the pose coordination constraint information for the current cycle is obtained.

[0050] The object operation instructions can be issued by the host computer or triggered by the robot through control buttons. This application embodiment does not limit this.

[0051] The preset spatial range indicates that the end effector of the main robotic arm moves towards the target object and the distance between them is less than a preset first distance threshold, or that the end effector of the main robotic arm moves away from the target object and the distance between them is less than a preset second distance threshold. In other words, the preset spatial range indicates the constraint space for the coordinated movement of the main robotic arm and the slave robotic arm. This application embodiment does not limit how the first and second distance thresholds are set; those skilled in the art can selectively set them according to actual conditions. By setting the preset spatial range, the robotic arm can be controlled in other ways outside the distance threshold range, and controlled in the manner described in this application range within the distance threshold range. Different control methods can further improve the robot's control efficiency. For example, within the constraint space, using the method described in this application embodiment, outside the constraint space (i.e., free space), the main robotic arm and the slave robotic arm complete free movement using their respective reference coordinate systems. For instance, the main robotic arm performs trajectory planning based on a first base coordinate system, and the slave robotic arm performs trajectory planning based on a second base coordinate system.

[0052] Understandably, after obtaining the second end-effector pose information from the end of the robotic arm in the current cycle, the method further includes at least one of the following: The first end pose information is updated based on the first pose error data of the current cycle; The second end pose information is updated based on the second pose error data of the current cycle; The third pose error data for the next cycle is determined based on the first end-effector pose information and / or the second end-effector pose information, so as to update the first end-effector pose information and / or the second end-effector pose information based on the third pose error data in the next cycle.

[0053] This application does not limit which one or more of the first pose error data, second pose error data, and third pose error data are used to update the pose information; those skilled in the art can choose to set them selectively.

[0054] The first pose error data is used to characterize the error between the theoretical pose and the actual pose of the end effector of the master robotic arm. For example, if the first end effector pose information obtained from the planning in cycle t1 is A1, and the actual pose after control based on A1 is B1, then for cycle t2, the first pose error data represents the pose error between A1 and B1. Similarly, the second pose error is used to characterize the error between the theoretical pose and the actual pose of the end effector of the robotic arm.

[0055] The third pose error data is used to indicate the pre-compensation data for the first and second end-effector pose information planned in the next cycle, so that the actual poses of the master and slave end-effectors in the next cycle match the theoretically expected poses. For example, in some embodiments, the third pose error data represents deviations caused by driving errors or usage time limits. The data used to update the first end-effector pose information in the next cycle can be determined by looking up the first pose information in the current cycle. Similarly, the third pose error data used to update the second end-effector pose in the next cycle can be determined by looking up the second pose information in the current cycle. In other embodiments, if the errors of the master and slave end-effectors are the same, after determining the third pose error data for the next cycle based on the first end-effector pose information, the first end-effector pose information can be directly updated based on the third pose error data, and then the second end-effector pose information can be obtained based on the updated second end-effector pose information. This application does not elaborate on how the third pose error data is compensated. Those skilled in the art can selectively set it according to actual conditions. The compensation in this application embodiment can also be dynamically configured by the user; however, this application does not elaborate on this aspect.

[0056] For example, reference is made below to this application. Figure 2 , Figure 3 , Figure 4 The robot coordination control method described in this application is illustrated in the example of a robot having one master robotic arm and one slave robotic arm. Figure 2 As shown, under the condition of satisfying the tight cooperation constraint, the target object is clamped and fixed by the master robot arm and the slave robot arm. At this time, the coordinate systems of each position of the master robot arm and the slave robot arm are as follows: Figure 2 As shown, where, as Figure 2 As shown, Indicates the world coordinate system. This represents the free coordinate system from the end effector of the robotic arm. This indicates the coordinate system from which the base of the robotic arm is located (i.e., the second base coordinate system in the above embodiment). This represents the free coordinate system where the end effector of the main robotic arm is located. The coordinate system representing the base of the master robotic arm (i.e., the first base coordinate system in the above embodiment) is used. Under the condition of tight cooperative constraints, when the master and slave robotic arms move, the positions of the end effectors of the master and slave robotic arms are relatively fixed. Therefore, the pose information of the second end effector can be obtained in real time based on the pose cooperative constraint information. The current time period is used as... For example, the specific steps are as follows: Figure 3 As shown: S1. Obtain pose coordination constraint information for the current cycle: 1) In some embodiments, the following first constraint matrix formula is used: (Formula 2); in, This indicates the pose information of the third robotic arm. This indicates the pose information of the fourth robotic arm; This represents the pose information of the fifth robotic arm, which is the pose information of the first base coordinate system of the main robotic arm in the world coordinate system during the current cycle. This represents the pose information of the sixth robotic arm, which is the pose information of the end effector of the main robotic arm in the first base coordinate system during the current cycle. This represents the pose information of the seventh robotic arm, which is the pose information of the robotic arm from the second base coordinate system in the world coordinate system during the previous cycle; This represents the pose information of the eighth robotic arm, that is, the pose information of the end of the robotic arm in the second base coordinate system within the current cycle.

[0057] At this point, the forward kinematic transformation matrix obtained by solving the current period is... , and known , Substituting these values ​​into the formula for the first constraint matrix yields the pose coordination constraint information for the current cycle.

[0058] Among them, reference Figure 2 The coordinate system constraint relationship shown indicates that the principle for obtaining the pose cooperative constraint information in Formula 2 is as follows: For closely coordinated tasks, since the two robotic arms need to grip an object during cooperation, there is no relative displacement between their ends and the object, and the relative velocity should be zero. Therefore, it is necessary to add positional constraints on the ends of the two robotic arms. Assume there is an object coordinate system. This object coordinate system can correspond to the coordinate system of the real object being transported, or it can correspond to a virtual object (that is, the target object in the above embodiment can be a real object or a virtual object). As a description of the relative position constraints of the two robotic arm ends, then... Figure 2 The coordinate systems shown satisfy the following formula three: (Formula 3); in, The transformation matrix represents the transformation from the world coordinate system to the centroid coordinate system where the centroid of the target object is located, which is also the pose information of the centroid coordinate system in the world coordinate system; , This represents the transformation matrix from the world coordinate system to the left and right arm base coordinate systems, respectively. , This represents the transformation matrix that transforms the base coordinate system of the master and slave robotic arms to their respective end-effector free coordinate systems; , This represents the transformation matrix that transforms the end-effector coordinate system of the master and slave robotic arms to the same centroid coordinate system.

[0059] At this point, Formula 4 can be derived from Formula 3 as follows: (Formula 4); In some embodiments, , It can be obtained through dual-arm robot model files or actual measurements during assembly; , It can be obtained by solving the robot's forward kinematics; , Depending on the object being manipulated and the task scenario, the coordinate system of the centroid of the object can be estimated using robot vision information. Relative to the world coordinate system Transformation matrix Therefore, the robot's end-effector pose when grasping an object can be used to calculate... , .

[0060] At this point, based on Formula 4, in , In the case of unknown circumstances, we can obtain: ; That is to say .

[0061] 2) In other embodiments, the following second constraint matrix formula may be used: (Formula 3); in, This indicates the pose information of the third robotic arm. This indicates the pose information of the fourth robotic arm. This provides the pose information for the ninth robotic arm. in, , It can be directly measured, and once measured, it can be substituted into Formula 3 to calculate the pose coordination constraint information.

[0062] This application does not restrict whether Formula 2 or Formula 3 is used for pose coordination constraint information. Those skilled in the art can selectively set the coordinate acquisition accuracy, whether the known pose information in Formula 2 or Formula 3 can be obtained, and user configuration. For example, in some embodiments, when the coordinate acquisition accuracy of the target object does not meet the conditions, such as too many interference conditions or no acquisition, Formula 2 is used. When the coordinate acquisition accuracy of the target object meets the preset acquisition accuracy conditions, Formula 3 can be used. This application does not restrict this. Those skilled in the art can selectively configure according to the actual operating environment, thereby adapting to different production environments.

[0063] S2. Under the condition of satisfying tight coordination constraints, the first end-effector pose information of the current main robotic arm is used in real time. Solve for the pose information of the second end point that satisfies the tight cooperative constraints. The details are as follows: Substitute the first end pose information obtained from the real-time planning of the current cycle into the following formula: ; At this point, the pose information of the second end effector can be obtained. .in, The pose information of the first robotic arm represents the pose information of the first base coordinate system in the world coordinate system; The pose information of the second robotic arm represents the pose information of the second base coordinate system in the world coordinate system.

[0064] In some embodiments, both the master robotic arm and the slave robotic arm have their own end-effector reference coordinate systems established by their respective end-effector reference objects. , When the robot's main arm completes trajectory planning and performs closely coordinated motion, the reference coordinate system of the end effector of the main robotic arm... It runs along the planned trajectory and uses pose-coordinated constraint information. And Formula 1 calculates in real time from the robotic arm's reference coordinate system. The transformation matrix relative to the base coordinate system of the robotic arm is as follows: ; in, This refers to the robot arm's pose information from the first end reference coordinate system to the first base coordinate system. This represents the robot arm pose information from the second end-effector reference coordinate system to the second base coordinate system. However, in practical applications, under the condition of tight coordination, , Since it is an identity matrix, under tight cooperative constraints, the pose information of the second end point can be directly used. As pose information from the end effector of the robotic arm, the first end effector pose information The pose information of the end effector of the main robotic arm.

[0065] S3. Under the condition of satisfying the loose coordination constraint, the first end-effector pose information of the current main manipulator is used in real time. Combined with pose and constraint information, the third end-effector pose information of the main robotic arm is obtained, and then the fourth end-effector pose information of the robotic arm is obtained; specifically as follows: Reference Figure 4 As shown, the free coordinate system at the end of the main robotic arm Around the reference coordinate system Taking motion as an example. From the perspective of the world coordinate system, the reference coordinate system... The origin It moves along the planned trajectory; however, from the perspective of the free coordinate system at the end of the main robotic arm, the reference coordinate system... The origin It is stationary. Then, as... Figure 4 The lighter-colored master and slave robotic arms are shown in the diagram. The free coordinate system at the end of the master robotic arm is shown. Relative to the reference coordinate system The transformation matrix is Similarly, the situation is similar from the end of the robotic arm, where there is a free coordinate system at the end of the robotic arm. Relative to the reference coordinate system The transformation matrix is At this point, the transformation matrices of the robot's master and slave arms relative to their respective base coordinate systems are: ; That is, the pose information of the third end effector. Fourth end-effector pose information .in, , , , , All of these are known quantities.

[0066] It is understood that in some embodiments, the first end reference coordinate system may coincide with the free coordinate system corresponding to the end of the main robotic arm; in other embodiments, the second end reference coordinate system may coincide with the free coordinate system corresponding to the end of the slave robotic arm; and in still other embodiments, the world coordinate system may coincide with the base coordinate system of the main robotic arm. In this application, the embodiments will not elaborate further on these points. Those skilled in the art may choose to have some coordinate systems coincide or not coincide at all. In this regard, the embodiments of this application do not impose any restrictions.

[0067] A robot controller according to an embodiment of this application includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the robot cooperative control method as described in any of the above embodiments, enabling the master robot arm and the slave robot arm to cooperate in control.

[0068] For example, please refer to Figure 5 , Figure 5 The hardware structure of a robot controller according to another embodiment is illustrated. The robot controller includes: The processor 501 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 502 can be a NAND flash, and the relevant program code is stored in the memory 502 and called by the processor 501 to execute the robot cooperative control method of the present application embodiment; The input / output interface 503 is used to implement information input and output; The communication interface 504 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 505 transmits information between various components of the device (e.g., processor 501, memory 502, input / output interface 503, and communication interface 504); The processor 501, memory 502, input / output interface 503, and communication interface 504 are connected to each other within the device via bus 505.

[0069] This application also provides a robot system, including: The main controller executes the above-described robot cooperative control method. Multiple target robots, each including a master robot arm, a slave robot arm, and a slave controller, with each slave controller communicating with the master controller; the slave controller controls the movement of the corresponding master robot arm based on the first end-effector pose information of the master controller, and controls the movement of the corresponding slave robot arm based on the second end-effector pose information of the master controller.

[0070] This application also provides a computer-readable storage medium that stores a computer program that, when executed by a processor, implements the above-described robot cooperative control method.

[0071] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0073] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0076] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0077] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0079] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A robot cooperative control method, characterized in that, The robot includes a master robotic arm and a slave robotic arm, and the method includes: Obtain pose coordination constraint information for the current cycle. The pose coordination constraint information represents the pose transformation relationship between the end effector of the slave robot and the end effector of the master robot within the current cycle, with the coordinate system where the centroid of the same target object is located as the reference coordinate system. Obtain the first end-effector pose information of the main robotic arm obtained in real time during the current cycle; Based on the first end-effector pose information and the pose coordination constraint information, the second end-effector pose information of the slave robot arm in the current cycle is obtained, so as to control the movement of the master robot arm based on the first end-effector pose information and control the movement of the slave robot arm based on the second end-effector pose information.

2. The robot cooperative control method according to claim 1, characterized in that, The step of obtaining the first end-effector pose information of the main robotic arm, which is planned in real time within the current cycle, includes: Obtain the first planned trajectory by performing path planning with the first end reference coordinate system of the main robotic arm as the path planning object; Determine the first end pose information for the current cycle from the first planned trajectory.

3. The robot cooperative control method according to claim 2, characterized in that, If the tight collaboration constraint condition is satisfied, the method further includes: The end effector of the main robotic arm is moved to the pose corresponding to the first end effector pose information. This moves the robot arm from its end to the pose corresponding to the second end pose information.

4. The robot cooperative control method according to claim 3, characterized in that, If the loose coordination constraint condition is satisfied, the method further includes: Obtain the first robotic arm reference pose information of the end effector in the first end effector reference coordinate system, and then... The first robotic arm reference pose information and the first end effector pose information are subjected to coordinate system pose transformation processing to obtain the third end effector pose information, and the end effector of the main robotic arm is moved to the pose corresponding to the third end effector pose information. The process involves obtaining the second robotic arm reference pose information of the end effector of the robotic arm in the second end effector reference coordinate system of the robotic arm; and performing coordinate system pose transformation processing on the second robotic arm reference pose information and the second end effector pose information to obtain the fourth end effector pose information, and moving the end effector of the robotic arm to the pose corresponding to the fourth end effector pose information.

5. The robot cooperative control method according to claim 1, characterized in that, The step of obtaining the second end-effector pose information of the robotic arm in the current cycle based on the first end-effector pose information and the pose coordination constraint information includes: Obtain the first robot arm pose information of the first base coordinate system in the world coordinate system; Obtain the pose information of the second robotic arm from the second base coordinate system in the world coordinate system; The inverse coordinate pose information of the second robotic arm pose information, the first robotic arm pose information, the pose coordination constraint information, and the first end effector pose information are subjected to coordinate system pose transformation processing to obtain the second end effector pose information.

6. The robot cooperative control method according to claim 1, characterized in that, The acquisition of pose coordination constraint information determined in real time for the current period includes: Based on the pose information of the third robotic arm in the current cycle, coordinate system pose transformation processing is performed on the pose information of the fourth robotic arm to obtain the pose cooperative constraint information. The pose information of the third robotic arm is obtained through one of the following steps: Parameters are extracted from the motion parameters of the current cycle; The coordinate system pose transformation is performed on the pose information of the fifth and sixth robotic arms in the motion parameters of the current cycle. The pose information of the fourth robotic arm is obtained through one of the following steps: The ninth robotic arm pose information of the motion parameters of the current cycle is obtained by inverse pose processing; The coordinate system pose transformation is performed on the pose information of the seventh and eighth robotic arms in the motion parameters of the current cycle. Wherein, the third robotic arm pose information is used to indicate the pose information of the target object's centroid coordinate system in the end-effector coordinate system of the main robotic arm within the current cycle, and the fourth robotic arm pose information is used to indicate the pose information of the slave robotic arm's end-effector free coordinate system in the target object's centroid coordinate system within the current cycle; or, the third robotic arm pose information is used to indicate the pose information of the world coordinate system in the end-effector free coordinate system of the main robotic arm within the current cycle, and the fourth robotic arm pose information indicates the pose information of the slave robotic arm's end-effector free coordinate system in the world coordinate system; Wherein, the pose information of the fifth robotic arm is the pose information of the first base coordinate system of the main robotic arm in the world coordinate system during the current cycle; the pose information of the sixth robotic arm is the pose information of the end effector of the main robotic arm in the first base coordinate system during the current cycle; the pose information of the seventh robotic arm is the pose information of the second base coordinate system of the slave robotic arm in the world coordinate system during the current cycle; and the pose information of the eighth robotic arm is the pose information of the end effector of the slave robotic arm in the second base coordinate system during the current cycle.

7. The robot cooperative control method according to any one of claims 4 to 6, characterized in that, The coordinate system pose transformation process means converting each pose information to be processed into a pose matrix and multiplying each pose matrix in sequence according to the pose matrix multiplication order corresponding to the pose information expected to be output by the coordinate system pose transformation process.

8. The robot cooperative control method according to claim 5, characterized in that, After obtaining the second end-effector pose information from the end of the robotic arm in the current cycle, the method further includes at least one of the following: The first end pose information is updated based on the first pose error data of the current cycle; The second end pose information is updated based on the second pose error data of the current cycle; The third pose error data for the next cycle is determined based on the first end pose information and / or the second end pose information, so as to update the first end pose information and / or the second end pose information based on the third pose error data in the next cycle.

9. A robot, comprising a robot controller, characterized in that, The robot controller stores a computer program, and when the robot controller executes the computer program, it implements the robot cooperative control method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the robot controller, it implements the robot cooperative control method according to any one of claims 1 to 8.

Images