Control methods and control devices for collaborative robots

Abstract

This invention provides a control method and control device for a collaborative robot. The method includes: calibrating the pose of a depth camera and a robotic arm; filtering the point cloud data acquired by the depth camera to obtain first sparse data; acquiring the joint angles corresponding to the current position of the robotic arm and the joint angles corresponding to future path points, and calculating the current and future robotic arm configurations respectively; deleting all points in the first sparse data corresponding to the current robotic arm configuration to obtain second sparse data; and controlling the speed of the robotic arm based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data. This invention does not distinguish between people and objects; as long as an obstacle intrudes into a pre-set area, deceleration or stopping is triggered. The entire process does not require prior training, has low computational requirements, and can operate in the dark, thus ensuring high safety.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a control method and control device for a collaborative robot. Background Technology

[0002] Collaborative robots, also known as collaborative robotic arms, are a type of robotic arm primarily used in scenarios where they work in cooperation with humans. Currently, collaborative robots cannot perceive changes in their surroundings using their own sensors during normal operation, and cannot effectively detect other nearby objects, making them prone to collisions (e.g., robots bumping into people).

[0003] In related technologies, collaborative robots mainly use visual recognition of the surrounding environment. They mainly use RGB (red, green, blue) images and deep learning technology to identify parts of the human body (hands, arms, or heads), define a range on the image, and determine whether the pixels representing the person are within the defined range, thereby controlling the collaborative robot to slow down or stop.

[0004] However, the above method has the following technical problems:

[0005] 1. RGB images are highly dependent on lighting conditions. Detection schemes based on RGB images cannot function properly when the light is weak or at night.

[0006] 2. Detection results depend on training samples. What objects a camera can recognize depends on what samples are used to train the network. Currently, most applications train on human samples. To recognize other objects, the samples need to be relabeled and retrained. The whole process is cumbersome and time-consuming.

[0007] 3. Training requires high computer performance. Training networks demands high computer performance and typically requires training on a server. Summary of the Invention

[0008] To solve the above-mentioned technical problems, this invention provides a control method for collaborative robots that does not distinguish between people and objects. As long as an obstacle intrudes into a pre-set area, it will trigger deceleration or stop. The entire process does not require prior training, has low requirements for computer computing power, and can work in the dark, thus having high safety.

[0009] The present invention also provides a control device for a collaborative robot.

[0010] The technical solution adopted in this invention is as follows:

[0011] A first aspect of the present invention provides a control method for a collaborative robot, the collaborative robot comprising: a robotic arm and a depth camera mounted on the robotic arm, the method comprising the following steps: performing pose calibration on the depth camera and the robotic arm; filtering the point cloud data acquired by the depth camera to obtain first sparse data; acquiring each joint angle corresponding to the current position of the robotic arm and each joint angle corresponding to a future path point; calculating the current configuration of the robotic arm based on the joint angles corresponding to the current position, and calculating the future configuration of the robotic arm based on the joint angles corresponding to the future path points; deleting all points in the first sparse data corresponding to the current configuration of the robotic arm to obtain second sparse data; and controlling the speed of the robotic arm based on the current configuration, the future configuration, and the second sparse data.

[0012] The control method for collaborative robots proposed in this invention also has the following additional technical features:

[0013] According to one embodiment of the present invention, the robotic arm configuration is specifically calculated according to the following formula:

[0014] T1=joint1; T2=T1*joint2; T3=T2*joint3;…; T6=T5*joint6;

[0015] G_arm = m0 + m1*T1 + m2*T2 + ... + m6*T6; where joint1-joint6 are the rotation matrices corresponding to the first to sixth joint angles of the robotic arm, T1-T6 are the transformation matrices of the first to sixth joint angles of the robotic arm, m0-m6 are the mechanical models of the first to seventh joints of the robotic arm, and G_arm is the configuration of the robotic arm.

[0016] According to one embodiment of the present invention, the speed of the robotic arm is controlled based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data. Specifically, this includes: calculating the distance between each point in the second sparse data and the current configuration of the robotic arm; obtaining the number of points in the second sparse data whose distance to the current configuration of the robotic arm is less than a first preset distance; if the number is greater than the first preset number, controlling the robotic arm to stop; calculating the distance between each point in the second sparse data and the future robotic arm configuration; determining whether there are points in the second sparse data whose distance to the future robotic arm configuration is less than the first preset distance; if so, further determining whether the distance between the future path point and the current position of the robotic arm is less than a second preset distance; if yes, controlling the robotic arm to stop; if no, controlling the robotic arm to decelerate.

[0017] According to one embodiment of the present invention, the depth camera comprises two.

[0018] According to one embodiment of the present invention, a voxel filtering algorithm is used to filter the point cloud data acquired by the depth camera.

[0019] A second aspect of the present invention provides a control device for a collaborative robot, the collaborative robot comprising: a robotic arm and a depth camera mounted on the robotic arm; the device comprising: a calibration module for calibrating the pose of the depth camera and the robotic arm; a filtering module for filtering point cloud data acquired by the depth camera to obtain first sparse data; an acquisition module for acquiring joint angles corresponding to the current position of the robotic arm and joint angles corresponding to future path points; a calculation module for calculating the current robotic arm configuration based on the joint angles corresponding to the current position and calculating the future robotic arm configuration based on the joint angles corresponding to the future path points; a processing module for deleting all points in the first sparse data corresponding to the current robotic arm configuration to obtain second sparse data; and a control module for controlling the speed of the robotic arm based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data.

[0020] The control device for the collaborative robot described above in this invention also has the following additional technical features:

[0021] According to one embodiment of the present invention, the calculation module specifically calculates the robotic arm configuration according to the following formula: T1 = joint1; T2 = T1 * joint2; T3 = T2 * joint3; ...; T6 = T5 * joint6; G_arm = m0 + m1 * T1 + m2 * T2 + ... + m6 * T6; where joint1-joint6 are the rotation matrices corresponding to the first to sixth joint angles of the robotic arm, T1-T6 are the transformation matrices of the first to sixth joint angles of the robotic arm, m0-m6 are the mechanical models of the first to seventh joints of the robotic arm, and G_arm is the robotic arm configuration.

[0022] According to one embodiment of the present invention, the control module is specifically configured to: calculate the distance between each point in the second sparse data and the current configuration of the robotic arm; obtain the number of points in the second sparse data whose distance to the current configuration of the robotic arm is less than a first preset distance; if the number is greater than the first preset number, control the robotic arm to stop; calculate the distance between each point in the second sparse data and the future configuration of the robotic arm; determine whether there are points in the second sparse data whose distance to the future configuration of the robotic arm is less than the first preset distance; if so, further determine whether the distance between the future path point and the current position of the robotic arm is less than a second preset distance; if so, control the robotic arm to stop; if not, control the robotic arm to decelerate.

[0023] According to one embodiment of the present invention, the depth camera comprises two.

[0024] According to one embodiment of the present invention, the filtering module uses a voxel filtering algorithm to filter the point cloud data acquired by the depth camera.

[0025] The beneficial effects of this invention are:

[0026] This invention controls the speed of the robotic arm based on depth camera data and real-time feedback information from the robotic arm, controlling the robotic arm to decelerate or stop. It does not distinguish between people and objects; as long as an obstacle intrudes into the pre-set area, it will trigger deceleration or stopping. The entire process does not require prior training, has low requirements for computer computing power, and can work in the dark, thus having high safety. Attached Figure Description

[0027] Figure 1 This is a flowchart of a control method for a collaborative robot according to an embodiment of the present invention;

[0028] Figure 2 This is a flowchart of a control method for a collaborative robot according to another embodiment of the present invention;

[0029] Figure 3 This is a block diagram of a control device for a collaborative robot according to an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Figure 1 This is a flowchart of a control method for a collaborative robot according to an embodiment of the present invention. The collaborative robot includes: a robotic arm and a depth camera mounted on the robotic arm, such as... Figure 1 As shown, the method includes the following steps:

[0032] S1 performs pose calibration of the depth camera and the robotic arm.

[0033] In embodiments of the present invention, there may be one or two depth cameras, with the second depth camera primarily used to compensate for the blind spots of the first depth camera.

[0034] The specific pose calibration process is as follows: First, the data obtained in real time from the depth camera is converted to the 0 coordinate system of the robotic arm. The conversion formula is as follows: Pa = T_arm_camera * Pc; where Pa is the converted depth camera data, T_arm_camera is the relative pose relationship between the depth camera and the robotic arm after calibration, and Pc is the original depth camera data.

[0035] The T_arm_camera format is as follows: Where R is the rotation matrix from the depth camera coordinate system to the robot arm coordinate system obtained from calibration, and t is the translation matrix from the depth camera coordinate system to the robot arm coordinate system.

[0036] S2 filters the point cloud data acquired by the depth camera to obtain the first sparse data Data1.

[0037] In one embodiment of the present invention, a voxel filtering algorithm is used to filter the point cloud data acquired by the depth camera. Filtering the point cloud data can reduce the density of the point cloud data and remove the ghosting problem caused by incomplete matching of data from two cameras. The main steps are to divide the space into equal-sized grids and to perform statistical analysis or sampling on the camera data in each grid to generate new sparse data.

[0038] S3, obtain the joint angles corresponding to the current position of the robotic arm and the joint angles corresponding to the future path points.

[0039] Joint angles consist of rotation and transformation matrices.

[0040] S4: Calculate the current robotic arm configuration based on the joint angles corresponding to the current position, and calculate the future robotic arm configuration based on the joint angles corresponding to the future path points.

[0041] According to a specific example of the present invention, the robotic arm configuration is calculated specifically according to the following formula:

[0042] T1=joint1; T2=T1*joint2; T3=T2*joint3;…; T6=T5*joint6;

[0043] G_arm=m0+m1*T1+m2*T2+…+m6*T6;

[0044] In this model, joint1-joint6 represent the rotation matrices corresponding to the first to sixth joint angles of the robotic arm, T1-T6 represent the transformation matrices corresponding to the first to sixth joint angles of the robotic arm, m0-m6 represent the mechanical models of the first to seventh joints of the robotic arm, and G_arm represents the configuration of the robotic arm. Specifically, m0-m6 can be the URDF (Unified Robot Description Format) model of the robotic arm.

[0045] Specifically, by substituting the rotation and transformation matrices corresponding to each joint angle of the current position of the robotic arm obtained in step S2 into joint1-joint6 and T1-T6 in the above formula, the current robotic arm configuration can be obtained. By substituting the rotation and transformation matrices corresponding to each joint angle of the future path point obtained in step S2 into joint1-joint6 and T1-T6 in the above formula, the future robotic arm configuration can be calculated for each joint angle of the future path point.

[0046] S5, delete all points in the first sparse data that correspond to the current configuration of the robotic arm, and obtain the second sparse data.

[0047] In other words, based on the current robotic arm configuration, the corresponding points are matched and found in the first sparse data Data1, and all these points are deleted. The deleted first sparse data Data1 is then used as the second sparse data Data2.

[0048] S6 controls the speed of the robotic arm based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data.

[0049] Therefore, the speed of the robotic arm is controlled based on depth camera data and real-time feedback information from the robotic arm, which can control the robotic arm to decelerate or stop. It does not distinguish between people and objects. As long as an obstacle enters the pre-set area, it will trigger deceleration or stop. The whole process does not require prior training, has low requirements for computer computing power, and can work in the dark, making it highly safe.

[0050] The following describes, with specific examples, how to control the speed of a robotic arm based on the current robotic arm configuration, the future robotic arm configuration, and second sparse data.

[0051] According to one embodiment of the present invention, such as Figure 2 As shown, the speed of the robotic arm is controlled based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data. Specifically, this includes:

[0052] S61, calculate the distance between each point in the second sparse data and the current configuration of the robotic arm, and obtain the number of points in the second sparse data whose distance from the current configuration of the robotic arm is less than the first preset distance.

[0053] The first preset distance can be set in advance, for example, it can be 0.05m.

[0054] S62, if the quantity is greater than the first preset quantity, control the robotic arm to stop.

[0055] The first preset quantity can be set in advance, for example, it can be 10.

[0056] S63, calculate the distance between each point in the second sparse data and the future robotic arm configuration.

[0057] S64, determine whether there are any points in the second sparse data whose distance to the future robotic arm configuration is less than the first preset distance. If there are, proceed to step S65; otherwise, return to step S61.

[0058] S65, further determine whether the distance between the future path point and the current position of the robotic arm is less than the second preset distance. If yes, proceed to step S66; otherwise, proceed to step S67.

[0059] The second preset distance is positively correlated with the current operating speed of the robotic arm. The specific relationship between the second preset distance and the current operating speed of the robotic arm is set and stored in advance and can be called directly.

[0060] S66 controls the robotic arm to stop.

[0061] S67 controls the deceleration of the robotic arm.

[0062] Specifically, if the point cloud data acquired by the depth camera contains additional points that are close to the current configuration of the robotic arm and the number exceeds a certain threshold, it can be assumed that an object has collided with the robotic arm, and a stopping speed is immediately issued to control the robotic arm to stop. If the point cloud data acquired by the depth camera contains points that are close to the robotic arm configuration corresponding to a future path point, it is further determined that the future path point is close to the current position of the robotic arm. If so, it means that an object collided with the robotic arm when it reached the future path point, and the robotic arm is directly controlled to stop. If not, it means that there is a risk of collision when the robotic arm reaches the future path point, and the robotic arm is controlled to decelerate.

[0063] In summary, the control method for the collaborative robot according to the embodiments of the present invention controls the speed of the robotic arm based on depth camera data and real-time feedback information from the robotic arm, controlling the robotic arm to decelerate or stop. It does not distinguish between people and objects; as long as an obstacle intrudes into the pre-set area, deceleration or stopping is triggered. The entire process does not require prior training, has low requirements for computer computing power, and can work in the dark, thus having high safety.

[0064] Corresponding to the control method for collaborative robots described above, this invention also proposes a control device for collaborative robots. Since the device embodiments of this invention correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to in the method embodiments described above, and will not be repeated here.

[0065] Figure 3 This is a block diagram of a control device for a collaborative robot according to an embodiment of the present invention. The collaborative robot includes: a robotic arm and a depth camera mounted on the robotic arm, such as... Figure 3 As shown, the device includes: calibration module 1, filtering module 2, acquisition module 3, calculation module 4, processing module 5, and control module 6.

[0066] The calibration module 1 is used to calibrate the pose of the depth camera and the robotic arm; the filtering module 2 is used to filter the point cloud data acquired by the depth camera to obtain the first sparse data; the acquisition module 3 is used to acquire the joint angles corresponding to the current position of the robotic arm and the joint angles corresponding to the future path points; the calculation module 4 is used to calculate the current robotic arm configuration based on the joint angles corresponding to the current position and the future robotic arm configuration based on the joint angles corresponding to the future path points; the processing module 5 is used to delete all points in the first sparse data that correspond to the current configuration of the robotic arm to obtain the second sparse data; and the control module 6 is used to control the speed of the robotic arm based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data.

[0067] According to one embodiment of the present invention, the calculation module 4 specifically calculates the robotic arm configuration according to the following formula: T1 = joint1; T2 = T1 * joint2; T3 = T2 * joint3; ...; T6 = T5 * joint6; G_arm = m0 + m1 * T1 + m2 * T2 + ... + m6 * T6; where joint1-joint6 are the rotation matrices corresponding to the first to sixth joint angles of the robotic arm, T1-T6 are the transformation matrices of the first to sixth joint angles of the robotic arm, m0-m6 are the mechanical models of the first to seventh joints of the robotic arm, and G_arm is the robotic arm configuration.

[0068] According to one embodiment of the present invention, the control module 6 is specifically configured to: calculate the distance between each point in the second sparse data and the current configuration of the robotic arm; obtain the number of points in the second sparse data whose distance to the current configuration of the robotic arm is less than a first preset distance; if the number is greater than the first preset number, control the robotic arm to stop; calculate the distance between each point in the second sparse data and the future configuration of the robotic arm; determine whether there are points in the second sparse data whose distance to the future configuration of the robotic arm is less than the first preset distance; if there are, further determine whether the distance between the future path point and the current position of the robotic arm is less than the second preset distance; if yes, control the robotic arm to stop; if no, control the robotic arm to decelerate.

[0069] According to one embodiment of the present invention, the depth camera includes two cameras.

[0070] According to one embodiment of the present invention, the filtering module 2 uses a voxel filtering algorithm to filter the point cloud data acquired by the depth camera.

[0071] In summary, the control device for the collaborative robot according to the embodiments of the present invention controls the speed of the robotic arm based on depth camera data and real-time feedback information from the robotic arm, controlling the robotic arm to decelerate or stop. It does not distinguish between people and objects; as long as an obstacle intrudes into the pre-set area, it will trigger deceleration or stopping. The entire process does not require prior training, has low requirements for computer computing power, and can work in the dark, thus having high safety.

[0072] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A control method for a collaborative robot, characterized in that, The collaborative robot includes a robotic arm and a depth camera mounted on the robotic arm. The method includes the following steps: The depth camera and the robotic arm are pose-calibrated. The point cloud data acquired by the depth camera is filtered to obtain the first sparse data; Obtain the joint angles corresponding to the current position of the robotic arm and the joint angles corresponding to the future path points; The current robotic arm configuration is calculated based on the joint angles corresponding to the current position, and the future robotic arm configuration is calculated based on the joint angles corresponding to the future path points. Delete all points in the first sparse data that correspond to the current robotic arm configuration to obtain the second sparse data; The speed of the robotic arm is controlled based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data; Based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data, the speed of the robotic arm is controlled, specifically including: Calculate the distance between each point in the second sparse data and the current configuration of the robotic arm, obtain the number of points in the second sparse data whose distance to the current configuration of the robotic arm is less than a first preset distance, and if the number is greater than the first preset number, control the robotic arm to stop. Calculate the distance between each point in the second sparse data and the future robotic arm configuration, determine whether there are any points in the second sparse data whose distance to the future robotic arm configuration is less than a first preset distance, if there are, further determine whether the distance between the future path point and the current position of the robotic arm is less than a second preset distance, if yes, control the robotic arm to stop, if no, control the robotic arm to decelerate.

2. The control method for a collaborative robot according to claim 1, characterized in that, The robotic arm configuration is calculated using the following formula: T1 = joint1; T2 = T1 joint2;T3 = T2 joint3; T6 = T5 joint6; G_arm = m0 + m1 T1 + m2 T2 + …+m6 T6; Where joint1-joint6 are the rotation matrices corresponding to the first to sixth joint angles of the robotic arm, T1-T6 are the transformation matrices corresponding to the first to sixth joint angles of the robotic arm, m0-m6 are the mechanical models of the first to seventh joints of the robotic arm URDF, and G_arm is the configuration of the robotic arm.

3. The control method for a collaborative robot according to claim 1, characterized in that, The depth camera comprises two.

4. The control method for a collaborative robot according to claim 1, characterized in that, A voxel filtering algorithm is used to filter the point cloud data acquired by the depth camera.

5. A control device for a collaborative robot, characterized in that, The collaborative robot includes: a robotic arm and a depth camera mounted on the robotic arm; the device includes: A calibration module is used to perform pose calibration on the depth camera and the robotic arm; A filtering module is used to filter the point cloud data acquired by the depth camera to obtain the first sparse data. The acquisition module is used to acquire the joint angles corresponding to the current position of the robotic arm and the joint angles corresponding to the future path points; The calculation module is used to calculate the current robotic arm configuration based on the joint angles corresponding to the current position, and to calculate the future robotic arm configuration based on the joint angles corresponding to the future path points. The processing module is used to delete all points in the first sparse data that correspond to the current robotic arm configuration to obtain the second sparse data; A control module is configured to control the speed of the robotic arm based on the current robotic arm configuration, the future robotic arm configuration, and the second sparse data. The control module is specifically used for: Calculate the distance between each point in the second sparse data and the current configuration of the robotic arm, obtain the number of points in the second sparse data whose distance to the current configuration of the robotic arm is less than a first preset distance, and if the number is greater than the first preset number, control the robotic arm to stop. Calculate the distance between each point in the second sparse data and the future robotic arm configuration, determine whether there are any points in the second sparse data whose distance to the future robotic arm configuration is less than a first preset distance, if there are, further determine whether the distance between the future path point and the current position of the robotic arm is less than a second preset distance, if yes, control the robotic arm to stop, if no, control the robotic arm to decelerate.

6. The control device for a collaborative robot according to claim 5, characterized in that, The calculation module specifically calculates the robotic arm configuration according to the following formula: T1 = joint1; T2 = T1 joint2;T3 = T2 joint3; T6 = T5 joint6; G_arm = m0 + m1 T1 + m2 T2 + …+m6 T6; Where joint1-joint6 are the rotation matrices corresponding to the first to sixth joint angles of the robotic arm, T1-T6 are the transformation matrices of the first to sixth joint angles of the robotic arm, m0-m6 are the mechanical models of the first to seventh joints of the robotic arm URDF, and G_arm is the configuration of the robotic arm.

7. The control device for a collaborative robot according to claim 5, characterized in that, The depth camera comprises two.

8. The control device for a collaborative robot according to claim 5, characterized in that, The filtering module uses a voxel filtering algorithm to filter the point cloud data acquired by the depth camera.

Images