A collaborative robot control system and method

Abstract

The present disclosure relates to a kind of collaborative robot control system and method, the system is based on the method implementation.A kind of collaborative robot control method, the method includes the following steps: obtaining image, the image has depth information, the image includes collaborator and collaborative robot same work scene;Based on the acquisition frequency of depth information and image, the joint pose and speed of collaborator are estimated;Based on the joint pose and speed, the impedance control model of current collaborator is predicted;In combination with the impedance control model of current collaborator and work scene in image, the impedance control model of collaborative robot is obtained, control instruction is generated and sent to collaborative robot, and the control instruction is used to drive collaborative robot work.The method of the present disclosure considers the difference of collaborative task and collaborator, and is widely applicable;It can be dynamically adjusted robot ontology in real time, and control is flexible and convenient;And collaborator does not need to wear external sensor, and operation is simple and efficient.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of collaborative robot control, in particular to a collaborative robot control system and method. BACKGROUND

[0002] Collaborative robots have great potential for growth in both industrial and non-industrial fields due to their safety, flexibility, and ease of use as an important supplement to traditional industrial robots. Collaborative robots generally have low rigidity for safety reasons, and to improve the accuracy of robot motion and reduce motion-induced jitter, more attention needs to be paid to the dynamics of the robot body. In human-robot collaboration, it is often necessary to dynamically adjust the impedance parameters. Adjusting the impedance parameters involves system stability, performance indicators, and even estimating the operator's intention. General collaborative robot impedance control methods often only consider drag teaching applications, and do not consider other application scenarios or distinguish between operators, so they cannot function in a wide range of application scenarios. SUMMARY

[0003] To solve the above problems, the present disclosure provides a collaborative robot control method. The technical solution of the present disclosure is as follows:

[0004] A collaborative robot control method, the method comprising the following steps:

[0005] S100, acquiring an image, the image having depth information, the image including a collaborative worker and a collaborative robot working in a working scene;

[0006] S200, estimating the joint pose and speed of the collaborative worker based on the depth information and the image acquisition frequency;

[0007] S300, predicting an impedance control model of the current collaborative worker based on the joint pose and speed;

[0008] S400, obtaining an impedance control model of the collaborative robot in combination with the impedance control model of the current collaborative worker and the working scene in the image, generating and sending a control instruction to the collaborative robot, the control instruction being used to drive the collaborative robot to work.

[0009] On the other hand, based on the above method, the present disclosure provides a collaborative robot control system, the collaborative robot control system comprising an outer loop control subsystem and an inner loop control subsystem;

[0010] The outer loop control subsystem comprises an image module and an impedance model module;

[0011] The inner loop control subsystem comprises a drive-integrated collaborative robot drive control module;

[0012] The driving integrated collaborative robot control module comprises a collaborative robot control module and a joint driving module.

[0013] The image module is configured to acquire images, the images having depth information and including a work scene of a collaborator and a collaborative robot.

[0014] The image module outputs the images to the impedance model module.

[0015] The impedance model module estimates the joint pose and velocity of the collaborator based on the depth information and the acquisition frequency of the images, predicts an impedance control model of the current collaborator based on the joint pose and velocity, and acquires an impedance control model of the collaborative robot based on the impedance control model of the current collaborator and the work scene in the images, and outputs the impedance control model to the collaborative robot control module.

[0016] The collaborative robot control module generates and sends control instructions to the joint driving module based on the received impedance control model, and the joint driving module drives the collaborative robot to work.

[0017] Compared with the prior art, the present disclosure has the following advantages:

[0018] (1) The differences between different collaborators and different collaborative tasks are considered, and the application range is wide.

[0019] (2) The impedance parameters of the collaborative robot can be dynamically and real-timely adjusted, so that the human-machine collaboration is flexible and convenient, and can be used in various scenes.

[0020] (3) The collaborator does not need to wear external sensors, and the operation is simple and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 Fig. 1 shows a system schematic diagram of an embodiment of the present disclosure.

[0022] Figure 2 Fig. 2 shows a structural schematic diagram of an embodiment of the present disclosure.

[0023] Figure 3 Fig. 3 shows a scene application schematic diagram in an embodiment of the present disclosure.

[0024] 1, image module; 2, impedance model module; 3, driving integrated collaborative robot control module; 4, collaborative robot; 5, collaborator. DETAILED DESCRIPTION

[0025] With reference to the drawings of the embodiments of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present disclosure.

[0026] With reference to Figure 1 The embodiments of the present disclosure provide a collaborative robot control method, which comprises the following steps:

[0027] S100, acquiring an image, the image having depth information, the image comprising a collaborator and a collaborative robot in a work scene;

[0028] S200, estimating joint poses and velocities of the collaborator based on the depth information and a frequency of acquisition of the image;

[0029] S300, predicting an impedance control model of the current collaborator based on the joint poses and velocities;

[0030] S400, acquiring an impedance control model of the collaborative robot in combination with the impedance control model of the current collaborator and the work scene in the image, generating and sending a control instruction to the collaborative robot, the control instruction being used to drive the collaborative robot to work.

[0031] In this embodiment, the image acquisition device can be a three-dimensional camera with depth information, or other devices capable of acquiring images with depth information. The image can be part or all of a video. The image can be acquired online in real time, or acquired in advance. For the image acquired in real time, the collaborative robot in the present embodiment can adjust the impedance parameters in real time, and has strong adaptability and high flexibility. For the image acquired in advance, the collaborative robot in the present embodiment can adjust the impedance parameters in advance, and directly enter the collaborative work state.

[0032] In this embodiment, the acquisition time is obtained by the acquisition frequency of the image, such as by the acquisition frequency of the image, the time is obtained as 0.1 seconds. Then the depth information in the image is obtained by analyzing the image of human-robot cooperation, combined with the obtained time, the joint pose and speed of the cooperators can be estimated, and then the impedance control model of the current cooperators can be obtained. The speed includes linear speed and angular speed. Using the impedance control model of the cooperators, combined with the human-robot cooperation scene identified from the image, the impedance control model of the collaborative robot can be predicted. According to the obtained impedance model of the collaborative robot, the impedance parameters of the collaborative robot can be dynamically adjusted. Under this control method, the cooperators no longer need to wear external sensors, and the operation is simple and efficient. When the cooperators or the cooperation tasks are changed, the human-robot cooperation image is re-acquired and analyzed, and the impedance parameters of the collaborative robot can be dynamically adjusted. Even if the cooperators and the cooperation tasks are different, the collaborative robot can still dynamically adjust the impedance parameters and operate normally.

[0033] The estimation of the joint pose and speed of the cooperators can be calculated by self-programming or using existing algorithms. When using existing algorithms, deep learning algorithms are preferred, especially convolutional neural network algorithms.

[0034] The human-robot cooperation scene identified from the image can be manually identified or identified using algorithms, and the latter is preferred. More preferably, convolutional neural networks are used for identification, and the joint pose and speed of the cooperators are used as input, and the impedance parameters are used as output.

[0035] The impedance model of the collaborative robot can be obtained by setting or adjusting the impedance parameters based on the impedance model of the collaborator, or by selecting the most suitable impedance model from the stored impedance models. For the latter, a collaborative task model library can be set up, in which impedance control models are stored to meet different collaborators and different collaborative tasks, and meet different collaborative scenarios. The impedance control models in the collaborative task model library can be added, deleted or changed as needed. If the current collaborative robot only needs to meet one collaborative scenario, only one human-robot collaborative impedance control model needs to be stored in this collaborative task model library, such as for the original application scenario that only needs to drag the demonstration. When the collaborative robot needs to be applied to other application scenarios, the human-robot impedance collaboration model for the corresponding scenario can be added to the collaborative task library. Further, when the collaborator needs to be replaced, the impedance control model after the replacement of the collaborator can also be stored or replaced in the collaborative task model library. In this case, according to the current image collected, the human-robot impedance collaboration model cooperating with the current collaborator is re-analyzed and identified, so that the impedance parameters of the collaborative robot are adjusted, so that the collaborative robot enters the collaborative working state in real time or waits to enter the collaborative working state. For the replacement of the collaborative task, the image is re-collected, the human-robot impedance collaboration model required by the current collaborative task is analyzed and identified, and the impedance parameters of the collaborative robot are adjusted, so that the collaborative robot enters the collaborative working state in real time or waits to enter the collaborative working state.

[0036] In one embodiment, the acquired image is analyzed using a deep learning algorithm. Preferably, a convolutional neural network is used to estimate the three-dimensional coordinates and velocities of the human joint key points, and at the same time another convolutional neural network is used to identify the human-robot collaboration scene to obtain the impedance collaboration model. When identifying the human-robot collaboration scene, the obtained human joint pose and velocity are used as input, and the impedance parameters are used as output, and then the optimal impedance control model for human-robot collaboration is recommended from the collaborative task model library. In other embodiments, the two convolutional neural networks can be a combination of different deep learning algorithms, or a combination of other same deep learning algorithms.

[0037] Using the impedance control model, the impedance parameters of the collaborative robot can be adjusted. Specifically, the impedance control model is used to plan the path of the collaborative robot, and then the trajectory of the collaborative robot is planned in combination with the physical constraints such as torque, power, and joint angle limit of the collaborative robot, and the dynamics model. Finally, based on the obtained trajectory, control commands are generated and sent to the collaborative robot.

[0038] In one embodiment, based on the above-mentioned collaborative robot control method, the present disclosure realizes a collaborative robot control system. As Figure 2As shown, the collaborative robot control system comprises an outer loop control subsystem, an inner loop control subsystem; the outer loop control subsystem comprises an image module, an impedance model module; the inner loop control subsystem comprises a drive-integrated collaborative robot drive control module; the drive-integrated collaborative robot drive control module comprises a collaborative robot control module, a joint drive module; the image module is used to acquire an image, the image has depth information, and the image includes a collaborator and a collaborative robot in a work scene; the image module outputs the image to the impedance model module; the impedance model module estimates the joint pose and velocity of the collaborator based on the depth information and the image acquisition frequency first; then predicts the impedance control model of the current collaborator based on the joint pose and velocity; finally, based on the impedance control model of the current collaborator and the work scene in the image, the impedance control model of the collaborative robot is acquired, and the impedance control model is output to the collaborative robot control module; the collaborative robot control module generates and sends control instructions to the joint drive module of the collaborative robot using the received impedance control model, and the joint drive module drives the collaborative robot to work.

[0039] In order to better understand the implementation of the system in the present disclosure, a common application scenario is shown in Figure 3 The connection between each module in the schematic diagram can be wired or wireless, and those skilled in the art can clearly understand that the present disclosure can be realized by means of software and necessary general hardware, and of course it can also be realized by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily realized by corresponding hardware, and the specific hardware structure for realizing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the present disclosure, software program implementation is a better implementation mode in most cases.

[0040] In Figure 3In the specific embodiment, the image module 1 collects images or image videos by using a three-dimensional camera with depth information. The image collection can be real-time or offline. The image module 1 is connected to the impedance model module 2, which constitutes the outer loop control subsystem of the system. The impedance model module 2 comprises a cooperative environment intelligent algorithm unit and a cooperative task model library. The algorithms that can be used in the cooperative environment algorithm unit include convolutional neural networks and other deep learning algorithms. The cooperative task model library comprises at least one human-machine cooperative impedance control model. The cooperative environment algorithm unit analyzes the images to obtain the depth information and the collection frequency of the images, and estimates the joint pose and speed of the collaborator. The speed includes linear speed and angular speed. In the estimation, the convolutional neural network is preferably used, and other deep learning algorithms can also be used. Using the joint pose and speed, in combination with the work scene identified from the images, the impedance parameters of the impedance control model can be obtained, and the optimal impedance control model for the current human-machine cooperation can be obtained from the cooperative task model library. In identifying the work scene, the convolutional neural network is preferably used, and other deep learning algorithms can also be used.

[0041] The impedance model module 2 is connected to the drive-integrated collaborative robot drive module 3. The impedance model module 2 recommends the optimal impedance control model for human-machine cooperation based on the obtained human impedance model, and sends it to the drive-integrated collaborative robot drive module 3. The drive-integrated collaborative robot control module 3 uses the received impedance control model to send control instructions to the joint drive module, and the joint drive module drives the collaborative robot 4 to work with the collaborator 5.

[0042] The drive-integrated collaborative robot drive module 3 comprises a collaborative robot control module and a joint drive module.

[0043] Further, the collaborative robot control module comprises a path planning unit. The path planning unit uses the received impedance control model to plan the motion path of the collaborative robot.

[0044] The collaborative robot control model further comprises a trajectory planning unit. The trajectory planning unit uses the motion path, in combination with the physical constraints such as torque, power, and joint angle limit of the collaborative robot, and the dynamics model, to plan the trajectory of the collaborative robot. Based on the obtained trajectory, control instructions are generated and sent to the collaborative robot.

[0045] As can be seen from the working mode of the above control system, the collaborator no longer needs to wear external sensors, and the operation is simple and efficient. When the collaborator is replaced or the collaborative task is changed, the human-machine cooperation image is re-collected and analyzed, and the impedance parameters of the collaborative robot are dynamically adjusted in real time. Even if the collaborator and the collaborative task are different, the collaborative robot can still work normally.

[0046] Although the embodiments of the present disclosure are described above with reference to the accompanying drawings, the present disclosure is not limited to the above-described specific embodiments and application fields, and the above-described specific embodiments are merely illustrative and instructive, but not restrictive. A person of ordinary skill in the art can make many forms under the inspiration of the present specification and without departing from the scope protected by the claims of the present disclosure, and these all belong to the protection of the present disclosure.

Claims

1. A method of collaborative robot control, characterized by: The method comprises the following steps: S100, acquiring an image, the image having depth information, the image comprising a current collaborator and a collaborative robot and a work scene; S200, estimating joint poses and speeds of the current collaborator based on the depth information and a frequency of acquisition of the image; S300, predicting an impedance control model of the current collaborator based on the joint poses and speeds, and acquiring an impedance control model of the collaborative robot from a collaborative task model library in combination with the impedance control model of the current collaborator and the work scene in the image; S400, dynamically adjusting impedance parameters of the collaborative robot by using the acquired impedance control model of the collaborative robot, and generating and sending a control instruction to the collaborative robot, the control instruction being used to drive the collaborative robot to work.

2. The method of claim 1, wherein: The method used for the estimation comprises a convolutional neural network or other deep learning algorithm.

3. The method of claim 1, wherein: The impedance control model of the collaborative robot is stored in a collaborative task model library; the collaborative task model library is used to store one or more impedance control models for enabling the collaborative robot to work.

4. The method of claim 1, wherein: The work scene is acquired by a convolutional neural network or other deep learning algorithm.

5. The method of claim 1, wherein: The step S400 further comprises the following before the control instruction is generated: S401, path planning for the collaborative robot; S402, after the path planning, trajectory planning for the collaborative robot in combination with a dynamics model of the collaborative robot and physical constraints.

6. A collaborative robot control system characterized by: The collaborative robot control system comprises an outer loop control subsystem and an inner loop control subsystem; The outer loop control subsystem comprises an image module and an impedance model module; The inner loop control subsystem comprises a drive-integrated collaborative robot drive control module; The drive-integrated collaborative robot drive control module comprises a collaborative robot control module and a joint drive module; The image module is used to acquire an image, the image having depth information, the image comprising a current collaborator and a collaborative robot and a work scene; The image module outputs the image to the impedance model module; The impedance model module estimates joint poses and speeds of the current collaborator based on the depth information and a frequency of acquisition of the image, predicts an impedance control model of the current collaborator based on the joint poses and speeds, and acquires an impedance control model of the collaborative robot based on the impedance control model of the current collaborator and the work scene in the image, and outputs the impedance control model to the collaborative robot control module; The collaborative robot control module dynamically adjusts impedance parameters of the collaborative robot by using the received impedance control model, and generates and sends a control instruction to the joint drive module of the collaborative robot, the joint drive module driving the collaborative robot to work.

7. The system according to claim 6, characterized in that: The impedance model module acquires the impedance control model of the current human-robot collaboration through a collaborative environment algorithm unit; The collaborative environment algorithm unit comprises a convolutional neural network or other deep learning algorithm.

8. The system according to claim 6, characterized in that: The impedance model module further comprises a collaborative task model library; The collaborative task model library is configured to store one or more impedance control models for a collaborative robot job.

9. The system of claim 6, wherein: The collaborative robot control module comprises a path planning unit. The path planning unit plans a motion path of the collaborative robot using the impedance control model output by the impedance model module.

10. The system of claim 9, wherein: The collaborative robot control module comprises a trajectory planning unit, which plans a trajectory of the collaborative robot using the motion path in combination with a dynamics model and physical constraints of the collaborative robot, and generates and sends a control instruction to the collaborative robot based on the obtained trajectory.

Images