Robot teaching method, system, and computer-readable storage medium

Abstract

The application relates to the field of robots, in particular to a teaching method of a robot, a robot teaching system and a computer readable storage medium. The teaching method of the robot comprises the following steps: a robot pose detection step, a first sensor arranged on the robot detects pose information of the robot; a terminal pose detection step, a second sensor arranged on a terminal detects pose information of the terminal; a simulation step, according to the pose information of the robot and the pose information of the terminal, a transformation matrix transforming the robot coordinate system into the terminal coordinate system is determined, and a simulation model of the robot is displayed on a terminal interface of the terminal, the pose of the simulation model is consistent with the pose of the robot, and the visual angle of the simulation model is consistent with the visual angle of the terminal for observing the robot; and a control movement step, based on the adjustment of the pose of the simulation model and the transformation matrix, the pose of the robot is adjusted so that the pose of the robot is consistent with the pose of the simulation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots, in particular to a teaching method of a robot, a robot teaching system and a computer readable storage medium. BACKGROUND

[0002] With the development of technology, robots have been more and more widely used. The coordinate system of a robot is the basis of motion control of the robot. Before a user operates the robot, the user needs to have certain understanding of the coordinate system of the robot. Common coordinate systems of the robot include a base coordinate system, a tool coordinate system and a wrist coordinate system, etc.

[0003] According to different application scenarios and installation requirements, the specific installation mode of the robot will also change. Typical installation modes of the robot include normal installation, side installation and upside-down installation. At present, various commonly used coordinate systems have certain limitations in different application scenarios. For example, the base coordinate system of an upside-down installed robot is opposite to the Z-axis of the geodetic coordinate system. At this time, it is difficult to set parameters based on the base coordinate system and it is easy to make parameter setting errors.

[0004] In the installation process of the robot of the prior art, one is that the user selects a fixed installation angle closest to the actual installation angle of the robot. At this time, several optional installation angles are provided in advance, such as 0°, ±45°, ±90, etc. The installation angle selected by this method is easy to deviate from the actual installation angle of the robot, thereby affecting the normal work of the robot. Another is that the user manually measures the actual installation angle of the robot. The process of manual measurement is extremely tedious and is easy to have errors. Both of the two methods need the user to manually input parameters. Before the robot is driven, the user needs to set the installation angle of the robot in the control system. Then, the robot control system determines various motion parameters according to the input installation angle. Finally, the robot is driven to move according to the various motion parameters. When the user operates the robot, the user needs to have sufficient understanding of the coordinate system used by the robot to be able to operate. When the user input fails, the robot may not work normally, which has certain safety risks.

[0005] In summary, there is an urgent need in the field to provide a teaching method of a robot and a robot teaching system, so that the user does not need to consider the coordinate system used by the robot and the installation mode, and can more intuitively, accurately and conveniently control the movement of the robot. SUMMARY

[0006] To address the problem that existing robots, due to their different coordinate systems and installation methods, make it difficult for users to intuitively, accurately, and conveniently control robot movement, this invention provides a robot teaching method, including: a robot pose detection step, in which a first sensor installed on the robot detects the robot's pose information; a terminal pose detection step, in which a second sensor installed on the terminal detects the terminal's pose information; a simulation step, in which, based on the robot's pose information and the terminal's pose information, a transformation matrix is ​​determined to transform the robot's coordinate system into the terminal's coordinate system, and a simulation model of the robot is displayed on the terminal's interface, wherein the pose of the simulation model is consistent with the robot's pose, and the viewpoint of the simulation model is consistent with the viewpoint of the robot observed from the terminal; and a movement control step, in which, based on the adjustment of the simulation model's pose and the transformation matrix, the robot's pose is adjusted to ensure that the robot's pose is consistent with the pose of the simulation model.

[0007] According to the above technical solution, in the robot pose detection step, the robot pose information is detected by the first sensor. The user can obtain the robot pose information without manually measuring the robot, thereby avoiding the errors or mistakes that may be caused by manual measurement. While ensuring accuracy, it is more convenient and faster to obtain the robot pose information.

[0008] In the simulation step, a simulation model is used to simulate the real robot. Based on the pose information of the robot and the terminal detected by the first and second sensors, after processing and calculation, the pose of the simulation model is made consistent with the pose of the robot without user configuration, and the viewpoint of the simulation model is consistent with the viewpoint of the robot observed from the terminal. In the movement control step, the user can intuitively control the robot's pose by operating the simulation model on the terminal interface. The user does not need to know the current coordinate system or installation method of the robot; they only need to control the movement of the simulation model from their own perspective to control the robot's movement, thus enabling more accurate and convenient robot movement control.

[0009] Preferably, the simulation step includes the following steps: determining a first transformation matrix between the world coordinate system and the robot coordinate system; determining a second transformation matrix between the world coordinate system and the terminal coordinate system; and determining a transformation matrix based on the first and second transformation matrices.

[0010] According to the above technical solution, by determining the first transformation matrix and the second transformation matrix, the robot coordinate system and the terminal coordinate system can use the world coordinate system as a reference to determine the transformation matrix, so that the robot's pose can be expressed in the terminal coordinate system, thus achieving the unification of the coordinate system.

[0011] Preferably, the first transformation matrix for transforming the robot coordinate system to the world coordinate system is T, and the second transformation matrix for transforming the terminal coordinate system to the world coordinate system is Ti. e T e -1 It is T e The inverse matrix of the matrix, the direction vector in the robot coordinate system is v, and the direction vector v transforms into the direction vector v in the terminal coordinate system. e v e =T e -1 *T*v.

[0012] According to the above technical solution, the direction vector v in the robot coordinate system is transformed to the world coordinate system, and then transformed to the direction vector v in the terminal coordinate system. e This allows the robot's pose information to be ultimately expressed using the terminal coordinate system. Since the poses of the robot and the terminal are expressed using the same coordinate system, it facilitates the analysis and judgment of their pose relationship, and makes robot simulation and operation easier.

[0013] Preferably, both the first sensor and the second sensor include an inertial sensor and a geomagnetic sensor.

[0014] According to the above technical solution, the inertial sensor can detect three-dimensional acceleration information, and the geomagnetic sensor can detect geomagnetic direction information. Based on the three-dimensional acceleration information and the geomagnetic direction information, the pose state can be calculated without the user actually measuring the robot's pose, which helps to improve the accuracy and efficiency of robot pose calculation.

[0015] Preferably, x, y, z are the three-dimensional acceleration information detected by the inertial sensor in the first sensor, a, b, c are the geomagnetic direction information detected by the geomagnetic sensor in the first sensor, and the first transformation matrix T is...

[0016]

[0017] According to the above technical solution, based on the three-dimensional acceleration information detected by the inertial sensor installed on the robot and the geomagnetic direction information detected by the geomagnetic sensor, the first transformation matrix can be calculated, thereby transforming the robot from being expressed in the robot coordinate system to being expressed in the world coordinate system.

[0018] Preferably, x1, y1, z1 are the three-dimensional acceleration information detected by the inertial sensor in the second sensor, a1, b1, c1 are the geomagnetic direction information detected by the geomagnetic sensor in the second sensor, and the second transformation matrix T e for

[0019]

[0020] According to the above technical solution, based on the three-dimensional acceleration information detected by the inertial sensor installed on the terminal and the geomagnetic direction information detected by the geomagnetic sensor, the second transformation matrix can be calculated, thereby transforming the coordinate information of the terminal from being expressed in the terminal coordinate system to being expressed in the world coordinate system.

[0021] Preferably, the pose information includes any one or more of position information, orientation information, and attitude information.

[0022] According to the above technical solution, position information, orientation information, and attitude information can be used to determine the specific pose of the robot or terminal, which helps to improve the accuracy of robot control.

[0023] The present invention also provides a robot teaching system, comprising: a robot having a first sensor that detects the robot's pose information; a terminal having a second sensor and a display, the second sensor detecting the terminal's pose information, the display displaying a simulation model of the robot, the pose of the simulation model being consistent with the robot's pose, and the viewpoint of the simulation model being consistent with the viewpoint of the terminal observing the robot; a storage unit storing a transformation matrix for transforming the robot's coordinate system to the terminal's coordinate system, the transformation matrix being determined based on the robot's pose information and the terminal's pose information; and a control and movement unit adjusting the robot's pose based on the adjustment of the simulation model's pose and the transformation matrix, so that the robot's pose is consistent with the simulation model's pose.

[0024] The present invention also provides a computer-readable storage medium storing processor-readable instructions, wherein one or more processors execute the robot according to any of the above-described technical solutions by running the processor-readable instructions. Attached Figure Description

[0025] Figure 1 This is a control module diagram of the robot teaching system according to the first embodiment of the present invention.

[0026] Figure 2 This is a flowchart of the robot teaching method according to the second embodiment of the present invention.

[0027] Figure 3 This is a block diagram of the robot teaching system according to the second embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of the robot and terminals in different positions according to the second embodiment of the present invention.

[0029] Reference numerals: 100 Robot Teaching System; 1 Robot; 2 First Sensor; 3 Terminal; 4 Second Sensor; 5 Display; 6 Storage Unit; 7 Control and Motion Unit; 8 Control Cabinet; 9 Terminal Interface; 10 Simulation Model; {A} Robot Coordinate System; {B} Terminal Coordinate System; {C} World Coordinate System; M First Position; N Second Position; P Third Position. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0031] First Implementation Method

[0032] Figure 1 This is a control module diagram of the robot teaching system 100 according to the first embodiment of the present invention, as shown below. Figure 1 As shown, this embodiment provides a robot teaching system 100, which includes a robot 1 and a terminal 3. In this embodiment, the terminal 3 can be a teaching pendant, or a device such as a mobile phone or tablet, and is not specifically limited here.

[0033] Furthermore, robot 1 has a first sensor 2 for detecting its pose information. Terminal 3 has a second sensor 4 for detecting its pose information. Both the first sensor 2 and the second sensor 4 include an inertial sensor and a geomagnetic sensor, whereby the inertial sensor can detect three-dimensional acceleration information and the geomagnetic sensor can detect geomagnetic direction information. Based on the three-dimensional acceleration information and the geomagnetic direction information, the specific pose can be calculated without requiring the user to actually measure the pose of robot 1 or terminal 3, which helps improve the accuracy and efficiency of pose calculation.

[0034] Specifically, pose information includes any one or more of position information, orientation information, and attitude information. The position information and orientation information can be used to locate the robot 1 or the terminal 3, and the attitude information can be used to obtain the specific state of the robot 1 or the terminal 3.

[0035] In this embodiment, the first sensor 2 is disposed on the base of the robot 1. In other embodiments of the present invention, the first sensor 2 may also be disposed at other locations on the robot 1, and there may be multiple first sensors 2, thereby enabling more accurate detection of the pose information of different parts of the robot 1.

[0036] In other embodiments of the present invention, the first sensor 2 and the second sensor 4 are not limited to the inertial sensor and geomagnetic sensor mentioned above, but may also include other types of sensors, and the detected information is not limited to three-dimensional acceleration information and geomagnetic direction information.

[0037] Furthermore, terminal 3 also includes a display 5, which displays terminal interface 9. On terminal interface 9, the user can see a simulation model 10 of robot 1. The pose of simulation model 10 is consistent with the pose of robot 1, and the viewpoint of simulation model 10 is consistent with the viewpoint from which robot 1 is observed at terminal 3.

[0038] In this embodiment, the simulation model 10 is a three-dimensional model of the robot 1, which is the same as or similar to the actual structure of the robot 1, allowing the user to control and adjust the simulation model 10 more intuitively and conveniently. In other embodiments of the present invention, the simulation model 10 is not limited to a three-dimensional model of the robot 1, and can also be other types of simulation models 10, such as URDF models (Unified Robot Description Format), etc., which are not specifically limited here.

[0039] In this embodiment, the robot teaching system 100 also includes a control cabinet 8, and the robot 1 is connected to the control cabinet 8 via a communication cable. The control cabinet 8 is equipped with a wireless router, and the terminal 3 has a network communication module, thereby enabling the terminal 3 to communicate with the control cabinet 8 and send control commands to the robot 1 through the control cabinet 8.

[0040] Furthermore, the control cabinet 8 includes a storage unit 6 and a control and movement unit 7. The storage unit 6 stores a transformation matrix that transforms the robot coordinate system {A} into the terminal coordinate system {B}. The transformation matrix is ​​determined based on the pose information of the robot 1 and the pose information of the terminal 3. The control and movement unit 7 adjusts the pose of the robot 1 based on the adjustment of the pose of the simulation model 10 and the transformation matrix, so that the pose of the robot 1 is consistent with the pose of the simulation model 10.

[0041] In other embodiments of the present invention, the storage unit 6 and the control and movement unit 7 are not limited to being located in the control cabinet 8, but can also be located in other locations, such as being built into the robot 1, without being specifically limited here.

[0042] Second Implementation Method

[0043] Figure 2 This is a flowchart of the teaching method for robot 1 according to the second embodiment of the present invention, as shown below. Figure 2As shown, this embodiment provides a teaching method for robot 1. The teaching method for robot 1 includes a robot pose detection step S1, an end-user pose detection step S2, a simulation step S3, and a control movement step S4. It should be noted that the actual order of steps S1 to S4 is not limited.

[0044] In the robot pose detection step S1, the first sensor 2 (including an inertial sensor and a geomagnetic sensor) installed on the robot 1 detects the three-dimensional acceleration information and geomagnetic direction information of the robot 1. The information detected by the first sensor 2 is transmitted to the control cabinet 8, and the control cabinet 8 processes the information detected by the first sensor 2 to obtain the pose information of the robot 1.

[0045] Similarly, in the terminal pose detection step S2, the second sensor 4 (including an inertial sensor and a geomagnetic sensor) installed on the terminal 3 detects the three-dimensional acceleration information and geomagnetic direction information of the terminal 3. The information detected by the second sensor 4 is transmitted to the control cabinet 8, and the control cabinet 8 processes the information detected by the second sensor 4 to obtain the pose information of the terminal 3.

[0046] After robot pose detection step S1 and terminal pose detection step S2, the simulation step S3 can be entered. Figure 3 This is a block diagram of the robot teaching system 100 according to the second embodiment of the present invention, as shown below. Figure 3 As shown, in simulation step S3, based on the pose information of robot 1 and terminal 3, the transformation matrix for transforming the robot coordinate system {A} into the terminal coordinate system {B} is determined. In this embodiment, the robot coordinate system {A} is the base coordinate system.

[0047] Specifically, in simulation step S3, it is necessary to determine the first transformation matrix between the world coordinate system and the robot coordinate system {A}. The first transformation matrix from the robot coordinate system {A} to the world coordinate system is T, where x, y, z are the three-dimensional acceleration information detected by the inertial sensor in the first sensor 2, and a, b, c are the geomagnetic direction information detected by the geomagnetic sensor in the first sensor 2. The first transformation matrix T is...

[0048]

[0049] The pose of robot 1 can be transformed from being expressed in robot coordinate system {A} to being expressed in world coordinate system using the first transformation matrix T.

[0050] Furthermore, it is necessary to determine the second transformation matrix between the world coordinate system and the terminal coordinate system {B}. The second transformation matrix from the terminal coordinate system {B} to the world coordinate system is T. ex1, y1, z1 represent the three-dimensional acceleration information detected by the inertial sensor in the second sensor 4, a1, b1, c1 represent the geomagnetic direction information detected by the geomagnetic sensor in the second sensor 4, and the second transformation matrix T e for

[0051]

[0052] Through the second transformation matrix T e It can transform the terminal's pose from being expressed in the terminal coordinate system {B} to being expressed in the world coordinate system.

[0053] First transformation matrix T and second transformation matrix T e Once determined, the transformation matrix can then be determined. T e -1 It is T e The inverse matrix, T e -1 It can transform the direction vector from a form expressed in the world coordinate system to a form expressed in the terminal coordinate system {B}. The direction vector in the robot coordinate system {A} is v, and the direction vector v is transformed into the direction vector vB in the terminal coordinate system {B}. e v e =T e -1 *T*v.

[0054] When T e When T is an orthogonal matrix, e The inverse matrix T e -1 Equal to the transpose matrix T e T Therefore, the direction vector v e It can also be represented as: v e =T e T *T*v.

[0055] Transform the direction vector v in the robot coordinate system {A} to the world coordinate system, and then transform it to the direction vector v in the terminal coordinate system {B}. e The direction vector v here can describe any position of robot 1. That is, any position of robot 1 can be transformed into a form expressed in the terminal coordinate system {B} through a transformation matrix. This unifies the coordinate systems referenced by robot 1 and terminal 3, thereby enabling the determination of the pose relationship of robot 1 relative to terminal 3, which is beneficial for subsequent simulation and operation of robot 1.

[0056] Furthermore, a simulation model 10 of robot 1 is displayed on the terminal interface 9 of terminal 3. By determining the pose relationship between robot 1 and terminal 3, the pose of simulation model 10 is made consistent with the pose of robot 1, and the viewpoint of simulation model 10 is consistent with the viewpoint of robot 1 observed from terminal 3. If robot 1 is mounted upside down, simulation model 10 in the terminal 3 display interface is also mounted upside down; if robot 1 is mounted on its side, simulation model 10 in the terminal 3 display interface is also mounted on its side.

[0057] In addition to seeing the simulation model 10 on the terminal interface 9, users can also view the pose information of robot 1, including the horizontal and vertical installation angles of robot 1 relative to the world coordinate system, the attitude of each motor axis of robot 1 relative to the base coordinate system, and so on.

[0058] In addition to the base coordinate system, robot 1 often uses other coordinate systems depending on the needs of its operation. For example, the end effector of robot 1's manipulator can use the wrist coordinate system, and the joints of robot 1 can use the joint coordinate system. These coordinate systems are usually determined directly or indirectly based on the base coordinate system. Therefore, when adjusting the pose of robot 1's links, wrists, and other parts, the positions of these coordinate systems can be expressed based on the base coordinate system, and then transformed using a transformation matrix.

[0059] In other embodiments of the present invention, the robot coordinate system {A} is not limited to the base coordinate system and can also be other coordinate systems. When the coordinate system used changes, the first transformation matrix and the second transformation matrix will also be adjusted accordingly, and are not limited to the matrices listed in this embodiment.

[0060] In the control movement step S4, the user teaches the simulation model 10 through the terminal 3 on the terminal interface 9, moving the simulation model 10 to the specified pose from the user's perspective. After the user's command is transmitted to the control cabinet 8, the control movement unit 7 analyzes the motion data of the simulation model 10, and, in conjunction with the motion command of the robot 1, sends the teaching process of the simulation model 10 to the robot 1, causing the robot 1 to move in the same direction.

[0061] Therefore, during teaching, the user does not need to know the coordinate system currently used by robot 1; they only need to control the movement direction of robot 1 based on the user's current viewpoint. After the user confirms the pose adjustment, the simulation model 10 moves and changes its pose according to the user's instructions. Based on the adjustment of the pose of the simulation model 10 and the transformation matrix, the real robot 1 makes corresponding adjustments to maintain consistency with the pose of the simulation model 10. Without complex operations, robot 1 can be easily and conveniently adjusted and changed to a specified pose, simplifying the teaching process.

[0062] In addition, users can observe whether the simulated actions of the simulation model 10 meet the requirements. If they do not meet the requirements, the teaching steps can be repeated until the requirements are met. Then, the posture data of the simulation model 10 is sent to the robot 1 control system to control the robot 1 to complete the final trajectory movement.

[0063] The following is a description of the application scenarios of the teaching method for robot 1 in this embodiment:

[0064] Figure 4 This is a schematic diagram of the robot according to the second embodiment of the present invention and terminals in different positions, as shown below. Figure 4 As shown, the first position M is slightly to the left of robot 1, the second position N is slightly to the right of robot 1, and the third position P is above robot 1. When the user is at the first position M, the view of the simulation model 10 seen by the user on the display interface of terminal 3 is consistent with the view of robot 1 observed from the first position M. When the user moves terminal 3 to the second position N or the third position P, the view of the simulation model 10 will change accordingly. Therefore, regardless of the user's location, the view of the simulation model 10 seen by the user on the display interface of terminal 3 remains consistent with the view of robot 1 observed from terminal 3.

[0065] The user teaches the simulation model 10 via mobile terminal 3, moving the simulation model 10 forward from the user's perspective. After the user's command is transmitted to control cabinet 8, robot 1 moves in the direction indicated by the simulation model 10.

[0066] Furthermore, this embodiment provides a computer-readable storage medium (not shown) storing processor (not shown) readable instructions, by which one or more processors execute the teaching method of robot 1 in this embodiment by running the processor-readable instructions.

[0067] Those skilled in the art will understand that specific technical features in various embodiments can be adaptively split or combined. Such splitting or combining of specific technical features will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will all fall within the protection scope of the present invention. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0068] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for teaching a robot, characterized in that, include: The robot pose detection step involves the robot's first sensor detecting the robot's pose information. The terminal pose detection step involves a second sensor installed on the terminal detecting the terminal's pose information. The simulation steps involve determining a transformation matrix to transform the robot coordinate system into the terminal coordinate system based on the robot's pose information and the terminal's pose information, and displaying the simulation model of the robot on the terminal interface. The pose of the simulation model is consistent with the pose of the robot, and the viewpoint of the simulation model is consistent with the viewpoint of the terminal observing the robot. The control movement step involves adjusting the robot's pose based on the pose of the simulation model and the transformation matrix, so that the robot's pose is consistent with the pose of the simulation model. The simulation step includes the following steps: Based on the pose information of the robot detected by the inertial sensor and the geomagnetic sensor in the first sensor, the first transformation matrix between the world coordinate system and the robot coordinate system is determined; Based on the pose information of the terminal detected by the inertial sensor and the geomagnetic sensor in the second sensor, a second transformation matrix between the world coordinate system and the terminal coordinate system is determined; The transformation matrix is ​​determined based on the first transformation matrix and the second transformation matrix. The first transformation matrix for transforming the robot coordinate system to the world coordinate system is T, and the second transformation matrix for transforming the terminal coordinate system to the world coordinate system is T. e T e -1 It is T e The inverse matrix, The direction vector of the robot coordinate system is v, and the direction vector v transforms into the direction vector v in the terminal coordinate system. e v e = T e -1 *T*v.

2. The robot teaching method as described in claim 1, characterized in that, The pose information includes any one or more of position information, orientation information, and posture information.

3. A robot teaching system, characterized in that, include: The robot has a first sensor, which detects the robot's pose information through an inertial sensor and a geomagnetic sensor. The terminal has a second sensor and a display. The inertial sensor and the geomagnetic sensor in the second sensor detect the pose information of the terminal. The display shows a simulation model of the robot. The pose of the simulation model is consistent with the pose of the robot. The viewpoint of the simulation model is consistent with the viewpoint of the terminal observing the robot. The storage unit stores a transformation matrix that transforms the robot coordinate system into the terminal coordinate system, the transformation matrix being determined based on the robot's pose information and the terminal's pose information. The control unit adjusts the robot's pose based on the pose adjustment of the simulation model and the transformation matrix, so that the robot's pose is consistent with the pose of the simulation model.

4. A computer-readable storage medium, characterized in that, The device stores processor-readable instructions, and one or more processors execute the teaching method of the robot as described in any one of claims 1-2 by running the processor-readable instructions.

Images