Calibration error compensation method of robot for marker and base movement in marker-based robot motion

Abstract

The present invention relates to a method for compensating calibration errors of a robot regarding marker and base movement. The method for compensating calibration errors of a robot regarding marker and base movement according to the present invention is a method for automatically compensating calibration errors caused by movement of a marker or a robot base, based on the premise that the robot's motion is taught based on a marker, and includes the steps of: generating a transformation matrix between an existing base and a new base; updating a robot trajectory generated based on the existing base to a robot trajectory based on the new base; generating a transformation matrix between an existing marker and a new marker; and updating a robot trajectory generated based on the existing marker to a robot trajectory based on the new marker.

Description

Technology Field

[0001] The present invention relates to a method for compensating for a robot's calibration error through coordinate system transformation. Background Technology

[0002] FIG. 1 shows a conceptual diagram of a robot's operation taught based on a marker. In FIG. 1, [b] represents the robot base coordinate system, and [m] represents the marker coordinate system. As shown in FIG. 1, the robot (manipulator) follows a set trajectory (P1~P n The task is performed by moving repeatedly. The above trajectory consists of one or more pre-stored target positions. However, if the target positions are generated based on the robot base coordinate system ([b]), there is a problem that if the position of the robot base changes, an error occurs in the previously generated target positions, making them unusable.

[0003] In particular, the above problem may occur in mobile manipulator systems with limited positional precision as shown in Fig. 2, or in systems where the user directly deploys the robot every time as shown in Fig. 3.

[0004] To solve this problem, markers are installed in the working environment as shown in FIG. 1 ([m]) and FIG. 3 (part indicated by arrows), and the robot trajectory (P1~P n A method of generating ) based on the marker coordinate system ([m]) rather than the robot base coordinate system ([b]) is widely used. In this case, there is an advantage that the previously generated trajectory can be utilized even if the robot base coordinate system [b] moves.

[0005] However, this method also has the problem that if the marker attached to the equipment moves or is damaged and needs to be replaced, an error occurs in the existing trajectory and the trajectory needs to be corrected. Prior art literature

[0006] Published Patent Application No. 10-2024-0011524 (Date of publication: January 26, 2024) The problem to be solved

[0007] The present invention aims to provide a method for compensating calibration errors of a robot that can automatically compensate for trajectory errors even if the position of the marker and the position of the base are changed in a robot system that generates a trajectory based on a marker.

[0008] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0009] A method for compensating calibration errors of a robot for marker and base movement according to an embodiment of the present invention comprises: a step in which a controller included in a robot system moves an end portion of a manipulator included in the robot system to a predetermined reference position; a step in which the controller generates a transformation matrix between a new base and a reference position based on the movement of the end portion; and a step in which the controller generates a transformation matrix between an existing base and a new base based on a previously generated transformation matrix between an existing base and a reference position and a transformation matrix between a new base and a reference position.

[0010] In one embodiment of the present invention, the error compensation method may further include the step of the controller updating the robot trajectory generated based on the existing base to a robot trajectory based on the new base using a transformation matrix between the existing base and the new base.

[0011] In one embodiment of the present invention, the error compensation method may further include: a step in which the controller controls the manipulator to set the transformation matrix between the second camera newly mounted on the manipulator and the new marker to be equal to the transformation matrix between the first camera previously mounted on the manipulator and the existing marker; and a step in which the controller generates a transformation matrix between the existing marker and the new marker based on the transformation matrix between the first camera and the existing marker and the transformation matrix between the existing base and the new base.

[0012] In one embodiment of the present invention, the error compensation method may further include the step of the controller updating the robot trajectory generated based on the existing marker to a robot trajectory based on the new marker using a transformation matrix between the existing marker and the new marker.

[0013] A robot system according to one embodiment of the present invention comprises: a base; a manipulator mounted on the base; and a controller for controlling the manipulator. The controller moves the end portion of the manipulator to a predetermined reference position and generates a transformation matrix between a new position of the base and a reference position based on the movement of the end portion. Based on the previously generated transformation matrix between the existing position of the base and the reference position and the transformation matrix between the new position of the base and the reference position, the controller generates a transformation matrix between the existing position of the base and the new position of the base.

[0014] In one embodiment of the present invention, the controller can update the robot trajectory generated based on the existing position of the base to the robot trajectory based on the new position of the base by using a transformation matrix between the existing position of the base and the new position of the base.

[0015] In one embodiment of the present invention, the controller controls the manipulator to set the transformation matrix between the second camera newly mounted on the manipulator and the new marker to be equal to the transformation matrix between the first camera previously mounted on the manipulator and the existing marker, and can generate a transformation matrix between the existing marker and the new marker based on the transformation matrix between the first camera and the existing marker and the transformation matrix between the existing position of the base and the new position of the base.

[0016] In one embodiment of the present invention, the controller can update the robot trajectory generated based on the existing marker to the robot trajectory based on the new marker by using a transformation matrix between the existing marker and the new marker. Effects of the invention

[0017] According to the present invention, the existing trajectory can be automatically corrected even when the marker or base moves, and it has the effect of being easily applicable even in situations where the marker needs to be replaced.

[0018] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0019] FIG. 1 is a conceptual diagram of a robot's operation taught based on a marker. Figure 2 is an example drawing of a mobile manipulator. Figure 3 is an example drawing of a system in which a user directly places a robot. FIG. 4 is a diagram illustrating coordinate system variation according to the movement of the marker and base. FIG. 5 is a block diagram showing the configuration of a robot system according to one embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for compensating calibration errors of a robot for marker and base movement according to an embodiment of the present invention. Specific details for implementing the invention

[0020] The present invention aims to provide a method for automatically compensating for calibration errors caused by movement of a marker or a robot base, based on the premise that the robot's motion is taught based on a marker.

[0021] As a difference from the prior art, the existing method required the user to manually modify the robot trajectory when the marker or base moved, but when the method proposed in the present invention is applied, the robot trajectory can be easily and automatically calculated. That is, the present invention presents a marker coordinate system-based robot trajectory calculation method that is robust even to the movement of the marker or base.

[0022] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other components, steps, actions, and / or elements in addition to the mentioned components, steps, actions, and / or elements.

[0023] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0024] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0025] In describing the present invention, detailed descriptions of related prior art are omitted if it is determined that such descriptions may unnecessarily obscure the essence of the invention.

[0026] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate an overall understanding in describing the present invention, the same reference numerals will be used for the same means regardless of the drawing number.

[0027] FIG. 4 is a diagram illustrating coordinate system changes due to the movement of a marker and a base. [b] and [c] in FIG. 4 represent the coordinate system (or position) of the robot base frame (hereinafter 'base') and the coordinate system (or position) of the camera mounted on the robot end, respectively. Furthermore, [b'] and [c'] represent the coordinate system (or position) resulting from the movement of the base, representing the new base (new position of the base) coordinate system and the new camera coordinate system, respectively. Additionally, [m] represents the position of the existing marker and the corresponding coordinate system, and [m'] represents the position of the moved marker and the corresponding coordinate system. The markers that can be used in the present invention may be any one of QR, ARToolkit, ARTag, AprilTag, ArUco, TRIP, RUNE-tag, ChromaTag, and CCTag. However, the markers used in the present invention are not limited to the examples described above, and any general markers commonly used in robots or industries may be applied.

[0028] T represents the transformation matrix between each coordinate system. r s is a matrix that transforms a vector in the r-coordinate system into a vector in the s-coordinate system (hereinafter referred to as the 'rs transformation matrix'). For example, T c m is a matrix that transforms the camera coordinate system into the marker coordinate system (camera-marker transformation matrix). The transformation matrix T can be defined as in Equation 1.

[0029]

[0030] In mathematical equation 1, R is a 3x3 rotation matrix, p is a 3x1 position vector, and 0 is a 1x3 zero vector.

[0031] When the transformation matrix T is expressed as in Equation 1, the inverse of the transformation matrix T can be expressed as in Equation 2.

[0032]

[0033] As described above, [m] in FIG. 4 represents the original marker position and the corresponding coordinate system, and [m'] represents the position of the moved marker and the corresponding coordinate system. Additionally, it is assumed that the position of the manipulator's base has also changed from [b], where [m] was first observed, to [b'] due to the mobile robot or the mobile stand. Even if the robot trajectory was initially generated based on the marker coordinate system [m], if the base and the marker move as described above, the marker coordinate system [m] is moved to the new marker coordinate system [m']. Therefore, if there is no transformation matrix between [m] and [m'], there is a problem in that the previously generated robot trajectory cannot be used.

[0034] Hereinafter, with reference to FIGS. 5 and 6, a method for compensating for calibration errors of a robot regarding marker and base movement proposed in the present invention will be described.

[0035] FIG. 5 is a block diagram showing the configuration of a robot system according to an embodiment of the present invention. A robot system (100) according to an embodiment of the present invention includes a base (110), a manipulator (120) mounted on the base (110), a controller (130) for controlling the manipulator (120), and a storage (140) for storing data (in particular, a transformation matrix T between coordinate systems) required for controlling the manipulator (120). The controller (130) includes a processor that performs matrix operations. The processor of the controller (130) can extract data stored in the storage (140) and utilize it for operations. The robot system (100) illustrated in FIG. 5 is according to an embodiment, and the components of the robot system (100) according to the present invention are not limited to the embodiment illustrated in FIG. 5 and may be added, changed, or deleted as needed.

[0036] FIG. 6 is a flowchart illustrating a method for compensating for calibration errors of a robot regarding marker and base movement according to an embodiment of the present invention.

[0037] Referring to FIG. 6, a method for compensating for a calibration error of a robot regarding marker and base movement according to one embodiment of the present invention (hereinafter referred to as the 'error compensation method') consists of steps S210 to S280. The error compensation method illustrated in FIG. 6 is according to one embodiment, and the steps of the error compensation method according to the present invention are not limited to the embodiment illustrated in FIG. 6 and may be added, changed, or deleted as necessary.

[0038] It is assumed that a robot trajectory (meaning the trajectory of the end portion of the manipulator (120)) has already been generated before the above error compensation method is executed. That is, an initial robot trajectory generation step (not shown) is performed by the controller (130) before the error compensation method is executed. The robot trajectory may be generated based on a base coordinate system ([b]) or based on a marker coordinate system (m). And, when the robot trajectory is generated, the controller (130) [uses] a base-camera transformation matrix (T b c ) and camera-marker transformation matrix (T c m ) is stored in the storage (140). In addition, the controller (130) stores the transformation matrix (T) from the base ([b]) to the reference position (p). b p ) is generated and stored in the storage (140). The reference position (p) may be, for example, a fixture or button installed on the work target equipment (e.g., a machine tool), or an external marker. The transformation matrices (T described above) b c , T c m , T b pSince ) can be easily obtained from the robot's kinematics and markers without separate additional equipment, a detailed explanation of the generation method for each transformation matrix is ​​omitted. In addition, the transformation matrix (T) between the base ([b]) and the marker ([m]) b m ) can also be easily calculated through the robot's kinematics and markers, so the explanation of the calculation method for this is also omitted. Similarly, when the base or marker moves, the aforementioned transformation matrices (T) with respect to the new base or new marker b' c' , T c' m' , T b' p, T b' m' A detailed explanation of how to generate ) is also omitted.

[0039] Hereinafter, steps S210 to S280 included in the error compensation method according to one embodiment of the present invention will be described.

[0040] First, the controller (130) moves the manipulator (120, specifically the end portion) to a reference position (p) (S210). That is, the controller (130) controls the manipulator (120) to move to the reference position (p) where it was moved during the initial robot trajectory generation. Then, the controller (130) [transforms] the new base ([b']) and the reference position (p) using a transformation matrix (T b' p ) is generated (S220) and stored in the storage (140). As described above, the transformation matrix (T) between the new base ([b']) and the reference position (p) b' p ) can be calculated by applying robot kinematics. By moving the manipulator (120) to a reference position (p) and applying kinematics, the transformation matrix (T) between the new base ([b']) and the reference position (p) b' p The reason for calculating ) is the transformation matrix (T) between the existing base ([b]) and the new base ([b']).b b' The purpose is to calculate ) using the mathematical formula 3 below.

[0041] The controller (130) is a transformation matrix (T) between the existing base ([b]) and the reference position (p). b p The transformation matrix (T) between ) and the new base ([b']) and the reference position (p) b' p The transformation matrix (T) between the existing base ([b]) and the new base ([b']) based on ) b b' ) can be generated as in Equation 3 (S230). The controller (130) has a transformation matrix (T) between the existing base ([b]) and the new base ([b']). b b' ) is stored in the storage (140).

[0042]

[0043] In mathematical formula 3, the inverse of the transformation matrix can be calculated by mathematical formula 2.

[0044] The controller (130) performs step S250 when the reference coordinate system of the robot trajectory to be updated is the base coordinate system ([b]), and performs steps S260 to S280 when it is the marker coordinate system ([m]).

[0045] When the reference coordinate system of the robot trajectory to be updated is the base coordinate system ([b]), the controller (130) [complies] with the transformation matrix (T) between the existing base ([b]) and the new base ([b']) on the robot trajectory. b b' Update the robot trajectory according to the new base coordinate system ([b']) by applying ) (S250).

[0046] When the reference coordinate system of the robot trajectory to be updated is a marker coordinate system ([m]), the controller (130) controls the manipulator (120) to transform the transformation matrix (T) between the new camera (which may mean a new position of the camera) and the new marker.c' m' ) is the previously stored camera-marker transformation matrix (T c m It is made to be the same as ). That is, the controller (130) makes the existing camera-existing marker transformation matrix (T c m A new camera-new marker transformation matrix (T) to become equal to ) c' m' ) is set. This can be implemented with a general manipulator position control algorithm (S260).

[0047] For reference, the controller (130) has a new camera-new marker transformation matrix (T c' m' ) is the existing camera-existing marker transformation matrix (T c m The process of setting it to be equal to ) is the transformation matrix T c' m' and T c m A method can be applied to calculate the error and control the manipulator (120) through a feedback control technique such as PID based on it. In addition, the controller (130) may use a feedforward control technique that generates a trajectory based on the error and causes the manipulator (120) to follow it. Furthermore, control through a combination of feedback and feedforward is also possible.

[0048] And, the controller (130) is T c' m' =T c m Using the conditions, a transformation matrix between the existing marker ([m]) and the new marker ([m']) is obtained (S270). The coordinate systems before the marker movement ([m]) and after the marker movement ([m']) can be expressed as Equations 4 to 6.

[0049]

[0050]

[0051] T c'm' =T c m Therefore, Equation 6 can be derived from Equation 4 and Equation 5.

[0052]

[0053] Based on mathematical formula 6, the controller (130) has a transformation matrix (T) between the existing marker coordinate system ([m]) and the new marker coordinate system ([m']). m m' ) can be obtained. For example, the controller (130) can obtain the transformation matrix (T) between the existing marker and the new marker through Equations 7 to 9. m m' ) can be obtained.

[0054] The base-marker transformation matrix (T) in Equation 6 b m ) can be expressed as in mathematical formula 7.

[0055]

[0056] And, the transformation matrix T in mathematical equation 7 b m' It can be expressed as in mathematical formula 8.

[0057]

[0058] Therefore, mathematical formula 6 can be expressed as mathematical formula 9.

[0059]

[0060] In Equation 9, the transformation matrix (T) between the new marker ([m']) and the new base ([b']) m' b' ) can be calculated by the controller (130) based on the robot's kinematics and marker recognition information. In addition, the transformation matrix (T) between the new base ([b']) and the corresponding camera position ([c']) c' b' Also, since the camera is attached to the end of the manipulator (120), the controller (130) applies robot kinematics to the transformation matrix Tc' b' can calculate.

[0061] Therefore, based on mathematical equation 9, T m m' It can be calculated as in mathematical formula 10.

[0062]

[0063] The controller (130) is a transformation matrix (T) between the existing marker and the new marker. m m' ) is calculated and stored in the storage (140). The controller (130) calculates the transformation matrix (T) between the existing marker and the new marker on the robot trajectory based on the marker coordinate system ([m]). m m' By applying ), the robot trajectory is updated based on the new marker (S280).

[0064] The aforementioned error compensation method has been described with reference to the flowchart presented in the drawings. For simplicity of explanation, the method has been illustrated and described in a series of blocks; however, the present invention is not limited to the order of said blocks, and some blocks may occur in a different order or simultaneously with other blocks as illustrated and described herein, and various other branches, flow paths, and sequences of blocks may be implemented to achieve the same or similar results. Furthermore, not all illustrated blocks may be required for the implementation of the method described herein.

[0065] Meanwhile, in the description with reference to FIG. 6, each step may be further divided into additional steps or combined into fewer steps, depending on an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order between steps may be changed.

[0066] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols

[0067] 100: Robot Systems 110: Bass 120: Manipulator 130: Controller 140: Storage

Claims

Claim 1 A method for compensating calibration errors of a robot for marker and base movement comprises: a step in which a controller included in a robot system moves an end portion of a manipulator included in the robot system to a predetermined reference position; a step in which the controller generates a first transformation matrix between a new base and a reference position based on the movement of the end portion; a step in which the controller generates a third transformation matrix between an existing base and a new base based on a previously generated second transformation matrix between an existing base and a reference position and the first transformation matrix; and a step of updating the robot trajectory according to a reference coordinate system of the robot trajectory to be updated, wherein, if the reference coordinate system is a marker coordinate system, the step of updating the robot trajectory comprises: a step in which the controller controls the manipulator to set a fourth transformation matrix between a second camera newly mounted on the manipulator and a new marker to be equal to a fifth transformation matrix between a first camera previously mounted on the manipulator and an existing marker. An error compensation method comprising: a step in which the controller generates a sixth transformation matrix between an existing marker and a new marker based on the fifth transformation matrix and the third transformation matrix by utilizing the condition that the fourth transformation matrix is ​​equal to the fifth transformation matrix; and a step in which the controller updates a robot trajectory generated based on the existing marker to a robot trajectory based on the new marker by utilizing the sixth transformation matrix. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 Base; manipulator mounted on the base; and includes a controller for controlling the manipulator, wherein the controller moves the end portion of the manipulator to a predetermined reference position and generates a first transformation matrix between the new position of the base and the reference position based on the movement of the end portion, generates a second transformation matrix between the existing position of the base and the reference position that was previously generated, and generates a third transformation matrix between the existing position of the base and the new position of the base based on the first transformation matrix, wherein when the reference coordinate system is a marker coordinate system, the controller controls the manipulator to set the fourth transformation matrix between the second camera newly mounted on the manipulator and the new marker to be equal to the fifth transformation matrix between the first camera previously mounted on the manipulator and the existing marker, and using the condition that the fourth transformation matrix is ​​equal to the fifth transformation matrix, generates a sixth transformation matrix between the existing marker and the new marker based on the fifth transformation matrix and the third transformation matrix, and uses the sixth transformation matrix to create a new robot trajectory based on the existing marker. A robot system that updates the robot trajectory based on a marker. Claim 6 delete Claim 7 delete Claim 8 delete

Images