Robot system having a robot and method for calibrating the robot

Abstract

The robot system includes a robot having a movable part with a calibration marker and at least a first joint and a second joint, a camera for capturing images of the calibration marker, and a computation unit including a robot control layer for controlling rotation of the first joint and the second joint based on a kinematic model of the robot to position the calibration at a plurality of positions, an image processing layer for processing images received from the camera to determine marker position data for each image, an axis and reduction ratio estimation layer for establishing a first error function based on the marker position data to determine a first rotation axis of the first joint, and a kinematic parameter update layer for establishing a second error function based on the at least the first rotation axis to determine a set of kinematic parameters for the kinematic model.

Description

[Technical Field]

[0001] The present disclosure relates generally to the field of robotics, and more particularly to a robotic system having a robot and a method for calibrating a robot. [Background technology]

[0002] In today's world, technological growth is occurring at a rapid pace. Due to this rapid advancement in technology, most manual tasks are now performed automatically, for example, by robots, and thus, robot manufacturing has grown exponentially. Various parameters require calibration during robot utilization, and one of these parameters is the kinematic parameters. A robot's kinematic parameters describe the geometry of the robot's motion and are used to relate the position of the robot's end-effector to the position of the robot's base via joint values. If kinematic parameters are not calibrated, they will have large errors due to robot manufacturing defects, continuous wear during use, and accumulated zero-position offsets of the joints. As a result, accurate calibration methods are desirable for robots. A widely used set of kinematic parameters is the Denavit-Hartenberg (DH) parameters, which allow the position of the robot's end-effector to be calculated based on joint values. Other sets of kinematic parameters include the transitions between the camera, robot base coordinate system, and the robot's gear reduction ratio.

[0003] Traditionally, calibration has been performed using laser tracker systems, where one part of the laser tracker system measures the robot's body position and another part of the laser tracker system measures the robot's end-effector position. However, such calibration is complex and inefficient due to its manual nature, and unnecessary equipment is costly. Another approach involves the introduction of cameras onto the robot, which are used at multiple positions on the robot. Furthermore, markers are placed close enough to the camera's beam to calculate accurate vectors for each robot position. However, all robot positions used for calibration must be close enough to the camera's beam. This means that motion parameters can only be calibrated within the robot's limited configuration space, which is a technical challenge.

[0004] In light of the above discussion, it is desirable to address the above-mentioned shortcomings associated with robot calibration. Summary of the Invention

[0005] The present disclosure relates to a robotic system having a robot, and a method for calibrating the robot. It is an object of the present disclosure to at least partially overcome problems encountered in the prior art by providing an improved robotic system having a robot, and an improved method for automatically calibrating the kinematic parameters of the robot, for example, to reduce errors in the position and orientation of the robot's end effector and to increase the accuracy of the robot's manipulation execution.

[0006] One or more objects of the present disclosure are achieved by the attached independent claims. Optional or alternative implementations of the present disclosure are further defined in the dependent claims.

[0007] In one aspect, the present disclosure provides a robot system. The robot system includes a robot further including a movable part, the movable part having at least one calibration marker provided thereon. The robot system further includes at least a first joint and a second joint disposed at one end of at least one link, the first joint and the second joint configured to generate a movement in the movable part controlled by a computing unit. The robot system further includes a camera configured to capture images of the at least one calibration marker at multiple positions. The robot system further includes a computing unit including a robot control layer, an image processing layer, an axis and reduction ratio estimation layer, and a motion parameter update layer. The robot control layer is configured to control rotation of the first joint and the second joint based on a kinematic model of the robot. The robot control layer is further configured to control rotation of the first joint and / or the second joint to the position of the at least one calibration marker at multiple positions. The image processing layer is configured to process images received from the camera to determine, for each image, a marker position of the at least one calibration marker in three-dimensional space, to be output as marker position data. The axis and reduction ratio estimation layer is configured to establish a first error function based on the marker position data and determine a first rotation axis of the first joint based on the first error function. The motion parameter update layer is configured to establish a second error function based on at least the first rotation axis and determine a set of motion parameters for the motion model based on the second error function.

[0008] The robot system includes a moving part of the robot. The moving part of the robot is provided with at least one calibration marker. The calibration marker serves to analyze the position of the moving part of the robot, and a camera is used to capture multiple positions of the calibration marker. Furthermore, a movement in the moving part is generated by a first joint and a second joint, one joint value being different from a zero position, and the other joint value being at the zero position during the entire collection procedure. Furthermore, the calculation of the rotation axis parameters and the calculation of the motion parameters include minimizing an error function. Furthermore, the first joint and the second joint are controlled by a calculation unit. The calculation unit includes a robot control layer that controls the rotation of the first joint and / or the second joint to change the position of one of the calibration markers. The calculation unit further includes an image processing layer that serves to provide marker position data by analyzing an image captured by the camera. The calculation unit calculates the first error function. function and an axis and reduction ratio estimation layer configured to analyze the marker position data to determine a first rotation axis of the first joint based on a first error function. function and a kinematic parameter update layer that analyzes the first rotation axis to establish a set of kinematic parameters for the kinematic model based on the second error function. Advantageously, the robotic system calibrates kinematic parameters of the robot, including robot link lengths, joint positions and orientations, robot joint zero positions, and joint-by-joint reduction ratios. Using the calibrated parameters, the robot can achieve better accuracy of the position and orientation of the end effector while performing a movement.

[0009] In an embodiment, the moving part further includes a first servo motor coupled to the first joint via a first gear system, a second servo motor coupled to the second joint via a second gear system, a first encoder provided on the first servo motor, and a second encoder provided on the second servo motor, wherein the first encoder and the second encoder are configured to collect servo values ​​of the first servo motor and the second servo motor, respectively.Furthermore, the computing unit has a clock synchronization layer configured to synchronize the camera with the first encoder and the second encoder, and optionally with the rotation angle provided by the encoder, so as to timestamp images captured by the camera.

[0010] The servo motors can rotate in both clockwise and counterclockwise directions, and a gear system helps transfer the motion of the servo motors to the joints.

[0011] Another aspect is a method for calibrating a robot including a movable part, the movable part having at least one calibration marker disposed thereon. Further, at least a first joint and a second joint are disposed at one end of at least one link, the first joint and the second joint configured to generate motion in the movable part controlled by a computing unit. The method further includes a data collection procedure including positioning the at least one calibration marker at multiple positions by rotating the first joint and / or the second joint. The method further includes acquiring multiple images of the at least one calibration marker, where the multiple images are captured by a camera while the at least one calibration marker is in multiple positions. A data processing procedure is further disclosed for processing the multiple images to determine, for each image of the multiple images, a marker position (and optionally, an orientation) of the at least one calibration marker in three-dimensional space, which is output as marker position data. The method further discloses establishing a first error function based on the marker position data. Thereafter, determining a first axis of rotation of the first joint and a second axis of rotation of the second joint based on the first error function. Further, a second error function is established based on at least the first rotation axis and the second rotation axis, and a set of motion parameters for a motion model of the robot is determined based on the second error function.

[0012] The method achieves at least some of the benefits and technical effects of the robotic system of the present disclosure.

[0013] It should be noted that all devices, elements, circuits, units, and means described herein may be implemented with software or hardware elements, or any combination thereof. The steps performed by various entities described herein and the functions described as being performed by various entities are intended to mean that each entity is adapted to or configured to perform the respective step or function. Even if, in the following description of specific embodiments, a specific function or step performed by an external entity is not reflected in the description of the specific detailed element of the entity that performs the specific step or function, it will be apparent to those skilled in the art that these methods and functions may be implemented with each software or hardware element, or any combination thereof. It will be understood that features of the present disclosure may be combined in various combinations without departing from the scope of the present application, as defined in the appended claims.

[0014] Further aspects, advantages, features and objects of the present disclosure will become apparent from the following detailed description of exemplary implementations and drawings, taken in conjunction with the appended claims. [Brief explanation of the drawings]

[0015] The foregoing summary and the following detailed description of exemplary embodiments will be better understood when read in conjunction with the accompanying drawings. For the purpose of illustrating the disclosure, exemplary structures of the disclosure are shown in the drawings. However, the disclosure is not limited to the specific methods and instrumentalities disclosed therein. Moreover, those skilled in the art will appreciate that the drawings are not to scale. Wherever possible, like elements are designated by like numerals.

[0016] Embodiments of the present disclosure will now be described, by way of example only, with reference to the following drawings, in which:

[0017] [Figure 1A]FIG. 1 is a schematic diagram of the architecture of a robotic system according to various embodiments of the present disclosure. [Figure 1B] FIG. 1 is a schematic diagram of the architecture of a robotic system according to various embodiments of the present disclosure. [Figure 2] FIG. 1 is a block diagram of a robotic system according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart of a method for calibrating a robot according to an embodiment of the present disclosure.

[0018] In the accompanying drawings, underlined numbers are used to identify the item the underlined number is above or next to. Numbers without underlines relate to the item identified by the line connecting the ununderlined number to the item. When a number is not underlined and has an associated arrow, the ununderlined number is used to identify the general item to which the arrow is pointing. DETAILED DESCRIPTION OF THE INVENTION

[0019] The following detailed description describes embodiments of the present disclosure and how they may be practiced. While several modes of carrying out the disclosure are disclosed, those skilled in the art will recognize that other embodiments are possible for carrying out or practicing the disclosure.

[0020] 1A and 1B are schematic diagrams of the architecture of a robot in a robotic system according to various embodiments of the present disclosure. FIGS. 1A and 1B are collectively described, with FIG. 1B being an embodiment of FIG. 1A. Referring to FIG. 1A, robots such as robot 100A (of FIG. 1A) and robot 100B (of FIG. 1B) are shown. Robot 100A includes a base 102, a movable part 104, a calibration marker 106A, a computing unit 108, a camera 110, first and second servo motors 112A and 112B, and first and second encoders 114A and 114B. Robot 100B includes all the same elements as robot 100A (computing unit 108 not shown), and additionally, robot 100B includes calibration markers 106B and 106C.

[0021] The disclosed robots 100A and 100B correspond to basic robots used to automatically calibrate the motion parameters of a robotic system by using an industrial camera, such as camera 110, and at least one reference marker, such as calibration marker 106A. Robot 100B further includes calibration markers 106B and 106C.

[0022] The base 102 of the robot 100A is used to provide balance to the moving part 104 of the robot 100A. The moving part 104 is the arm of the robot 100A.

[0023] Calibration markers 106A, 106B, and 106C are used to provide marker position data. Calibration markers 106A, 106B, and 106C are attached to the end effector of robot 100A. At least one calibration marker is used for calibration purposes.

[0024] Computing unit 108 may include suitable logic, circuitry, interfaces, and / or code configured to control robot 100 A. Examples of implementations of computing unit 108 may include, but are not limited to, a central data processing device, a microprocessor, a microcontroller, a multiple instruction set computing (CISC) processor, an application specific integrated circuit (ASIC) processor, a reduced instruction set (RISC) processor, a very long instruction word (VLIW) processor, a state machine, and other processor or control circuitry.

[0025] The camera 110 may include appropriate logic, circuitry, interfaces, and / or code configured to capture an image of the position of the calibration marker 106A.

[0026] The first servo motor 112A and the second servo motor 112B are 100A The first servo motor 112A and the second servo motor 112B are used to provide movement to the arms of the first servo motor 112A and the second servo motor 112B. The first servo motor 112A and the second servo motor 112B can move in clockwise and counterclockwise directions. Furthermore, the first encoder 114A and the second encoder 114B are used to determine the rotation (e.g., by calculating the angle of rotation) of the first servo motor 112A and the second servo motor 112B, respectively.

[0027] In one aspect, a robot system is provided. The robot system includes a robot 100A further including a movable part 104. The movable part 104 has at least one calibration marker provided thereon. The robot system further includes at least a first joint and a second joint disposed at one end of at least one link, the first joint and the second joint configured to generate a movement in the movable part 104 controlled by a computing unit 108. In other words, the movable part 104 corresponds to an arm of the robot 100A. The movable part 104 includes any number of joints disposed at one end of the at least one link. The movement of the at least one link is performed by rotating at least one pre-joint. Furthermore, the computing unit 108 controls at least the first joint and the second joint to cause a movement in the movable part 104, which is further beneficial for the movement of at least one calibration marker, such as the calibration marker 106A. In an implementation, the computing unit 108 includes a robot control layer that stores a set of predetermined commands for the movement of the first joint and the second joint. It should be noted that the robot control layer and the computing unit 108 can be the same device or separate devices in a distributed system. Furthermore, the robot control layer sends a set of predetermined commands to the first servo motor 112A and the second servo motor 112B to cause movement therein.

[0028] According to the embodiment, the first servo motor 112A is coupled to the first joint via a first gear system, and the second servo motor 112B is coupled to the second joint via a second gear system. Furthermore, a first encoder 114A is provided on the first servo motor 112A, and a second encoder 114B is provided on the second servo motor 112B. Furthermore, the first encoder 114A and the second encoder 114B are configured to collect servo values ​​(e.g., angles) of the first servo motor 112A and the second servo motor 112B, respectively. Furthermore, the calculation unit 108 further includes a clock synchronization layer configured to synchronize the camera 110 with the first encoder 114A and the second encoder 114B so as to timestamp the images and servo values ​​captured by the camera 110. In other words, at least the first joint and the second joint are connected to the first servo motor 112A and the second servo motor 112B, respectively, via a gear system with a reduction ratio for each joint as a system parameter. The motions of the first servo motor 112A and the second servo motor 112B are transmitted to at least the first joint and the second joint with some reduction ratio coefficient. Furthermore, a clock synchronization layer present in the calculation unit 108 performs synchronization for time stamps. Furthermore, the clock synchronization layer synchronizes the camera 110 with the first encoder 114A and the second encoder 114B.

[0029] Furthermore, the robotic system includes a camera 110 configured to capture images of at least one calibration marker at multiple positions. In an implementation, the camera 110 includes a perception layer. Furthermore, synchronization between the camera 110 and the first and second encoders 114A, 114B is performed by time stamping the images and encoder data captured by the camera 110. For example, the motion M J During this time, the camera 110 captures multiple images of at least one calibration marker, such as the calibration marker 106A. For example, the camera 110 captures multiple images I1-I2. T where T is the motion M JThe image is then sent to the calculation unit 108 for further processing. J and M J+1 During this time, the first encoder 114A and the second encoder 114B collect information from the first servo motor 112A of joint J and the second servo motor 112B of joint J+1 and send the information to the calculation unit 108. Furthermore, synchronization is performed between the encoder clock and the camera clock. In the example, the angle values ​​(e.g., values ​​a1 to a T ) is available without any synchronization. In implementation, synchronization is performed so that time tags for each image and each angle are obtained to establish the first error function.

[0030] 1A further includes a robot control layer, an image processing layer, an axis and reduction ratio estimation layer, a motion parameter update layer, and a clock synchronization layer. The robot control layer is configured to control the rotation of the first joint and the second joint based on a motion model of the robot 100A, and the robot control layer is configured to control the rotation of the first joint and / or the second joint to position at least one calibration marker at a plurality of positions. In other words, the computation unit 108 includes various layers used to perform a calibration process in the robot system. Furthermore, the robot control layer provides motion to the arm of the robot 100A to change the position of at least one calibration marker, such as the calibration marker 106A. For example, the robot control layer controls the rotation of several reference markers Δ i (e.g., i∈[1,2]) is statically attached to the end effector of the robot arm at an arbitrary position and orientation. Furthermore, at least one calibration marker, such as calibration marker 106A, allows for calibration of the position and orientation of robot 100A in three-dimensional (3D) space based on its image, e.g., taking into account the intrinsic parameters of camera 110 and real-world marker dimensions.

[0031] The image processing layer is configured to process images received from the camera 110 to determine, for each image, the marker position and (optionally) the orientation of at least one calibration marker in 3D space, which is to be output as marker position data. In other words, the image processing is performed by acquiring data from the camera 110 and outputting a set of position / orientation data for each observed marker along with a corresponding timestamp. Computational Unit 108 The image processing layer is, for example, image data I1 to I T , and participate in the calculation of the position and orientation of the calibration markers 106A, 106B, 106C relative to the camera 110. For example, without loss of generality, the reference marker Δ i The calibration markers 106A, 106B, and 106C are used for all images I1 to I T In other cases, the image processing layer i is detected by the camera 110 i1 ~I iT Furthermore, the intrinsic parameters of the camera 110 may be assumed to be accurate and known. In implementation, a camera calibration procedure is also performed on the intrinsic parameters of the camera 110. Then, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, C t i Frame I t Marker Δ i In the example, the pre-computed intrinsic parameters of the camera 110 and the corner coordinates C t I is the 3D position P of the marker center relative to the camera 110 using the perspective-n-point (PnP) method. t i and 3D marker orientation t iFurthermore, P t i The representation of is performed by a three-dimensional vector, O t i The representation of is performed by a 3x3 orthonormal basis matrix.

[0032] Furthermore, the axis and reduction ratio estimation layer is configured to establish a first error function based on the marker position data, and determine a first rotation axis of the first joint based on the first error function. In other words, the axis and reduction ratio estimation layer is responsible for processing the marker position / orientation data as well as the encoder data, and outputs an estimated rotation axis for each joint. For example, from the geometric properties of rotation, for each i, the 3D marker coordinate P1 i ,···,P T i But, Q j In a circle centered at V j on a plane perpendicular to and at an angle r j α1,···,r j α T are arranged according to the direction base O1 i ,···,O T i is the angle r j α1,···,r j α T According to V j Furthermore, the rotation axis vector V j , rotation axis point Q j and turnover rate r j For example, an optimization method can be applied to a loss function L(V, Q, r) that reflects the magnitude to which the above geometric properties are not satisfied for a given set of 3D vectors V, 3D points Q, and a scalar r. j ,Q j ,r j )=argminL(V,Q,r) to find the desired value.

[0033] Furthermore, the motion parameter update layer is configured to establish a second error function based on at least the first rotation axis, and determine a set of motion parameters for the motion model based on the second error function. In other words, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint and outputting calibrated motion parameters. For example, L represents the set of motion parameters to be calibrated. In implementation, the estimated rotation axis A1 to A2 of each joint are j Given the initial values ​​of the motion parameters L0, the motion parameter update layer estimates the actual parameters L1, L2, L3, L4, L5, L6, L7, L8, L9, L10, L11, L12, L13, L14, L15, L16, L17, L18, L19, L20, L21, L22, L23, L24, L25, L26, L27, L28, L29, L30, L31, L32, L40, L41, L42, L43, L54, L55, L66, L77, L88, L99, L109, L110, L121, L132, L142, L153, ​​L164, L175, L186, L197, L198, L199, L209, L210, L221, L232, L242, L254, L265, L276, L287, L298, L299, L309, L41, L42, L43, L54, L65, L109, L110, L121, L222, L232, L242, L254, L265, L309, L41, L42, L43, L55, L66, L109, L110, L121, L222, L232, L242, L254, L265, L309, L41, L42, L55, L66, L109, L275, L287, L298, L299, L309, L41, L42, L55, L67, L109, L209, L210, L221

number

number

number

number

number

number

number

number

[0034] The robot system calibrates the kinematic parameters of the robot 100A, including robot link lengths, joint positions and orientations, robot joint zero positions, reduction ratios per joint position and orientation, and reduction ratios per joint. Using the calibrated parameters, the robot 100A can achieve improved accuracy of the end effector position and orientation during kinematic execution. According to an embodiment, the calibration procedure may be performed without encoder data. Multiple images containing markers are sufficient to provide kinematic model parameters and hand-eye calibration.

[0035] According to an embodiment, a special fiducial marker type, for example, a 3x3 ChArUco marker, can be attached to the end effector to automatically determine the intrinsic parameters of the camera 110 using motion information from the marker's keypoints. In an embodiment, the marker's keypoints are stored in the image data I1-I2. T Furthermore, by knowing the ground truth information about the relative positions of the marker keypoints, the intrinsic camera parameters of the camera 110 are obtained. Furthermore, by using the obtained camera parameters and the positions of the marker keypoints on the image, the 3D position P of the marker center relative to the camera 110 can be calculated. t i and 3D marker orientation t i is calculated by the PnP method.

[0036] According to another embodiment, at least one marker is attached to each link of the robot. This setup allows arbitrary movements of the robot during calibration and online kinematic parameter calibration. If arbitrary movements are applied, encoder values ​​may be required for zero-position calibration. This embodiment can be applied to any robot architecture with revolute joints that allow arbitrary link connections. The difference between the embodiment and the main disclosure is that the movements of the robot arm can be arbitrary. In this embodiment, the camera 110 captures images I1,...,I T where T is the total number of frames. Then, the rotary encoder data α1 for joint j=1, , J is captured. j ,···,α T j Furthermore, the position P1 of the marker attached to the link j=1,...,J is collected. j ,···P Tj and orientation O1 j ,···,O T j Furthermore, the rotation axis and reduction ratio estimation is performed by using data P1 from the first joint to obtain V1, Q1, and r1 of the first joint.1 ,···,P T 1 ;O1 1 ,···,O T 1 ;α1 1 ,···,α T 1 For each t = 1, , T, the angle value (-rα t 1 ) a 3D affine transformation τ corresponding to a rotation of 3D space around the axis A1 = (V1,Q1) by t 1 Furthermore, for t=1, ,T,

number

number

number

number

[0037] In another embodiment, the camera 110 is mounted to the end effector and the fiducial markers are not mounted to the end effector but are mounted in a fixed workspace visible to the camera 110. In this embodiment, the image processing is performed by capturing a series of marker positions P1,...,P T and a set of marker orientation bases O1,...,O T Then, the rotation axis and reduction ratio estimation is performed to obtain -P1,...,-P T and O1 -1 ,···,O T -1 and finally, motion parameter adjustment is performed.

[0038] In other embodiments, multiple cameras are used in case a marker cannot be photographed due to spatial obstructions. Camera 110 is preferably in a stationary position. In other embodiments, instead of using multiple cameras, camera 110 can have a predefined trajectory in case there are spatial obstructions.

[0039] FIG. 2 is a block diagram of a robotic system according to an embodiment of the present disclosure. FIG. 2 is described with elements from FIGS. 1A and 1B. Referring to FIG. 2, a robotic system 200 for calibrating a robot is shown. The robotic system 200 includes a robot control 204, a robot servo 206, an encoder layer 208, sensing 210, image processing 212, clock synchronization 214, axis and reduction ratio estimation 216, and motion parameter update 218. The robotic system 200 further includes a robot 100A (or 100B), a computing unit 108, and a camera 110. FIG. 2 is described step by step through steps 202A to 202K.

[0040] The robot control 204 of the robot system 200 is used to control the robot 100A by providing commands for each movement. The encoder layer 208 in the robot system 200 is used to capture angles and movements. The sensing 210 may include the camera 110 for gathering visual information. The image processing 212 is a layer present in the computation unit 108 for analyzing images captured by the camera 110. The clock synchronization 214 is a layer present in the computation unit 108 for synchronizing the camera 110 with the encoder layer 208. Furthermore, the axis and reduction ratio estimation 216 should provide estimated rotation axes for each joint. Furthermore, the motion parameter update 218 is used to provide calibrated motion parameters.

[0041] In step 202A, a robot control 204 in the computing unit 108 provides commands for movement. The robot control 204 stores a set of predetermined commands for movement of at least a first joint and a second joint. Note that the robot control 204 and the computing unit 108 may be the same device.

[0042] In step 202B, the robot control 204 sends a set of predetermined commands to the robot servo 206 to cause movement in the robot servo 206.

[0043] In step 202C, the angle of the robot servo 206 is captured by the encoder layer 208. In other words, at least the first joint and the second joint are connected to the robot servo 206 via a gear system with a gear reduction ratio for each joint as a system parameter. The movement of the robot servo 206 is transmitted to at least the first joint and the second joint with some reduction ratio coefficient. Furthermore, the clock synchronization 214 performs synchronization for time stamps.

[0044] In step 202D, the vision method is collected by sensing 210. In step 202E, clock synchronization 214 synchronizes the camera 110 with the encoder layer 208. In other words, the perception layer 112 is within the camera 110. The synchronization between the camera 110 and the encoder layer 208 is performed by time stamping the images captured by the camera 110. For example, the motion M J During this time, the camera captures multiple images I1 to I2 of at least one calibration marker. T where T is the motion M J The images are sent to the calculation unit 108 for processing. J During this time, the encoder layer 208 collects information from the robot servos 206 of the joints J and transmits the information to the computation unit 108 The clock synchronization 214 sends the servo angles a1 to a T The encoder clock and the camera clock are synchronized to obtain the

[0045] In step 202F, image processing 212 is performed, which involves acquiring data from camera 110 and outputting a set of position / orientation data for each observed marker along with a corresponding timestamp. The image processing in computation unit 108 generates image data I1-I T is used to calculate the position and orientation of the marker relative to the camera 110. Without loss of generality, the reference marker Δ i is all images I1 to I T In other cases, the image processing layer i Frame I where i1 ~I iT The intrinsic parameters of the camera 110 can be assumed to be accurate and known. If not, a camera calibration procedure is performed. Furthermore, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C Ti At this time, C t i Frame I t Marker Δ i Furthermore, the pre-calculated intrinsic parameters of the camera 110 and the corner coordinates C t I is the 3D position P of the marker center relative to the camera 110 using the perspective-n-point (PnP) method. t i and 3D marker orientation t i In the example, the marker center P t i The representation of is performed by a three-dimensional vector, O t i The representation of is performed by a 3x3 orthonormal basis matrix.

[0046] In step 202G, the visual information collected by sensing 210 is time-stamped by clock synchronization 214. Collectively in steps 202H and 202I, axis and reduction ratio estimation is performed after calculating the position and orientation of the calibration markers 106A, 106B, 106C relative to the camera 110. Axis and reduction ratio estimation 216 involves processing the calibration marker position / orientation data and encoder data and outputs estimated rotation axes for each joint.

[0047] In step 202J, the estimated rotation axis for each joint is updated using the motion parameters. 218 The motion parameter update 218 involves processing the estimated rotation axis for each joint and outputs the calibrated motion parameters.

[0048] In step 202K, the motion parameter calibration procedure is completed. The updated motion parameters and transformation operators complete the motion parameter calibration procedure. The transformation between the marker frame at the zero position and the robot base frame obtained by the calibrated motion parameters, together with the transformation between the camera frame obtained by image processing and the marker frame at the zero position, forms the transformation between the camera 110 and the base 102, thus completing the hand-eye calibration.

[0049] The robot system 200 calibrates the motion parameters of the robot 100, including robot link lengths, joint positions and orientations, robot joint zero positions, reduction ratios for each joint position and orientation, and robot joint zero positions. Using the calibrated parameters, the robot system 200 can achieve improved accuracy in the position and orientation of the end effector during motion execution.

[0050] Figure 3 is a flowchart of a method for calibrating a robot according to an embodiment of the present disclosure. Figure 3 is described in conjunction with elements from Figures 1A, 1B, and 2. Figure 3 is described step by step from steps 302 to 304.

[0051] The method 300 is used to calibrate a robot 100A having a movable part 104, the movable part 104 having at least one calibration marker thereon, and at least a first joint and a second joint disposed at one end of at least one link, the first joint and the second joint configured to generate a movement in the movable part 104 controlled by a computing unit 108. In other words, the movable part 104 forms an arm of the robot 100A. The movable part 104 may include any number of joints, the joints being disposed at either end of at least one link. The movement of at least one link is performed by rotating an adjacent joint. The computing unit 108 controls the first joint and the second joint to impart movement to the movable part 104, and data is collected by evaluating the movement of the at least one calibration marker. For example, in FIG. 3 , the method 300 begins at step 302 by performing a data collection procedure. The data collection procedure 302 includes, in step 302A, positioning at least one calibration marker at multiple positions by rotating the first joint and / or the second joint. In other words, data is collected by changing the position of at least one calibration marker, such as calibration marker 106A. Furthermore, the first joint and / or the second joint may be connected to the first servo motor 112A and the second servo motor 112B via a gear system with system parameters including a gear reduction ratio for each joint. The movement of the first servo motor 112A and the second servo motor 112B is transferred to the first joint and / or the second joint, which in turn transfers the movement to the at least one calibration marker.

[0052] The data collection procedure 302 further includes, in step 302B, acquiring multiple images of the at least one calibration marker. In implementation, the multiple images are captured by the camera 110 when the at least one calibration marker is in multiple positions. In other words, synchronization between the camera 110 and the first and second encoders 114A, 114B is performed by time stamping the images captured by the camera 110. For example, the motion M J During this time, the camera 110 captures multiple images I1 to I2 of at least one calibration marker. T where T is the motion M J The images are sent to the calculation unit 108 for processing. J During this time, the first encoder 114A and the second encoder 114B collect information from the first servo motor 112A and the second servo motor 112B of the joint J and send the information to the computing unit 108.

[0053] The method 300 further comprises performing a data processing procedure at step 304. In other words, the collected data is further processed by evaluating the movement of the at least one calibration marker. The procedure 304 is further divided into various steps, such as steps 304A, 304B, 304C, 304D, and 304E.

[0054] The data processing procedure 304 includes, at 304A, processing a plurality of images to determine, for each image of the plurality of images, a marker position of at least one calibration marker in three-dimensional space to be output as marker position data. In other words, the image processing is performed by acquiring data from the camera 110 and outputting a series of position / orientation data for each observed marker along with a corresponding timestamp. In implementation, the computation unit 108 The image processing layer is, for example, image data I1 to I T, and is responsible for calculating the position and orientation of the marker relative to the camera 110. In implementation, without loss of generality, the reference marker Δ i is all images I1 to I T In other implementations, the image processing Ya is , Δ i Frame I where i1 ~I iT The intrinsic parameters of the camera 110 can be assumed to be accurate and known. If not, a camera calibration procedure is performed. Furthermore, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, C t i Frame I t Marker Δ i is a set of (x,y) coordinates of the corners of the camera. Furthermore, the pre-computed intrinsic parameters of the camera and the corner coordinates C t I is the 3D position P of the marker center relative to the camera using the perspective-n-point (PnP) method. t i and 3D marker orientation t i In the example, the marker center P t i The representation of is performed by a three-dimensional vector, O t i The representation of is performed by a 3x3 orthonormal basis matrix.

[0055] The data processing procedure 304 further includes, in step 304B, establishing a first error function based on the marker position data. In other words, the marker position data is essentially data obtained by image processing. During image processing, the positions of at least one calibration marker in the image captured by the camera 110 are marked. These marked positions of the at least one calibration marker are referred to as marker position data. Furthermore, the axis and reduction ratio estimation layer is responsible for processing the marker position / orientation data and the encoder data, and outputs an estimated rotation axis for each joint. For example, from the geometric properties of rotation, for each i, the 3D marker coordinate P1 i ,···,P T i But, Q j In a circle centered at V j on a plane perpendicular to and at an angle r j α1,···,r j α T are arranged according to the direction base O1 i ,···,O T i is the angle r j α1,···,r j α T According to V j Furthermore, the rotation axis vector V j , rotation axis point Q j and turnover rate r j For example, an optimization method can be applied to a loss function L(V, Q, r) that reflects the magnitude to which the above geometric properties are not satisfied for a given set of 3D vectors V, 3D points Q, and a scalar r. j ,Q j ,r j )=argminL(V,Q,r) to find the desired value.

[0056] The data processing procedure 304 further includes determining a first rotation axis of the first joint and a second rotation axis of the second joint based on the first error function at step 304C. In other words, based on the calculated marker position / orientation data and the servo motor rotation data collected by the first encoder 114A and the second encoder 114B, the information is processed to determine the rotation axis for each joint and the reduction ratio after the joint.

[0057] The data processing procedure 304 further includes, at step 304D, establishing a second error function based on at least the first and second rotational axes. After determining the first and second rotational axes of the first and second joints based on the first error function, a second error function is established using the first and second rotational axes. In particular, one or more error functions may be established based on the calculated 3D positions of the markers. Furthermore, parameters are calculated to define a set of rotational axes of the robotic system 200 based on the error function.

[0058] The data processing procedure 304 further includes, at step 304F, determining a set of motion parameters for the motion model of the robot 100A based on the second error function. In other words, after determining the second error function, the set of motion parameters is determined using the second error function. In particular, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint and outputs calibrated motion parameters.

[0059] According to an embodiment, the method 300 includes calibrating a kinematic model of the robot 100A using the determined set of kinematic parameters. In other words, the set of kinematic parameters is used to calibrate the kinematic model of the robot 100A. Further, calibration is performed including robot link lengths, joint positions and orientations, robot joint zero positions, and reduction ratios for each joint.

[0060] According to the embodiment, the movable part 104 further includes a first servo motor 112A configured to move the first joint, a second servo motor 112B configured to move the second joint, a first encoder 114A provided on the first servo motor 112A, and a second encoder 114B provided on the second servo motor 112B. Furthermore, the camera 110 is synchronized with the first encoder 114A and the second encoder 114B, and the data collection procedure further includes time-stamping each image of the plurality of images. Furthermore, the time-stamp corresponds to a time step within the series of time steps. In other words, a continuous circular movement of each joint is performed. During the joint movement, motor rotary encoder data is collected, and images of markers are placed on the end effector. Further, image processing and calculation of the position and orientation of the markers are performed. Furthermore, the calculated marker position / orientation data is based on the motor rotation data collected by the first encoder 114A and the second encoder 114B. The information is further processed to determine the axis of rotation for each joint and the reduction ratio for each joint.

[0061] According to the embodiment, the first encoder 114A and the second encoder 114B are configured to collect servo values ​​of the first servo motor 112A and the second servo motor 112B, respectively. The data processing procedure further includes determining, for each of the first servo motor 112A and the second servo motor 112B, a plurality of servo motor angles for each servo motor based on the servo values ​​collected by the first encoder 114A and the second encoder 114B, respectively. In other words, at least the first joint and the second joint are connected to the first servo motor 112A and the second servo motor 112B via a gear system having a gear reduction ratio for each joint as a system parameter. The movement of the first servo motor 112A and the second servo motor 112B is transmitted to at least the first joint and the second joint with some reduction ratio coefficient.

[0062] According to an embodiment, the marker positions are determined by extracting, for each image of the plurality of images, a plurality of corner coordinates of at least one calibration marker from the image, each of the plurality of corner coordinates corresponding to a position approximately at a corner of the at least one calibration marker. 108 The image processing layer of T is used to calculate the position and orientation of the calibration marker relative to the camera 110. Furthermore, without loss of generality, the reference marker Δ i is all images I1 to I T Visible and detectable.

[0063] According to an embodiment, the marker positions are determined by calculating, for each image of the plurality of images, a marker center coordinate using a plurality of corner coordinates of at least one calibration marker on that image and one or more intrinsic camera parameters. In other words, the at least one calibration marker performs a circular motion while the joint rotates. Furthermore, the angular trajectory of the joint is determined by the camera 110 moving the marker Δ i Furthermore, for each j = 1,...,J, the sequence of data collection, image processing, and rotation axis and reduction ratio estimation is j ,Q j ) for each joint A j is repeated to obtain an estimate of the position and orientation of the rotation axis for V j is the 3D rotation vector that defines the orientation of the jth rotation axis, and Q j is the position of the rotating shaft and the reduction ratio r j are 3D points that define

[0064] According to an embodiment, the method 300 includes processing a plurality of images, where processing the plurality of images further includes determining, for each image of the plurality of images, a marker orientation of at least one calibration marker in three-dimensional space based on a plurality of corner coordinates of the at least one calibration marker on the image and one or more intrinsic camera parameters, to output as marker position data. For example, without loss of generality, i is all images I1 to I T In implementation, the image processing layer i Frame I where i1 ~I iT The intrinsic parameters of the camera 110 can be assumed to be accurate and known. If not, a camera calibration procedure is performed. Furthermore, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, C t i Frame I t Marker Δ i is a set of (x,y) coordinates of the corners of the camera. Furthermore, the pre-computed intrinsic parameters of the camera and the corner coordinates C t I is the 3D position P of the marker center relative to the camera using the perspective-n-point (PnP) method. t i and 3D marker orientation t i In the example, the marker center P t i The representation of is performed by a three-dimensional vector, O t i The representation of is performed by a 3x3 orthonormal basis matrix.

[0065] According to an embodiment, the method 300 comprises establishing a first error function comprising an arc fitting of a marker center coordinate and a marker orientation of the at least one calibration marker corresponding to a circular movement of the at least one marker relative to the coordinate system of the camera 110. In other words, the error function is based on an arc fitting of a marker's 3D position and a 3D marker orientation corresponding to a circular movement of the at least one calibration marker relative to another marker or the camera coordinate system, and optionally also based on encoder data and fitting the encoder data to the 3D orientation of the at least one calibration marker.

[0066] According to the embodiment, the movable part 104 includes a first servo motor 112A coupled to a first joint via a first gear system having a gear reduction ratio. The method 300 further includes determining a first rotation axis of the first joint, which includes establishing a first error function based on marker center coordinates and marker orientations of at least one calibration marker determined from the multiple images, and applying an optimization function to the first error function to determine a joint rotation axis origin Q, a rotation axis vector V, and a gear reduction ratio r. In other words, the movable part 104 includes at least two joints. The joints are disposed at one end of at least one link, and at least one link movement is performed by rotating at least one pre-joint. Furthermore, at least the first joint and the second joint are connected to the first servo motor 112A and the second servo motor 112B via a gear system having a gear reduction ratio for each joint as a system parameter. The motions of the first servo motor 112A and the second servo motor 112B are transmitted to at least the first joint and the second joint with some reduction ratio coefficient. Further, the axis and reduction ratio estimation layer is responsible for processing the marker position / orientation data and the encoder data, and outputs the estimated rotation axis for each joint. Further, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint, and outputs the calibrated motion parameters.

[0067] According to the embodiment, a first encoder 114A is provided to the first servo motor 112A, configured to output encoder data including a servo value of the first servo motor 112A, and a gear reduction ratio is determined based on the marker orientation of the at least one calibration marker determined from the plurality of images and the encoder data of the first servo motor 112A. In other words, a gear system having a gear reduction ratio for each joint as a system parameter is used to couple at least the first joint and the second joint with the first servo motor 112A and the second servo motor 112B. The at least the first joint and the second joint are moved by the movement of the first servo motor 112A and the second servo motor 112B. Furthermore, any reduction ratio coefficient is also applied with the movement.

[0068] According to an embodiment, the at least one calibration marker includes one or more marker keypoints. Further, the marker keypoints correspond to uniquely identifiable points on the at least one calibration marker, and the data processing procedure further includes extracting the one or more marker keypoints from the plurality of images. For example, a special fiducial marker type, e.g., a 3x3 ChArUco marker, can be attached to the end effector to automatically determine the intrinsic parameters of the camera 110 using motion information from the marker keypoints.

[0069] According to an embodiment, the method includes determining one or more intrinsic camera parameters based on the relative positions of the one or more marker keypoints. For example, the image data I1,...,I T Then, by knowing the ground truth information about the relative positions of the marker keypoints, the intrinsic camera parameters of the camera 110 need to be obtained. Then, by using the obtained camera parameters and the positions of the marker keypoints on the image, the 3D position P of the marker center relative to the camera 110 can be calculated. ti and 3D marker orientation t i is calculated using the PnP method.

[0070] According to an embodiment, the movable part 104 includes a plurality of J joints arranged to be rotated by a corresponding plurality of J servo motors. Furthermore, positioning at least one calibration marker may involve movements M1, M2,...,M to rotate the plurality of J joints. J In other words, a set of predefined commands stored in the computing unit 108 is sent to the first servo motor 112A and the second servo motor 112B. The first servo motor 112A and the second servo motor 112B perform movements M1,...,M J where J is the number of joints in the moving part 104, and M j is determined by moving the jth robot joint while the other joints remain stationary at their zero positions. The joints move and at least one calibration marker, such as calibration marker 106A, is statically attached at an arbitrary position and orientation relative to the moving part of the robot. As a result, the at least one calibration marker undergoes a circular motion while the joint rotates. The angular trajectory of the joint is determined by the camera 110 tracking the marker Δ i Furthermore, for each j = 1,...,J, the sequence of data collection, image processing, and rotation axis and reduction ratio estimation is j ,Q j ) for each joint A j This is repeated to obtain an estimate of the position and orientation of the rotation axis for V. j is the 3D rotation vector that defines the orientation of the jth rotation axis, and Q j is the position of the rotating shaft and the reduction ratio r j are 3D points that define

[0071] According to an embodiment, the movement M of the nth joint, where 1≦n≦J n is determined by controlling the n-th servo motor to rotate the n-th joint while keeping the remaining joints of the plurality of J joints at their zero positions. In other words, the number of joints is equal to the number of servo motors, and as the servo motors move, movement is imparted to the joints. This movement is controlled by the calculation unit 108. When only one servo motor is rotating, only one joint is moving, and in this case, the remaining joints are considered to be at their zero positions.

[0072] According to an embodiment, at least one calibration marker is provided on one or more links connecting two adjacent joints of the plurality of J joints, and an encoder is provided on each servo motor of the plurality of J servo motors, each of the plurality of encoders configured to output encoder data corresponding to each joint of the plurality of J joints. Further, the encoder data includes a servo value (e.g., an angle) of the corresponding servo motor. The movement of the servo motor translates into a rotation of the corresponding joint. When representing the movement, the encoder generates the encoder data, where the encoder estimates the angle of rotation of the joint movement, which corresponds to the servo value.

[0073] According to an embodiment, the movements M1, M2,...,M J is arbitrary, and processing the plurality of images includes, for each n-th joint of the plurality of J joints, transforming, for each image of the plurality of images, a position of at least one calibration marker between the (n-1)-th joint and the n-th joint from a motion frame of the (n-1)-th joint to a stationary frame of the (n-1)-th joint in which the (n-1)-th joint is stationary and at a zero position, using the encoder data of the (n-1)-th joint.

[0074] According to an embodiment, the method 300 includes determining a set of motion parameters, where determining the set of motion parameters includes determining a link length between a first joint and a second joint. In other words, the link length is determined between the first joint and the second joint by a process executed in the computing unit 108. Furthermore, motion parameters related to all links and joints located between the robot base and the position of the at least one calibration marker are calibrated, except for the first and last link lengths, respectively.

[0075] According to an embodiment, determining the set of motion parameters includes determining a marker-based transformation between a frame of at least one calibration marker at a zero position and a frame of the base 102 of the robot 110A using the set of motion parameters; determining a marker-camera transformation between the frame of at least one calibration marker at a zero position and a frame of the camera 110 by processing a plurality of images; and calculating a camera-based transformation between the frame of the camera 110 and the frame of the robot base based on the determined marker-based transformation and the determined marker-camera transformation. In other words, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint and outputs calibrated motion parameters. For example, the set of motion parameters to be calibrated is denoted as L. Furthermore, the estimated rotation axes A1 to A2 of each joint are calculated based on the determined marker-based transformation and the determined marker-camera transformation. j Given the initial values ​​of the motion parameters L0, the motion parameter update layer estimates the actual parameters L1, L2, L3, L4, L5, L6, L7, L8, L9, L10, L11, L12, L13, L14, L15, L16, L17, L18, L19, L20, L21, L22, L23, L24, L25, L26, L27, L28, L29, L30, L31, L32, L40, L41, L42, L43, L54, L55, L66, L77, L88, L99, L109, L110, L121, L132, L142, L153, ​​L164, L175, L186, L197, L198, L199, L209, L210, L221, L232, L242, L254, L265, L276, L287, L298, L299, L309, L41, L42, L43, L54, L65, L109, L110, L121, L222, L232, L242, L254, L265, L309, L41, L42, L43, L55, L66, L109, L110, L121, L222, L232, L242, L254, L265, L309, L41, L42, L55, L66, L109, L275, L287, L298, L299, L309, L41, L42, L55, L67, L109, L209, L210, L221

number

number

number

number

number

[0076] The method 300 calibrates robot kinematic parameters, including robot link lengths, joint positions and orientations, robot joint zero positions, reduction ratios for each joint position and orientation, robot joint zero positions, and reduction ratios for each joint. Using the calibrated parameters, the method 300 can achieve improved accuracy of the end effector position and orientation during kinematic execution.

[0077] Steps 302 and 304 are merely exemplary, and other alternatives may be provided, in which one or more steps are added, one or more steps are removed, or one or more steps are provided in a different order, without departing from the scope of the claims herein.

[0078] A non-transitory computer-readable storage medium is further provided, having machine-readable code that, when executed by a processor, causes the processor to position at least one calibration marker at a plurality of positions by rotating a first joint and / or a second joint. The processor further acquires a plurality of images of the at least one calibration marker, the plurality of images being captured by the camera 110 when the at least one calibration marker is in the plurality of positions. The processor further processes the plurality of images to determine, for each image of the plurality of images, a marker position (and optionally, an orientation) of the at least one calibration marker in three-dimensional space to be output as marker position data. The processor further establishes a first error function based on the marker position data. The processor further determines a first rotation axis of the first joint and a second rotation axis of the second joint based on the first error function. The processor further establishes a second error function based on at least the first rotation axis and the second rotation axis. The processor further determines a set of motion parameters for a motion model of the robot 110A based on the second error function. In an example, the instructions are embodied in a computer-readable medium, including, but not limited to, an electrically erasable programmable read-only memory (EEPROM), a random access memory (RAM), a read-only memory (ROM), a hard disk drive (HDD), flash memory, a secure digital (SD) card, a solid-state drive (SSD), a computer-readable storage medium, and / or a CPU cache memory. In an example, the instructions are generated by a computer program, and the computer program is executed in view of method 300 for use in implementing method 300 by a processor.

[0079] Modifications to the embodiments of the present disclosure described above are possible without departing from the scope of the present disclosure, as defined by the appended claims. The terms "comprising," "having," "incorporating," "having," "being," and the like, used to describe and claim the present disclosure, are intended to interpret the disclosure in a non-exhaustive manner, i.e., there may be items, components, or elements not expressly described. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described as "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments or to exclude the incorporation of features from other embodiments. The word "optionally" is used herein to mean "provided in some embodiments but not in other embodiments." It will be understood that certain features of the present disclosure, while described in the context of separate embodiments for clarity, may also be provided in combination in a single embodiment. Conversely, various features of the invention, while described in the context of a single embodiment for brevity, may also be provided separately or in any suitable combination or with any other described embodiment of the present disclosure, as appropriate.

Claims

1. a robot having a movable part, the movable part having at least one calibration marker thereon, and at least a first joint and a second joint disposed at one end of at least one link, the first joint and the second joint configured to generate a movement in the movable part controlled by a computing unit; a camera configured to capture a plurality of images of the at least one calibration marker at a plurality of positions; the computation unit including a robot control layer, an image processing layer, an axis and reduction ratio estimation layer, and a motion parameter update layer; the robot control layer is configured to control rotations of the first joint and the second joint based on a kinematic model of the robot, and the robot control layer is configured to control rotations of the first joint and / or the second joint to position the at least one calibration marker at the plurality of positions; the image processing layer is configured to process the plurality of images received from the camera to determine, for each image of the plurality of images, a marker position of the at least one calibration marker in three-dimensional space to be output as marker position data; the axis and reduction ratio estimation layer is configured to establish a first error function based on the marker position data and determine a first axis of rotation of the first joint based on the first error function; the motion parameter update layer is configured to establish a second error function based on at least the first rotation axis, and determine a set of motion parameters for the motion model based on the second error function; the image processing layer is further configured to, for each image of the plurality of images, determine the marker position by extracting from the image a plurality of corner coordinates of the at least one calibration marker, each of the plurality of corner coordinates corresponding to a position that is approximately at a corner of the at least one calibration marker. Robot system.

2. the movable part further comprises a first servo motor coupled to the first joint via a first gear system, a second servo motor coupled to the second joint via a second gear system, a first encoder provided on the first servo motor, and a second encoder provided on the second servo motor, the first encoder and the second encoder being configured to collect servo values ​​of the first servo motor and the second servo motor, respectively; and the calculation unit further comprises a clock synchronization layer configured to synchronize the camera with the first encoder and the second encoder so as to timestamp each image of the plurality of images captured by the camera. The robot system of claim 1 .

3. 1. A method for calibrating a robot having a movable part, the movable part having at least one calibration marker provided thereon, and at least a first joint and a second joint arranged at one end of at least one link, the first joint and the second joint being configured to generate a movement in the movable part controlled by a computing unit, the method comprising: In the data collection procedure, positioning the at least one calibration marker at a plurality of positions by rotating the first joint and / or the second joint; acquiring a plurality of images of the at least one calibration marker, the plurality of images being captured by a camera when the at least one calibration marker is in the plurality of positions; In the data processing procedure, processing the plurality of images to determine, for each image of the plurality of images, a marker position of the at least one calibration marker in three-dimensional space to be output as marker position data; establishing a first error function based on the marker position data; determining a first axis of rotation of the first joint and a second axis of rotation of the second joint based on the first error function; establishing a second error function based on at least the first axis of rotation and the second axis of rotation; determining a set of motion parameters for a motion model of the robot based on the second error function; the marker positions are determined by, for each image of the plurality of images, extracting from the image a plurality of corner coordinates of the at least one calibration marker, each of the plurality of corner coordinates corresponding to a position approximately at a corner of the at least one calibration marker. method.

4. calibrating the kinematic model of the robot using the determined set of kinematic parameters. The method of claim 3.

5. the movable part further comprises a first servo motor configured to move the first joint, a second servo motor configured to move the second joint, a first encoder provided on the first servo motor, and a second encoder provided on the second servo motor, the camera is synchronized with the first encoder and the second encoder, and the data collection procedure further comprises assigning a time stamp to each image of the plurality of images, the time stamp corresponding to a time step within a series of time steps. The method according to claim 3 or 4.

6. the first encoder and the second encoder are configured to collect servo values ​​of the first servo motor and the second servo motor, respectively, and the data processing procedure further comprises, for each of the first servo motor and the second servo motor, determining a plurality of servo motor angles for each servo motor based on the servo values ​​collected by the first encoder and the second encoder, respectively. The method of claim 5.

7. the marker positions are determined by calculating, for each image of the plurality of images, a marker center coordinate using the plurality of corner coordinates of the at least one calibration marker on that image and one or more intrinsic camera parameters. The method of claim 5.

8. and processing the plurality of images further comprises determining, for each image of the plurality of images, a marker orientation of the at least one calibration marker in three-dimensional space based on the plurality of corner coordinates of the at least one calibration marker on that image and one or more intrinsic camera parameters, to output as the marker position data. The method of claim 7.

9. establishing the first error function includes arc-fitting the marker center coordinates and the marker orientation of the at least one calibration marker corresponding to a circular motion of the at least one calibration marker relative to a coordinate system of the camera. The method of claim 8.

10. the movable part includes a first servo motor coupled to the first joint via a first gear system having a gear reduction ratio, and determining the first rotation axis of the first joint includes establishing the first error function based on the marker center coordinates and the marker orientation of the at least one calibration marker determined from the plurality of images, and applying an optimization function to the first error function to determine a joint rotation axis origin, a rotation axis vector, and the gear reduction ratio.

10. The method according to claim 8 or 9.

11. a first encoder is provided on the first servo motor configured to output encoder data including a servo value of the first servo motor, and the gear reduction ratio is determined based on the marker orientation of the at least one calibration marker determined from the plurality of images and the encoder data of the first servo motor. The method of claim 10.

12. the at least one calibration marker includes one or more marker keypoints, the marker keypoints corresponding to uniquely identifiable points on the at least one calibration marker, and the data processing procedure comprises: extracting the one or more marker keypoints from the plurality of images; determining one or more intrinsic camera parameters based on the relative positions of the one or more marker keypoints; Further comprising:

12. The method according to any one of claims 3 to 11.

13. The movable part has a plurality of J joints arranged to be rotated by a corresponding plurality of J servo motors, and positioning the at least one calibration marker involves movement M to rotate the plurality of J joints. １ , M ２ , ..., M Ｊ controlling the plurality of J servo motors to execute a set of 13. The method according to any one of claims 3 to 12.

14. For 1≦n≦J, the movement M of the nth joint ｎ is determined by controlling an n-th servo motor to rotate the n-th joint while keeping the remaining joints of the plurality of J joints at their respective zero positions. The method of claim 13.

15. at least one calibration marker is provided on one or more links connecting two adjacent joints of the plurality of J joints, and a plurality of encoders are provided on corresponding servo motors of the plurality of J servo motors, each of the plurality of encoders being configured to output encoder data corresponding to each joint of the plurality of J joints, the encoder data including a servo value of the corresponding servo motor; 15. The method of claim 13 or 14.

16. Movement M １ , M ２ , ..., M Ｊ is any set, and processing the plurality of images includes, for each n-th joint of the plurality of J joints, transforming, for each image of the plurality of images, a position of the at least one calibration marker between the (n-1)-th joint and the n-th joint from a motion frame of the (n-1)-th joint to a stationary frame of the (n-1)-th joint in which the (n-1)-th joint is stationary and at a zero position, using the encoder data of the (n-1)-th joint.

16. The method of claim 15.

17. determining the set of motion parameters includes determining a link length between the first joint and the second joint; 17. The method according to any one of claims 3 to 16.

18. determining the set of motion parameters includes determining a marker-based transformation between a frame of the at least one calibration marker at a zero position and a frame of a base of the robot using the set of motion parameters, determining a marker-to-camera transformation between the frame of the at least one calibration marker at the zero position and a frame of the camera by processing the plurality of images, and calculating a camera-based transformation between the frame of the camera and the frame of the base of the robot based on the determined marker-based transformation and the determined marker-to-camera transformation.

18. The method according to any one of claims 3 to 17.

19. a processor that executes the program; and a program that executes the program. The program includes machine-readable code that, when executed by a processor, causes the processor to: positioning at least one calibration marker provided on a movable part of a robot at a plurality of positions by rotating a first joint and / or a second joint, the movable part having at least the first joint and the second joint disposed at one end of at least one link, the first joint and the second joint configured to produce a movement in the movable part controlled by the processor; acquiring a plurality of images of the at least one calibration marker, the plurality of images being captured by a camera when the at least one calibration marker is in the plurality of positions; processing the plurality of images to determine, for each image of the plurality of images, a marker position of the at least one calibration marker in three-dimensional space to be output as marker position data; establishing a first error function based on the marker position data; determining a first axis of rotation of the first joint and a second axis of rotation of the second joint based on the first error function; establishing a second error function based on at least the first axis of rotation and the second axis of rotation; determining a set of motion parameters for a motion model of the robot based on the second error function; Execute the marker positions are determined by, for each image of the plurality of images, extracting from the image a plurality of corner coordinates of the at least one calibration marker, each of the plurality of corner coordinates corresponding to a position approximately at a corner of the at least one calibration marker. A non-transitory computer-readable storage medium.

20. A program which, when executed by a processor, causes the processor to carry out a method according to any one of claims 3 to 18.

Images