ROBOT SYSTEM HAVING ROBOT AND CALIBRATION METHOD FOR ROBOT

Abstract

The robot system includes a robot having a movable part with a calibration marker and at least a first joint and a second joint, a camera for capturing images of the calibration marker, and a computation unit including a robot control layer for controlling rotation of the first joint and the second joint based on a kinematic model of the robot to position the calibration at a plurality of positions, an image processing layer for processing images received from the camera to determine marker position data for each image, an axis and reduction ratio estimation layer for establishing a first error function based on the marker position data to determine a first rotation axis of the first joint, and a kinematic parameter update layer for establishing a second error function based on the at least the first rotation axis to determine a set of kinematic parameters for the kinematic model.

Description

[Technical field]

[0001] The present disclosure relates generally to the field of robotics, and more specifically to a robotic system having a robot and a method for calibrating a robot. [Background technology]

[0002] In today's world, the growth of technology is progressing at a rapid rate. With this rapid advancement in technology, most of the manual tasks are now performed automatically, for example, by robots, and hence, the manufacturing of robots has grown at a tremendous rate. Various parameters require calibration during the utilization of a robot, and one of these parameters is the kinematic parameters. The kinematic parameters of a robot describe the geometry of the robot's movements and are used to relate the position of the robot's end effector to the position of the robot's base through the joint values. If the kinematic parameters are not calibrated, they will have large errors due to the robot's manufacturing defects, continuous wear during use, and accumulated zero position offsets of the joints. As a result, accurate calibration methods are desired for robots. A widely used set of kinematic parameters are the Denavit-Hartenberg (DH) parameters, which allow the position of the robot's end effector to be calculated based on the joint values. Other sets of kinematic parameters include the transition between the camera, the robot base coordinate system, and the gear reduction ratio of the robot.

[0003] Traditionally, calibration is done by laser tracker systems, one part of which measures the robot's body position, and another part of the laser tracker system measures the robot's end-effector position. However, such calibration is complex and inefficient due to its manual nature, and unnecessary equipment is costly. Other approaches include the introduction of cameras on the robot, which are used at multiple positions on the robot. Furthermore, markers are placed close enough to the camera's beam to compute an accurate vector for each robot position. However, all positions of the robot used for calibration need to be close enough to the camera's beam. This means that the motion parameters are only subject to calibration within a limited configuration space of the robot, which is a technical challenge.

[0004] In light of the above discussion, it is desirable to address the above-mentioned shortcomings associated with robot calibration. Summary of the Invention

[0005] The present disclosure relates to a robotic system having a robot, and a method for calibrating the robot. It is an object of the present disclosure to at least partially overcome problems encountered in the prior art by providing an improved robotic system having a robot, and an improved method for automatically calibrating the kinematic parameters of the robot, for example to reduce errors in the position and orientation of the robot's end effector, and to increase the accuracy of the robot's execution of operations.

[0006] One or more objects of the present disclosure are achieved by the attached independent claims. Optional or alternative implementations of the present disclosure are further defined in the dependent claims.

[0007] In one aspect, the present disclosure provides a robot system. The robot system includes a robot further including a moving part, the moving part being provided with at least one calibration marker thereon. The robot system further includes at least a first joint and a second joint disposed at one end of at least one link, the first joint and the second joint configured to generate a movement in the moving part controlled by a computing unit. The robot system further includes a camera configured to capture images of the at least one calibration marker at a plurality of positions. The robot system further includes a computing unit including a robot control layer, an image processing layer, an axis and reduction ratio estimation layer, and a motion parameter update layer. The robot control layer is configured to control rotation of the first joint and the second joint based on a motion model of the robot. The robot control layer is further configured to control rotation of the first joint and / or the second joint to a position of the at least one calibration marker at a plurality of positions. The image processing layer is configured to process images received from the camera to determine, for each image, a marker position of the at least one calibration marker in a three-dimensional space to be output as marker position data. The axis and reduction ratio estimation layer is configured to establish a first error function based on the marker position data and determine a first rotation axis of the first joint based on the first error function. The motion parameter update layer is configured to establish a second error function based on at least the first rotation axis and determine a set of motion parameters for the motion model based on the second error function.

[0008] The robot system includes a moving part of the robot. The moving part of the robot is provided with at least one calibration marker. The calibration marker serves to analyze a position of the moving part of the robot, and the camera is used to capture multiple positions of the calibration marker. Furthermore, the movement at the moving part is generated by a first joint and a second joint, one joint value being different from a zero position and the other joint value being at the zero position during all collection procedures. Furthermore, the calculation of the rotation axis parameters and the calculation of the motion parameters include minimization of an error function. Furthermore, the first joint and the second joint are controlled by a calculation unit. The calculation unit includes a robot control layer that controls the rotation of the first joint and / or the second joint to change the position of one of the calibration markers. The calculation unit further includes an image processing layer that serves to provide marker position data by analyzing images captured by the camera. The calculation unit further includes an axis and reduction ratio estimation layer that analyzes the marker position data to establish a first error and to determine a first rotation axis of the first joint based on the first error function. The computation unit further includes a motion parameter update layer that analyzes the first rotation axis to establish a second error and determine a set of motion parameters for the motion model based on the second error function. Advantageously, the robot system calibrates the robot's motion parameters including robot link lengths, joint positions and orientations, robot joint zero positions, and reduction ratios per joint. With the calibrated parameters, the robot can achieve better accuracy of the position and orientation of the end effector during motion execution.

[0009] In an embodiment, the moving part further comprises a first servo motor coupled to the first joint via a first gear system, a second servo motor coupled to the second joint via a second gear system, a first encoder provided on the first servo motor, and a second encoder provided on the second servo motor, the first encoder and the second encoder configured to collect servo values ​​of the first servo motor and the second servo motor, respectively.Furthermore, the calculation unit comprises a clock synchronization layer configured to synchronize the camera with the first encoder and the second encoder, and optionally with the rotation angle provided by the encoder, to timestamp images captured by the camera.

[0010] The servo motors can rotate in both clockwise and counterclockwise directions, and a gear system helps to transmit the motion of the servo motors to the joints.

[0011] In another aspect, there is a method of calibrating a robot including a moving part, the moving part being provided with at least one calibration marker thereon. Furthermore, at least a first joint and a second joint are disposed at one end of at least one link, the first joint and the second joint being configured to generate a movement in the moving part controlled by a computing unit. The method further includes a data acquisition procedure including positioning the at least one calibration marker at a plurality of positions by rotating the first joint and / or the second joint. Further, the method includes acquiring a plurality of images of the at least one calibration marker, where the plurality of images are captured by a camera when the at least one calibration marker is in a plurality of positions. Further, a data processing procedure is disclosed for processing the plurality of images to determine, for each image of the plurality of images, a marker position (and optionally an orientation) of the at least one calibration marker in three-dimensional space to be output as marker position data. The method further discloses for establishing a first error function based on the marker position data. Thereafter, determining a first axis of rotation of the first joint and a second axis of rotation of the second joint based on the first error function. Further, a second error function is established based on at least the first rotation axis and the second rotation axis, and a set of motion parameters for a motion model of the robot is determined based on the second error function.

[0012] The method achieves at least some of the benefits and technical effects of the robotic system of the present disclosure.

[0013] It should be noted that all devices, elements, circuits, units and means described herein may be implemented with software or hardware elements or any kind of combination thereof. The steps performed by various entities described herein and the functions described as being performed by various entities are intended to mean that each entity is adapted or configured to perform each step or function. Even if in the following description of specific embodiments, a specific function or step performed by an external entity is not reflected in the description of the specific detailed elements of the entity performing the specific step or function, it will be clear to a person skilled in the art that these methods and functions may be implemented with each software or hardware element or any kind of combination thereof. It will be understood that the features of the present disclosure may be combined in various combinations without departing from the scope of the present application as defined in the appended claims.

[0014] Further aspects, advantages, features and objects of the present disclosure will become apparent from the following detailed description of exemplary implementations and drawings taken in conjunction with the appended claims. [Brief description of the drawings]

[0015] The foregoing summary and the following detailed description of exemplary embodiments will be better understood when read in conjunction with the accompanying drawings. For the purpose of illustrating the disclosure, exemplary structures of the disclosure are shown in the drawings. However, the disclosure is not limited to the specific methods and instrumentalities disclosed therein. In addition, persons skilled in the art will appreciate that the drawings are not to scale. Wherever possible, like elements are designated by like numerals.

[0016] Embodiments of the present disclosure will now be described, by way of example only, with reference to the following drawings, in which:

[0017] [Figure 1A]FIG. 1 is a schematic diagram of an architecture of a robotic system according to various embodiments of the present disclosure. [Figure 1B] FIG. 1 is a schematic diagram of an architecture of a robotic system according to various embodiments of the present disclosure. [Diagram 2] FIG. 1 is a block diagram of a robotic system according to an embodiment of the present disclosure. [Diagram 3] 1 is a flowchart of a method for calibrating a robot according to an embodiment of the present disclosure.

[0018] In the accompanying drawings, underlined numbers are used to identify the item the underlined number is above or next to. Numbers without underlines relate to the item identified by the line connecting the ununderlined number to the item. When a number is not underlined and has an associated arrow, the ununderlined number is used to identify the general item to which the arrow is pointing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The following detailed description describes embodiments of the present disclosure and how they may be practiced. Although several modes of carrying out the disclosure are disclosed, those skilled in the art will recognize that other embodiments are possible for carrying out or practicing the disclosure.

[0020] 1A and 1B are schematic diagrams of the architecture of a robot of a robotic system according to various embodiments of the present disclosure. 1A and 1B are collectively described, and 1B is an embodiment of 1A. Referring to 1A, robots such as robot 100A (of 1A) and robot 100B (of 1B) are shown. Robot 100A includes a base 102, a moving part 104, a calibration marker 106A, a computing unit 108, a camera 110, a first servo motor 112A and a second servo motor 112B, and a first encoder 114A and a second encoder 114B. Robot 100B includes all the same elements as robot 100A (computing unit 108 is not shown), and in addition, robot 100B further includes calibration markers 106B and 106C.

[0021] The disclosed robots 100A and 100B correspond to basic robots used to automatically calibrate the motion parameters of a robotic system by using an industrial camera, such as camera 110, and at least one reference marker, such as calibration marker 106A. Robot 100B further includes calibration markers 106B and 106C.

[0022] The base 102 of the robot 100A is used to provide balance to the moving part 104 of the robot 100A. The moving part 104 is an arm of the robot 100A.

[0023] Calibration markers 106A, 106B, and 106C are used to provide marker position data. Calibration markers 106A, 106B, and 106C are attached to the end effector of robot 100A. At least one calibration marker is used for calibration purposes.

[0024] The computing unit 108 may include suitable logic, circuitry, interfaces, and / or code configured to control the robot 100A. Example implementations of the computing unit 108 may include, but are not limited to, a central data processing device, a microprocessor, a microcontroller, a multiple instruction set computing (CISC) processor, an application specific integrated circuit (ASIC) processor, a reduced instruction set (RISC) processor, a very long instruction word (VLIW) processor, a state machine, and other processors or control circuits.

[0025] The camera 110 may include suitable logic, circuitry, interfaces, and / or code configured to capture an image of the position of the calibration marker 106A.

[0026] The first servo motor 112A and the second servo motor 112B are used to provide motion to the arm of the robot 110A. The first servo motor 112A and the second servo motor 112B can move in clockwise and counterclockwise directions. Additionally, the first encoder 114A and the second encoder 114B are used to determine the rotation (e.g., by calculating the rotation angle) of the first servo motor 112A and the second servo motor 112B, respectively.

[0027] In one aspect, a robot system is provided. The robot system includes a robot 100A further including a moving part 104. The moving part 104 is provided with at least one calibration marker thereon. The robot system further includes at least a first joint and a second joint disposed at one end of at least one link, the first joint and the second joint configured to generate a movement in the moving part 104 controlled by a computing unit 108. In other words, the moving part 104 corresponds to an arm of the robot 100A. The moving part 104 includes any number of joints disposed at one end of the at least one link. The movement of the at least one link is performed by the rotation of at least one pre-joint. Furthermore, the computing unit 108 controls at least the first joint and the second joint to cause a movement in the moving part 104, which is further beneficial for the movement of the at least one calibration marker, such as the movement of the calibration marker 106A. In an implementation, the computing unit 108 includes a robot control layer that stores a set of predefined commands for the movement of the first joint and the second joint. It should be noted that the robot control layer and the computing unit 108 can be the same device or separate devices in a distributed system. Furthermore, the robot control layer sends a set of predefined commands to the first servo motor 112A and the second servo motor 112B to cause movements therein.

[0028] According to the embodiment, the first servo motor 112A is coupled to the first joint via a first gear system, and the second servo motor 112B is coupled to the second joint via a second gear system. Furthermore, a first encoder 114A is provided on the first servo motor 112A, and a second encoder 114B is provided on the second servo motor 112B. Furthermore, the first encoder 114A and the second encoder 114B are configured to collect servo values ​​(e.g., angles) of the first servo motor 112A and the second servo motor 112B, respectively. Furthermore, the calculation unit 108 further includes a clock synchronization layer configured to synchronize the camera 110 with the first encoder 114A and the second encoder 114B so as to timestamp the images and servo values ​​captured by the camera 110. In other words, at least the first joint and the second joint are connected to the first servo motor 112A and the second servo motor 112B, respectively, via a gear system with a reduction ratio for each joint as a system parameter. The motions of the first servo motor 112A and the second servo motor 112B are transferred to at least the first joint and the second joint with some reduction ratio coefficient. Furthermore, a clock synchronization layer present in the calculation unit 108 performs synchronization for time stamps. Furthermore, the clock synchronization layer synchronizes the camera 110 with the first encoder 114A and the second encoder 114B.

[0029] Furthermore, the robotic system includes a camera 110 configured to capture images of at least one calibration marker in multiple positions. In an implementation, the camera 110 includes a perception layer. Furthermore, synchronization of the camera 110 with the first encoder 114A and the second encoder 114B is performed by time stamping the images and encoder data captured by the camera 110. For example, the motion M J During this time, the camera 110 captures multiple images of at least one calibration marker, such as the calibration marker 106A. For example, the camera 110 captures multiple images I1-I T Then, T is the motion M JThe image is then sent to the calculation unit 108 for further processing. J and M J+1 During the period, the first encoder 114A and the second encoder 114B collect information from the first servo motor 112A of joint J and the second servo motor 112B of joint J+1 and send the information to the calculation unit 108. Furthermore, synchronization is performed between the encoder clock and the camera clock. In the example, the angle values ​​(e.g., values ​​a1 to a T ) is available without any synchronization. In implementation, synchronization is performed so that a time tag for each image and each angle is obtained to establish the first error function.

[0030] The computation unit 108 in FIG. 1A further includes a robot control layer, an image processing layer, an axis and reduction ratio estimation layer, a motion parameter update layer, and a clock synchronization layer. The robot control layer is configured to control the rotation of the first joint and the second joint based on a motion model of the robot 100A, and the robot control layer is configured to control the rotation of the first joint and / or the second joint to position at least one calibration marker at a plurality of positions. In other words, there are various layers in the computation unit 108 that are used to perform a calibration process in a robot system. Furthermore, the robot control layer provides a movement to the arm of the robot 100A to change the position of at least one calibration marker, such as the calibration marker 106A. For example, the robot control layer controls the position of several reference markers Δ i (e.g., i∈[1,2]) is statically attached to the end effector of the robot arm at an arbitrary position and orientation. Furthermore, at least one calibration marker, such as calibration marker 106A, allows calibration of the position and orientation of robot 100A in three-dimensional (3D) space based on its image, e.g., taking into account the intrinsic parameters of camera 110 and real-world marker dimensions.

[0031] The image processing layer is configured to process images received from the camera 110 to determine, for each image, a marker position and (optionally) an orientation of at least one calibration marker in 3D space, which should be output as marker position data. In other words, the image processing is performed by acquiring data from the camera 110 and outputting a set of position / orientation data for each observed marker together with a corresponding timestamp. The image processing layer of the computation unit 102A converts, for example, image data I1-I T By using the reference markers Δ i The calibration markers 106A, 106B, and 106C are all aligned in all images I1 to I T In other cases, the image processing layer may i is detected by the camera 110. i1 ~I iT Furthermore, the intrinsic parameters of the camera 110 may be assumed to be accurate and known. In implementation, a camera calibration procedure is also performed on the intrinsic parameters of the camera 110. Then, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, C t i Frame I t Marker Δ i In the example, the pre-computed intrinsic parameters of the camera 110 and the corner coordinates C t I is the 3D position P of the marker center relative to the camera 110 using the perspective-n-point (PnP) method. t i and 3D marker orientation O t i It is used to calculate Pt i The representation of is carried out by a three-dimensional vector, O t i The representation of is carried out using a 3 × 3 orthonormal basis matrix.

[0032] Further, the axis and reduction ratio estimation layer is configured to establish a first error function based on the marker position data, and determine a first rotation axis of the first joint based on the first error function. In other words, the axis and reduction ratio estimation layer is responsible for processing the marker position / orientation data as well as the encoder data, and outputs an estimated rotation axis for each joint. For example, from the geometric properties of rotation, for each i, the 3D marker coordinate P1 i ,···,P T i But, Q j In a circle centered at, V j on a plane perpendicular to and at an angle r j α1,...,r j α T are arranged according to the direction base O1 i ,···,O T i is the angle r j α1,...,r j α T According to V j Furthermore, the rotation axis vector V j , rotation axis point Q j and turnover rate r j For example, an optimization method can be applied to a loss function L(V,Q,r) that reflects the degree to which the above geometric properties are not satisfied for a given set of 3D vectors V, 3D points Q, and a scalar r. j ,Q j ,r j )=argminL(V,Q,r) to find the desired value.

[0033] Further, the motion parameter update layer is configured to establish a second error function based on at least the first rotation axis, and determine a set of motion parameters for the motion model based on the second error function. In other words, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint, and outputs the calibrated motion parameters. For example, L represents a set of motion parameters to be calibrated. In implementation, the estimated rotation axis A1 to A2 of each joint are j Given the initial values ​​of the motion parameters L0, the motion parameter update layer estimates the actual parameters

number

number

number

number

number

number

number

number

[0034] The robot system calibrates the kinematic parameters of the robot 100A, including robot link lengths, joint positions and orientations, robot joint zero positions, reduction ratios per joint position and orientation, and reduction ratios per joint. With the calibrated parameters, the robot 100A can achieve improved accuracy of the end effector position and orientation during motion execution. According to an embodiment, the calibration procedure may be performed without encoder data. Multiple images with markers are sufficient to provide the kinematic model parameters and hand-eye calibration.

[0035] According to an embodiment, a special fiducial marker type, for example, a 3×3 ChArUco marker, can be attached to the end effector to automatically determine the intrinsic parameters of the camera 110 using motion information from the marker keypoints. In an embodiment, the marker keypoints are stored in the image data I1-I T Furthermore, by knowing the ground truth information about the relative positions of the marker's keypoints, the intrinsic camera parameters of the camera 110 are obtained. Furthermore, by using the obtained camera parameters and the positions of the marker's keypoints on the image, the 3D position P t i and 3D marker orientation O t i is calculated by the PnP method.

[0036] According to another embodiment, at least one marker is attached to each link of the robot. This setup allows arbitrary movements of the robot during calibration and online kinematic parameter calibration. If arbitrary movements are applied, encoder values ​​may be needed for zero position calibration. This embodiment can be applied to any robot architecture with revolute joints allowing arbitrary link coupling. The difference between the embodiment and the main disclosure is that the movements of the robot arm can be arbitrary. In this embodiment, the camera 110 captures images I1,...,I T Then, the rotary encoder data α1 for joint j=1,...,J is captured, where T is the total number of frames. j ,...,α T j In addition, the position P1 of the marker attached to the link j=1,...,J is collected. j , P Tj and orientation O1 j ,···,O T j Further, the rotation axis and reduction ratio estimation is performed by obtaining data P1 from the first joint in order to obtain V1, Q1, and r1 of the first joint.1 ,···,P T 1 ;O1 1 ,···,O T 1 ;α1 1 ,...,α T 1 For each t = 1,...,T, the angle value (-r1α t 1 ) 3D affine transformation τ corresponding to the rotation of 3D space around axis A1 = (V1, Q1) by t 1 Furthermore, for t=1, ,T,

number

number

number

number

[0037] In another embodiment, the camera 110 is mounted to the end effector and the fiducial markers are mounted not to the end effector but to a fixed workspace visible to the camera 110. In this embodiment, the image processing is performed by capturing a series of marker positions P1,...,P T and a set of marker orientation bases O1,...,O T Then, the rotation axis and reduction ratio estimation is performed to obtain -P1,...,-P T and O1 -1 ,···,O T -1 etc., and finally, motion parameter adjustment is performed.

[0038] In other embodiments, multiple cameras are used in case the markers cannot be captured due to spatial obstacles. The camera 110 is preferably in a stationary position. In other embodiments, instead of using multiple cameras, the camera 110 can have a predefined trajectory in case there are spatial obstacles.

[0039] FIG. 2 is a block diagram of a robot system according to an embodiment of the present disclosure. FIG. 2 is described with elements from FIG. 1A and FIG. 1B. Referring to FIG. 2, a robot system 200 for calibrating a robot is shown. The robot system 200 includes a robot control 204, a robot servo 206, an encoder layer 208, sensing 210, image processing 212, clock synchronization 214, axis and reduction ratio estimation 216, and motion parameter update 218. The robot system 200 further includes a robot 100A (or 100B), a computation unit 108, and a camera 110. FIG. 2 is step-by-step described from step 202A to 202K.

[0040] The robot control 204 of the robot system 200 is used to control the robot 100A by providing commands for each movement. The encoder layer 208 in the robot system 200 is used to capture angles and movements. The sensing 210 may include the camera 110 to gather visual information. The image processing 212 is a layer present in the computation unit 108 to analyze images captured by the camera 110. The clock synchronization 214 is a layer present in the computation unit 108 to synchronize the camera 110 with the encoder layer 208. Furthermore, the axis and reduction ratio estimation 216 should provide the estimated rotation axis of each joint. Furthermore, the motion parameter update 218 is used to provide the calibrated motion parameters.

[0041] In step 202A, the robot control 204 in the computing unit 108 provides commands for movement. The robot control 204 stores a set of predefined commands for the movement of at least a first joint and a second joint. It should be noted that the robot control 204 and the computing unit 108 may be the same device.

[0042] In step 202B, the robot control 204 sends a set of predetermined commands to the robot servo 206 to cause movement in the robot servo 206.

[0043] In step 202C, the angle of the robot servo 206 is captured by the encoder layer 208. In other words, at least the first joint and the second joint are connected to the robot servo 206 via a gear system with a gear reduction ratio for each joint as a system parameter. The movement of the robot servo 206 is transmitted to at least the first joint and the second joint with some reduction ratio coefficient. Furthermore, the clock synchronization 214 performs synchronization for time stamps.

[0044] In step 202D, the vision method is collected by sensing 210. In step 202E, clock synchronization 214 synchronizes the camera 110 with the encoder layer 208. In other words, the perception layer 112 is within the camera 110. The synchronization between the camera 110 and the encoder layer 208 is performed by time stamping the images captured by the camera 110. For example, the motion M J During this time, the camera captures multiple images I1 to I2 of at least one calibration marker. T Then, T is the motion M J The images are sent to the computation unit 108 for processing. J During this time, the encoder layer 208 collects information from the robot servo 206 of the joint J and sends the information to the computation unit 102A. The clock synchronization 214 synchronizes the servo angles a1 to a T The encoder clock and the camera clock are synchronized to obtain

[0045] In step 202F, image processing 212 is performed, which involves acquiring data from the camera 110 and outputs a set of position / orientation data for each observed marker along with a corresponding timestamp. The image processing of the computation unit 108 converts the image data I1 to I T is used to calculate the position and orientation of the marker relative to the camera 110. Without loss of generality, the reference marker Δ i is all images I1 to I T In other cases, the image processing layer may i Frame I where i1 ~I iT The intrinsic parameters of the camera 110 may be assumed to be accurate and known. If not, a camera calibration procedure is performed. Furthermore, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, Ct i Frame I t Marker Δ i Furthermore, the pre-computed intrinsic parameters of the camera 110 and the corner coordinates C t I is the 3D position P of the marker center relative to the camera 110 using the perspective-n-point (PnP) method. t i and 3D marker orientation O t i In the example, the marker center P t i The representation of is carried out by a three-dimensional vector, O t i The representation of is carried out using a 3 × 3 orthonormal basis matrix.

[0046] In step 202G, the visual information collected by sensing 210 is time-stamped by clock synchronization 214. Collectively in steps 202H and 202I, axis and reduction ratio estimation is performed after calculating the positions and orientations of the calibration markers 106A, 106B, 106C with respect to the camera 110. The axis and reduction ratio estimation 216 involves processing the calibration marker position / orientation data and the encoder data, and outputs estimated rotation axes for each joint.

[0047] In step 202J, the estimated rotation axes for each joint are processed by the motion parameter update 120. The motion parameter update 218 is responsible for processing the estimated rotation axes for each joint and outputs calibrated motion parameters.

[0048] At step 202K, the motion parameter calibration procedure is completed. The updated motion parameters and transformation operators complete the motion parameter calibration procedure. The transformation between the marker frame at the zero position and the robot base frame obtained by the calibrated motion parameters, together with the transformation between the camera frame obtained by image processing and the marker frame at the zero position, forms the transformation between the camera 110 and the base 102, thus completing the hand-eye calibration.

[0049] The robotic system 200 calibrates the motion parameters of the robot 100, including robot link lengths, joint positions and orientations, robot joint zero positions, reduction ratios for each joint position and orientation, and robot joint zero positions. With the calibrated parameters, the robotic system 200 can achieve improved accuracy of the position and orientation of the end effector during motion execution.

[0050] Fig. 3 is a flow chart of a method for calibrating a robot according to an embodiment of the present disclosure. Fig. 3 is described in conjunction with elements from Figs. 1A, 1B, and 2. Fig. 3 is described step by step from steps 302 to 304.

[0051] The method 300 is used to calibrate a robot 100A having a moving part 104, the moving part 104 being provided with at least one calibration marker thereon, the moving part 104 having at least a first joint and a second joint arranged at one end of at least one link, the first joint and the second joint being configured to generate a movement in the moving part 104 controlled by a computing unit 108. In other words, the moving part 104 forms an arm of the robot 100A. The moving part 104 comprises any number of joints. The joints are arranged at either end of at least one link. The movement of at least one link is performed by a rotation of an adjacent joint. The computing unit 108 controls the first joint and the second joint to bring about a movement in the moving part 104, and data is collected by evaluation of the movement of the at least one calibration marker. For example, in FIG. 3, in step 302, the method 300 starts with performing a data collection procedure. The data collection procedure 302 includes positioning at least one calibration marker at multiple positions by rotating the first joint and / or the second joint at step 302A. In other words, data is collected by changing the position of at least one calibration marker, such as the calibration marker 106A. Furthermore, the first joint and / or the second joint may be connected to the first servo motor 112A and the second servo motor 112B via a gear system with system parameters including a gear reduction ratio for each joint. The movement of the first servo motor 112A and the second servo motor 112B is transferred to the first joint and / or the second joint, which in turn transfers the movement to the at least one calibration marker.

[0052] The data collection procedure 302 further includes, at step 302B, acquiring multiple images of the at least one calibration marker. In implementation, the multiple images are captured by the camera 110 when the at least one calibration marker is in multiple positions. In other words, synchronization of the camera 110 with the first encoder 114A and the second encoder 114B is performed by time stamping the images captured by the camera 110. For example, the motion M J During this time, the camera 110 captures multiple images I1 to I2 of at least one calibration marker. T Then, T is the motion M J The images are sent to the computation unit 108 for processing. J During this time, the first encoder 114A and the second encoder 114B collect information from the first servo motor 112A and the second servo motor 112B of the joint J and send the information to the computing unit 108.

[0053] The method 300 further comprises performing a data processing procedure at step 304. In other words, the collected data is further processed by evaluating the movement of the at least one calibration marker. The procedure 304 is further divided into various steps such as steps 304A, 304B, 304C, 304D, and 304E.

[0054] The data processing procedure 304 comprises, at 304A, processing the plurality of images to determine, for each image of the plurality of images, a marker position of at least one calibration marker in three-dimensional space to be output as marker position data. In other words, the image processing is performed by acquiring data from the camera 110 and outputting a set of position / orientation data with a corresponding timestamp for each observed marker. In implementation, the image processing layer of the computation unit 102A processes, for example, image data I1-I TIn the implementation, without loss of generality, the reference marker Δ i is all images I1 to I T In another implementation, the image processing layer 114 may i Frame I where i1 ~I iT The intrinsic parameters of the camera 110 may be assumed to be accurate and known. If not, a camera calibration procedure is performed. Furthermore, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, C t i Frame I t Marker Δ i In addition, the pre-computed intrinsic parameters of the camera and the corner coordinates C t I is the 3D position P of the marker center relative to the camera using the perspective-n-point (PnP) method. t i and 3D marker orientation O t i In the example, the marker center P t i The representation of is carried out by a three-dimensional vector, O t i The representation of is carried out using a 3 × 3 orthonormal basis matrix.

[0055] The data processing procedure 304 further comprises, in step 304B, establishing a first error function based on the marker position data. In other words, the marker position data is essentially data acquired by image processing. During image processing, the positions of at least one calibration marker in the image captured by the camera 110 are marked. These marked positions of the at least one calibration marker are referred to as marker position data. Furthermore, the axis and reduction ratio estimation layer is responsible for processing the marker position / orientation data and the encoder data, and outputs an estimated rotation axis for each joint. For example, from the geometric properties of rotation, for each i, the 3D marker coordinate P1 i ,···,P T i But, Q j In a circle centered at, V j on a plane perpendicular to and at an angle r j α1,...,r j α T are arranged according to the direction base O1 i ,···,O T i is the angle r j α1,...,r j α T According to V j Furthermore, the rotation axis vector V j , rotation axis point Q j and turnover rate r j For example, an optimization method can be applied to a loss function L(V,Q,r) that reflects the degree to which the above geometric properties are not satisfied for a given set of 3D vectors V, 3D points Q, and a scalar r. j ,Q j ,r j )=argminL(V,Q,r) to find the desired value.

[0056] The data processing procedure 304 further includes determining a first rotation axis of the first joint and a second rotation axis of the second joint based on the first error function at step 304C. In other words, based on the calculated marker position / orientation data and the servo motor rotation data collected by the first encoder 114A and the second encoder 114B, the information is processed to determine the rotation axis for each joint and the reduction ratio after the joint.

[0057] The data processing procedure 304 further includes, at step 304D, establishing a second error function based on at least the first rotation axis and the second rotation axis. After determining the first rotation axis of the first joint and the second rotation axis of the second joint based on the first error function, a second error function is established using the first rotation axis and the second rotation axis. In particular, one or more error functions may be established based on the calculated 3D positions of the marker. Further, parameters are calculated to define a set of rotation axes of the robotic system 200 based on the error function.

[0058] The data processing procedure 304 further includes, at step 304F, determining a set of motion parameters for the motion model of the robot 100A based on the second error function. In other words, after determining the second error function, a set of motion parameters is determined using the second error function. In particular, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint and outputs the calibrated motion parameters.

[0059] According to an embodiment, the method 300 includes calibrating a kinematic model of the robot 100A using the determined set of kinematic parameters. In other words, the set of kinematic parameters is used to calibrate a kinematic model of the robot 100A. Further calibration is performed including robot link lengths, joint positions and orientations, robot joint zero positions, and reduction ratios per joint.

[0060] According to an embodiment, the movable part 104 further comprises a first servo motor 112A configured to move the first joint, a second servo motor 112B configured to move the second joint, a first encoder 114A provided on the first servo motor 112A, and a second encoder 114B provided on the second servo motor 112B. Furthermore, the camera 110 is synchronized with the first encoder 114A and the second encoder 114B, and the data collection procedure further comprises time stamping each image of the plurality of images. Furthermore, the time stamp corresponds to a time step in the sequence of time steps. In other words, a continuous circular movement of each joint is performed. During the movement of the joints, motor rotary encoder data is collected and images of markers are placed on the end effector. Further, image processing and calculation of the position and orientation of the markers are performed. Further, the calculated marker position / orientation data is based on the motor rotation data collected by the first encoder 114A and the second encoder 114B. The information is further processed to determine the rotation axis for each joint and the reduction ratio for each joint.

[0061] According to an embodiment, the first encoder 114A and the second encoder 114B are configured to collect servo values ​​of the first servo motor 112A and the second servo motor 112B, respectively. The data processing procedure further comprises determining, for each of the first servo motor 112A and the second servo motor 112B, a plurality of servo motor angles for each servo motor based on the servo values ​​collected by the first encoder 114A and the second encoder 114B, respectively. In other words, at least the first joint and the second joint are connected to the first servo motor 112A and the second servo motor 112B through a gear system with a gear reduction ratio for each joint as a system parameter. The movement of the first servo motor 112A and the second servo motor 112B is transmitted to at least the first joint and the second joint with some reduction ratio coefficient.

[0062] According to an embodiment, the marker positions are determined for each image of the plurality of images by extracting a plurality of corner coordinates of at least one calibration marker from the image. Further, each of the plurality of corner coordinates corresponds to a position approximately at a corner of the at least one calibration marker. In other words, the image processing layer of the calculation unit 102A extracts the image data I1-I T , and participate in the calculation of the position and orientation of the calibration markers relative to the camera 110. Furthermore, without loss of generality, the reference marker Δ i is all images I1 to I T Visible and detectable.

[0063] According to an embodiment, the marker positions are determined by calculating, for each image of the multiple images, a marker center coordinate using multiple corner coordinates of at least one calibration marker on the image and one or more intrinsic camera parameters. In other words, the at least one calibration marker performs a circular motion while the joint rotates. Furthermore, the angular trajectory of the joint is calculated by the camera 110 moving the marker Δ i Furthermore, for each j = 1,...,J, the sequence of data collection, image processing, and rotation axis and reduction ratio estimation is j ,Q j ) for each joint A j Iteratively, the rotation axis position and orientation estimates for V j is the 3D rotation vector that defines the orientation of the jth rotation axis, and Q j is the position of the rotating shaft and the reduction ratio r j is a 3D point that defines

[0064] According to an embodiment, the method 300 includes processing a plurality of images, where processing the plurality of images further includes determining, for each image of the plurality of images, a marker orientation of at least one calibration marker in three-dimensional space based on a plurality of corner coordinates of the at least one calibration marker on the image and one or more intrinsic camera parameters to output as marker position data. For example, without loss of generality, i is all images I1 to I T In implementation, the image processing layer i Frame I where i1 ~I iT The intrinsic parameters of the camera 110 may be assumed to be accurate and known. If not, a camera calibration procedure is performed. Furthermore, the images I1 to I T The result of the detection procedure is the sequence C1 I ~C T i At this time, C t i Frame I t Marker Δ i In addition, the pre-computed intrinsic parameters of the camera and the corner coordinates C t I is the 3D position P of the marker center relative to the camera using the perspective-n-point (PnP) method. t i and 3D marker orientation O t i In the example, the marker center P t i The representation of is carried out by a three-dimensional vector, O t i The representation of is carried out using a 3 × 3 orthonormal basis matrix.

[0065] According to an embodiment, the method 300 comprises establishing a first error function comprising an arc fitting of a marker center coordinate and a marker orientation of the at least one calibration marker corresponding to a circular movement of the at least one marker relative to the coordinate system of the camera 110. In other words, the error function is based on an arc fitting of a marker's 3D position and a 3D marker orientation corresponding to a circular movement of the at least one calibration marker relative to another marker or the camera coordinate system, and optionally also on encoder data and fitting of the encoder data to the 3D orientation of the at least one calibration marker.

[0066] According to an embodiment, the moving part includes a first servo motor 112A coupled to the first joint via a first gear system having a gear reduction ratio. The method 300 further includes determining a first rotation axis of the first joint, which includes establishing a first error function based on the marker center coordinates and the marker orientation of the at least one calibration marker determined from the multiple images, and applying an optimization function to the first error function to determine a joint rotation axis origin Q, a rotation axis vector V, and a gear reduction ratio r. In other words, the moving part 104 includes at least two joints. The joints are disposed at one of the ends of at least one link, and at least one link movement is performed by the rotation of at least one pre-joint. Furthermore, at least the first joint and the second joint are connected to the first servo motor 112A and the second servo motor 112B via a gear system having a gear reduction ratio for each joint as a system parameter. The motion of the first servo motor 112A and the second servo motor 112B is transferred to at least the first joint and the second joint with some reduction ratio coefficient. Further, the axis and reduction ratio estimation layer is responsible for processing the marker position / orientation data and the encoder data, and outputs the estimated rotation axis for each joint. Further, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint, and outputs the calibrated motion parameters.

[0067] According to an embodiment, a first encoder 114A is provided to the first servo motor 112A, configured to output encoder data including a servo value of the first servo motor 112A, and a gear reduction ratio is determined based on the marker orientation of at least one calibration marker determined from a plurality of images and the encoder data of the first servo motor 112A. In other words, a gear system with a gear reduction ratio per joint as a system parameter is used to couple at least the first joint and the second joint with the first servo motor 112A and the second servo motor 112B. At least the first joint and the second joint are moved by the movement of the first servo motor 112A and the second servo motor 112B. Furthermore, some reduction ratio coefficient is also added with the movement.

[0068] According to an embodiment, the at least one calibration marker includes one or more marker keypoints. Further, the marker keypoints correspond to uniquely identifiable points on the at least one calibration marker, and the data processing procedure further includes extracting the one or more marker keypoints from the multiple images. For example, a special fiducial marker type, e.g., a 3×3 ChArUco marker, can be attached to the end effector to automatically determine the intrinsic parameters of the camera 110 using the motion information from the marker keypoints.

[0069] According to an embodiment, the method includes determining one or more intrinsic camera parameters based on the relative positions of the one or more marker keypoints. For example, image data I1,...,I T Then, by knowing the ground truth information about the relative positions of the marker keypoints, the intrinsic camera parameters of the camera 110 need to be obtained. Then, by using the obtained camera parameters and the positions of the marker keypoints on the image, the 3D position P ti and 3D marker orientation O t i is calculated using the PnP method.

[0070] According to an embodiment, the movable part 104 includes a plurality of J joints arranged to be rotated by a corresponding plurality of J servo motors. Further, positioning at least one calibration marker includes movements M1, M2, . . . , M3 to rotate the plurality of J joints. J In other words, a set of predefined commands stored in the computing unit 108 is sent to the first servo motor 112A and the second servo motor 112B. The first servo motor 112A and the second servo motor 112B perform the movements M1, . . . , M J where J is the number of joints in the moving part 104, and M j is determined by moving the jth robot joint while the other joints are stationary in their zero positions. The joints move and at least one calibration marker, such as calibration marker 106A, is statically attached at an arbitrary position and orientation relative to the moving parts of the robot. As a result, the at least one calibration marker performs a circular motion while the joint is rotating. The angular trajectory of the joint is determined by observing the camera 110 moving the marker Δ i Furthermore, for each j = 1,...,J, the sequence of data collection, image processing, and rotation axis and reduction ratio estimation is j ,Q j ) for each joint A j This is repeated to obtain an estimate of the position and orientation of the rotation axis for V. j is the 3D rotation vector that defines the orientation of the jth rotation axis, and Q j is the position of the rotating shaft and the reduction ratio r j is a 3D point that defines

[0071] According to an embodiment, the movement M of the nth joint, where 1≦n≦J n is determined by controlling the n-th servo motor to rotate the n-th joint while keeping the remaining joints of the plurality of J joints at their zero positions. In other words, the number of joints is equal to the number of servo motors, and as the servo motors move, motion is imparted to the joints. This motion is controlled by the calculation unit 108. When only one servo motor is rotating, only one joint is moving, and in this case the remaining joints are considered to be at their zero positions.

[0072] According to an embodiment, at least one calibration marker is provided on one or more links each connecting two adjacent joints of the plurality of J joints, and an encoder is provided on each servo motor of the plurality of J servo motors, each of the plurality of encoders configured to output encoder data corresponding to each joint of the plurality of J joints. Further, the encoder data includes a servo value (e.g., an angle) of the corresponding servo motor. A motion of the servo motor translates into a rotation of the corresponding joint. When representing the motion, the encoder generates the encoder data, where the encoder estimates an angle of rotation of the motion of the joint, which corresponds to the servo value.

[0073] According to an embodiment, the movements M1, M2, . . . , M J is arbitrary, and processing the plurality of images includes, for each n-th joint of the plurality of J joints, transforming, for each image of the plurality of images, a position of at least one calibration marker between the (n-1)-th joint and the n-th joint from the motion frame of the (n-1)-th joint to a stationary frame of the (n-1)-th joint, in which the (n-1)-th joint is stationary and in a zero position, using the encoder data of the (n-1)-th joint.

[0074] According to an embodiment, the method 300 comprises determining a set of motion parameters, where determining the set of motion parameters comprises determining a link length between a first joint and a second joint. In other words, the link length is determined between the first joint and the second joint by a process executed in the computing unit 108. Furthermore, the motion parameters related to all links and joints located between the robot base and the position of the at least one calibration marker are subjected to calibration, except for the first and last link lengths, respectively.

[0075] According to an embodiment, determining the set of motion parameters includes: determining a marker-based transformation between a frame of at least one calibration marker at a zero position and a frame of the base 102 of the robot 110A using the set of motion parameters; determining a marker-camera transformation between a frame of at least one calibration marker at a zero position and a frame of the camera 110 by processing a plurality of images; and calculating a camera-based transformation between the frame of the camera 110 and a frame of the base of the robot based on the determined marker-based transformation and the determined marker-camera transformation. In other words, the motion parameter update layer is responsible for processing the estimated rotation axis for each joint and outputs calibrated motion parameters. For example, the set of motion parameters to be calibrated is denoted as L. Furthermore, the estimated rotation axes A1 to A of each joint are calculated based on the determined marker-based transformation and the determined marker-camera transformation. j Given the initial values ​​of the motion parameters L0, the motion parameter update layer estimates the actual parameters

number

number

number

number

number

[0076] The method 300 calibrates the robot's motion parameters including robot link lengths, joint positions and orientations, robot joint zero positions, reduction ratios for each joint position and orientation, robot joint zero positions, reduction ratios for each joint, etc. Using the calibrated parameters, the method 300 can achieve improved accuracy of the position and orientation of the end effector during motion execution.

[0077] Steps 302 and 304 are merely exemplary and other alternatives may be provided, in which one or more steps are added, one or more steps are removed, or one or more steps are provided in a different order, without departing from the scope of the claims herein.

[0078] A non-transitory computer readable storage medium is further provided having machine readable code that, when executed by a processor, causes the processor to position at least one calibration marker at a plurality of positions by rotating a first joint and / or a second joint. The processor further obtains a plurality of images of the at least one calibration marker, the plurality of images being captured by the camera 110 when the at least one calibration marker is in the plurality of positions. The processor further processes the plurality of images to determine, for each image of the plurality of images, a marker position (and optionally an orientation) of the at least one calibration marker in the three-dimensional space to be output as marker position data. The processor further establishes a first error function based on the marker position data. The processor further determines a first rotation axis of the first joint and a second rotation axis of the second joint based on the first error function. The processor further establishes a second error function based on at least the first rotation axis and the second rotation axis. The processor further determines a set of motion parameters for a motion model of the robot 110A based on the second error function. In an example, the instructions are embodied in a computer readable medium, including but not limited to an Electrically Erasable Programmable Read Only Memory (EEPROM), a Random Access Memory (RAM), a Read Only Memory (ROM), a Hard Disk Drive (HDD), a Flash Memory, a Secure Digital (SD) Card, a Solid State Drive (SSD), a computer readable storage medium, and / or a CPU cache memory. In an example, the instructions are generated by a computer program, and the computer program is executed in view of method 300 for use in implementing method 300 by a processor.

[0079] Modifications to the embodiments of the present disclosure described above are possible without departing from the scope of the present disclosure, which is defined by the appended claims. The terms "comprising," "having," "incorporating," "having," "being," and the like, used to describe and claim the present disclosure, are intended to interpret the present disclosure in a non-exhaustive manner, i.e., there may be items, components, or elements not expressly described. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described as being "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments or to exclude the incorporation of features from other embodiments. The word "optionally" is used herein to mean "provided in some embodiments but not in other embodiments." It will be understood that certain features of the present disclosure, although described in connection with separate embodiments for clarity, may also be provided in combination in a single embodiment. Conversely, various features of the present invention, although described in connection with a single embodiment for brevity, may also be provided separately or in any suitable combination or in any other described embodiment of the present disclosure, as appropriate.

Claims

1. a robot (100A, 100B) configured to generate a movement in the movable part (104) having at least one calibration marker thereon and at least a first joint and a second joint arranged at one end of at least one link, the first joint and the second joint being controlled by a computing unit (108); a camera (110) configured to capture images of the at least one calibration marker in a plurality of positions; the calculation unit (108) including a robot control layer, an image processing layer, an axis and reduction ratio estimation layer, and a motion parameter update layer; the robot control layer is configured to control rotations of the first joint and the second joint based on a kinematic model of the robot (100A, 100B), the robot control layer is configured to control rotations of the first joint and / or the second joint to position the at least one calibration marker at the multiple positions; the image processing layer is configured to process images received from the camera (110) to determine, for each image, a marker position of the at least one calibration marker in three-dimensional space to be output as marker position data; the axis and reduction ratio estimation layer is configured to establish a first error function based on the marker position data and determine a first axis of rotation of the first joint based on the first error function; the motion parameter update layer is configured to establish a second error function based on at least the first rotation axis, and to determine a set of motion parameters for the motion model based on the second error function. A robot system (200).

2. the moving part (104) further comprises a first servo motor (112A) coupled to the first joint via a first gear system, a second servo motor (112B) coupled to the second joint via a second gear system, a first encoder (114A) provided on the first servo motor (112A) and a second encoder (114B) provided on the second servo motor (112B), the first encoder and the second encoder (114A, 114B) being configured to collect servo values ​​of the first servo motor and the second servo motor (112A, 112B), respectively; and the calculation unit (108) further comprises a clock synchronization layer configured to synchronize the camera (110) with the first encoder and the second encoder (114A, 114B) to time stamp images captured by the camera (110). The robotic system (200) of claim 1.

3. A method (300) for calibrating a robot (100A, 100B) having a movable part (104) provided with at least one calibration marker thereon and at least a first joint and a second joint arranged at one end of at least one link, the first joint and the second joint being configured to generate a movement in the movable part (104) controlled by a computing unit (108), comprising: In the data collection procedure, positioning the at least one calibration marker at a plurality of positions by rotating the first joint and / or the second joint; acquiring a plurality of images of the at least one calibration marker, the plurality of images being captured by a camera (110) when the at least one calibration marker is in the plurality of positions; In the data processing procedure, processing the plurality of images to determine, for each image of the plurality of images, a marker position of the at least one calibration marker in three dimensional space to be output as marker position data; establishing a first error function based on the marker position data; determining a first rotation axis of the first joint and a second rotation axis of the second joint based on the first error function; establishing a second error function based on at least the first axis of rotation and the second axis of rotation; determining a set of motion parameters for a motion model of the robot (110A, 110b) based on the second error function; Method (300).

4. calibrating the kinematic model of the robot (110A, 110b) using the determined set of kinematic parameters.

4. The method of claim 3 (300).

5. the movable part (104) further comprises a first servo motor (112A) configured to move the first joint, a second servo motor (112B) configured to move the second joint, a first encoder (114A) provided on the first servo motor (112A) and a second encoder (114B) provided on the second servo motor (112B), the camera (110) is synchronized with the first encoder and the second encoder (114A, 114B), the data collection procedure further comprises time stamping each image of the plurality of images, the time stamp corresponding to a time step in a series of time steps.

5. The method (300) of claim 3 or 4.

6. the first and second encoders (114A, 114B) are configured to collect servo values ​​of the first and second servo motors (112A, 112B), respectively, and the data processing procedure further comprises, for each of the first and second servo motors (112A, 112B), determining a plurality of servo motor angles for each servo motor based on the servo values ​​collected by the first and second encoders (114A, 114B), respectively.

6. The method (300) of claim 5.

7. the marker positions are determined, for each image of the plurality of images, by extracting from that image a plurality of corner coordinates of the at least one calibration marker, each of the plurality of corner coordinates corresponding to a position that is approximately at a corner of the at least one calibration marker. The method (300) according to any of claims 3 to 6.

8. the marker positions are determined by calculating, for each image of the plurality of images, a marker center coordinate using the plurality of corner coordinates of the at least one calibration marker on that image and one or more intrinsic camera parameters.

6. The method (300) of claim 5.

9. and processing the plurality of images further comprises determining, for each image of the plurality of images, a marker orientation of the at least one calibration marker in three-dimensional space based on the plurality of corner coordinates of the at least one calibration marker on that image and one or more intrinsic camera parameters to output as the marker position data.

9. The method (300) of claim 7 or 8.

10. establishing a first error function includes arc-fitting the marker center coordinates and the marker orientation of the at least one calibration marker corresponding to a circular motion of the at least one marker with respect to a coordinate system of the camera (110).

10. The method (300) of claim 9.

11. the movable part has a first servo motor (112A) coupled to the first joint via a first gear system having a gear reduction ratio, and determining a first rotation axis of the first joint includes establishing the first error function based on the marker center coordinates and the marker orientation of the at least one calibration marker determined from the plurality of images, and applying an optimization function to the first error function to determine a joint rotation axis origin Q, a rotation axis vector V, and the gear reduction ratio r.

11. The method (300) of claim 9 or 10.

12. a first encoder (114A) is provided on the first servo motor (112A) configured to output encoder data including a servo value of the first servo motor (112A), and the gear reduction ratio is determined based on the marker orientation of the at least one calibration marker determined from the plurality of images and the encoder data of the first servo motor (112A); 12. The method (300) of claim 11.

13. the at least one calibration marker includes one or more marker keypoints, the marker keypoints corresponding to uniquely identifiable points on the at least one calibration marker, and the data processing procedure further comprises: extracting the one or more marker keypoints from the plurality of images; determining one or more intrinsic camera parameters based on the relative positions of the one or more marker keypoints; Further comprising The method (300) according to any of claims 3 to 12.

14. The movable part (104) has a plurality of J joints arranged to be rotated by a corresponding plurality of J servo motors, and positioning the at least one calibration marker (106A, 106B, 106C) includes a motion M to rotate the plurality of J joints. １ , M ２ , ..., M Ｊ controlling the plurality of J servo motors to execute a set of A method (300) according to any of claims 3 to 13.

15. For 1≦n≦J, the movement M of the nth joint ｎ is determined by controlling an n-th servo motor to rotate the n-th joint while keeping the remaining joints of the plurality of J joints at their respective zero positions.

15. The method (300) of claim 14.

16. At least one calibration marker is provided on one or more links each connecting two adjacent joints of the plurality of J joints, and an encoder is provided on each servo motor of the plurality of J servo motors, each of the plurality of encoders is configured to output encoder data corresponding to each joint of the plurality of J joints, and the encoder data includes a servo value of the corresponding servo motor.

16. The method (300) of claim 14 or 15.

17. Movement M １ , M ２ , ..., M Ｊ is arbitrary, and processing the plurality of images includes, for each n-th joint of the plurality of J joints, transforming, for each image of the plurality of images, a position of the at least one calibration marker between the (n-1)-th joint and the n-th joint from a motion frame of the (n-1)-th joint to a stationary frame of the (n-1)-th joint, in which the (n-1)-th joint is stationary and in a zero position, using the encoder data of the (n-1)-th joint.

20. The method (300) of claim 16.

18. Determining a set of motion parameters includes determining a link length between the first joint and the second joint. A method (300) according to any of claims 3 to 17.

19. Determining a set of motion parameters includes determining a marker-based transformation between a frame of the at least one calibration marker at a zero position and a frame of a base (102) of the robot (110A, 110b) using the set of motion parameters, determining a marker-to-camera transformation between the frame of the at least one calibration marker at the zero position and a frame of the camera (110) by processing the plurality of images, and calculating a camera-based transformation between the frame of the camera (110) and the frame of the robot base based on the determined marker-based transformation and the determined marker-to-camera transformation. The method (300) according to any of claims 3 to 18.

20. a processor that executes the program and a program code that, when executed by a processor, causes the processor to: positioning the at least one calibration marker at a plurality of positions by rotating the first joint and / or the second joint; acquiring a plurality of images of the at least one calibration marker, the plurality of images being captured by a camera (110) when the at least one calibration marker is in the plurality of positions; processing the plurality of images to determine, for each image of the plurality of images, a marker position of the at least one calibration marker in three dimensional space to be output as marker position data; establishing a first error function based on the marker position data; determining a first axis of rotation of the first joint and a second axis of rotation of the second joint based on the first error function; establishing a second error function based on at least the first axis of rotation and the second axis of rotation; determining a set of motion parameters for a motion model of the robot (110A, 110b) based on the second error function; Execute the A non-transitory computer-readable storage medium.

Images