Method and Electronic Device, System, and Computer-Readable Medium for Calibration

Abstract

A method, an electronic device, a system, and a computer-readable medium for time calibration are disclosed. The method (600) includes triggering a robot to perform a plurality of transformations in response to sensing data received from a sensor indicating a deviation of a tool from an object to be operated on by the robot using the tool, each transformation causing the tool to contact the object at a reference position (610); determining, based on feedback data received from the robot, joint positions of joints of the robot that hold the tool or the object after the plurality of transformations, the feedback data recording the plurality of transformations of the robot (620); and determining a positional relationship between the tool and the robot at least partially based on the joint positions and the reference position (630). In this way, a visual servo method can be implemented, so that a tool coordinate system can be defined and the tool can be automatically calibrated.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to methods, electronic devices, systems, and computer-readable media for calibration. Background Art

[0002] In industrial automation technology, it may be necessary for an industrial robot to cooperate with a tool to operate an object. As an option, the robot can hold the tool and cause the tool to perform a specific operation on the object. For example, the tool can be a glue gun. The robot can hold the glue gun to spray an adhesive onto the object.

[0003] As another option, the tool can be stationary. For example, the tool can be located in the reachable area of the robot, and the robot can hold the object and move to a position close to the tool so that the object is operated by the tool. For example, the tool can be a pin. The robot can hold the object so that the object is pierced by the pin.

[0004] There are multiple coordinate systems in the scenario where the robot and the tool cooperate with each other. For example, the Tool0 (without tool) coordinate system can be referred to as a Cartesian coordinate system with the origin at the joint attached to the tool or the object. In addition, the tool coordinate system can be referred to as a Cartesian coordinate system with the origin at the tool center point (TCP).

[0005] Since the position of the robot's joints can be recorded by the robot controller when the joints are triggered and transformed, tool calibration is required to set the TCP for a specific tool. Therefore, it is desirable to determine the mapping relationship between the tool coordinate system and the Tool0 coordinate system. Summary of the Invention

[0006] Embodiments of the present disclosure provide a method, an electronic device, a system, and a computer-readable medium for calibration.

[0007] In a first aspect, a method for time calibration is provided. The method includes triggering the robot to perform a plurality of transformations in response to sensing data received from a sensor indicating a deviation between the tool and the object to be operated by the robot using the tool, each transformation causing the tool to contact the object at a reference position; determining feedback data recording the plurality of transformations of the robot based on the feedback data received from the robot; and determining the positional relationship between the tool and the robot at least partially based on the joint position and the reference position.

[0008] In some embodiments, triggering the robot includes: triggering the joints of the robot to perform at least one of the following: horizontal movement in the robot coordinate system, the origin of the robot coordinate system being the initial position of the joint before the plurality of transformations; and rotational movement around the coordinate axes in the robot coordinate system.

[0009] In some embodiments, determining the joint position includes: receiving feedback data from the robot; obtaining, from the feedback data, a first plurality of coordinate parameters of the joints of the robot in a robot coordinate system in a set of transformed poses, the robot having the set of transformed poses after performing a plurality of transformations; and determining the joint position in the robot coordinate system based on the first plurality of coordinate parameters of the joints.

[0010] In some embodiments, determining the position relationship includes: determining a mapping between a sensor coordinate system and the robot coordinate system based on the sensing data and a first plurality of coordinate parameters of the joints of the robot in a robot coordinate system in a set of transformed poses, the robot having the set of transformed poses after performing a plurality of transformations; determining first reference position information of a reference position in the sensor coordinate system based on the sensing data; and determining the position relationship based on the mapping, the first plurality of coordinate parameters, and the first reference position information.

[0011] In some embodiments, determining the mapping includes: determining a second plurality of coordinate parameters of the joints of the robot in the sensor coordinate system in the set of transformed poses based on the sensing data; and determining the mapping based on the first plurality of coordinate parameters and the second plurality of coordinate parameters.

[0012] In some embodiments, determining the position relationship includes: determining second reference position information of the reference position in the robot coordinate system based on the first reference position information and the mapping; determining first tool position information in the robot coordinate system based on the second reference position information; and determining the position relationship in the robot coordinate system based on the first tool position information and the first plurality of coordinate parameters.

[0013] In this way, a visual servo method can be implemented. The robot can be guided to perform transformations with different poses, causing the tool to contact the object to be operated at one or more feature points on the object or the tool. After each transformation, the joint positions of the robot can be recorded. According to the positions of the feature points and the recorded joint positions of the robot, the transformation between the tool coordinate system and the Tool0 coordinate system can be determined. Therefore, the tool coordinate system can be defined and the tool can be automatically calibrated.

[0014] In a second aspect, an electronic device is provided. The device includes a processing unit and a memory coupled to the processing unit and having instructions stored thereon that, when executed by the processing unit, cause the device to perform the following actions: in response to sensing data received from a sensor indicating a deviation of a tool from an object to be operated on by a robot using the tool, trigger the robot to perform a plurality of transformations, each transformation causing the tool to contact the object at a reference position; based on feedback data received from the robot, determine the joint positions of the joints of the robot that hold the tool or the object after the plurality of transformations, the feedback data recording the plurality of transformations of the robot; and determine the positional relationship between the tool and the robot at least in part based on the joint positions and the reference position.

[0015] In some embodiments, triggering the robot includes: triggering the joints of the robot to perform at least one of the following: horizontal movement in the robot coordinate system, the origin of the robot coordinate system being the initial position of the joint before the plurality of transformations; and rotational movement about the coordinate axes in the robot coordinate system.

[0016] In some embodiments, determining the joint positions includes: receiving feedback data from the robot; obtaining from the feedback data a first plurality of coordinate parameters of the joints of the robot in the robot coordinate system in a set of post-transform poses, the robot having the set of post-transform poses after performing the plurality of transformations; and determining the joint positions in the robot coordinate system based on the first plurality of coordinate parameters of the joints.

[0017] In some embodiments, determining the positional relationship includes: determining a mapping between the sensor coordinate system and the robot coordinate system based on the sensing data and a first plurality of coordinate parameters of the joints of the robot in the robot coordinate system in a set of post-transform poses, the robot having the set of post-transform poses after performing the plurality of transformations; determining first reference position information of the reference position in the sensor coordinate system based on the sensing data; and determining the positional relationship based on the mapping, the first plurality of coordinate parameters, and the first reference position information.

[0018] In some embodiments, determining the mapping includes: determining a second plurality of coordinate parameters of the joints of the robot in the sensor coordinate system in the set of post-transform poses based on the sensing data; and determining the mapping based on the first plurality of coordinate parameters and the second plurality of coordinate parameters.

[0019] In some embodiments, determining the positional relationship includes: determining second reference position information of the reference position in the robot coordinate system based on the first reference position information and the mapping; determining first tool position information in the robot coordinate system based on the second reference position information; and determining the positional relationship in the robot coordinate system based on the first tool position information and the first plurality of coordinate parameters.

[0020] In a third aspect, a system for calibration is provided. The system includes a sensor configured to sense a deviation of a tool from an object to be manipulated by a robot using the tool and to sense joint positions of joints of a robot holding the tool or the object when the robot is triggered to perform a plurality of transformations; and a controller coupled to the robot and the sensor and configured to perform the actions of the first aspect.

[0021] In a fourth aspect, a computer-readable medium is provided. The computer-readable medium includes program instructions for causing an electronic device to perform at least the actions of the first aspect.

[0022] It should be understood that the Summary of the Invention is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become readily apparent through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description of example embodiments of the present disclosure in conjunction with the accompanying drawings, in which like reference numerals generally represent like components in the example embodiments of the present disclosure.

[0024] Figure 1 An example coordinate system in an example operating environment in which embodiments of the present disclosure may be implemented is shown;

[0025] Figure 2 An example operating environment in which embodiments of the present disclosure may be implemented is shown;

[0026] Figures 3A - 3D A schematic diagram of a transformation of a joint of a robot is shown;

[0027] Figure 4 An example operating environment in which embodiments of the present disclosure may be implemented is shown;

[0028] Figures 5A - 5D A schematic diagram of a transformation of a joint of a robot is shown;

[0029] Figure 6 A flowchart illustrating a calibration method according to an embodiment of the present disclosure is shown;

[0030] Figure 7 A block diagram of an example computer-readable medium according to some example embodiments of the present disclosure is shown.

[0031] Throughout the drawings, the same or similar reference numerals are used to indicate the same or similar elements. DETAILED DESCRIPTION

[0032] The present disclosure will now be discussed with reference to several exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the present disclosure, rather than implying any limitation on the scope of the subject matter.

[0033] As used herein, the term "comprising" and its variants should be construed as open terms, meaning "including but not limited to". The term "based on" should be construed as "at least partially based on". The terms "one embodiment" and "embodiment" should be construed as "at least one embodiment". The term "another embodiment" should be construed as "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless the context clearly dictates otherwise, the definitions of the terms are consistent throughout the description.

[0034] Unless otherwise specified or limited, the terms "mounted", "connected", "supported", and "coupled" and their variants are used broadly and encompass direct and indirect mounting, connection, support, and coupling. In addition, "connected" and "coupled" are not limited to physical or mechanical connection or coupling. In the following description, the same reference numerals and marks are used to describe the same, similar, or corresponding parts in the figures. Other definitions, whether explicit or implicit, may be included below.

[0035] As described above, in industrial automation technology, it may be necessary for an industrial robot to cooperate with a tool to operate an object. As an option, the robot can hold the tool and cause the tool to perform a specific operation on the object. As another option, the tool can be stationary. For example, the tool can be located in the reachable area of the robot, and the robot can hold the object and move to a position close to the tool so that the object can be operated by the tool.

[0036] Figure 1 An exemplary operating environment 100 in which embodiments of the present disclosure can be implemented is shown. In this operating environment 100, a tool 120 is attached to a joint 111 of a robot 110. An object 130 is stationary and will be operated by the robot 110 using the tool 120. A sensor 140 can be disposed in the operating environment 100 and can observe the joint 111, the tool 120, and the object 130 at any time.

[0037] There are multiple coordinate systems in the operating environment 100. For example, the Tool0 (without tool) coordinate system (which may also be referred to as the robot coordinate system hereafter) can be referred to as a Cartesian coordinate system formed by axes X1, Y1, and Z1 with the joint 111 as the origin. In addition, the tool coordinate system can be referred to as a Cartesian coordinate system formed by axes X2, Y2, and Z2 with the tool center point (TCP) as the origin.

[0038] The term "TCP" in this document can be used to describe a tool in space. As mentioned above, the origin of the Tool Coordinate System (TCS) can be considered as the TCP. In the TCS, six degrees of freedom or six pieces of information are usually required to fully define the pose of the joints of a robot, because it can move along the X2, Y2, and Z2 axes in space and can also rotate around the X2, Y2, and Z2 axes.

[0039] In addition, the base coordinate system can be referred to as a Cartesian coordinate system formed by the axes X3, Y3, and Z3, and its origin is located at the base 112 of the robot 110. The sensor coordinate system can be referred to as a Cartesian coordinate system formed by the axes X4, Y4, and Z4, which has any point that always remains stationary relative to the robot as the origin.

[0040] Since the position of the joints of the robot can be recorded by the robot controller when the joints are triggered and transformed, tool calibration is required to set the TCP for a specific tool. Therefore, it is desirable to determine the mapping relationship between the tool coordinate system and the Tool0 coordinate system.

[0041] In the traditional method, the calibration between the robot coordinate system and the tool coordinate system is performed manually. For example, the feature points of an object are manually touched by the tool attached to the robot joint. The accuracy of this calibration process depends on the operator's skills, and the calibration process is cumbersome.

[0042] Therefore, the embodiments of the present disclosure propose a method and device for tool calibration. A sensor can be used to sense the deviation from the tool and the object and trigger the robot to perform a transformation so that the tool and the object can touch each other at the reference position. Based on the reference position and the joint position of the robot, the tool coordinate system can be calibrated.

[0043] Refer to the following Figure 2 to FIG. 5 for a detailed description of the principles and implementations of the present disclosure. As mentioned above, the tool calibration for the scenario where the tool 120 is held by the robot 110 can be referred to Figures 2 - 3D to describe. Figure 2 An example operating environment 200 in which the embodiments of the present disclosure can be implemented is shown. It should be understood that the multiple coordinate systems in the operating environment 200 can be similar to those in the operating environment 100.

[0044] With Figure 1In contrast, the operating environment 200 also includes a controller 150 and a sensor 140 coupled to the robot. The robot 110 can transmit feedback data to the controller 150, which can be obtained by recording the movement of the robot. The controller 150 can also transmit commands to the robot 110 to trigger the movement of the robot 110. In addition, the sensor 140 can also transmit sensed data to the controller 150, which can be obtained by sensing the movement of the robot or the positional relationship between the robot, the tool, and the object. It should be understood that the sensor 140 and the controller 150 can be integrated into the calibration device. In addition, the controller 150 can be considered as the calibration device and the sensor 120 can be considered as an external device coupled to the calibration device.

[0045] Before the process of fine tool calibration described in detail Figures 2 - 3D below, rough calibration may be required. In combination with Figure 1 referring to Figure 2 , for example, for rough calibration, the joints 111 of the robot 110 can be triggered to perform orthogonal movement along any one of the axes X1, Y1, and Z1, or the base 112 of the robot 110 can be triggered to perform orthogonal movement along any one of the axes X3, Y3, and Z3. The robot 110 can transmit feedback data recording the position of the robot after movement in the robot coordinate system or in the base coordinate system to the controller 150, and the sensor 140 can also transmit sensed data recording the position of the robot after movement in the sensed coordinate system to the controller 150.

[0046] In addition, for rough calibration, the joints 111 of the robot 110 can also be triggered to perform rotational movement around any axis among the axes X1, Y1, and Z1. Similarly, the robot 110 can transmit feedback data recording the position of the robot after movement in the robot coordinate system or in the base coordinate system to the controller 150, and the sensor 140 can also transmit sensed data recording the position of the robot after movement in the sensor coordinate system to the controller 150.

[0047] In this way, the conversion relationship between the sensor coordinate system and the robot coordinate system and the base coordinate system can be used for calibration.

[0048] Then the process of fine tool calibration will be further described. As Figure 2 shown, the sensor 140 can sense the deviation between the tool 120 and the object 130. The sensor 140 can transmit the sensed data to the controller 150. Then the controller 150 can trigger the robot to perform multiple transformations. After each transformation, the tool 120 can contact the object 130, for example, at the reference position Pr. This reference position can be considered as the position of the feature point of the object 130. As Figure 2As shown, object 130 can be a cube and the feature points can be any corner points of the cube. Tool 120 can also contact object 130 at different reference positions after each transformation. For example, tool 120 can contact object 130 at a set of corner points of the cube.

[0049] In some embodiments, triggering the robot to perform multiple transformations can include performing a horizontal movement along any one of axes X1, Y1, and Z1 in the robot (Tool0) coordinate system, as Figure 1 shown. Triggering the robot to perform multiple transformations can include performing a rotational movement about any one of axes X1, Y1, and Z1 in the robot (Tool0) coordinate system.

[0050] After each transformation, the joints of the robot can be transformed from the initial pose to the post-transformation pose. Figures 3A - 3D A schematic diagram showing the transformation of the joints of the robot is shown. As Figure 3A shown, after the first transformation, joint 111 can be transformed from the initial pose P0 to the post-transformation pose P1. Similarly, as Figures 3B - 3D shown, after the second, third, and fourth transformations, joint 111 can be transformed from the initial pose P0 to the post-transformation poses P2, P3, and P4 respectively. After each transformation of the joint, tool 120 can contact object 130 at the reference position Pr. During the fine calibration process, the joint can be transformed to at least four different poses to determine the relationship between the tool and the joints of the robot. It should be understood that robot 110 can be triggered to perform more than four transformations.

[0051] Then the robot can transmit feedback recording the multiple transformations of the robot to controller 150. Controller 150 can obtain a first plurality of coordinate parameters of joint 111 in the robot coordinate system in a set of post-transformation poses P1, P2, P3, and P4 after the transformation from the feedback data. Based on the first plurality of coordinate parameters of joint 111, controller 150 can determine the joint position of joint 111 of robot 110 that holds tool 111 after four transformations.

[0052] Then controller 150 can determine the positional relationship between tool 111 and robot 110 (i.e., joint 111) at least partially based on the joint position and reference position Pr.

[0053] In some embodiments, the controller 150 may determine the mapping between the sensor coordinate system and the robot coordinate system. For example, the controller 150 may determine a second plurality of coordinate parameters of the robot joints in the sensor coordinate system in a set of transformed poses P1, P2, P3, and P4 after transformation based on the sensing data of the sensor 140. Based on the first plurality of coordinate parameters and the second plurality of coordinate parameters of the joint 111 in the robot coordinate system in a set of transformed poses P1, P2, P3, and P4 after transformation that can be obtained from the feedback data received from the robot 110, the mapping between the sensor coordinate system and the robot coordinate system can be determined.

[0054] Then the controller 150 may also determine first reference position information of the reference position Pr in the sensor coordinate system based on the sensing data. The controller 150 may also determine second reference position information of the reference position in the robot coordinate system based on the first reference position information and the mapping between the sensor coordinate system and the robot coordinate system.

[0055] Since the tool and the object are in contact with each other at the reference position Pr, the second reference position information of the reference position in the robot coordinate system can be regarded as the tool position in the robot coordinate system. Therefore, the controller 150 may determine the positional relationship between the tool and the robot in the robot coordinate system based on the second reference position information of the reference position in the robot coordinate system and the second plurality of coordinate parameters of the joint 111 in the robot coordinate system in a set of transformed poses P1, P2, P3, and P4 after transformation. In this way, fine tool calibration can be automatically achieved.

[0056] As mentioned above, in another alternative, the tool may be stationary. For example, the tool may be located in the reachable area of the robot. The object may be held by the robot and moved to a position close to the tool as the robot moves, so that the object can be operated by the tool.

[0057] The tool calibration for the scenario where the tool 120 is stationary can be referred to Figure 4 - Figure 5 for description. Figure 4 An example operating environment 400 in which embodiments of the present disclosure may be implemented is shown. It should be understood that the multiple coordinate systems in the operating environment 400 may be similar to those in the operating environment 100.

[0058] As Figure 4 shown, the robot 111 may grasp the object 130. To better describe the present disclosure, the offset between the joint and the object 130 caused by the gripper may be ignored. The object 130 may be considered to be directly attached to the joint 111. The tool 120 may be stationary and located on the ground. In Figure 4 the same components as in Figure 2 will not be described in detail here.

[0059] The Figure 1 coarse calibration described with Figure 2 can also be performed in the operating environment 400 to calibrate the transformation relationship between the sensor coordinate system, the robot coordinate system, and the base coordinate system.

[0060] Then the process of fine tool calibration will be further described. As Figure 4 shown, the sensor 140 can sense the deviation of the tool 120 from the object 130. The sensor 140 can transmit the sensed data to the controller 150. Then the controller 150 can trigger the robot to perform multiple transformations. After each transformation, for example, at the reference position Pr of the tool 120, the object 130 can contact the tool 120. This reference position can be considered as the position of the feature point of the tool 120. As Figure 4 shown, the tool 120 can be a cone and the feature point can be the corner point of the cone.

[0061] The object 130 can also contact the tool 120 using the feature position of the object 130. For example, as shown in the figure, the object 130 is a cone and the object 130 can contact the tool 120 using the corner point of the cone. In this case, the reference position can be the position of the feature point of the object 130 after the transformation of the robot.

[0062] In some embodiments, triggering the robot to perform multiple transformations can include performing a horizontal movement along any one of the axes X1, Y1, and Z1 in the robot (Tool0) coordinate system, as Figure 1 shown. Triggering the robot to perform multiple transformations can include performing a rotational movement around any one of the axes X1, Y1, and Z1 in the robot (Tool0) coordinate system.

[0063] After each transformation, the joint 111 of the robot can be transformed from the initial pose to the transformed pose. Figures 5A - 5D A schematic diagram showing the transformation of the joint 111 of the robot 110 is shown. As Figure 5A shown, after the first transformation, the joint 111 can be transformed from the initial pose P0 to the transformed pose P1. Similarly, as Figures 5B - 5D shown, after the second, third, and fourth transformations, the joint 111 can be transformed from the initial pose P0 to the transformed poses P2, P3, and P4 respectively. After each transformation of the joint, the object 130 can contact the tool 120 at the reference position Pr. During the fine calibration process, the joint can be transformed to at least four different poses to determine the relationship between the tool and the joints of the robot. It should be understood that the robot 110 can be triggered to perform more than four transformations.

[0064] Then the robot can transmit feedback recording multiple transformations of the robot to the controller 150. The controller 150 can obtain from the feedback data a first plurality of coordinate parameters of joint 111 in the robot coordinate system in a set of transformed postures P1, P2, P3, and P4 after the transformation. Based on the first plurality of coordinate parameters of joint 111, the controller 150 can determine the joint position of joint 111 of the robot 110 holding the object 130 after four transformations.

[0065] Then the controller 150 can determine the positional relationship between the tool 111 and the robot 110 (i.e., joint 111) at least partially based on the joint position and the reference position Pr.

[0066] In some embodiments, the controller 150 can determine the mapping between the sensor coordinate system and the robot coordinate system. For example, the controller 150 can determine a second plurality of coordinate parameters of the robot joints in the sensor coordinate system in a set of transformed postures P1, P2, P3, and P4 after the transformation based on the sensing data of the sensor 140. Based on the first plurality of coordinate parameters and the second plurality of coordinate parameters of joint 111 in the robot coordinate system in a set of transformed postures P1, P2, P3, and P4 obtained from the feedback data that can be received from the robot 110, the mapping between the sensor coordinate system and the robot coordinate system can be determined.

[0067] Then the controller 150 can also determine first reference position information of the reference position Pr in the sensor coordinate system based on the sensing data. The controller 150 can also determine second reference position information of the reference position in the robot coordinate system based on the first reference position information and the mapping between the sensor coordinate system and the robot coordinate system.

[0068] Since the tool and the object are in contact with each other at the reference position Pr, the second reference position information of the reference position in the robot coordinate system can be regarded as the tool position in the robot coordinate system. Therefore, the controller 150 can determine the positional relationship between the tool and the robot in the robot coordinate system based on the second reference position information of the reference position in the robot coordinate system and the second plurality of coordinate parameters of joint 111 in the robot coordinate system in a set of transformed postures P1, P2, P3, and P4 after the transformation. In this way, a visual servo method can be implemented. The robot can be guided to perform transformations through different postures, resulting in the tool contacting the object to be operated at one or more feature points on the object or the tool. After each transformation, the joint position of the robot can be recorded. According to the position of the feature points and the recorded joint position of the robot, the transformation between the tool coordinate system and the Tool0 coordinate system can be determined. Therefore, the tool coordinate system can be defined and the tool can be automatically calibrated.

[0069] Figure 6FIG. 600 is a flow chart illustrating a method for time calibration according to an embodiment of the present disclosure. For purposes of discussion, method 600 will be described with reference to Figure 1 , Figure 2 and Figure 4 . It should be understood that although method 400 has been described in the operating environment 100 of Figure 1 , the operating environment 200 of Figure 2 and the operating environment 400 of Figure 4 , method 600 can equally be applied to other operating environments.

[0070] At 610, if the controller 150 receives sensed data from a sensor indicating a deviation of the tool from an object to be operated on by the robot using the tool, the controller 150 triggers the robot to perform a plurality of transformations. Each transformation may cause the tool to contact the object at a reference position.

[0071] In some embodiments, triggering the robot includes: triggering the joints of the robot to perform at least one of the following: horizontal movement in the robot coordinate system, the origin of the robot coordinate system being the initial position of the joints prior to the plurality of transformations; and rotational movement about an axis in the robot coordinate system.

[0072] At 620, the controller 150 determines the joint positions of the joints of the robot holding the tool or the object after the plurality of transformations based on feedback data received from the robot. The feedback data may record the plurality of transformations of the robot.

[0073] In some embodiments, determining the joint positions includes: receiving feedback data from the robot; obtaining from the feedback data a first plurality of coordinate parameters of the joints of the robot in a post-transformation pose in the robot coordinate system, the robot having the post-transformation pose after performing the plurality of transformations; and determining the joint positions in the robot coordinate system based on the first plurality of coordinate parameters of the joints.

[0074] At 630, the controller 150 determines the positional relationship between the tool and the robot based at least in part on the joint positions and the reference position.

[0075] In some embodiments, determining the positional relationship includes: determining a mapping between the sensor coordinate system and the robot coordinate system based on the sensed data and a first plurality of coordinate parameters of the joints of the robot in a post-transformation pose in the robot coordinate system, the robot having the post-transformation pose after performing the plurality of transformations; determining first reference position information of the reference position in the sensor coordinate system based on the sensed data; and determining the positional relationship based on the mapping, the first plurality of coordinate parameters and the first reference position information.

[0076] In some embodiments, determining the mapping includes: determining a second plurality of coordinate parameters of the joints of the robot in the sensor coordinate system in the set of transformed postures based on the sensed data; and determining the mapping based on the first plurality of coordinate parameters and the second plurality of coordinate parameters.

[0077] In some embodiments, determining the positional relationship includes: determining second reference position information of the reference position in the robot coordinate system based on the first reference position information and the mapping; determining first tool position information in the robot coordinate system based on the second reference position information; and determining the positional relationship in the robot coordinate system based on the first tool position information and the first plurality of coordinate parameters.

[0078] In addition, the present disclosure may also propose a system for calibration. The system includes a sensor configured to sense the deviation of the tool from the object to be operated by the robot using the tool and sense the joint positions of the joints of the robot holding the tool or the object when the robot is triggered to perform a plurality of transformations; a controller coupled to the robot and the sensor and configured to execute Figure 6 the method 600 shown in

[0079] It should be understood that the system can be implemented with any hardware and software. For example, the system can be implemented as the controller 150 shown in Figure 1 , Figure 2 and Figure 4 . For example, the system can be implemented as the controller 150 and the sensor 140 shown in Figure 1 , Figure 2 and Figure 4 . It should be understood that the number of sensors shown in Figure 1 , Figure 2 and Figure 4 is given for illustrative purposes and does not imply any limitation. The operating environment 100 may include any suitable number of sensors.

[0080] The system can also be implemented as an integrated chip. The components of the system can be considered as entities capable of performing certain functions, such as data collectors, estimators, instruction generators, etc. The components in the system can also be considered as virtual modules capable of implementing certain functions.

[0081] Figure 7 is a simplified block diagram of a device 700 suitable for implementing the embodiments of the present disclosure. The device 700 can be provided to implement Figure 6 the method 600 shown in Figure 7As illustrated, device 700 may include a computer processor 710 coupled to a computer-readable memory unit 720, and the memory unit 720 includes instructions 722. When executed by the computer processor 710, the instructions 722 may implement the method for operating a robot as described in the previous paragraph, and details thereof will be omitted hereinafter.

[0082] In some embodiments of the present disclosure, a computer-readable medium for simulating at least one object in a production line is provided. Instructions are stored thereon, which when executed on at least one processor, may cause the at least one processor to execute the method for operating a robot as described in the previous paragraph, and details thereof will be omitted hereinafter.

[0083] Generally, the various embodiments of the present disclosure may be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software executable by a controller, microprocessor, or other computing device. Although the various aspects of the embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0084] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, which are executed in a device on a target real or virtual processor to perform the processes or methods described above with reference to Figure 6 description. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of the program modules may be combined or split as desired among the program modules. The machine-executable instructions for the program modules may be executed within a local device or a distributed device. In a distributed device, the program modules may be located in local and remote storage media.

[0085] Program code for performing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] The above program code can be embodied on a machine-readable medium, which can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0087] Furthermore, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order, or that all of the illustrated operations be performed to achieve the desired result. In certain scenarios, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to a particular embodiment. Certain features described in the context of separate embodiments can also be combined and implemented in a single embodiment. On the other hand, various features described in the context of a single embodiment can also be separately implemented in multiple embodiments or in any suitable sub-combination.

[0088] Although the subject matter has been described in language specific to structural features and / or method acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the above specific features or acts. Rather, the above specific features and acts are disclosed as example forms for implementing the claims.

Claims

1. A calibration method, comprising: In response to sensing data received from a sensor indicating a deviation of a tool from an object to be operated on by a robot using the tool, triggering the robot to automatically perform a plurality of transformations, wherein the plurality of transformations includes transformations of the joints of the robot from an initial pose to different post-transformation poses, each transformation causing the tool to contact the object at a reference position; Receiving feedback data from the robot, the feedback data recording the plurality of transformations of the robot; Obtaining, from the feedback data, a first plurality of coordinate parameters of the joints of the robot in a robot coordinate system in a set of post-transformation poses, the robot having the set of post-transformation poses after performing the plurality of transformations; Determining, based on the first plurality of coordinate parameters of the joints, the joint positions of the robot holding the tool or the object in the robot coordinate system after the plurality of transformations; Determining, based on the sensing data, a second plurality of coordinate parameters of the joints of the robot in a sensor coordinate system in the set of post-transformation poses; Determining a mapping between the sensor coordinate system and the robot coordinate system based on the first plurality of coordinate parameters and the second plurality of coordinate parameters; Determining, based on the sensing data, first reference position information of the reference position in the sensor coordinate system; Determining, based on the first reference position information and the mapping, second reference position information of the reference position in the robot coordinate system; Determining, based on the second reference position information, first tool position information in the robot coordinate system; And determining a positional relationship between the tool and the robot at least partially based on the first tool position information and the first plurality of coordinate parameters.

2. The method according to claim 1, wherein triggering the robot includes triggering the joints of the robot to perform at least one of the following: Horizontal movement in a robot coordinate system, the origin of the robot coordinate system being the initial position of the joints before the plurality of transformations; and Rotational movement about an axis in the robot coordinate system.

3. An electronic device, comprising: A processing unit; And A memory coupled to the processing unit and having instructions stored thereon, the instructions causing the device to perform actions when executed by the processing unit, the actions including: In response to sensing data received from a sensor indicating a deviation of a tool from an object to be operated on by a robot using the tool, triggering the robot to automatically perform a plurality of transformations, wherein the plurality of transformations includes transformations of the joints of the robot from an initial pose to different post-transformation poses, each transformation causing the tool to contact the object at a reference position; Receiving feedback data from the robot, the feedback data recording the plurality of transformations of the robot; Obtaining, from the feedback data, a first plurality of coordinate parameters of the joints of the robot in a robot coordinate system in a set of post-transformation poses, the robot having the set of post-transformation poses after performing the plurality of transformations; Determine the joint positions of the joints of the robot holding the tool or the object after the plurality of transformations in the robot coordinate system based on the first plurality of coordinate parameters of the joints; Determine a second plurality of coordinate parameters of the joints of the robot in the sensor coordinate system in the pose after the set of transformations based on the sensed data; Determine the mapping between the sensor coordinate system and the robot coordinate system based on the first plurality of coordinate parameters and the second plurality of coordinate parameters; Determine first reference position information of the reference position in the sensor coordinate system based on the sensed data; Determine second reference position information of the reference position in the robot coordinate system based on the first reference position information and the mapping; Determine first tool position information in the robot coordinate system based on the second reference position information; and Determine the positional relationship between the tool and the robot at least in part based on the first tool position information and the first plurality of coordinate parameters.

4. The apparatus according to claim 3, wherein triggering the robot includes triggering the joints of the robot to perform at least one of the following: Horizontal movement in the robot coordinate system, the origin of the robot coordinate system being the initial position of the joints before the plurality of transformations; and Rotational movement about the coordinate axes in the robot coordinate system.

5. A system for calibration, comprising: A sensor configured to sense the deviation of the tool from the object to be operated by the robot using the tool and to sense the joint positions of the joints of the robot holding the tool or the object when the robot is triggered to perform a plurality of transformations; And A controller coupled to the robot and the sensor and configured to execute the method according to any one of claims 1-2.

6. A computer-readable medium comprising program instructions for causing an electronic device to at least execute the method according to any one of claims 1-2.

Images