Actuate mechanical machine calibration to stationary mark

Abstract

A system and method for calibrating the position of a machine having stationary parts to a stationary marker. The process first images the machine and the marker, then identifies the visible part of the machine in the 3D modeled image. The process then uses the modeled position of the visible part and the known kinematics and position of the machine to calculate the position of the stationary part of the machine. The process then identifies the stationary marker in the image and establishes a relationship between the stationary marker and the stationary part of the machine that can be used for calibration purposes. In one embodiment, the machine is a robot and the process is performed by an AR application.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of the filing date of U.S. Provisional Application No. 62 / 843,120, filed May 3, 2019, entitled “Actuating Mechanical Machine Calibration to a Stationary Mark.” Technical Field

[0003] The present disclosure generally relates to a method for establishing a relationship between a machine and a fixed marker, and more specifically, to a method for calibrating a robot's position relative to a stationary augmented reality marker by identifying a relationship between a modeled visible portion of the robot, a stationary portion of the robot, and the marker. Background Art

[0004] Augmented reality (AR) has been described as an interactive experience of a real-world environment in which objects residing in the real world are augmented in a virtual world by computer-generated sensory information. It is known in the art to use AR systems to simulate the operation of industrial robots for calibration purposes, teaching purposes, etc. For example, an AR system can be used to teach a robot how to perform certain operations, such as painting a vehicle, welding body panels, etc., where a skilled operator demonstrates the operation using the AR system and the robot learns the actions involved in a manner familiar to those skilled in the art. AR systems can also be used for other teaching activities, such as establishing a virtual safe zone that the robot must not intrude.

[0005] The 3D model of the object can be used to find the object in the real world using an imaging system (e.g. a camera) and suitable algorithms, and can therefore be used for calibration, i.e. finding the offset or distance between a fixed marker and the object. For a moving mechanical system, such as a robotic system, it is sometimes desirable to know the offset from a fixed marker to a static part of the robot, for example for calibration purposes. However, this requires that the static part of the system (e.g. the base of the robot) is visible to the camera and has features sufficient to be detected by the camera. In the case where the static part is not visible or does not have sufficient features, it is difficult to use the moving part of the robot for calibration. Summary of the invention

[0006] The following discussion discloses and describes a system and method for calibrating the position of a machine having a stationary portion to a stationary marker. The process first images the machine and the marker and then identifies the visible portion of the machine in multiple images that have been 3D modeled. The process then uses the modeled position of the visible portion and the known kinematics and position of the machine to calculate the position of the stationary portion of the machine. The process then identifies the stationary marker in the image and establishes a relationship between the stationary marker and the stationary portion of the machine that can be used for calibration purposes. In one embodiment, the machine is a robot and the process is performed by an AR application.

[0007] Other features of the present disclosure will become apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is an isometric view of a workstation including a robot and a fixed stationary marker; and

[0009] Figure 2 is a flow chart illustrating a process for establishing a calibration relationship between a robot and a stationary marker. DETAILED DESCRIPTION

[0010] The following discussion of embodiments of the present disclosure directed to methods for establishing a calibration relationship between a machine and a fixed marker is merely exemplary in nature and is in no way intended to limit the present disclosure or its applications or uses.

[0011] The present disclosure proposes a system and method for calibrating a mechanical machine such as a robot relative to a fixed marker. The method requires knowledge of the current position of the machine, the kinematics of the machine's motion, and the relationship between all components of the machine and a stationary part of the machine, at least one of which is 3D modeled, significant, and sufficiently visible. An AR application images the marker and the modeled components, uses kinematics to associate the modeled components with the stationary part, and then establishes a relationship between the stationary part and the marker.

[0012] Figure 1is an isometric view of a workstation 10 including a machine, specifically a robot 12, having a base portion 14, an extension link 16 coupled to the base portion 14 by a rotating and pivoting joint 18, a working link 20 coupled to the extension link 16 by an elbow pivoting joint 22 opposite the base portion 14, and an end effector 24 to which a tool (not shown), such as a coater, will be coupled. In the following discussion, the extension link 16 will be described as a visible part of the robot 12 that has sufficient features and has been modeled by a known 3D model. Other parts of the robot 12 may also be used for this purpose, as long as they have sufficient features to be modeled and identified and are stationary during the calibration process. The base portion 14 will generally not be one of these components, as the base portion 14 generally does not have sufficient features and / or is not sufficiently visible. The robot 12 can be any multi-axis industrial robot suitable for the purposes discussed herein, such as a six-axis robot that can be programmed to perform various operations in a manufacturing environment, such as material cutting, welding, part selection / movement / replacement, painting, etc. The robot 12 has known kinematics and geometry between the base portion 14, the extension links 16 and the working links 20, and the end effector 24, as well as a known configuration, i.e., its current position. Although the robot 12 is employed in this non-limiting embodiment, the calibration techniques discussed herein will have application for calibrating other machines to a stationary mark.

[0013] A safety fence 28 is provided at the workstation 10, positioned around the robot 12 for safety purposes, and operates as a non-moving component separate from the robot 12 in this discussion. A stationary augmented reality marker 30, such as an image, model, or other indicia, having a plurality of identifiable features, is fixed to the fence 28 at a visible location thereon, wherein in other embodiments, the marker may be fixed to other stationary components separate from the robot 12. A user 32 is shown standing at the workstation 10 and holding a tablet 34 on which an AR application, the kinematics of the robot 12, and a 3D model of the extension link 16 have been downloaded. Many AR algorithms suitable for the purposes discussed herein are known in the art. The tablet 34 has, among other things, a camera 36 that captures an image of the workstation 10 provided to the AR application, a display 38 that displays the movement of the robot 12 in the workstation 10. Instead of the tablet 34, other AR devices, such as AR glasses, smartphones, etc., may also be employed.

[0014] The AR application operates an algorithm that can be used to establish a positional relationship between the stationary augmented reality marker 30 and the robot 12 to display information relative to the stationary augmented reality marker 30 in the frame of the robot 12 so that the simulated movement of the robot 12 can be viewed on the display 38. To do this, the user 32 points the camera 36 at the robot 12 so that the extension link 16 and the stationary augmented reality marker 30 are visible in the field of view of the camera 36. The algorithm uses the 3D model of the extension link 16 to identify or recognize the extension link 16 and uses the kinematics or movement of the robot 12 to determine the position of the base portion 14, which does not move during operation of the robot 12, relative to the extension link 16. The user 32 also positions the stationary augmented reality marker 30 in the field of view of the camera 36 so that the stationary augmented reality marker 30 and the extension link 16 are visible in the display 38. The algorithm then determines a mathematical relationship or calibration between the stationary augmented reality marker 30 and the base portion 14. Therefore, information known in the frame of the robot 12 can be displayed relative to the stationary augmented reality marker 30. In particular, by knowing the relationship between the stationary augmented reality marker 30 and the base portion 14 and between the base portion 14 and other parts of the robot 12, it is possible to display points on the robot 12 relative to the stationary augmented reality marker 30 to display the movement of the robot 12 on the display 38, for example to view simulated operation of the robot 12.

[0015] Figure 2 4 is a flow chart 40 showing a process for calibrating the position of the robot 12 relative to the stationary augmented reality marker 30 as described above. At box 42, the process first uses the camera 36 to image the workstation 10 so that the robot 12 and the stationary augmented reality marker 30 are visible in the image. The process then identifies the visible and discernible portion of the robot 12 in the image that has been 3D modeled at box 44, here the extension link 16. The process then calculates the position of the stationary portion of the robot 12, here the position of the base portion 14 of the robot 12, based on the modeled position of the extension link 16 and the known kinematics and position of the robot 12 at box 46. The process then identifies the stationary augmented reality marker 30 in the image at box 48 and establishes a relationship between the stationary augmented reality marker 30 and the base portion 14 of the robot 12 at box 50, which relationship can be used for calibration and simulation purposes as discussed.

[0016] The foregoing discussion discloses and describes only exemplary embodiments of the present disclosure. Those skilled in the art will readily recognize from such discussion and from the accompanying drawings and claims that various changes, modifications and variations may be made therein without departing from the spirit and scope of the present disclosure as defined in the appended claims.

Claims

1. A method for calibrating the position of a machine relative to a stationary marker, the method comprising: acquiring a plurality of images including the machine and the marking; identifying locations of visible portions of the machine in the plurality of images; calculating the position of a stationary part of the machine using the position of the visible part of the machine and the known kinematics of the machine; identifying a location of the stationary marker in the plurality of images; establishing a relationship between the stationary marker and the stationary portion of the machine; and A plurality of points on the machine are displayed using the relationship between the stationary marker and the stationary portion of the machine to illustrate simulated operation of the machine.

2. The method according to claim 1, wherein: Identifying the location of the visible portion of the machine includes using a 3D model of the visible portion.

3. The method according to claim 1, wherein: The method is performed using an augmented reality algorithm.

4. The method according to claim 1, wherein: The machine is a robot.

5. The method according to claim 4, wherein: The markers are located on the safety fence surrounding the robot's workspace.

6. The method according to claim 4, wherein: The visible part is an intermediate link of the robot, and the stationary part is a base part of the robot, wherein the intermediate link is coupled to the base part. The method of claim 1 , wherein acquiring an image comprises using a camera.

8. The method according to claim 7, wherein: The camera is part of a tablet, a smartphone or augmented reality AR glasses.

9. A method for calibrating a robot's position relative to a stationary augmented reality marker using an augmented reality (AR) algorithm, the method comprising: 3D modeling the visible part of the robot; acquiring, using a camera, a plurality of images including the robot and the stationary augmented reality marker; locating the visible portion of the robot in the plurality of images using the 3D modeled model; calculating the position of a stationary part of the robot using the position of the visible part and the known kinematics of the robot; locating the stationary augmented reality marker in the plurality of images; establishing a relationship between the stationary augmented reality marker and the stationary portion of the robot; and A plurality of points on the robot are displayed using the relationship between the stationary augmented reality marker and the stationary portion of the robot to illustrate simulated operation of the robot.

10. The method according to claim 9, wherein: The stationary augmented reality marker is located on a safety fence surrounding the robot's workspace.

11. The method according to claim 9, wherein: The visible part is an intermediate link of the robot, and the stationary part is a base part of the robot, wherein the intermediate link is coupled to the base part.

12. The method according to claim 9, wherein: The camera is part of a tablet, a smartphone or augmented reality AR glasses.

13. A system for calibrating the position of a machine relative to a stationary marker, the system comprising: means for acquiring a plurality of images including said machine and said marking; means for identifying the location of visible parts of the machine in the plurality of images; means for calculating the position of a stationary part of the machine using the position of the visible part of the machine and the known kinematics of the machine; means for identifying a location of the stationary marker in the plurality of images; means for establishing a relationship between said stationary marker and said stationary part of said machine; and Means for displaying a plurality of points on the machine using the relationship between the stationary marker and the stationary portion of the machine to illustrate simulated operation of the machine.

14. The system according to claim 13, wherein: The means for identifying the location of the visible part of the machine in the plurality of images uses a 3D model of the visible part.

15. The system of claim 13, wherein: The machine is a robot, the visible part is an intermediate link of the robot, and the stationary part is a base part of the robot, wherein the intermediate link is coupled to the base part.

16. The system of claim 15, wherein: The markers are located on the safety fence surrounding the robot's workspace.

Images