Method and system for programming a robot

Abstract

A method for programming a robot includes the following steps: a) providing a 3D representation of a workpiece to be handled by the robot; b) providing a 3D representation of a working environment, the 3D representation of the working environment including an initial position at which each workpiece is to be grasped by the robot and a final position at which the workpiece is to be installed by the robot; c) synthesizing and displaying a view of the working environment, the view of the working environment including an image of the workpiece at the corresponding initial position; d) enabling a user to select one of the displayed workpieces; e) identifying matching features of the selected workpiece and the working environment that can cooperate to hold the workpiece in the final position in the working environment, and skills that can enable the matching features to cooperate; f) based on the skills and the final position, identifying an intermediate position, applying the skills to the workpiece from the intermediate position to move the workpiece to the final position; g) adding routines to a motion program for the robot for moving the workpiece from its initial position to the intermediate position and for applying the skills to the workpiece at the intermediate position.

Description

Technical Field

[0001] The present invention relates to a method for programming a robot and a system for performing the method. Background Art

[0002] Programming industrial robots is a time-consuming task, especially for applications in which several workpieces must be assembled into a product.

[0003] Traditional CAD tools can provide very detailed information about the workpieces to be assembled into a given product. However, due to the geometric features of the different workpieces that may have to be joined to each other during the assembly process, the CAD data format, and the diversity of unknown parameters such as material properties, design tolerances, etc., currently no system can directly derive an assembly program for the robot from the CAD data of the workpieces to be assembled.

[0004] In the automation industry, there are various software products that support programming industrial robots, such as ABB PowerPac. Such software can help the user define the workspace, the work objects, and focus on automatically generating paths for the robot to machine a single fixed workpiece, e.g., by machining or welding, but provides only limited support for assembly processes involving placing workpieces. Summary of the Invention

[0005] The object of the present invention is to provide a method that facilitates programming of the assembly tasks performed by a robot.

[0006] This object is achieved by a method for programming a robot, the method comprising the following steps

[0007] a) providing a 3D representation of the workpiece to be handled by the robot;

[0008] b) providing a 3D representation of the working environment, the 3D representation of the working environment including the initial position of each workpiece to be grasped by the robot and the final position where the workpiece is to be installed by the robot;

[0009] c) synthesizing and displaying a view of the working environment, the view of the working environment including an image of the workpiece at the respective initial position;

[0010] d) enabling the user to select one of the displayed workpieces;

[0011] e) identifying matching features of the selected workpiece and the working environment that can cooperate to hold the workpiece in the final position in the working environment, and skills that can enable the matching features to cooperate;

[0012] f) based on the skills and the final position, identifying intermediate positions from which applying the skills to the workpiece will move the workpiece to the final position;

[0013] g) Add routines to the motion program for the robot to move the workpiece from its initial position to an intermediate position and to apply skills to the workpiece at the intermediate position.

[0014] In this method, what the user needs to do is to define the order of assembling the workpiece. Determining the routine for the robot to move the selected workpiece to the intermediate position and controlling the skills for the robot to bring the selected workpiece from the intermediate position to the final position, these tasks can be automated.

[0015] Displaying the currently selected workpiece at the intermediate or final position can be helpful as it enables the user to check whether the system is planning to install the workpiece at the position where it actually belongs. This is especially important if there are several identical workpieces and there is a possibility of installing one workpiece at the final position where it will obstruct the subsequent installation of other workpieces.

[0016] Therefore, this method can only proceed from step f) to step g) after the user approves the match.

[0017] If the method allows the user to drag an image of the workpiece to the desired position, assuming that the user is actually dragging the workpiece towards the position where it should be installed, this can help the method identify a suitable intermediate position.

[0018] On the other hand, if the user drags the workpiece away from the currently displayed intermediate position, it is obvious that the user does not approve of the intermediate position and wishes to install the workpiece elsewhere.

[0019] When the final position of the first workpiece is determined, the working environment should be updated by including the workpiece at its final position in the working environment. Therefore, when the user selects the second workpiece, the search for the matching features of the second workpiece and the working environment can automatically ignore the features occupied by the first workpiece, and the calculation of the path for the robot to move the second workpiece from its initial position to its intermediate position can take into account the profile of the working environment modified by adding the first workpiece.

[0020] The 3D representation of the workpiece for synthesizing the view and finding the matching features is preferably derived from the CAD data of the workpiece in the preparation step.

[0021] If features of a workpiece are marked in the CAD data, it is possible to facilitate finding such features that can match features of the working environment. Such tags can unambiguously characterize the feature by: the way it connects to the matching feature in the working environment, or by referring to the skill with which it is to be connected to its mating feature, i.e., by defining the feature as, for example, an external or internal thread, a welding surface, a plug, a socket, etc., or it can simply specify that the feature is expected to connect to some matching feature in the working environment, leaving the task of identifying the matching feature and the appropriate skill (e.g., based on the geometric properties of the feature) to a computer system or to the user who sees the features shown in the view of the working environment.

[0022] If the CAD data includes a 3D representation of the product to be assembled by the workpiece, the orientation of the workpiece in the product can be extracted from the CAD data. In this case, the user's task can be simplified by showing the user in the view of the working environment all the orientations that the workpieces will have in the assembled product.

[0023] Obviously, there are as many different types of matching features as there are skills for joining the workpieces, and in principle, the present invention can be applied to any of these. By way of illustration, the matching feature can be

[0024] - projections and recesses that can be joined in a given direction. In this case, the associated skill would be to push the workpiece in the given direction. Optionally, if the projections and recesses have the same cross-section, they can be considered a match. Alternatively; the matching feature can be

[0025] - external and internal threads, in which case the associated skill is to tighten; or

[0026] - flat surfaces, in which case the associated skills can be gluing, welding, etc.

[0027] For a user who sees the workpiece in a synthetic view, the skill of installing the workpiece in the working environment is often immediately obvious. For example, when the workpiece is a screw, it is obvious to a human user that it must be tightened, and the only question may be to find the correct hole for the screw in a complex environment. Thus, if the user specifies to the computer system the skill of installing the workpiece, this greatly reduces the system's selection of candidates for the matching feature, enabling the matching feature pair of the workpiece and the working environment to be found much more quickly.

[0028] The present invention can also be embodied in a computer system that includes a computer, a display, and a coordinate input device, where the computer is programmed to perform the above method based on user input provided via the coordinate input device, or in a computer program that, when executed by a computer system, causes the computer system to perform the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Further features and advantages will become apparent from the subsequent description of its embodiments with reference to the accompanying drawings.

[0030] Figure 1 is a block diagram of a computer system;

[0031] Figures 2 to 5 is a view of the working environment generated during the execution of the method of the present invention by the computer system. Detailed Description of the Invention

[0032] The computer system of the present invention includes a general-purpose computer 1 having a CPU 2, program and data storage devices 3, 4, a display 5, and a coordinate input device 6. The program storage device 3 stores a program whose instructions enable the computer to execute the method described below. The data storage device 4 stores 3D representations of the initial working environment, the product to be assembled, and the workpieces to be assembled into the product, typically CAD data. These representations include all the data required to generate a realistic or at least clearly identifiable image of each workpiece on the display 5. They also include detailed information about the features of the workpieces that are to be connected to the environment or to each other, through which the computer can determine whether two such features can be connected to each other. The robot for which the system is to generate a program that will enable it to assemble the physical workpieces is not necessarily part of the system.

[0033] In the basic case, the initial working environment is a physical surface 7, such as a tabletop, and in the first step of the method, the first workpiece 8 is virtually fixed on the surface by the computer 1, thereby obtaining a secondary working environment. The computer 1 synthesizes a view of the secondary working environment and some of the workpieces 9 - 14 that have not yet been installed, as Figure 2 shown and displays it on the display 5.

[0034] In this view, some of the virtual workpieces 9 - 14 are shown in orientations in which their physical counterparts would be unstable on the surface of the working environment. The reason is that the computer 1 derives the orientations that the workpieces 9 - 14 will have in the product from the 3D representation of the product to be assembled and displays them in those orientations. In this way, it is easier for the user to discern from the view the ways in which the workpieces can be installed.

[0035] For a human user, it is easy to see that the workpieces 9 - 14 are of different types and will be connected to the workpiece 8 by different skills. In this example, the workpiece 8 has matching features for each of the workpieces 9 - 14; in a more complex case, there may be workpieces that have not yet been installed and for which there are no matching features in the working environment yet, but which will be formed during the installation of other workpieces; in such a case, there will be workpieces in the view that cannot yet be installed, and the user must select the workpieces that can be installed.

[0036] For the assembly of a product, several workpieces of the same type may be required, such as screws. In this case, there will be several positions available in the working environment where the screws can be installed, but the computer 1 usually has no criteria for deciding which particular screw should be installed in which position. This decision should be made by the user and entered into the computer system, as will be described below.

[0037] The workpiece 9 is a screw. If in the 3D representation mentioned above, the workpiece is explicitly marked as a screw, the computer 1 can be aware of this fact. Alternatively, the computer can be programmed to identify it as a screw based on the geometric characteristics of the workpiece 9. Further alternatively, the information that the workpiece 9 is a screw can be entered by the user, for example when selecting it or in a preliminary step in which all the workpieces 9 - 14 are successively characterized.

[0038] The user selects the workpiece 9 in the usual way by the following steps: placing the cursor 15 on it in the view on the display 5, using the coordinate input device 6, and pressing a key. When the workpiece 9 is selected, when the user further moves the cursor 15, the image of the workpiece 9 will move as if attached to the cursor 15.

[0039] The coordinate input device 6 can be a 3D input device, commonly known as a "space mouse", through which not only the three coordinates of the workpiece in the coordinate system of the working environment can be specified, but also the orientation angle of the workpiece can be specified. It is preferable to use a simpler and cheaper input device. For example, in the current case, the device for specifying the orientation angle can be omitted because the orientation of the workpiece shown in the view does not have to be changed, or because, if rotation is necessary, the computer determines the rotation without the need for input from the user. In addition, it may be sufficient to only input two spatial coordinates because the computer 1 can select the third coordinate such that the workpiece is positioned adjacent to the surface of the working environment shown in the view.

[0040] Suppose the user uses the coordinate input device 6 to drag the screw towards the hole 16 of the workpiece 8 ( Figure 3 ). Based on the 3D representation, the computer 1 checks whether the screw fits into the hole 16. If so, the user will be aware of this fact, for example by the image of the screw flashing, changing its color, etc. If the user realizes that the screw 9 should not enter the hole 16, he will further drag the screw, and the image of the screw will return to normal.

[0041] When the screw 9 is moved near the hole 17, the system again detects that the screw may fit and makes the user aware of this. The user confirms that the screw 9 will enter the hole 17, for example by releasing or pressing again the key previously used to select the workpiece.

[0042] Inserting the solid screw 9 into the hole 17 requires the robot to perform a tightening action. Based on the coordinates of the hole 17, the computer 1 calculates an intermediate position 9'( Figure 4 ), from which the screw can be inserted into the hole 17, i.e., a position close to the surface of the workpiece 8, where the axes of the screw 9 and the hole 17 are aligned. Then, the computer calculates a routine through which the robot can first move the solid screw from its initial position to the said intermediate position adjacent to the workpiece 8 and screw it in from there, and append it to the working program for the robot.

[0043] The position where the user drags the image of the screw near the hole 17 and where the computer 1 detects that the screw can fit into the hole 17 generally will not be the same as the above-mentioned intermediate position. Therefore, in order to make the user aware of the possible fit, the computer 1 can, in addition to or as an alternative to the above method, suddenly move the image of the screw (or any other workpiece just selected) from the position set by the user to the intermediate position 9'. Since the screw thus moves relative to the cursor 15 - in Figure 4 which it is actually separated from the cursor 15 - the user cannot fail to notice the displacement, even if it is a small one.

[0044] When the virtual screw 9 is inserted into the hole 17 and thus reaches Figure 5 its final position 9" shown, the hole 17 is no longer available for inserting a workpiece therein, and the presence of the screw head outside the hole 17 may affect how other workpieces can access the workpiece 8 and connect to its other features. Therefore, a new secondary working environment is calculated, which includes not only the workpiece 8 but also the screw 9, and which will be used for machining the next workpiece selected by the user.

[0045] The workpiece 10 is a rectangular plug. Once this fact is recognized by the system, based on the stored 3D representation or input from the user, the computer 1 starts searching the working environment for a suitable socket. This process can be accelerated by the user selecting the workpiece 10 and dragging it towards the socket 18, thus indicating to the computer 1 the area in the working environment where the final position of the workpiece 10 may be found, as well as the best starting position for the search for that final position. When the match between the workpiece 10 in the working environment and its associated features such as the socket 18 is confirmed, the computer 1 autonomously calculates an intermediate position adjacent to the socket 18, where the longitudinal axes of the plug and the socket 18 are aligned, such that by the linear displacement of the robot, the solid plug can be pressed from the intermediate position into its final position in the socket 18 of the solid workpiece 8, and the computer 1 places the image of the workpiece 10 at the said intermediate position in the view shown on the display 5 to make the user aware of the match.

[0046] Based on the 3D representation, computer 1 can identify the location where the workpiece must be installed in a very short time, or can even identify this location before the user selects the workpiece. This is possible, especially if the workpiece, such as a plug, only appears once in the product to be assembled. In this case, the computer will move the image of the workpiece to its intermediate or final position while the user selects the workpiece.

[0047] Workpiece 11 is a fixture. A human user will easily recognize that among all the features of workpiece 8, only the fixture can enter hole 19. The computer will not do so in advance because if only geometric features are compared, it will consider the barb 20 of fixture 11 not suitable for entering hole 19. Here, explicitly marking workpiece 11 as an elastic fixture, either through the label included in the 3D representation or through user input, enables the system to ignore the barb 20, recognize that the shank 21 of the fixture is indeed suitable for the cross-section of hole 19, and make the user aware of this fact in either of the above ways. Based on this information, the system can further program the robot such that when the fixture moves from its intermediate position in front of hole 19 to its final position inside the hole, sufficient pressure is applied to deflect the barbs 20 so that they will enter hole 19.

[0048] Workpiece 13 is a cylindrical rod. The user selects the rod, drags it to and finally inserts it into hole 16, which can be performed according to the above principles. However, the system cannot pre-judge from the geometric characteristics of workpiece 13 whether it cannot move after installation, or whether it is to be rotatably installed. Again, such information must be provided in the 3D representation of workpiece 13 or workpiece 8, or by user input. Depending on this information, computer 1 determines whether the robot program for installing the rod includes skills such as welding, ultrasonic welding, or friction welding in addition to the skill of pushing the rod into hole 16.

[0049] Reference numerals

[0050] 1 Computer

[0051] 2 CPU

[0052] 3 Data storage device

[0053] 4 Data storage device

[0054] 5 Display

[0055] 6 Coordinate input device

[0056] 7 Solid surface

[0057] 8 - 14 Workpieces

[0058] 15 Cursor

[0059] 16 Hole

[0060] 17 holes

[0061] 18 sockets

[0062] 19 holes

[0063] 20 barbs

[0064] 21 shank.

Claims

1. A method for programming a robot, comprising the steps of: a) providing a 3D representation of at least one workpiece to be handled by the robot; b) providing a 3D representation of the working environment, the 3D representation of the working environment including an initial position where each workpiece is to be grasped by the robot and a final position where the workpiece is to be installed by the robot; c) synthesizing and displaying a view of the working environment, the view of the working environment including an image of the workpiece (9 - 14) at the corresponding initial position; d) enabling a user to select one of the displayed workpieces (9 - 14); e) identifying matching features (17) of the selected workpiece (9 - 14) and the working environment that can cooperate to hold the workpiece (9 - 14) in the final position (9") of the working environment, and skills that can enable the matching features to cooperate and enable the workpiece to be installed in the working environment; f) based on the skills and the final position (9"), identifying an intermediate position (9'), applying the skills to the workpiece (9 - 14) from the intermediate position to move the workpiece to the final position (9"); g) adding a routine to a motion program for the robot to move the workpiece (9 - 14) from its initial position to the intermediate position and to apply the skills to the workpiece (9 - 14) at the intermediate position.

2. The method according to claim 1, wherein step f) includes displaying the workpiece (9 - 14) at the intermediate position.

3. The method according to claim 2, wherein the method proceeds from step f) to step g) only after the user approves the match.

4. The method according to claim 2 or 3, further comprising enabling the user to drag the image of the workpiece (9 - 14) to a desired position.

5. The method according to claim 4, wherein if the user drags the image of the workpiece away from the intermediate position, the match is considered not to be approved by the user.

6. The method according to any one of the preceding claims 1 - 3, further comprising the following steps: h) updating the working environment by including the workpiece at the final position of the workpiece in the working environment.

7. The method according to claim 6, wherein after step h), the method returns to step c).

8. The method according to any one of the preceding claims 1 - 3, including a preparatory step of deriving the 3D representation of the workpiece from CAD data.

9. The method according to claim 8, wherein the features of the workpiece (9 - 14) to be matched with the features (17) of the working environment are identified in the CAD data.

10. The method according to claim 8, wherein the CAD data includes a 3D representation of the product to be assembled by the workpiece, and in step c), each workpiece (9 - 14) is displayed in the orientation it has in the product.

11. The method according to any one of the preceding claims 1 - 3, wherein the matching features include: - Protrusions and grooves that can be joined in a given direction, and the associated skill is to push the workpiece in the given direction; Or - External threads and internal threads, and the associated skill is to screw; - Flat surfaces, and the associated skill is to place the surfaces in contact.

12. The method according to claim 11, wherein the protrusions and grooves have the same cross-section.

13. The method according to claim 11, wherein the placing of the surfaces in contact is accompanied by pressing and / or heating.

14. The method according to any one of the preceding claims 1-3, further comprising the step of identifying matching features of the workpiece and the working environment, taking into account the user-specified skill.

15. A computer system comprising a computer, a display, and a coordinate input device, wherein the computer is programmed to perform the method according to any one of claims 1 to 14 based on user input provided via the coordinate input device.

16. A computer program product comprising a computer program or instructions that, when executed by a computer system, cause the computer system to perform the method according to any one of claims 1 to 14.

Images