Robot teaching method and robot operation method

Abstract

A robot teaching method includes a pre-registration process, a robot movement process, and a teaching process. In the pre-registration process, a relative self-position of a measuring device (31) with respect to a surrounding environment is determined by measuring the surrounding environment with the measuring device (31), and a teaching point of a robot (10) determined using the relative self-position of the measuring device (31) with respect to the surrounding environment, i.e., an environmental teaching point, is registered. In the robot movement process, the robot (10) is automatically moved so as to make the relative self-position of the robot (10) with respect to the surrounding environment coincide with the environmental teaching point, based on a measurement result of the surrounding environment measured by the measuring device (31), in a state where the robot (10) is equipped with the measuring device (31). In the teaching process, a detection value of a position and a posture of the robot (10) detected by an internal sensor (12) is registered as teaching information in a state where the relative self-position of the robot (10) with respect to the surrounding environment coincides with the environmental teaching point.

Description

TECHNICAL FIELD

[0001] The present application relates to a teaching method for an industrial robot. BACKGROUND

[0002] As a teaching method for an industrial robot, there is known a method in which a robot is actually moved by an operation device until a teaching point, and data (detection values of internal sensors) about a position and a posture of the robot are registered.

[0003] Further, Patent Literature 1 discloses a method in which an industrial robot is caused to perform work by visual servoing. Visual servoing is a control in which a camera is attached to a robot, and the robot is caused to move so that an image captured by the camera during work coincides with a teaching image registered in advance. That is, in the case of visual servoing, teaching of a robot is performed by capturing a work site and a workpiece by a camera.

[0004] Here, in the method in which teaching is performed by actually moving a robot, the robot needs to be operated by an operation device, and thus it is necessary to be skilled. Further, in the case of using visual servoing, it is not possible to simply determine how the robot is caused to move during work so as to approach a teaching image.

[0005]

Patent Literature

[0006]

Patent Literature 1

[0007] In view of the above, an object of the present application is to provide a robot teaching method capable of registering useful teaching information with a simple method.

[0008] The technical problem to be solved by the present application is as described above, and the technical solutions for solving the technical problem and the effects thereof are described below.

[0009] Based on the present invention, the following robot teaching method for teaching an industrial robot is provided. The robot teaching method includes a pre-registration process, a robot movement process, and a teaching process. In the pre-registration process, a relative self-position of a measuring device with respect to a surrounding environment is determined by measuring the surrounding environment with the measuring device; and an environmental teaching point, which is a teaching point of the robot, is registered using the relative self-position with respect to the surrounding environment. In the robot movement process, the robot is automatically moved in a state where the measuring device is attached to the robot so that the relative self-position of the robot with respect to the surrounding environment coincides with the environmental teaching point based on a measurement result of the surrounding environment detected by the measuring device. In the teaching process, a detection value of a position and an attitude of the robot detected by a sensor is registered as teaching information in a state where the relative self-position of the robot with respect to the surrounding environment coincides with the environmental teaching point.

[0010] Thus, the robot can be taught by the operator only by performing the operation of specifying the environmental teaching point, and therefore, the robot can be taught in a simple manner compared to a teaching method in which the operator actually moves the robot. In addition, the teaching information registered in the present invention is a detection value of a sensor, and therefore, the position and the attitude of the robot can be grasped accurately compared to a case where the teaching information of visual servo is an image.

[0011] Effects of the Invention

[0012] Based on the present invention, a robot teaching method in which useful teaching information can be registered in a simple manner can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a configuration diagram of a robot system.

[0014] Figure 2 is a process diagram of a robot teaching method.

[0015] Figure 3 is an explanatory diagram of an environmental map creation process.

[0016] Figure 4 is an explanatory diagram of a pre-registration process.

[0017] Figure 5 is an explanatory diagram of a robot movement process.

[0018] Figure 6 is a flowchart of an operation in which a measuring device is not attached to a robot.

[0019] Figure 7 is a flowchart of an operation in which a measuring device is attached to a robot.

[0020] Figure 8 is an explanatory view showing a case where a virtual auxiliary image is displayed on an information terminal.

[0021] Figure 9 is an explanatory view showing a case where an AR auxiliary image is displayed on an information terminal. DETAILED DESCRIPTION

[0022] Hereinafter, an embodiment of the present application will be described with reference to the drawings. First, referring to Figure 1 , a robot system 1 of the present embodiment will be described. Figure 1 is a block diagram of the robot system 1.

[0023] The robot system 1 is configured in a facility such as a factory. The robot system 1 is a system for causing a robot 10 to work. As Figure 1 indicated, the robot system 1 includes the robot 10, a control device 20, and a measurement unit 30. Each device is connected to each other through a wired or wireless network. In Figure 1 , only one set of the robot 10 and the control device 20 is described, but the robot system 1 can also be configured to have a plurality of sets of the robot 10 and the control device 20.

[0024] The robot 10 is an industrial robot. The work performed by the robot 10 is, for example, assembly, welding, painting, washing, and the like. The robot 10 is of a teach-playback type. The teach-playback type refers to a type in which a worker pre-teaches a motion and work of the robot 10 by hand, and the robot 10 is caused to repeatedly perform the taught motion and work. However, the robot 10 can also be of a type other than the teach-playback type.

[0025] The robot 10 has an arm 11 mounted on a base. The arm 11 has a plurality of joints, each of which has an actuator mounted thereon. The robot 10 changes the position and posture of the arm 11 by causing the actuators to act in accordance with an operation command input from the outside. An internal sensor 12 is provided on each joint. The internal sensor 12 is mounted on the robot 10 and is a sensor that detects the position and posture of the robot 10. Specifically, the internal sensor 12 detects the rotation angle or angular velocity of each joint of the arm 11. The internal sensor 12 is also sometimes referred to as an encoder. A terminal actuator 13 corresponding to the work content is mounted on the tip of the arm 11. The robot 10 performs work by causing the terminal actuator 13 to act in accordance with an operation command input from the outside. In addition, the position and orientation of the terminal actuator 13 can be determined from the detection value of the internal sensor 12. In addition, on the arm 11, a mounting structure (for example, a mounting hole or the like) for mounting the measurement unit 30 is provided. The mounting structure is provided in the vicinity of the terminal actuator 13, that is, on the tip side of the joint on the most tip side (terminal actuator 13 side) of the robot 10. In other words, the mounting structure is provided at a position that moves and rotates integrally with the terminal actuator 13. In addition, the mounting position of the measurement unit 30 is not limited to the tip of the robot 10, and can be provided at any other position as long as the positional relationship with the tip of the robot 10 (terminal actuator 13) can be determined.

[0026] The control device 20 is constituted by a known computer and has a calculation device (CPU or the like) and a storage device (for example, a flash memory or an HDD or the like). By causing the calculation device to read and execute a program stored in the storage device, the control device 20 can realize various functions. For example, the control device 20 causes the robot 10 to act by making an operation command of the robot 10 and sending it to the robot 10. In the operation command of the robot 10, an instruction to cause the actuators provided on each joint of the arm 11 to act and an instruction to cause the terminal actuator 13 to act are included.

[0027] The measurement unit 30 is mainly a device for teaching the robot 10. The measurement unit 30 is held and operated by a worker (hereinafter, simply referred to as a worker) who performs teaching, and can be detachably attached to the robot 10. The measurement unit 30 is provided with a measurement device 31, a processing device 32, and a pointer 33. The measurement device 31, the processing device 32, and the pointer 33 are fixed in the same housing or the like, and the worker can perform processing integrally. However, in the case where the measurement device 31 and the processing device 32 can perform wireless communication, the processing device 32 can be provided at a position apart from the measurement device 31. That is, the measurement unit 30 of the present embodiment adopts a structure in which the measurement device 31 and the processing device 32 are integrated, but can also adopt a structure in which the processing device 32 is an external device. In addition, the measurement unit 30 of the present embodiment is a dedicated product for performing teaching of the robot 10, but can also be a general-purpose information processing terminal such as a smartphone. In this case, a camera provided on the smartphone corresponds to the measurement device 31, a SOC provided in the smartphone corresponds to the processing device 32, and a pointer (a rod-shaped object or a laser pointer) attached to the smartphone corresponds to the pointer 33.

[0028] The measurement device 31 measures the position and shape of an object existing around the measurement device 31 (surrounding environment) and detects surrounding environment data indicating the surrounding environment. The measurement device 31 detects the surrounding environment data at a predetermined time interval. The measurement device 31 of the present embodiment is a stereo camera. The stereo camera is provided with a pair of imaging elements (for example, CCDs) disposed at an appropriate distance from each other. Each of the imaging elements performs imaging and acquires an image. By comparing the images acquired by the respective imaging elements, the position of an object or the like included in the acquired images can be calculated. In addition, the measurement device 31 can also be a LiDAR (Laser Imaging Detection and Ranging). The LiDAR is one of three-dimensional measurement sensors that acquire the position and shape of an object or the like around by emitting an electric wave and measuring the time until a reflected wave of the electric wave is received. In addition, the measurement device 31 is not limited to the stereo camera and the LiDAR, and can also be a monocular camera or a ToF (Time of Flight) camera.

[0029] The processing device 32 processes the surrounding environment data acquired by the measurement device 31. The processing device 32 has the arithmetic device and the storage device as described above. The processing device 32 performs SLAM (Simultaneous Localization and Mapping) processing on the surrounding environment data. The SLAM processing is known, and thus only a brief description will be given. That is, a feature point is extracted from the surrounding environment data acquired by the measurement device 31, and a change in the position and the posture of the measurement device 31 is acquired by analyzing the movement amount of the feature point between a plurality of pieces of surrounding environment data, and an environmental map is created based on the position of the feature point. The environmental map is data indicating the three-dimensional position and the three-dimensional shape of equipment, machines, workpieces, and the like arranged around the measurement device 31 (in this embodiment, the place where the robot 10 performs work, that is, the work site). Note that the processing device 32 can not be installed in the measurement unit 30, but can be arranged at a position apart from the measurement unit 30.

[0030] The pointer 33 is a member for making the work of the worker to specify a teaching point of the robot 10 easier. The pointer 33 is a rod, and thus makes the work of specifying the teaching point accurate. In addition, the pointer 33 protrudes further forward than the measurement device 31, and thus the work of specifying the teaching point can be easily performed. The measurement unit 30 aims to acquire a self-position at a state where the pointer 33 indicates a teaching point. Thus, the relative position between the measurement device 31 and the pointer 33 is fixed. In addition, the pointer 33 can be detachably attached to the measurement unit 30. The rod-shaped pointer 33 is not limited to a structure in which a rod-shaped member is used to indicate, and for example, a structure in which a specific position or direction is indicated by irradiating laser light can be adopted.

[0031] Next, the robot teaching method according to the present embodiment will be described with reference to Figures 2 to 5 to the drawings. Figure 2 is a flowchart of the robot teaching method. Figures 3 to 5 is an explanatory view of each step of the robot teaching method.

[0032] First, as shown in Figure 3As shown, the measuring unit 30 is used to create an environmental map of the work area (S101, preparation step). Specifically, the operator holds the measuring unit 30 and detects the surrounding environmental data of the work area by changing the position and orientation of the measuring unit 30. The processing device 32 performs SLAM processing on this data to create an environmental map of the work area. In addition, in order to create an environmental map including the workpiece 40, a preparation step is required with the workpiece 40 positioned in the work's working position. Alternatively, the robot 10 can be used to move the measuring unit 30 instead of the operator. Specifically, the measuring unit 30 can be mounted on the robot 10, and the robot 10 can be made to move in a way that draws a predetermined trajectory to create an environmental map of the work area.

[0033] Secondly, such as Figure 4 As shown, environmental teaching points are registered on the environmental map (S102, pre-registration process). An environmental teaching point refers to a teaching point of the robot 10 determined by its relative position to the surrounding environment (hereinafter, the surrounding environment) within the work area. In this embodiment, the environmental teaching point is recorded in the coordinate system of the environmental map. That is, in this embodiment, coordinate values ​​from the environmental map are used as an example of "relative position to the surrounding environment." Furthermore, the environmental teaching point includes not only a three-dimensional position but also the concept of orientation (e.g., rotation angle around three axes). Registering the environmental teaching point means storing the environmental teaching point (coordinate values ​​from the environmental map) in, for example, the storage unit of the measurement unit 30 or the control device 20.

[0034] In this embodiment, environmental teaching points are designated in the actual space. Specifically, such as... Figure 4As shown, the operator holds the measurement unit 30 and moves the measurement unit 30 to a position desired to be registered as a teaching point. At this time, the operator moves the measurement unit 30 while looking at the position and orientation of the pointer 33. The positional relationship between the pointer 33 and the end effector 13 is determined in advance. For example, it can be determined that the position of the tip of the pointer 33 corresponds to the position of the tip of the end effector 13, or it can be determined that the position deviated from the tip of the end effector 13 by a prescribed distance (e.g., 5 cm below) corresponds to the tip of the end effector 13. Then, the operator aligns the measurement unit 30 with the appropriate position in consideration of the positional relationship. Thereafter, the operator acquires the self-position on the environmental map in the state by operating the operation section (not shown) of the measurement unit 30. At this time, if the positional relationship between the pointer 33 and the end effector 13 in the state where the measurement unit 30 is attached to the robot 10 coincides with the above-described determined positional relationship between the pointer 33 and the end effector 13, the self-position acquired at this time is registered as the environmental teaching point. In contrast, if the positional relationships are different, the coordinate value calculated by adding a correction value for eliminating the difference to the above-described acquired self-position is registered as the environmental teaching point.

[0035] In addition, the environmental teaching point can also be specified by displaying the environmental map. In this case, the control device 20 displays the environmental map on a prescribed display (a display of the control device 20, or a display of an information terminal possessed by the operator, etc.) and accepts input of the environmental teaching point by the operator. The operator specifies the environmental teaching point by confirming the environmental map while, for example, clicking the environmental teaching point on the environmental map or inputting the coordinate value of the environmental coordinate, etc. The environmental teaching point thus specified is registered. Since the environmental map is created by the SLAM processing, the three-dimensional model of the work site and the robot 10 is not necessary, and thus it is possible to reduce the working hours spent by the operator. In addition, the acquired environmental map can be edited by the operator. The operator can invalidate the information within a specific region by specifying the region on the environmental map, or can cause the region to be preferentially used in the estimation of the self-position. In addition, a specific region can also be specified by setting a marker on the environmental map.

[0036] In addition, the creation of the environmental map is not necessary. That is, as long as the self-position with respect to the surrounding environment can be determined, the self-position can be used as the environmental teaching point, and thus the environmental teaching point can be specified without using the environmental map. In this case, the preparation process can also be omitted. Specifically, by using the measurement device 31 of the present embodiment, the surrounding environment can be measured, and thus the self-position with respect to the surrounding environment can be determined. In addition, in order to improve the accuracy of the determination of the self-position, it is preferable to register information about objects disposed around in advance, or to dispose markers around, etc.

[0037] In addition, in the present embodiment, the image acquired by the measuring device 31 until the operator aligns the measuring unit 30 with the teaching point is stored in the control device 20 or an external server or the like. In this image, the intention of the operator who determined the teaching point taking into account which aspects is sometimes reflected. Thus, by not deleting this image after the teaching is completed and keeping the image at least for a certain period after the teaching, the intention of the teaching can be confirmed thereafter. In addition, since a video is an arrangement of images acquired at time intervals, the expression "storage of the image" or the like also includes storage of a video or the like.

[0038] Next, the robot 10 is automatically operated so that the self-position of the measuring unit 30 coincides with the environmental teaching point in a state where the measuring unit 30 is attached to the robot 10 (S103, robot operation step). In addition, since the measuring unit 30 is attached to the robot 10, the self-position of the measuring unit 30 is included in the self-position of the robot 10 (a point different from the measuring unit 30 can also be used as the position of the robot 10). In addition, the robot 10 can be automatically operated by the following method. That is, the processing device 32 can calculate the current self-position on the environmental map (in other words, can calculate the coordinate value in the environmental coordinates). In addition, the control device 20 can calculate the current position and posture of the robot 10 (in other words, can calculate the coordinate value in the robot coordinates) from the detection value of the internal sensor 12. Thus, the correspondence relationship between the coordinate value in the environmental coordinates and the coordinate value in the robot coordinates can be acquired. By changing the position or posture of the robot 10 and acquiring the coordinate values in the environmental coordinates and the robot coordinates, a plurality of the correspondence relationships can be acquired. In addition, the environmental coordinates and the robot coordinates substantially correspond one-to-one. Thus, from a plurality of the correspondence relationships, a conversion matrix (conversion data) for performing conversion between the environmental coordinates and the robot coordinates can be created.

[0039] Thus, for example, the self-position calculated by the measuring unit 30 can be converted into the coordinate value in the robot coordinates, and thus the robot 10 can be controlled on the basis of the robot coordinates. In addition, in order to improve the accuracy of the conversion matrix, a new correspondence relationship can be acquired at a prescribed time point, and the conversion matrix can be updated on the basis of the new correspondence relationship. In addition, the value after the environmental teaching point is converted into the coordinate value in the robot coordinates by the conversion matrix can be used as a target value. In addition, the method of automatically operating the robot 10 is not limited to this, and for example, the robot 10 can be controlled on the basis of the environmental coordinates, or the robot 10 can be controlled by visual servoing.

[0040] Next, the detected value of the internal sensor 12 of the robot 10 in the state where the self-position coincides with the environment teaching point is registered as the teaching information (S104, teaching process). The registration of the teaching information means that the teaching information is stored in the control device 20.

[0041] In addition, the position and the posture of the robot 10 in the embodiment can be detected by a sensor other than the internal sensor, for example, an external sensor that measures the robot 10 from the outside. In the case of using the external sensor, the detected value of the position and the posture of the robot 10 detected by the external sensor becomes the teaching information. In addition, in the case of using the external sensor, it is preferable to register the detected value of the position and the posture of the robot 10 detected by the external sensor (or the difference between the detected value of the target and the current detected value) as information for conversion into the motion command that the control device 20 outputs to the robot 10.

[0042] In the teaching method of the embodiment, the robot 10 is automatically moved in the robot motion process by registering the environment teaching point in advance. In contrast, in the general method of the related art, the robot needs to be moved to the teaching point using the operation device. In addition, the operation of the operation device needs to be skilled. Thus, the robot can be taught in a simple method compared with the teaching method of the related art. In addition, the teaching information registered in the embodiment is the same as the teaching information obtained by the teaching method of the related art.

[0043] The above teaching method is not limited to the case of registering new teaching information, and can be used in the case of updating the existing teaching information.

[0044] Next, the method of operating the robot 10 using the teaching information registered by the above method will be described with reference to Figure 6 and Figure 7

[0045] Since the teaching information of the robot 10 is the detected value of the internal sensor 12, the robot 10 can operate without the measurement unit 30. However, there are cases where it is preferable to attach the measurement unit 30 to operate under certain conditions.

[0046] First, the process performed by the control device 20 in the case of operating without the measurement unit 30 will be described. The control device 20 compares the coordinate value of the robot coordinate indicated by the teaching information with the coordinate value of the current robot coordinate, and calculates the difference (x, y, z) between the coordinate values. Figure 6 ​The control device 20 determines whether the difference is below the threshold value (S202). In the case where the difference is not below the threshold value, the control device 20 causes the robot 10 to act in such a manner that the difference becomes smaller (S203). In the case where the difference is below the threshold value, the control device 20 causes the work to be performed using the end effector 13 (S204). Thereafter, the control device 20 causes the robot 10 to move to the prescribed standby position (S205), and the same processing is performed again on the new workpiece 40.

[0047] An advantage of performing the work without the measurement unit 30 is that the introduction cost of the robot system 1 is lower compared to the case of using visual servoing. That is, the measurement unit 30 is required at the time of teaching, but is not required at the time of work. Therefore, by performing teaching of a plurality of robots 10 with one measurement unit 30, the number of measurement units 30 required can be reduced.

[0048] In addition, in the case where a plurality of robots 10 of the same kind are introduced, and each robot 10 performs the same work, the teaching information created at the time of teaching one robot 10 can be transferred to another robot 10. Specifically, the teaching information registered in the control device 20 that controls the robot 10 on which teaching has been performed is registered in the control device 20 that controls another robot 10. Thereby, the man-hours taken for teaching can be reduced. In addition, instead of transferring the teaching information, the environmental teaching points can be transferred. In this case, the work of mounting the measurement unit 30 on the robot 10 and causing it to automatically act (robot action process) is required for each robot, but the work of registering the environmental teaching points by the worker using the measurement unit 30 (pre-registration process) is required only once. By this method, the teaching information can be created taking into account the individual differences of the robots 10.

[0049] Next, the processing performed by the control device 20 in the case where the work is performed with the measurement unit 30 mounted will be described. Figure 7 The processing in the case where the work is performed with the measurement unit 30 mounted is described in FIG. 21. Among these, the processing other than step S210 is the same as in the case where the work is performed without the measurement unit 30 mounted (i.e., the processing of S201 to S205 is common processing, and is independent of whether the measurement unit 30 is mounted or not). By mounting the measurement unit 30, the position of the workpiece 40 can be detected. Therefore, for example, in the case where the position of the workpiece 40 slightly deviates from the prescribed position, the work can be performed correctly. Specifically, after the control device 20 determines that the difference is below the threshold value (i.e., after step S202), the position and posture of the robot are adjusted so that the position and orientation of the end effector 13 with respect to the workpiece 40 are appropriate (S210).

[0050] Specifically, in the teaching process (S104), the relative position of the workpiece 40 with respect to the ego position in a state where the ego position coincides with the environment teaching point is additionally registered. The relative position of the workpiece 40 is used to indicate the appropriate positional relationship between the robot 10 (end effector 13) and the workpiece 40. Then, in step S210, the relative position of the workpiece 40 with respect to the current ego position is determined based on the measurement result of the measurement device 31. Next, the current relative position of the workpiece 40 is compared with the previously registered relative position of the workpiece 40 to find the difference, and the robot 10 is operated in such a manner that the difference is reduced. Thus, even if the position of the workpiece 40 slightly deviates from the intended position, the work can be performed correctly. In addition, in the case where the pointer 33 is detachable, the pointer 33 can be detached during the work.

[0051] Next, an application example of the present embodiment will be described. First, referring to Figure 8 An application example in which a virtual assistance image is displayed on the information terminal 51 in the pre-registration process will be described.

[0052] The virtual assistance image refers to an image in which the robot 10 is disposed in a virtual space corresponding to the position and orientation of the current measurement unit 30. By looking at the virtual assistance image, the worker can easily confirm whether the robot 10 interferes with other objects in the work site. In addition, the worker can also confirm whether the indicated position and orientation are possible (whether the robot 10 can become the specified posture). In addition, the virtual assistance image is not only a specific posture of the robot 10, but can also be a range of postures that the robot 10 can reach, in other words, can be an image indicating the possible angles or positions of the robot 10 (especially the hand of the robot 10).

[0053] In addition, in the present embodiment, the virtual assistance image is displayed on the display (output device) of the information terminal 51 (portable terminal) that the worker can hold, but for example, in the case where there is another display in the work site, the virtual assistance image can also be displayed on the display. In addition, it is not limited to a display, but various devices (for example, a projector or a head-mounted display, etc.) that can output an image can also be used to output the virtual assistance image. In the present embodiment, the information terminal 51 performs the process of creating the virtual assistance image, but it can also be that another device creates the virtual assistance image and sends it to the information terminal 51.

[0054] The following describes an example of a method for creating a virtual auxiliary image. To create the virtual auxiliary image, a 3D model (3D CG data) of the work area (specifically, workpiece 40) and robot 10 is prepared in advance and configured in a virtual space. The position of robot 10 at that point in time is a temporarily determined position. Furthermore, it is assumed that the information terminal 51 has a program that outputs the rotation angles of each joint to implement the input state after the position and orientation of the end effector 13 and the setting position of robot 10 are input.

[0055] In the pre-registration process, the measurement unit 30 calculates its own position on the environmental map and outputs it to the information terminal 51. The information terminal 51 obtains the correspondence between the positions of various parts of the work area contained in the environmental map and the positions of various parts of the work area contained in the virtual space. Based on these correspondences, the information terminal 51 generates conversion data that converts the environmental coordinates into coordinates in the virtual space. By converting the self-position on the environmental map using this conversion data, the information terminal 51 determines the position and orientation of the measurement unit 30 in the virtual space, thereby determining the position and orientation of the end effector 13 in the virtual space. Then, the information terminal 51 obtains the rotation angle of each joint of the robot 10 by applying the above procedure to the determined position and orientation of the end effector 13. In addition, if the robot 10 cannot achieve the specified posture, the information terminal 51 displays this information on the display. If the robot 10 can achieve the specified posture, the information terminal 51 depicts this state in the virtual space. Thus, a virtual auxiliary image is created. In addition, this method is only one example, and other methods (such as using three-dimensional markers to create conversion data) can also be used.

[0056] Below, refer to Figure 9 An application example of displaying AR-assisted images on information terminal 51 will be explained. AR stands for Augmented Reality.

[0057] AR-assisted images are images obtained by overlaying the robot 10 (especially the end effector 13) corresponding to the current position and orientation of the measurement unit 30 onto an image captured by the camera of the information terminal 51. By viewing the AR-assisted image, the operator can intuitively grasp the position and orientation of the end effector 13 while registering environmental teaching points. In addition, the variations shown in the virtual assistive image can also be applied to AR-assisted images.

[0058] Next, the method of creating the AR assist image will be described. The method of superimposing another image on the camera is known, and various methods of creating the image are available. One example of the method will be described below, but other methods can also be used. In order to create the AR assist image, it is necessary to determine how to depict the position, posture, size, and the like of the robot 10 when the robot 10 is superimposed on the image captured by the camera. In order to determine these, the AR marker 53 is arranged in the work site. The position (coordinates in the actual space) at which the AR marker 53 is arranged and the size of the AR marker 53 are registered in advance. When the AR marker 53 is included in the image captured by the camera, the information terminal 51 determines the direction in which the AR marker 53 exists based on the position at which the AR marker 53 is displayed, the orientation of the AR marker 53 based on the orientation at which the AR marker 53 is displayed, and the distance to the AR marker 53 based on the size of the AR marker 53 on the image. In addition, as described above, the position and orientation of the AR marker 53 in the work site are determined in advance. In addition, the position and posture of the robot 10 in the work site can be determined based on the self-position on the environmental map. Thus, the position, orientation, and size of the robot 10 when the robot 10 is superimposed on the image captured by the camera of the information terminal 51 can be determined, and thus the AR assist image can be created.

[0059] As described above, the robot teaching method of the present embodiment includes a registration step, a robot operation step, and a teaching step. In the registration step, a registration point of the robot 10 determined based on the relative self-position of the measuring device 31 with respect to the surrounding environment is registered by measuring the surrounding environment with the measuring device 31 to determine the relative self-position of the measuring device 31 with respect to the surrounding environment. In the robot operation step, the robot 10 is automatically operated so that the relative self-position of the robot 10 with respect to the surrounding environment coincides with the environmental registration point based on the measurement result of the surrounding environment measured by the measuring device 31 while the measuring device 31 is attached to the robot 10. In the teaching step, the detection value of the position and posture of the robot 10 detected by the internal sensor 12 is registered as teaching information in a state where the relative self-position of the robot 10 with respect to the surrounding environment coincides with the environmental registration point.

[0060] Thus, the operator can teach the robot 10 by performing only the operation of specifying the environmental registration point, and thus the robot 10 can be taught in a simple manner compared to a teaching method in which the operator actually operates the robot 10. In addition, the teaching information of the visual servo is an image, and the teaching information registered in the present embodiment is a detection value of a sensor, and thus the position and posture of the robot 10 can be grasped accurately.

[0061] In the robot teaching method of the present embodiment, in the pre-registration process, the worker holds the measurement device 31 and registers the environmental teaching point based on the self position at the time of moving the measurement device 31 to the teaching point of the robot 10 in the actual space.

[0062] Thus, the worker can perform the registration of the environmental teaching point by simply moving the measurement device 31 to the teaching point.

[0063] In the robot teaching method of the present embodiment, the measurement device 31 is a stereo camera, and the image captured in the pre-registration process is continuously saved as an image representing the teaching intention of the worker after the end of the teaching process.

[0064] Thus, the teaching intention of the worker can be confirmed.

[0065] In the robot teaching method of the present embodiment, before the pre-registration process, a preparation process of creating an environmental map of the surrounding environment by performing SLAM processing after measuring the surrounding environment with the measurement device 31 is performed. In the pre-registration process, the environmental map is displayed, and input of the environmental teaching point by the worker is accepted, and the environmental teaching point is registered based on the input content of the worker.

[0066] Thus, the registration of the environmental teaching point can be performed by simply specifying the environmental teaching point on the environmental map.

[0067] In the robot teaching method of the present embodiment, in the robot operation process, a plurality of correspondence relations between the coordinates of the environmental map of the surrounding environment (i.e., environmental coordinates) and the coordinates of the robot 10 based on the detection values of the internal sensor 12 (i.e., robot coordinates) are obtained. Next, conversion data for converting the environmental coordinates into the robot coordinates is created based on the plurality of correspondence relations. Next, the self position on the environmental map obtained by performing SLAM processing on the measurement result of the measurement device 31 is converted using the conversion data, and the current coordinate value in the robot coordinates is calculated. Next, the robot 10 is controlled based on the current coordinate value in the robot coordinates so that the self position in the environmental map coincides with the environmental teaching point.

[0068] Thus, the robot 10 can be controlled based on the robot coordinates, and thus the robot 10 can be caused to move to the teaching point appropriately.

[0069] In the robot teaching method of the present embodiment, after the creation of the conversion data, the correspondence relation between the environmental coordinates and the robot coordinates is obtained again, and the conversion data is updated based on the correspondence relation.

[0070] Thus, the precision of the conversion data can be improved, and thus the robot 10 can be caused to move using more accurate robot coordinates.

[0071] In the robot teaching method of the present embodiment, the sensor used in the teaching step to detect the position and posture of the robot 10 is the internal sensor 12 of the robot 10.

[0072] The detection value of the internal sensor 12 is the value itself indicating the position and posture of the robot 10, and thus the direction in which the position and posture of the robot 10 should be moved in the work can be correctly grasped.

[0073] In the robot teaching method of the present embodiment, the robot 10 is of the teaching playback type.

[0074] Therefore, the present application can be applied to general industrial robots.

[0075] In the robot teaching method of the present embodiment, in the pre-registration step, an image of the work site in which the robot 10 is disposed in the virtual space corresponding to the position and orientation of the current measuring device 31 (i.e., a virtual assistance image) is output using an output device.

[0076] Thus, it is possible to simply confirm whether the work site interferes with the robot 10 or not, and the like.

[0077] In the robot teaching method of the present embodiment, in the pre-registration step, an image of the robot 10 corresponding to the position and orientation of the current measuring device 31 is displayed on the image captured by the camera of the information terminal 51 (i.e., an AR assistance image) on the display of the information terminal 51.

[0078] Thus, the operator can register the environmental teaching points while intuitively grasping the position and orientation of the end effector 13 by looking at the AR assistance image.

[0079] In the robot work method of the present embodiment, the robot 10 performs work without the measuring device 31.

[0080] Therefore, compared with visual servoing in which the robot 10 is provided with a camera, the introduction cost can be reduced.

[0081] In the robot work method of the present embodiment, the robot 10 performs work with the measuring device 31 attached. The control device 20 performs processing to adjust the position and posture of the robot 10 with respect to the work object, the workpiece 40.

[0082] Therefore, even in the case where the position of the workpiece 40 slightly deviates from the predetermined position, the work can be correctly performed.

[0083] Explanation of Reference Numerals

[0084] 1 robot system

[0085] 10 robot

[0086] 20 control device

[0087] 30 measuring unit

[0088] 31 measuring device

[0089] 32 processing device

[0090] 33 pointer

Claims

1. A robot teaching method, characterized in that, Comprising: a pre-registration process of registering an environmental teaching point determined as a teaching point of the robot with respect to the surrounding environment by measuring the surrounding environment with a measuring device to determine a relative self-position of the measuring device with respect to the surrounding environment; a robot action process of automatically moving the robot in a state where the relative self-position of the robot with respect to the surrounding environment coincides with the environmental teaching point, based on a measurement result of the surrounding environment detected by the measuring device, in a state where the measuring device is attached to the robot; and a teaching process of registering a detection value of a position and a posture of the robot detected by a sensor as teaching information in a state where the relative self-position of the robot with respect to the surrounding environment coincides with the environmental teaching point, in the robot action process, a plurality of correspondence relations between environmental coordinates that are coordinates of an environmental map obtained by performing SLAM processing on a measurement result of the measuring device and robot coordinates that are coordinates of the robot based on the detection value of the sensor are acquired; conversion data for converting the environmental coordinates into the robot coordinates is created based on the plurality of correspondence relations; and a current coordinate value in the robot coordinates is calculated by calculating a self-position on an environmental map obtained by performing SLAM processing on a measurement result of the measuring device using the conversion data, and the robot is controlled based on the coordinate value to make the self-position on the environmental map coincide with the environmental teaching point.

2. The robot teaching method according to claim 1, wherein: in the pre-registration process, an operator holds the measuring device and registers the environmental teaching point based on a self-position of the measuring device when the measuring device is moved to a teaching point of the robot in an actual space.

3. The robot teaching method according to claim 2, wherein: the measuring device is a stereo camera, and images captured in the pre-registration process are continuously saved as images representing a teaching intention of the operator after the teaching process ends.

4. The robot teaching method according to claim 1, wherein: before the pre-registration process, a preparation process of creating an environmental map of the surrounding environment by performing SLAM processing after measuring the surrounding environment with the measuring device is performed, in the pre-registration process, the environmental map is displayed and input of the environmental teaching point by an operator is received, and the environmental teaching point is registered based on the input content of the operator.

5. The robot teaching method according to any one of claims 1 to 4, wherein: after the conversion data is created, a correspondence relation between the environmental coordinates and the robot coordinates is determined again, and the conversion data is updated based on the correspondence relation.

6. The robot teaching method according to any one of claims 1 to 4, wherein: the sensor used in the teaching process to detect the position and the posture of the robot is an internal sensor of the robot.

7. The robot teaching method according to any one of claims 1 to 4, characterized in that: the robot is of a teaching playback type.

8. The robot teaching method according to any one of claims 1 to 4, characterized in that: in the pre-registration process, an image of a work site in which the robot is disposed in a virtual space, i.e., a virtual assist image, corresponding to the current position and orientation of the measuring device is output using an output device.

9. The robot teaching method according to any one of claims 1 to 4, characterized in that: in the pre-registration process, an image of the robot, i.e., an AR assist image, corresponding to the current position and orientation of the measuring device is superimposed on an image captured by a camera of an information terminal and displayed on a display of the information terminal.

10. A robot work method in which a robot uses the teaching information registered using the robot teaching method according to any one of claims 1 to 9 to perform work, characterized in that: the robot performs work without the measuring device.

11. A robot work method in which a robot uses the teaching information registered using the robot teaching method according to any one of claims 1 to 9 to perform work, characterized in that: the robot performs work with the measuring device attached, and performs processing of adjusting the position and posture of the robot with respect to a workpiece as a work object based on a measurement structure of the measuring device.

Images