Instructional data storage device

Abstract

To solve such a problem that if simulation of a linear body attached to a robot is performed after completion of a series of teachings, a teaching result could be wasted.SOLUTION: A teaching data accumulation apparatus 3 includes: a positional relation acquisition unit 33 which acquires a relative positional relation between a real environment and a display device 4; a reception unit 34 which receives operation and teaching instructions of a three-dimensional model 10 of a virtual robot; an image generation unit 35 which generates a display image that displays the operated three-dimensional model 10 on the basis of the three-dimensional model and the relative positional relation; an output unit 36 which outputs the display image to the display device 4; an accumulation unit 37 which accumulates teaching data at the time of reception of the teaching instructions; and a torsional amount acquisition unit 38 which acquires a torsional amount of a linear body attached to the three-dimensional model of the robot body of the virtual robot. The output unit 36 performs an output on the acquired torsional amount. In this way, it is possible to perform teaching while confirming the torsional amount of the linear body.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a teaching data storage device that stores teaching data. [Background Art]

[0002] Conventionally, systems that perform simulation on the posture or the like of linear bodies such as cables and hoses are known. Further, such simulation is sometimes performed on a linear body attached to a robot (see, for example, Patent Document 1). [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2016-087750 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, simulation for a linear body attached to a robot is normally performed after the motion of the robot is determined, that is, after teaching is completed. Therefore, when a simulation result reveals that the linear body undergoes twisting exceeding an allowable range, it is necessary to perform new teaching again, which poses a problem that the previous teaching may become wasted.

[0005] The present invention has been made to solve the above problem, and an object of the present invention is to provide a teaching data storage device capable of preventing a linear body attached to a robot body from undergoing twisting exceeding an allowable range when the robot operates in accordance with teaching data. [Means for Solving the Problem]

[0006] To achieve the above objective, a teaching data storage device according to one aspect of the present invention includes: a model storage unit that stores a 3D model of a virtual robot corresponding to a real robot; a teaching data storage unit that stores teaching data for a real robot; a position relationship acquisition unit that acquires the relative position relationship between the real environment and a display device that overlays an image onto an image of the real environment or the real environment itself; a reception unit that receives instructions for operation and teaching of the 3D model of the virtual robot; an image generation unit that generates a display image for displaying the operated 3D model of the virtual robot at a predetermined position in the real environment based on the relative position relationship with the 3D model of the virtual robot; an output unit that outputs the display image generated by the image generation unit to a display device; a storage unit that stores teaching data in the teaching data storage unit according to the 3D model of the virtual robot when the reception unit receives instructions for teaching the 3D model of the virtual robot; and a torsion amount acquisition unit that acquires the torsion amount of a linear body attached to the 3D model of the robot body of the virtual robot, wherein the output unit also outputs the torsion amount acquired by the torsion amount acquisition unit. [Effects of the Invention]

[0007] According to a teaching data storage device in one aspect of the present invention, the amount of twist of a linear body attached to the robot body can be known during teaching. Therefore, for example, if the amount of twist of the linear body is large, the amount of twist of the linear body can be reduced by changing the handling of the linear body or changing the teaching method. As a result, when the robot operates according to the stored teaching data, it is possible to prevent twisting exceeding an acceptable range from occurring in the linear body attached to the robot body. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram showing the configuration of a teaching data storage device according to an embodiment of the present invention. [Figure 2] A flowchart illustrating the operation of the teaching data storage device according to this embodiment. [Figure 3]This figure shows an example of the display of the 3D model and torsion amount of the virtual robot in the same embodiment. [Figure 4] A diagram showing an example of the amount of torsion obtained in the same embodiment. [Figure 5] Block diagram showing another example of the configuration of the teaching data storage device according to the same embodiment. [Modes for carrying out the invention]

[0009] The teaching data storage device according to the present invention will be described below using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and their repeated explanation may be omitted. The teaching data storage device according to this embodiment acquires the amount of twist of a linear body attached to the robot body during teaching and outputs information regarding that amount of twist.

[0010] Figure 1 is a schematic diagram showing the configuration of an information processing system 100 including a teaching data storage device 3 according to this embodiment. As shown in Figure 1, the information processing system 100 comprises a robot 1, a robot control device 2, a teaching data storage device 3, and a display device 4. The devices may be connected to each other by, for example, wired or wireless connections. In this embodiment, the description mainly focuses on the case where the robot 1, which is a real robot, exists in a real environment as shown in Figure 1, but this is not required. For example, the robot 1 may not exist in the real environment. In this case, the information processing system 100 may not include, for example, the robot 1 or the robot control device 2.

[0011] Robot 1 is typically an industrial robot and may be a manipulator having multiple arms (i.e., links) connected by motor-driven joints. Robot 1 may be, for example, a vertical articulated robot or a horizontal articulated robot. Furthermore, Robot 1 may be, for example, a transport robot, a welding robot, an assembly robot, a painting robot, or a robot for other purposes. In this embodiment, the case where Robot 1 is a welding robot will be mainly described. Robot 1 also includes a robot body and a linear body 1a attached to the robot body. Note that the part of Robot 1 other than the linear body 1a may be considered as the robot body. The linear body 1a may be, for example, a wiring cable for supplying power, a wiring cable for transmitting control signals, a conduit cable through which a welding wire passes, a pipe for supplying liquid or gas, or other linear configuration attached to the robot body. Preferably, the linear body 1a changes shape according to the posture of Robot 1, i.e., it is flexible. Figure 1 shows the case where the linear body 1a is fixed to the housing of the robot 1 at both ends 1b and 1c, but the linear body 1a may also be fixed to the housing of the robot 1 at locations other than the ends 1b and 1c. The positions where the linear body 1a is fixed to the housing of the robot 1 are sometimes called fixing positions. Also, Figure 1 shows, as an example, the case where a part of the linear body 1a passes inside the housing of the robot 1, but this is not required. The linear body 1a may be entirely outside the housing, or entirely inside the housing. Note that since the robot 1 is a robot that exists in a real environment, it is sometimes called a real robot to distinguish it from a virtual robot that is a 3D model existing in a virtual environment. A real environment refers to the environment of real space.

[0012] The robot control device 2 controls the operation of the robot 1 using a teaching playback method. The robot control device 2, which operates the robot 1 based on teaching data, is already publicly known, and a detailed explanation thereof will be omitted. The information processing system 100 may further include, for example, a teaching pendant for teaching the robot 1, and if the robot 1 is a welding robot, it may further include a welding power supply, a wire feeder, etc., as needed.

[0013] The teaching data storage device 3 stores teaching data for robot 1 using a 3D model 10 of a virtual robot corresponding to robot 1. Details of the teaching data storage device 3 will be described later.

[0014] The display device 4 displays an image superimposed on an image of the real environment or the real environment itself. That is, the instructor can view both the real environment and the virtual environment images using the display device 4. The display device 4 may be a wearable display device worn on the head of the instructor, or it may be a portable information processing terminal such as a tablet. The wearable display device may be, for example, a head-mounted display. The display device 4 may also have, for example, a transparent display. In this case, the display device 4 will display the image superimposed on the real environment itself. Examples of wearable display devices 4 with transparent displays include HoloLens®. Such a display device 4 with a transparent display can also be considered a display device for realizing mixed reality (MR). The display device 4 may also have, for example, an opaque display. In this case, the display device 4 will display the image superimposed on an image of the real environment. Therefore, it is preferable that the display device 4 with an opaque display has a camera for capturing the real environment, or is connected to a camera for capturing the real environment. Images of the real environment captured by the camera are displayed in real time on an opaque display. Examples of wearable display devices 4 having an opaque display include the Oculus Quest. Such a display device 4 having an opaque display can also be considered a display device for realizing augmented reality (AR). A portable information processing terminal such as a tablet device may have, for example, a camera and a display, and may display images of the real environment captured by the camera on the display in real time. In this embodiment, the case in which the display device 4 is a head-mounted display having a transparent display will be mainly described.

[0015] As shown in Figure 1, the teaching data storage device 3 is connected to the robot control device 2 and the display device 4, and comprises a model storage unit 31, a teaching data storage unit 32, a position relationship acquisition unit 33, a reception unit 34, an image generation unit 35, an output unit 36, a storage unit 37, and a twist amount acquisition unit 38.

[0016] The model memory unit 31 stores a 3D model 10 of a virtual robot corresponding to the actual robot, robot 1. The 3D model 10 of the virtual robot is the same size and configuration as the actual robot, robot 1, except that it is composed of a 3D model. In addition, the 3D model 10, like robot 1, allows for the modification of the angles of the joints of multiple arms. Furthermore, the 3D model 10 of the virtual robot, like robot 1, has linear bodies 10a fixed to the housing at both ends 10b and 10c.

[0017] The process by which information about the 3D model of the virtual robot is stored in the model storage unit 31 is not specified. For example, the information may be stored in the model storage unit 31 via a recording medium, or the information transmitted via a communication line or the like may be stored in the model storage unit 31. The model storage unit 31 is preferably implemented using a non-volatile recording medium, but it may also be implemented using a volatile recording medium. The recording medium may be, for example, a semiconductor memory or a magnetic disk. Furthermore, the shape of the 3D model 10 stored in the model storage unit 31 and the angles of each joint may be changed, for example, according to the received operation.

[0018] The teaching data storage unit 32 stores teaching data for the actual robot, robot 1. The teaching data for robot 1 is stored in the teaching data storage unit 32 by the storage unit 37. The teaching data stored in the teaching data storage unit 32 is the teaching data used to control the actual robot, robot 1. Therefore, for example, the teaching data stored in the teaching data storage unit 32 may be output to the robot control device 2. In this case, the teaching data storage device 3 may have, for example, a teaching data output unit (not shown) for outputting the teaching data stored in the teaching data storage unit 32 to the robot control device 2, other devices, a removable recording medium, etc. Also, for example, when the robot control device 2 controls robot 1, it may access the teaching data stored in the teaching data storage unit 32. The teaching data may, for example, indicate the position and orientation of robot 1 for each teaching point. The information indicating the position and orientation of robot 1 may, for example, be information indicating the position and orientation of the end effector of robot 1, or it may be information indicating the angle in each axis of robot 1. Furthermore, the teaching data may also include information indicating interpolation methods between teaching points, such as linear interpolation, curve interpolation, or circular arc interpolation. The teaching data may also include, for example, teaching lines with interpolated teaching points. When the teaching data includes positional information such as teaching points, this positional information may be, for example, information in the robot's local coordinate system or information in the world coordinate system. The former is preferred when the 3D model 10 of the virtual robot is displayed at a different position from the actual robot 1. In the latter case, the teaching data is typically used for robot 1 positioned at a predetermined location in the world coordinate system. The teaching data storage unit 32 is preferably implemented using a non-volatile recording medium, but may also be implemented using a volatile recording medium. The recording medium may be, for example, a semiconductor memory or a magnetic disk.

[0019] Furthermore, the model storage unit 31 and the teaching data storage unit 32 may be implemented by the same recording medium, or may be implemented by separate recording media. In the former case, for example, the area storing the three-dimensional model 10 of the virtual robot serves as the model storage unit 31, and the area storing teaching data serves as the teaching data storage unit 32.

[0020] The positional relationship acquisition unit 33 acquires the relative positional relationship between the real environment and the display device 4. The real environment may be, for example, a real environment where the robot 1 is present. This case is mainly described in the present embodiment. Acquiring the relative positional relationship between the real environment and the display device 4 may be, for example, acquiring the relative positional relationship between a world coordinate system, which is the coordinate system of the real environment, and a display coordinate system, which is the local coordinate system of the display device 4. The relative positional relationship may be represented by, for example, a homogeneous transformation matrix representing transformation between the two coordinate systems. Note that the relative positional relationship between the real environment and the display device 4 may be, for example, the relative positional relationship between the display device 4 and the robot 1 or other objects present in the real environment, or a reference marker described later, and may also be information indicating the position and posture of the display device 4 in the real environment.

[0021] There is no limitation on the method by which the positional relationship acquisition unit 33 acquires this relative positional relationship. For example, when an environmental map of the real environment is prepared, the positional relationship acquisition unit 33 may use the environmental map of the real environment, distances to surrounding objects acquired by the display device 4, and surrounding images, and use SLAM (Simultaneous Localization and Mapping) or Visual-SLAM techniques to acquire the relative positional relationship, which is information indicating the position and orientation of the display device 4 in the three-dimensional real environment. In this case, for example, the environmental map is stored in the teaching data storage device 3, and the positional relationship acquisition unit 33 may acquire the relative positional relationship by using the distances to surrounding objects and surrounding images received from the display device 4 and the environmental map. Furthermore, for example, the acquisition of the relative positional relationship using SLAM or Visual-SLAM techniques may be performed by the display device 4, and the obtained relative positional relationship may be transmitted to the positional relationship acquisition unit 33. In this case, the acquisition of the relative positional relationship by the positional relationship acquisition unit 33 may be the reception of the relative positional relationship. In addition, in this case, the display device 4 may include a depth sensor capable of measuring the distance to a surrounding object, and a camera capable of acquiring an image of the surroundings. Furthermore, the positional relationship acquisition unit 33 may receive an image captured by the camera of the display device 4, and acquire a homogeneous transformation matrix representing the transformation between the world coordinate system and the display coordinate system using three or more feature points of the robot 1, other objects included in the image, or reference markers described later.

[0022] Furthermore, the position relationship acquisition unit 33 may acquire the relative position relationship between the real environment and the display device 4 by methods other than those described above. For an example of such a method, please refer to, for example, the following references 1 to 3. Also, the relative position relationship may be acquired by the position relationship acquisition unit 33 as described above each time a display image is generated, or it may not be. In the latter case, for example, the relative position relationship between the real environment and the display device 4 may be updated using changes in the position and orientation of the display device 4 after the relative position relationship has been acquired. This update may be performed, for example, by the position relationship acquisition unit 33. Furthermore, if this update is performed, the display device 4 may be equipped with, for example, an acceleration sensor or a sensor for acquiring the orientation of the display device 4, and changes in position and orientation may be acquired using these sensors. The values ​​acquired by these sensors may be passed to a component that updates the relative position relationship, such as the position relationship acquisition unit 33. The sensor for acquiring the orientation of the display device 4 may be, for example, a gyro sensor or an orientation sensor. Reference 1: Japanese Patent Publication No. 2017-100234 Reference 2: Japanese Patent Publication No. 2020-055075 Reference 3: Japanese Patent Publication No. 2020-069538

[0023] The reception unit 34 receives instructions for operations on the 3D model 10 of the virtual robot. The reception unit 34 also receives instructions for teaching the 3D model of the virtual robot. For example, instructions for operations and teaching the 3D model 10 may be received from an operator viewing the display image of the display device 4. Operations on the 3D model 10 may include, for example, operations to change the position or orientation of at least a part of the 3D model 10. Instructions for operations and teaching may be received, for example, via an input device such as a teaching pendant present in the real environment, or via a virtual input interface such as a virtual button or virtual teaching pendant displayed on the display of the display device 4. In the latter case, for example, instructions for operations and teaching the 3D model 10 of the virtual robot may be received, for example, via the display device 4. The virtual input interface may be displayed on the display of the display device 4 in response to an action such as an air tap, and when a button or the like is selected by the operator's finger or pointing device, input corresponding to the operation of that button or the like may be passed from the display device 4 to the reception unit 34. Furthermore, the operator may change the position and orientation of at least a part of the 3D model 10 of the virtual robot (for example, an end effector such as a tool) by gestures of the operator's hands. In this case, for example, the operator may perform a pinching motion (holding motion) of the 3D model 10 displayed on the display, thereby identifying the pinched part as the target of operation. By changing the position and orientation of the hand, an operation is performed to change the position and orientation of the identified target, and by ending the pinching motion, the operation on the identified target is completed. In this case, for example, information indicating the target of operation (for example, information indicating the position or part of the 3D model 10) and information indicating the content of the operation (for example, information indicating a change in position or orientation) may be passed from the display device 4 to the reception unit 34, or the results of hand tracking acquired by the display device 4 may be passed to the reception unit 34, and the target of operation and the content of the operation may be identified in the teaching data storage device 3.If the display device 4 acquires information indicating the object of the operation and information indicating the content of the operation, the display device 4 may be able to access information on the current 3D model 10 of the virtual robot in the virtual space, which is held in the teaching data storage device 3. If the operation is performed using the operator's hands, the display device 4 may have a camera, and the position of the operator's hands on the display may be determined by hand tracking of the operator's hands captured by the camera. Furthermore, methods for converting operations on the displayed image of the 3D model in response to gestures such as those of the operator's hands into changes in the position and orientation of the 3D model in the 3D virtual space are already known, and a detailed explanation thereof will be omitted.

[0024] Furthermore, the reception unit 34 may accept changes to the handling of the linear body 10a attached to the 3D model 10 of the robot body of the virtual robot. The reception unit 34 may accept changes to the handling of the linear body 10a input via, for example, an input device in the real environment or a virtual input interface. Changes to the handling of the linear body 10a may include, for example, at least one of the following: a change in the initial twist amount of the linear body 10a, a change in the length of the linear body 10a, or a change in the fixed position of the linear body 10a. The initial twist amount of the linear body 10a may be the angle obtained by rotating the linear body 10a from its initial angle at at least one of the fixed positions of the ends 10b, 10c of the linear body 10a. If the change in the handling of the linear body 10a is a change in the initial twist amount, the reception unit 34 may accept the changed initial twist amount. The length of the linear body 10a may be the length along the linear body 10a from one fixed position end 10b to the other fixed position end 10c. If the change in the routing of the linear body 10a is a change in length, the reception unit 34 may accept the changed length. The fixed position of the linear body 10a may be, for example, the fixed position of the end of the linear body 10a. The fixed position may be changeable within a predetermined range, for example. For example, if the linear body 10a is a wiring cable, the fixed position of the end may be changed by a few centimeters by adding an extension adapter to the end of the linear body 10a. If the change in the routing of the linear body 10a is a change in the fixed position, the reception unit 34 may accept an identifier that identifies the changed fixed position. In this case, it is preferable that the correspondence between the fixed position of the linear body 10a and the identifier that identifies that fixed position is stored in advance in the teaching data storage device 3. Furthermore, if the change in the handling of the linear body 10a is a change in the length of the linear body 10a or a change in its fixed position, the portion of the linear body 10a in the 3D model 10 of the virtual robot may be changed in response to the acceptance of such change. This change may be performed, for example, by the image generation unit 35. In this embodiment, the case in which the change in the handling of the linear body 10a is a change in the initial amount of twist of the linear body 10a will be mainly described.

[0025] The reception unit 34 may accept information other than that described above. For example, the reception unit 34 may accept information indicating that teaching has been completed. The reception unit 34 may accept information input from an input device or display device 4, or it may receive information transmitted via a wired or wireless communication line. The reception unit 34 may or may not include a device for receiving information (for example, an input device or a communication device). Furthermore, the reception unit 34 may be implemented by hardware or by software such as a driver that drives a predetermined device.

[0026] The image generation unit 35 generates a display image for displaying the 3D model 10 at a predetermined location in the real environment, based on the 3D model 10 stored in the model storage unit 31 and the relative positional relationship acquired by the positional relationship acquisition unit 33. Furthermore, when an operation on the 3D model 10 of the virtual robot is received, the image generation unit 35 generates a display image for displaying the operated 3D model 10. The image generation unit 35 may, for example, change the angles of each joint of the 3D model 10 stored in the model storage unit 31 according to the received operation. Displaying the 3D model 10 at a predetermined location in the real environment may mean, for example, displaying the 3D model 10 such that the 3D model 10 of the virtual robot and the robot 1 (which is the real robot) are in a predetermined positional relationship, or, as will be described later, displaying the 3D model 10 such that the 3D model 10 of the virtual robot is in a predetermined positional relationship with a reference marker. The predetermined positional relationship may be, for example, a predetermined positional relationship, or a positional relationship that can be changed by the operator. The 3D model 10 of the virtual robot may be displayed, for example, at the position of robot 1 or a reference marker, or at a position different from robot 1 or a reference marker. Displaying the 3D model 10 of the virtual robot at the position of robot 1 means displaying the 3D model 10 so that the virtual robot is positioned at the same location as robot 1. For example, this may be done by displaying the 3D model 10 so that its base end (e.g., the part that attaches to the floor) overlaps with the base end of robot 1. Displaying the 3D model 10 of the virtual robot at the position of a reference marker means displaying the 3D model 10 so that the situation in which the actual robot is placed at the position of the reference marker is virtually reproduced. For example, this may be done by displaying the 3D model 10 so that its base end face (e.g., the surface that attaches to the floor) coincides with the surface of the reference marker. When the 3D model 10 of the virtual robot is displayed at a position different from robot 1 or a reference marker, for example, the 3D model may be displayed next to robot 1 or a reference marker.In any case, the 3D model 10 of the virtual robot will be displayed in a way that does not change its position in the real environment. Therefore, even if the operator changes the orientation of the display device 4, the display position of the 3D model 10 of the virtual robot in the real environment will not change.

[0027] The positions of robot 1 and the reference marker in the world coordinate system are known. Furthermore, the positional relationship between robot 1, the reference marker, and the 3D model 10 of the virtual robot is also determined. Therefore, using this information, the image generation unit 35 can position the 3D model 10 of the virtual robot in a virtual space where the positions of robot 1 and the reference marker are predetermined, so that it has a predetermined positional relationship with robot 1. The angles of each joint in the 3D model 10 of the virtual robot are initial values ​​if no operations are performed, and post-operation values ​​if operations are performed. The post-operation angles of each joint may be calculated, for example, by inverse kinematics using the position and orientation of the end-effector in the 3D model 10 of the virtual robot after operation, similar to a real robot. Additionally, the image generation unit 35 can determine the relationship between the world coordinate system and the display coordinate system, which is the local coordinate system of the display device 4, based on the relative positional relationships acquired by the positional relationship acquisition unit 33, and thus can identify the position and orientation of the display device 4 in the virtual space. Therefore, the image generation unit 35 can generate a two-dimensional display image for displaying the 3D model 10 of the virtual robot in the virtual space by rendering the 3D model 10 based on the position and orientation of the display device 4. When the 3D model 10 of the virtual robot is manipulated, the shape of the 3D model 10 of the virtual robot in the virtual space will change accordingly, as described above. Also, if the position or orientation of the display device 4 changes in the real environment, the position and orientation of the viewpoint in the virtual space will change accordingly. Then, rendering is performed after the change, generating a display image of the 3D model 10 after the manipulation and a display image of the 3D model 10 after the change in the position and orientation of the display device 4. The image generation unit 35 generates the display image so that when the display image of the 3D model 10 of the virtual robot is displayed on the display of the display device 4, the size of the display image matches that of the real environment.In other words, the display image is generated such that the 3D model 10 of the virtual robot displayed on the display device 4 and the actual robot placed in the real environment so as to have the same relative positional relationship as the 3D model 10 appear to be the same size when viewed through the display device 4.

[0028] It should be noted that it may not be possible to operate the virtual robot in response to the accepted operation. For example, an operation to move the end-effector of the 3D model 10 of the virtual robot beyond its movable range or rotate it beyond its rotatable range may be accepted. In such cases, the image generation unit 35 does not need to generate a display image corresponding to the operation, or it may generate a display image of the 3D model 10 of the virtual robot that has been moved or rotated within the possible range.

[0029] The image generation unit 35 may generate a display image that also displays the teaching position corresponding to the teaching data. That is, the teaching position may also be displayed in the display image. By displaying the teaching position, the operator will be able to know the taught position. Displaying the teaching position may, for example, mean displaying the teaching point, or it may mean displaying the teaching line, which is the interpolation result of multiple teaching points. Note that whether the position information included in the teaching data is information from the robot's local coordinate system or information from the world coordinate system, the positional relationship between the 3D model 10 of the virtual robot and the teaching position can be known from the teaching data. Therefore, since the teaching position in the virtual space is determined, the image generation unit 35 can generate a display image that includes the 3D model 10 of the virtual robot and the teaching position in the same manner as described above.

[0030] The image generation unit 35 may also generate a display image to show the amount of twist acquired by the twist amount acquisition unit 38, for example. The amount of twist may be displayed as a number or a string, or as a color. In the former case, for example, a string such as "Twist amount: 14 degrees" may be displayed. In the latter case, the image generation unit 35 may generate a display image that displays the amount of twist by color. For example, the linear body 10a portion of the 3D model 10 of the virtual robot may be displayed in a color corresponding to the amount of twist, or the entire 3D model 10 of the virtual robot may be displayed in a color corresponding to the amount of twist. For example, the color corresponding to the amount of twist may be blue if the absolute value of the amount of twist is 0 degrees or more and less than 10 degrees, green if it is 10 degrees or more and less than 15 degrees, yellow if it is 15 degrees or more and less than 20 degrees, and red if it is 20 degrees or more. The image generation unit 35 may also generate a display image to show alerts, for example, which will be described later. The alert may be displayed as text such as "WARNING" or as a graphic, or as a highlighting of part or all of a display image (for example, by making it flash or changing it to a warning color such as red).

[0031] The output unit 36 ​​outputs the display image generated by the image generation unit 35 to the display device 4. The output unit 36 ​​may also output only the display image. In this case, the display image will be superimposed on the image of the real environment or the real environment itself on the display device 4. On the other hand, if the display device 4 has a non-transparent display and the image of the real environment captured by the display device 4 is received by the teaching data storage device 3, the result of combining the image of the real environment and the display image may be output to the display device 4. This combination may be performed, for example, by a combination unit (not shown) in the teaching data storage device 3.

[0032] Furthermore, the output unit 36 ​​may output information regarding the torsion amount acquired by the torsion amount acquisition unit 38. This output may be, for example, the torsion amount itself, or it may be an alert indicating that the absolute value of the torsion amount has exceeded a threshold. The output of the torsion amount itself may be, for example, a display image for displaying the torsion amount, or an output indicating the torsion amount by sound or the like. The alert output may be, for example, a display image for displaying the alert, or an output indicating the alert by sound or the illumination of a warning light. Note that, for example, if the absolute value of the torsion amount does not exceed a threshold, an alert output may not be performed. The display image output by the output unit 36 ​​may be, for example, generated by the image generation unit 35.

[0033] When the receiving unit 34 receives a teaching instruction for the 3D model 10 of the virtual robot, the storage unit 37 stores teaching data in the teaching data storage unit 32 according to the 3D model 10 of the virtual robot. That is, when the teaching instruction is received, the storage unit 37 stores information indicating the position and orientation of the 3D model 10 as teaching data in the teaching data storage unit 32. Typically, the teaching data stored in response to the receipt of a single teaching instruction corresponds to one teaching point. As this storage of teaching data is repeated, for example, teaching data corresponding to the path of the robot's end effector will be stored in the teaching data storage unit 32. The teaching data to be stored may, as described above, be information indicating the position and orientation of the end effector of the 3D model 10 for each teaching point, or it may be information indicating the angle of each joint of the 3D model 10 for each teaching point. The teaching data may also include information indicating the teaching position, for example, in order to enable the display of the teaching position. The storage unit 37 may obtain information necessary for storing teaching data, such as the position and orientation of the end effector in the robot coordinate system, which is the local coordinate system of the 3D model 10 of the virtual robot, or the angles of each joint, from the image generation unit 35, from the 3D model 10 stored in the model storage unit 31, or from other configurations. The position and orientation of the end effector in the robot coordinate system of the 3D model 10 of the virtual robot can be obtained, for example, by transforming the position and orientation of the end effector in the virtual space using a homogeneous transformation matrix that shows the transformation between the virtual space and the robot coordinate system of the 3D model 10 of the virtual robot.

[0034] The torsion amount acquisition unit 38 acquires the torsion amount of the linear body 10a attached to the 3D model of the robot body of the virtual robot. The torsion amount to be acquired is the torsion amount of the linear body 10a as a whole. The torsion amount may be acquired, for example, by performing a simulation on the linear body 10a. This simulation may be, for example, the simulation described in the above-mentioned Patent Document 1, a simulation using a commercially available simulation tool for linear bodies such as cables and hoses, or other physical simulations. This simulation may be performed, for example, using a 3D model of the linear body 10a. The 3D model of the linear body 10a may be, for example, a part of the 3D model 10 of the virtual robot, or a separate 3D model. When the torsion amount is acquired by performing a simulation on the linear body 10a, parameters of the linear body 10a necessary for the simulation, such as length, thickness, and parameters indicating elastic properties such as Young's modulus, may be set in advance, and the simulation may be performed using these parameters. The torsion amount acquisition unit 38 can determine the position and orientation of the fixed positions at both ends of the linear body 10a in the current 3D model 10. The torsion amount acquisition unit 38 may then acquire the torsion amount of the linear body 10a by performing a simulation of the linear body 10a using the determined position and orientation. For example, as shown in Figure 1, if the linear body 10a passes inside the housing of the 3D model 10 of the virtual robot, and the position of a part of the linear body 10a is determined even in addition to the fixed position, the simulation may be performed using this as a constraint condition. As an example, the torsion amount acquisition unit 38 may acquire the torsion amount at predetermined intervals along the longitudinal direction of the linear body 10a and acquire the total torsion amount of the linear body 10a by accumulating the torsion amounts at predetermined intervals.The torsion amount acquisition unit 38 may acquire the torsion amount in real time, for example, to determine the torsion amount corresponding to the current shape of the 3D model 10, or it may acquire the torsion amount at predetermined time intervals or each time a predetermined event occurs (for example, receiving a teaching instruction). In this embodiment, the case in which the torsion amount is acquired in real time will be mainly described. Even when the torsion amount is acquired in real time, the torsion amount does not change when no operation is being performed on the 3D model 10. Therefore, the torsion amount acquisition unit 38 may acquire a new torsion amount each time an operation on the 3D model 10 is received. The acquired torsion amount may also be stored in the teaching data storage unit 32 in association with the teaching data when teaching data is accumulated. The torsion amount stored in association with the teaching data is the torsion amount corresponding to the shape of the 3D model 10 corresponding to that teaching data. Furthermore, the sign of the torsion amount may be defined such that, for example, clockwise torsion is positive and counterclockwise torsion is negative in the longitudinal direction of the linear body 10a, from the base end 10b to the hand end 10c, or vice versa.

[0035] Furthermore, the twist amount acquisition unit 38 may acquire the twist amount of the linear body 10a corresponding to the accepted change in handling when the receiving unit 34 accepts the change in handling of the linear body 10a. For example, if the initial twist amount is changed, the twist amount acquisition unit 38 may acquire the twist amount of the linear body 10a corresponding to the changed initial twist amount. More specifically, if the twist amount of the linear body 10a acquired for an initial twist amount of 0 degrees was 15 degrees, and the initial twist amount is subsequently changed to -10 degrees, the twist amount acquisition unit 38 may acquire a twist amount of 5 degrees (=15 degrees - 10 degrees) of the linear body 10a corresponding to the changed initial twist amount. Note that the twist amount after the change in handling may be acquired, for example, by performing another simulation for the linear body 10a after the change in handling. If the change in handling is, for example, a change in the length of the linear body 10a or a change in the fixing position of the linear body 10a, it is preferable to perform the simulation again to obtain the amount of twist corresponding to the changed handling.

[0036] Next, the operation of the teaching data storage device 3 will be explained using the flowchart in Figure 2. (Step S101) The position relationship acquisition unit 33 acquires the relative position relationship between the actual environment and the display device 4.

[0037] (Step S102) The twist amount acquisition unit 38 acquires the twist amount of the linear body 10a according to the shape of the 3D model 10 of the virtual robot at this point. If a change in the handling of the linear body 10a is accepted, the twist amount of the linear body 10a corresponding to the changed handling will be acquired.

[0038] (Step S103) The image generation unit 35 uses the 3D model 10 of the virtual robot, the relative positional relationship acquired in step S101, and the acquired twist amount to generate a display image for displaying the 3D model 10 of the virtual robot, which displays according to the twist amount of the linear body 10a. Note that the 3D model 10 to be displayed is in its initial shape before the operation is accepted, and becomes the shape after the operation is accepted.

[0039] (Step S104) The output unit 36 ​​outputs the generated display image to the display device 4.

[0040] (Step S105) The reception unit 34 determines whether it has received an operation request for the 3D model 10 of the virtual robot. If it has received an operation request for the 3D model 10 of the virtual robot, it proceeds to step S106; otherwise, it proceeds to step S110.

[0041] (Step S106) The position relationship acquisition unit 33 acquires the relative position relationship between the actual environment and the display device 4.

[0042] (Step S107) The twist amount acquisition unit 38 acquires the twist amount of the linear body 10a according to the shape of the 3D model 10 of the virtual robot at this point.

[0043] (Step S108) The image generation unit 35 uses the 3D model 10 of the virtual robot, the relative positional relationship acquired in step S106, and the amount of twist acquired in step S107 to generate a display image for displaying the 3D model 10 of the virtual robot after the operation, which displays according to the amount of twist of the linear body 10a.

[0044] (Step S109) The output unit 36 ​​outputs the generated display image to the display device 4. Then the process returns to step S105.

[0045] (Step S110) The reception unit 34 determines whether it has received the instruction to teach. If it has received the instruction to teach, it proceeds to step S111; otherwise, it proceeds to step S112.

[0046] (Step S111) The storage unit 37 stores teaching data corresponding to the position and orientation of the virtual robot's 3D model 10 at that time in the teaching data storage unit 32. Then, the process returns to step S105.

[0047] (Step S112) The image generation unit 35 determines whether to generate a display image. If it decides to generate a display image, it returns to step S101; otherwise, it proceeds to step S113. The image generation unit 35 may, for example, periodically decide whether to generate a display image. This decision ensures that, for example, even if no operation is performed, if the position or orientation of the display device 4 is changed, the display image corresponding to the changed position or orientation will be displayed on the display device 4.

[0048] (Step S113) The reception unit 34 determines whether it has received a request to change the routing of the linear body 10a. If it has received a request to change the routing, it returns to step S101; otherwise, it proceeds to step S114.

[0049] (Step S114) The image generation unit 35 determines whether the teaching regarding the teaching data has been completed. If the teaching has been completed, the series of processes for generating the teaching data is completed; otherwise, the process returns to step S105. The image generation unit 35 may also determine that the teaching regarding the teaching data has been completed, for example, when the reception unit 34 receives information that the accumulation of teaching data has been completed.

[0050] In addition, in the flowchart of Figure 2, teaching data may be output when it is determined that teaching has finished. Furthermore, the order of processing in the flowchart of Figure 2 is just one example, and the order of each step may be changed if the same result can be obtained.

[0051] Next, the operation of the teaching data storage device 3 according to this embodiment will be explained using a specific example. In this example, the case in which an operator wearing a head-mounted display device 4 performs teaching along a weld line will be described. Furthermore, the initial twist amount of the linear body 10a is set to "0 degrees" at the start of the teaching operation.

[0052] First, when the operator starts the teaching data storage device 3, the position relationship acquisition unit 33 uses the captured image received from the display device 4 to acquire the relative position relationship between the robot 1 placed in the real environment and the display device 4, and passes it to the image generation unit 35 (step S101). The twist amount acquisition unit 38 acquires the twist amount of the linear body 10a attached to the 3D model of the robot body of the virtual robot at that time (step S102). Then, the image generation unit 35 uses the relative position relationship, the 3D model 10 stored in the model storage unit 31, and the acquired twist amount to generate a display image for displaying the 3D model 10 of the virtual robot at the position of the reference marker 6, and passes it to the output unit 36 ​​(step S103). Upon receiving the display image, the output unit 36 ​​outputs the display image to the display device 4 (step S104). As a result, the operator can see the 3D model 10 displayed at the position of the reference marker 6. The displayed image may be updated periodically (steps S112, S101-S104).

[0053] Here, the reference marker 6 is a predetermined two-dimensional image. The reference marker 6 may be, for example, an AR marker, a QR code (registered trademark), or any other two-dimensional image with a predetermined shape. The size of the reference marker 6 may be predetermined, for example. The reference marker 6 may be printed on, for example, paper or a resin sheet. When the three-dimensional model 10 of the virtual robot is to be displayed at the position of the reference marker 6, the operator can position the reference marker 6 to display the three-dimensional model 10 at the position of the reference marker 6.

[0054] The 3D model 10 may be displayed at a location other than the reference marker 6. In this case, however, it is preferable that the 3D model 10 be displayed in a predetermined positional relationship with the reference marker 6. For example, the 3D model 10 may be displayed next to the reference marker 6.

[0055] Figure 3 shows an example of the display of the 3D model 10 of the virtual robot. As shown in Figure 3, the operator can see the 3D model 10 of the virtual robot displayed at the position of the reference marker 6, and the display 51 showing the amount of twist. In this way, the operator can know the amount of twist corresponding to the shape of the 3D model at that time.

[0056] Subsequently, the operator manipulates the 3D model 10 using an input device such as a teaching pendant or gestures to position the tip of the welding torch on the 3D model 10 at the location of the weld line. As a result, a display image showing the 3D model 10 after the manipulation and the amount of twist is output to the display device 4, allowing the operator to confirm the position and orientation of the welding torch on the 3D model 10 after the manipulation, and the corresponding amount of twist of the linear body 10a (steps S105-S109). Subsequently, when the operator inputs teaching instructions using an input device such as a teaching pendant or gestures, teaching data corresponding to the teaching point is stored in the teaching data storage unit 32 according to the teaching instructions (steps S110, S111). The amount of twist at that time may also be stored in association with the storage of teaching data. Subsequently, teaching data is stored in the same manner for other teaching points on the weld line (steps S105-S111).

[0057] Furthermore, the operator can determine how much the linear body 10a is twisted by the displayed twist amount. For example, if the absolute value of the twist amount exceeds the allowable range, the operator can change the orientation of the welding torch in the 3D model 10 to teach a position in which the absolute value of the twist amount is smaller. Also, if changing the orientation of the welding torch alone is not enough to reduce the absolute value of the twist amount, the operator may change the handling of the linear body 10a. For example, if the operator wants the absolute value of the twist amount to be 20 degrees or less, and the twist amount in the orientation of the 3D model 10 corresponding to the new teaching point is 21 degrees, the operator may input an initial twist amount of "-10 degrees" via an input device in the real environment or a virtual input interface. The input change in handling is received by the reception unit 34 and passed to the twist amount acquisition unit 38 (step S113). Furthermore, when the torsion amount acquisition unit 38 acquires a new torsion amount, it may acquire a torsion amount of "11 degrees" (=21 degrees - 10 degrees) in accordance with the initial torsion amount of "-10 degrees" which is the modified handling, and display an image including that torsion amount (steps S101 to S104).

[0058] In this way, the operator can perform teaching while appropriately changing the handling of the linear body 10a during the teaching process. Therefore, it is possible to avoid a situation where all teaching data is wasted because it is found in the simulation after teaching that the absolute value of the torsion amount exceeds the allowable range. Furthermore, if the absolute value of the torsion amount cannot be kept within the allowable range even by changing the orientation of the welding torch or changing the handling of the linear body 10a, the teaching process may be stopped, for example, and new teaching may be performed by changing the positional relationship between the 3D model 10 of the virtual robot and the workpiece to be welded. Even in this case, for example, the amount of wasted teaching data can be reduced. After teaching is completed (step S114), the teaching data stored in the teaching data storage unit 32 may be output. Also, if the handling of the linear body 10a is changed during teaching, the teaching data may be output together with information indicating the changed handling (for example, the initial torsion amount).

[0059] Furthermore, when the reception unit 34 receives a change in the handling of the linear body 10a, the torsion amount acquisition unit 38 may also acquire a representative value of the torsion amount corresponding to the teaching data stored in the teaching data storage unit 32, corresponding to the changed handling. That is, when the handling of the linear body 10a is changed, the torsion amount acquisition unit 38 may acquire the torsion amount of the linear body 10a according to the position and orientation of the 3D model 10 of the virtual robot corresponding to each teaching data accumulated up to that point, and acquire a representative value of one or more of the acquired torsion amounts. The representative value may be, for example, the maximum value and the minimum value, or it may be the maximum value of the absolute value of the torsion amount. In this case, the output unit 36 ​​may also output the representative value of the torsion amount acquired by the torsion amount acquisition unit 38. The output regarding the representative value may be, for example, the output of the representative value itself, or it may be the output of an alert indicating that the representative value has exceeded a threshold. Furthermore, if display images for showing representative values ​​or display images for showing alerts are output, these display images may be generated, for example, by the image generation unit 35. Also, for example, if the amount of twist is stored in association with teaching data, when a new amount of twist is obtained for each of the one or more stored teaching data according to the modified handling of the linear body 10a, the new amount of twist may also be stored in association with the teaching data. In this case, for example, the amount of twist may be overwritten or not. Even in the latter case, it is preferable that the latest amount of twist be stored in a way that allows for distinction. Furthermore, the new amount of twist may be stored in association with information indicating the modified handling, for example.

[0060] For example, in the above specific example, suppose that when the current twist amount becomes "21 degrees", the teaching data storage unit 32 stores information associating the teaching data with the twist amount, as shown in Figure 4. Then, when an initial twist amount of "-10 degrees" is received, the twist amount acquisition unit 38 may subtract 10 degrees from each twist amount associated with the teaching data in Figure 4, overwrite and store them, and acquire the maximum value "9 degrees" and minimum value "-3 degrees" of the subtraction result. Then, a display image including the maximum value "9 degrees" and minimum value "-3 degrees" may be generated and output. In this way, the operator can check not only the current twist amount corresponding to the change in the handling of the linear body 10a, but also the twist amount corresponding to the previous teaching data. Therefore, for example, even if the current amount of twist comes within an acceptable range due to a change in the handling of the linear body 10a, if the amount of twist according to the previous teaching data does not come within an acceptable range, the handling of the linear body 10a can be changed again to adjust both the current amount of twist and the past amount of twist so that they all come within the desired range. For example, information indicating the received changes in handling may also be associated with the teaching data and stored in the teaching data storage unit 32.

[0061] As described above, the teaching data storage device 3 according to this embodiment allows us to know the amount of twist of the linear body 10a when teaching using the 3D model 10 of the virtual robot. Therefore, if the absolute value of the amount of twist is large, for example, the teaching can be performed in a way that reduces the absolute value of the amount of twist by changing the posture of the end-user of the 3D model 10. Furthermore, if the absolute value of the amount of twist cannot be reduced by changing the posture alone, the absolute value of the amount of twist can also be reduced by changing the handling of the linear body 10a. In this case, for example, a representative value of the amount of twist corresponding to the teaching data for the taught teaching point is also acquired and output, so that the amount of twist for each taught teaching point can also be made to fall within the desired range. In addition, if the amount of twist is displayed in color, the operator can intuitively understand the amount of twist without having to check the numerical value indicating the amount of twist.

[0062] In this embodiment, the case in which the operator changes the handling of the linear body 10a through trial and error has been mainly described, but this is not required. For example, a change in the handling of the linear body 10a may be proposed so that the representative value of the twist amount corresponding to the teaching data approaches zero, and the handling may be changed accordingly. In this case, the teaching data storage device 3 may further include a specification unit 39, as shown in Figure 5, for example. This specification unit 39 may specify a change in the handling of the linear body 10a so that the representative value of the twist amount corresponding to the teaching data stored in the teaching data storage unit 32 approaches zero. A change in the handling of the linear body 10a so that the representative value of the twist amount corresponding to the teaching data approaches zero may be a change in handling such that the representative value of the twist amount corresponding to the teaching data after the change is closer to zero than the representative value of the twist amount corresponding to the teaching data before the change. The specification unit 39 may perform the specification, for example, in response to a specific instruction being received by the reception unit 34. If the identification unit 39 identifies a change in the handling of the linear body 10a, the output unit 36 ​​may also output information regarding the change in the handling of the linear body 10a identified by the identification unit 39. This output may be, for example, information indicating the identified change in handling. This output may be, for example, a display image indicating the identified change in handling, or an output of sound indicating the identified change in handling. The display image may be generated, for example, by the image generation unit 35.

[0063] As an example, the identification unit 39 may determine a change in the handling of the linear body 10a by optimizing an objective function corresponding to the representative value of the twist amount after the change for each teaching data. That is, the identification unit 39 may identify the change in handling, which is the optimal solution for optimizing the objective function, as a change in the handling of the linear body 10a so that the representative value of the twist amount corresponding to the teaching data approaches zero. The optimization of the objective function may be, for example, minimization of the objective function. If the representative value of the twist amount is the maximum value of the absolute value of the twist amount, the objective function may be, for example, an increasing function of the maximum value of the absolute value of the twist amount. For example, if the twist amount for each teaching data is obtained as shown in Figure 4, the identification unit 39 may identify an initial twist amount of "-13 degrees" so as to minimize the maximum value of the absolute value of the twist amount. Depending on the output of the identification result, the operator may input the initial twist amount of "-13 degrees" to the teaching data storage device 3 as the changed handling. In this way, workers can make appropriate changes to the handling of the linear body 10a without having to go through trial and error.

[0064] Furthermore, in this embodiment, the image generation unit 35 may generate a display image that shows, for example, the linear body 10a located inside the housing of the 3D model 10 of the virtual robot as if it were transparent. In this way, the operator can confirm the shape of the linear body 10a, including the inside of the housing, and can, for example, provide teaching instructions to prevent excessive force from being applied to the linear body 10a. Note that the linear body 10a located inside the housing may be, for example, a linear body 10a that is not visible from the outside of the housing. Also, the linear body 10a located inside the housing may be, for example, a part of the linear body 10a.

[0065] Furthermore, although this embodiment mainly describes the case where the torsion amount is acquired in real time by the torsion amount acquisition unit 38, this is not required. For example, the torsion amount may be acquired each time teaching data is accumulated. In this case, if the operator determines that the torsion amount corresponding to the teaching exceeds the acceptable range after inputting a teaching instruction, they may cancel the latest teaching and perform teaching again.

[0066] Furthermore, although this embodiment mainly describes the case where teaching is performed for welding, it goes without saying that teaching may be performed for other purposes. When teaching is performed for purposes other than welding, the welding line described above may be, for example, a work line or the movement path of the robot's end effector.

[0067] Furthermore, although this embodiment mainly describes the case where the reference marker 6 is a predetermined two-dimensional image, this is not required. Similar to markerless AR, an object existing in the real environment may be used as the reference marker. The object existing in the real environment is not particularly limited, but for example, it may be the base used to install the actual robot or a jig located near the position where the robot is placed. Alternatively, a marker on a three-dimensional object (for example, a rectangular prism shape) with markers attached to each surface may be used as the reference marker. In addition, the reference marker may be three-dimensional, such as a base or a jig.

[0068] Furthermore, although this embodiment mainly describes the case where the handling of the linear body 10a can be changed, this is not required. The handling of the linear body 10a does not need to be changed. Even in this case, teaching that reduces the amount of twisting can be performed by appropriately changing the orientation of the end-effector of the 3D model 10.

[0069] Furthermore, in the above embodiment, each process or function may be implemented by centralized processing by a single device or a single system, or by distributed processing by multiple devices or multiple systems. For example, at least a part of the configuration of the teaching data storage device 3 may be physically included in a device having a display, etc. Therefore, the device divisions shown in Figure 1 may be considered to be for convenience based on function rather than corresponding to physical devices.

[0070] Furthermore, in the above embodiment, if two or more components included in the teaching data storage device 3 have communication devices, input devices, etc., the two or more components may have a single physical device or they may have separate devices.

[0071] Furthermore, in the above embodiment, each component may be configured with dedicated hardware, or, if it is a component that can be implemented by software, it may be implemented by executing a program. For example, each component can be implemented by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. During execution, the program execution unit may execute the program while accessing the storage unit or recording medium. The program may also be executed by being downloaded from a server or the like, or by being executed by reading a program recorded on a predetermined recording medium (e.g., an optical disk, magnetic disk, semiconductor memory, etc.). Furthermore, this program may be used as a program that constitutes a program product. Furthermore, the computer executing the program may be one or multiple computers. That is, centralized processing may be performed, or distributed processing may be performed.

[0072] Furthermore, the embodiments described above are illustrative examples for specifically carrying out the present invention and do not limit the technical scope of the present invention. The technical scope of the present invention is indicated by the claims rather than by the description of the embodiments, and modifications within the literal scope and equivalent meaning of the claims are intended. [Explanation of Symbols]

[0073] 3 Teaching data storage device, 31 Model storage unit, 32 Teaching data storage unit, 33 Position relationship acquisition unit, 34 Reception unit, 35 Image generation unit, 36 Output unit, 37 Storage unit, 38 Torsion amount acquisition unit, 39 Identification unit

Claims

1. A model memory unit that stores a 3D model of a virtual robot corresponding to a real robot, A teaching data storage unit in which the teaching data of the actual robot is stored, A position relationship acquisition unit that acquires the relative positional relationship between the real environment and a display device that overlays an image onto the image of the real environment or the real environment itself, A reception unit that receives instructions for operation and teaching of the three-dimensional model of the virtual robot, An image generation unit generates a display image for displaying the 3D model of the operated virtual robot at a predetermined location in the real environment, based on the 3D model of the virtual robot and the relative positional relationship. An output unit that outputs the display image generated by the image generation unit to the display device, When the receiving unit receives a teaching instruction for the 3D model of the virtual robot, the storage unit stores teaching data in the teaching data storage unit according to the 3D model of the virtual robot. It includes a torsion amount acquisition unit that acquires the amount of torsion of a linear body attached to a 3D model of the robot body of a virtual robot, The reception unit also accepts changes to the routing of the linear body. When the reception unit receives a change in the handling of the linear body, the torsion amount acquisition unit acquires the torsion amount of the linear body corresponding to the changed handling, and also acquires a representative value of the torsion amount corresponding to the teaching data stored in the teaching data storage unit, corresponding to the changed handling. The representative values ​​of the torsion amount are the maximum and minimum values ​​of the torsion amount, or the maximum value of the absolute value of the torsion amount. The output unit is a teaching data storage device that also outputs the amount of torsion and a representative value of the amount of torsion obtained by the torsion amount acquisition unit.

2. A model storage unit that stores a 3D model of a virtual robot corresponding to an actual robot, A teaching data storage unit in which the teaching data of the actual robot is stored, A position relationship acquisition unit that acquires the relative positional relationship between the real environment and a display device that overlays an image onto the image of the real environment or the real environment itself, A reception unit that receives instructions for operation and teaching of the three-dimensional model of the virtual robot, An image generation unit generates a display image for displaying the 3D model of the operated virtual robot at a predetermined location in the real environment, based on the 3D model of the virtual robot and the relative positional relationship. An output unit that outputs the display image generated by the image generation unit to the display device, When the receiving unit receives a teaching instruction for the 3D model of the virtual robot, the storage unit stores teaching data in the teaching data storage unit according to the 3D model of the virtual robot. It includes a torsion amount acquisition unit that acquires the amount of torsion of a linear body attached to a 3D model of the robot body of a virtual robot, The reception unit also accepts changes to the routing of the linear body. The torsion amount acquisition unit acquires the torsion amount of the linear body corresponding to the modified handling received by the reception unit, The system further includes a specification unit that identifies a change in the handling of the linear body so that the representative value of the amount of twist corresponding to the teaching data stored in the teaching data storage unit approaches zero. The representative values ​​of the torsion amount are the maximum and minimum values ​​of the torsion amount, or the maximum value of the absolute value of the torsion amount. The output unit is a teaching data storage device that also outputs a torsion amount acquired by the torsion amount acquisition unit, and an output related to a change in the handling of the linear body identified by the identification unit.

3. The teaching data storage device according to claim 1 or claim 2, wherein the change in the handling of the linear body is at least one of changing the initial amount of twist of the linear body, changing the length of the linear body, or changing the fixing position of the linear body.

4. The teaching data storage device according to claim 1 or 2, wherein the image generation unit generates a display image that displays the amount of twist acquired by the amount of twist acquisition unit using color.

Images