Virtual robot display device and virtual robot display method

Abstract

The objective is to provide a virtual robot display device and a virtual robot display method that appropriately display virtual robots using AR devices and MR devices so as not to deviate from the operating speed of actual robots. [Solution] The virtual robot display device 100 includes a storage unit 120 that stores axis angle information indicating the axis angles corresponding to each time interval of a robot operating according to a work program, a control unit 130 that generates a virtual robot which is an image showing the posture of the robot corresponding to one of the time intervals based on the axis angle information, and a display unit 140 that sequentially displays the virtual robots generated by the control unit 130. The control unit 130 includes a time calculation unit 135 that calculates the time in the axis angle information corresponding to the current time, and an axis angle selection unit 136 that selects the axis angles corresponding to the time in the axis angle information calculated by the time calculation unit 135.

Description

Technical Field

[0001] The present invention relates to a virtual robot display device and a virtual robot display method.

Background Art

[0002] Conventionally, technologies that connect the real space and the virtual space, such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality), have been utilized in many fields.

[0003] In the field of industrial robots, an AR device or an MR device is used to operate or teach a robot. Furthermore, by operating a robot image (virtual robot) displayed using an AR device or an MR device according to a work program, an operator can visualize the operation before actually introducing the robot (for example, Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when a virtual robot that operates according to a work program is displayed using an AR device and an MR device, the operation speed of the virtual robot may be different from the operation speed of an actual robot that operates according to the work program. That is, the operation of the virtual robot displayed using an AR device and an MR device is slower than the operation of the actual robot, and real-time performance is not guaranteed.

[0006] Therefore, the present invention aims to provide a virtual robot display device and a virtual robot display method that appropriately display virtual robots using AR devices and MR devices so as not to deviate from the operating speed of actual robots. [Means for solving the problem]

[0007] A virtual robot display device according to one aspect of the present invention is a virtual robot display device that visualizes a robot virtually in real space, comprising: a storage unit that stores axis angle information indicating each axis angle corresponding to each time interval of a robot operating according to a work program; a control unit that generates a virtual robot, which is an image showing the posture of the robot corresponding to one of the time intervals based on the axis angle information; and a display unit that sequentially displays the virtual robots generated by the control unit, wherein the control unit includes a time calculation unit that calculates the time in the axis angle information corresponding to the current time, and an axis angle selection unit that selects each axis angle corresponding to the time in the axis angle information calculated by the time calculation unit.

[0008] According to this embodiment, the control unit generates a virtual robot corresponding to one of the times at predetermined intervals based on the axis angle information, and the display unit sequentially displays the virtual robots generated by the control unit. The virtual robot is generated when the time calculation unit calculates the time in the axis angle information corresponding to the current time, and the axis angle corresponding to the time in the calculated axis angle information is selected by the axis angle selection unit. As a result, the virtual robot displayed using the AR device and MR device can be displayed appropriately so as not to deviate from the operating speed of the actual robot.

[0009] In the above embodiment, the storage unit stores a work program, and the control unit may further include an axis angle information generation unit that generates axis angle information based on the work program and stores it in the storage unit.

[0010] According to this embodiment, each axis angle information generation unit generates and stores axis angle information based on the work program stored in the storage unit, thereby enabling the appropriate generation of axis angle information according to the work program.

[0011] In the above embodiment, each axis angle information generation unit may include a trajectory generation unit that generates a robot trajectory based on the position of the robot operating according to a work program, a posture generation unit that generates the posture of the robot based on the robot trajectory generated by the trajectory generation unit, and an inverse kinematics calculation unit that calculates each axis angle of the robot corresponding to each time interval at a predetermined interval based on the posture of the robot generated by the posture generation unit.

[0012] In this embodiment, the trajectory generation unit generates a robot trajectory based on the robot's position as it operates according to a work program, the attitude generation unit generates the robot's attitude based on the robot's trajectory, and the inverse kinematics calculation unit calculates the axis angles of the robot corresponding to each time interval based on the robot's attitude. This makes it possible to appropriately generate axis angle information that shows the axis angles of the robot as it operates according to a work program as it operates according to a predetermined time interval.

[0013] In the above embodiment, the system further includes a reception unit for receiving start operations from the user, and if the axis angle information generation unit receives a start operation from the user by the reception unit and the axis angle information is not stored in the storage unit, it may generate the axis angle information based on a work program and store it in the storage unit.

[0014] According to this embodiment, when the reception unit receives a start operation from the user, if the axis angle information for each axis is not stored in the storage unit, it generates the axis angle information for each axis and stores it in the storage unit, so it does not need to generate the axis angle information each time.

[0015] In the above embodiment, the system may further include a reception unit that receives a setting operation for a predetermined interval from the user, and when the reception unit receives a setting operation for a predetermined interval from the user, each axis angle information generation unit may generate axis angle information indicating each axis angle corresponding to each time in the received predetermined interval based on a work program and store it in a storage unit.

[0016] According to this embodiment, each axis angle information generation unit generates axis angle information indicating the axis angle corresponding to each time interval received from the user. Therefore, the user can set the settings considering the memory capacity and the accuracy of the displayed virtual robot.

[0017] In the above embodiment, the control unit may further include a speed determination unit that calculates the rotational speed of each axis based on the axis angle corresponding to each time in the axis angle information for each axis, and determines whether the calculated rotational speed of each axis exceeds the upper limit speed for each axis, and a speed conversion unit that, if the speed determination unit determines that there is an over-speed axis among the axes that exceeds the upper limit speed, converts the speed of the over-speed axis to the upper limit speed or less.

[0018] In this embodiment, the speed determination unit calculates the rotational speed of each axis and determines whether the calculated rotational speed of each axis exceeds the upper limit speed for each axis. If the speed conversion unit determines that there is an axis exceeding the upper limit speed, it converts the speed of the exceeding axis to below the upper limit speed. This makes it possible to display the virtual robot more appropriately so as not to deviate from the actual operating speed of the welding robot.

[0019] In the above embodiment, the speed conversion unit may convert the speeds of each shaft other than the speed-over-speed shaft in accordance with converting the speed of the speed-over-speed shaft to a speed below the upper limit speed.

[0020] According to this embodiment, the speed conversion unit converts the speed of each axis other than the speed-over-speed axis in accordance with converting the speed of the speed-over-speed axis to a speed below the upper limit, thereby enabling a more accurate display of the virtual robot in accordance with the actual operation of the welding robot.

[0021] A virtual robot display method according to an aspect of the present invention is a virtual robot display method executed by a virtual robot display device that virtually visualizes a robot in a real space. Based on axis angle information indicating axis angles corresponding to each time at a predetermined interval of a robot that operates according to a work program and is stored in a storage unit, a control step of generating a virtual robot that is an image showing the posture of the robot corresponding to any one of the times at a predetermined interval, and a display step of sequentially displaying the virtual robot generated in the control step, the control step includes a time calculation step of calculating the time in the axis angle information corresponding to the current time, and an axis angle selection step of selecting axis angles corresponding to the time in the axis angle information calculated in the time calculation step.

[0022] According to this aspect, in the control step, based on the axis angle information, a virtual robot corresponding to any one of the times at a predetermined interval is generated, and in the display step, the virtual robot generated by the control unit is sequentially displayed. The virtual robot is generated by calculating the time in the axis angle information corresponding to the current time in the time calculation unit step, and selecting the axis angles corresponding to the time in the calculated axis angle information in the axis angle selection step. As a result, the virtual robot displayed using the AR device and the MR device can be appropriately displayed so as not to deviate from the operating speed of the actual robot.

Advantages of the Invention

[0023] According to the present invention, it is possible to provide a virtual robot display device and a virtual robot display method for appropriately displaying a virtual robot displayed using an AR device and an MR device so as not to deviate from the operating speed of an actual robot.

Brief Description of the Drawings

[0024] [Figure 1] It is a schematic diagram showing an MR device 10 according to the first embodiment of the present invention. [Figure 2]It is a functional block diagram showing each function of the virtual robot display device 100 according to the first embodiment of the present invention. [Figure 3] It is an example of axis angle information showing each axis angle of a welding robot operating according to a work program every T seconds, generated by the axis angle information generation unit 131. [Figure 4] It is an example of a virtual robot displayed by the display unit 140. [Figure 5] It is a flowchart showing the processing flow of the virtual robot display method M100 executed by the virtual robot display device 100 according to the first embodiment of the present invention. [Figure 6] It is a functional block diagram showing each function of the virtual robot display device 200 according to the second embodiment of the present invention. [Figure 7] It is a flowchart showing the processing flow of the virtual robot display method M200 executed by the virtual robot display device 200 according to the second embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0025] Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. Note that the embodiments described below are merely specific examples for carrying out the present invention and do not limit the interpretation of the present invention. Also, for ease of understanding of the description, the same reference numerals are attached to the same components in each drawing as much as possible, and duplicate descriptions may be omitted.

[0026] <First Embodiment> [System Overview] FIG. 1 is a schematic diagram showing the MR device 10 according to the first embodiment of the present invention. As shown in FIG. 1, the MR device 10 (virtual robot display device) includes a processor 11, a bus 12, a work memory 13, a storage memory 14, a driver 15, a reception unit 16, a display unit 17, and a photographing unit 18.

[0027] In the MR device 10, the processor 11, work memory 13, storage memory 14, and driver 15 are connected via a bus 12 so that they can exchange data with each other.

[0028] The work memory 13 is a volatile memory device such as DRAM (Dynamic Random Access Memory).

[0029] The storage memory 14 is a read-write, non-volatile storage device such as flash memory, where the program (code) is stored. The program can be installed externally. The program is distributed in a state where it is stored on a storage medium readable by the MR device 10, such as the storage memory 14, HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or read-only CD-ROM (Compact Disc Read Only Memory) or DVD-ROM (Digital Versatile Disc-Read Only Memory). The program may also be distributed over the internet connected via a communication interface.

[0030] When a program is executed, the processor 11 transfers the program stored in the storage memory 14 and the data necessary for its execution to the work memory 13. The processor 11 reads the processing instructions and data necessary for the program execution from the work memory 13 and executes the arithmetic processing according to the contents of the processing instructions. At this time, the processor 11 may generate new data necessary for the program execution and store it in the work memory 13. Note that the processor 11 is not limited to a configuration in which it obtains the program and data from the storage memory 14, but may also be configured to obtain them from a server or the like via the internet.

[0031] The driver 15 outputs various commands and data to the reception unit 16, display unit 17, and imaging unit 18 according to instructions from the processor 11. The driver 15 also receives various data output from the reception unit 16, display unit 17, and imaging unit 18, and outputs the received data to the processor 11.

[0032] The display unit 17 is, for example, an optically semi-transparent head-mounted display, which is connected to the harness. The user wears the harness on their head so that the display is positioned in front of their line of sight. The user can view the images displayed on the display unit 17 while looking at the real world through the display unit 17.

[0033] In this embodiment, for example, while the user views the actual space including the workpiece W (object to be welded), the display unit 17 displays at least a portion of a virtual robot R, which is an image showing the posture of the welding robot.

[0034] The shooting unit 18 is, for example, a digital camera including a lens and an image sensor, which converts the light from the subject received by the lens into an electrical signal (image data). The digital camera is connected to the harness so that objects in the direction of the user's line of sight are photographed. The digital camera is, for example, a three-dimensional camera. However, the digital camera may also be a two-dimensional camera.

[0035] The reception unit 16 is, for example, a pointing device or a joystick. The reception unit 16 detects user operations and outputs data indicating the detection result to the driver 15.

[0036] Furthermore, the reception unit 16 may, for example, when the user's hand is located within the field of view of the shooting unit 18, track the movement of the user's hand in the image captured by the shooting unit 18 to detect the shape and position of the user's hand as an operation by the user, and output data indicating the detection result to the driver 15.

[0037] When the processor 11 receives data from the reception unit 16 via the driver 15, it generates an image showing the posture of the welding robot to be displayed on the display unit 17 based on the received data. The processor 11 then controls the display unit 17 to sequentially display the virtual robot R as if it were operating. Details of the process by which the virtual robot R is generated and sequentially displayed as if it were operating will be described later.

[0038] The user can observe the movements of the virtual robot R displayed on the display unit 17 while viewing the real space.

[0039] [Configuration of the virtual robot display device] Figure 2 is a functional block diagram showing the functions of the virtual robot display device 100 according to the first embodiment of the present invention. As shown in Figure 2, the virtual robot display device 100 comprises a reception unit 110, a storage unit 120, a control unit 130, and a display unit 140.

[0040] The virtual robot display device 100 is implemented, for example, in the MR device 10 shown in Figure 1. The control unit 130 includes an axis angle information generation unit 131, a time calculation unit 135, and an axis angle selection unit 136. Furthermore, the axis angle information generation unit 131 includes a trajectory generation unit 132, an attitude generation unit 133, and an inverse kinematics calculation unit 134. Here, the control unit 130 of the virtual robot display device 100 is processed by the processor 11 in the MR device 10 to realize each function.

[0041] The reception unit 110 receives start operations from the user. For example, the user uses the MR device 10 to view the real space and perform an operation to start playback of the virtual robot so that the welding robot is virtually visualized in the real space.

[0042] The memory unit 120 stores the work program for operating the welding robot.

[0043] The axis angle information generation unit 131 of the control unit 130 generates axis angle information that indicates the axis angles of the welding robot, which operates according to the work program, at each time interval, based on the work program stored in the storage unit 120.

[0044] Specifically, the trajectory generation unit 132 generates a trajectory for the welding robot based on the position of the welding robot operating according to the work program. For example, the trajectory generation unit 132 obtains the coordinates of the TCP (Tool Center Point) and the movement speed between coordinates based on teaching information including teaching points registered in the work program, and generates a trajectory for the TCP. Note that, for example, if the robot is a welding robot and the tool placed at its tip is a welding torch, the TCP may be the center position of the mounting member of the welding torch, or the tip position of the welding torch.

[0045] The attitude generation unit 133 generates the attitude of the welding robot corresponding to each time interval based on the trajectory of the welding robot generated by the trajectory generation unit 132. For example, the attitude generation unit 133 generates the attitude of the TCP by linearly interpolating based on the trajectory of the TCP generated by the trajectory generation unit 132.

[0046] The inverse kinematics calculation unit 134 calculates the axis angles of the welding robot corresponding to each time interval based on the welding robot's posture generated by the posture generation unit 133. For example, the inverse kinematics calculation unit 134 calculates the axis angles of the welding robot corresponding to the coordinates of TCP in the welding robot's posture every T seconds (e.g., 1 ms to 10 ms) using inverse kinematics, based on the coordinates of TCP and the posture of the tool (welding torch).

[0047] Figure 3 is an example of axis angle information generated by the axis angle information generation unit 131, showing the axis angles of a welding robot operating according to a work program every T seconds. As shown in Figure 3, axis states R0 (A0, B0, C0, D0, E0, F0) are registered, where A0 is the angle of the first axis, B0 is the angle of the second axis, C0 is the angle of the third axis, D0 is the angle of the fourth axis, E0 is the angle of the fifth axis, and F0 is the angle of the sixth axis of the welding robot corresponding to time t0.

[0048] The axis states R1 (A1, B1, C1, D1, E1, F1) of the welding robot are registered, with the angles of the first axis A1, second axis B1, third axis C1, fourth axis D1, fifth axis E1, and sixth axis F1 corresponding to time t1, T seconds after time t0.

[0049] Similarly, Rn(An,Bn,Cn,Dn,En,Fn) is registered for each axis angle of the welding robot from the 1st to the 6th axis, corresponding to time tn every T seconds.

[0050] Returning to the explanation of Figure 2, the time calculation unit 135 calculates the time for each axis angle information corresponding to the current time. For example, after the start operation from the user is received by the reception unit 110, the time calculation unit 135 obtains the current time and associates this current time with the time for each axis angle information shown in Figure 3. Then, each time the control unit 130 generates a virtual robot, the time calculation unit 135 obtains the current time and associates this current time with the time for each axis angle information shown in Figure 3.

[0051] More specifically, the time calculation unit 135 may, after receiving a start operation from the user by the reception unit 110, acquire the current time and associate this current time with the first time t0 among the axis angle information shown in Figure 3 as the start time. Then, each time the control unit 130 generates a virtual robot, the time calculation unit 135 acquires the current time, calculates the elapsed time from the start time (data acquisition time) based on the difference between the current time and the start time, and associates this data acquisition time with the time in the axis angle information shown in Figure 3.

[0052] The axis angle selection unit 136 selects the axis angle corresponding to the time in the axis angle information calculated by the time calculation unit 135. For example, the axis angle selection unit 136 associates the data acquisition time calculated by the time calculation unit 135 with the time in the axis angle information and selects the axis angle state corresponding to the closest time among the times t0 to ten.

[0053] The display unit 140 displays a virtual robot, which is an image showing the posture of the welding robot based on the axis angles selected by the axis angle selection unit 136.

[0054] After the virtual robot is displayed on the display unit 140, the time calculation unit 135 obtains the current time and calculates the data acquisition time, the axis angle selection unit 136 selects the axis angle state corresponding to the data acquisition time, and the display unit 140 displays the virtual robot again based on the selected axis angles, and these processes are repeated.

[0055] Figure 4 shows an example of a virtual robot displayed by the display unit 140. As shown in Figure 4, the display unit 140 displays virtual robots corresponding to each axis angle state R1, R5, ...Rn, ...Ren from the axis angle states R0 to Ren shown in Figure 3.

[0056] The display unit 140 does not sequentially display the virtual robots corresponding to each axis angle state R0 to Ren shown in Figure 3, but rather displays the virtual robot corresponding to each axis angle state R1, then sequentially displays the virtual robot corresponding to each axis angle state R5, ..., the virtual robot corresponding to each axis angle state Rn, ..., the virtual robot corresponding to each axis angle state Ren.

[0057] As described above, the time calculation unit 135 obtains the current time and calculates the data acquisition time, the axis angle selection unit 136 selects the axis angle state corresponding to the data acquisition time, and the display unit 140 displays the virtual robot based on the axis angle, resulting in the display of the virtual robots corresponding to the axis angle states R1, R5, ...Rn, ...Ren shown in Figure 3.

[0058] [How to display virtual robots]

[0059] Figure 5 is a flowchart showing the processing flow of the virtual robot display method M100 executed by the virtual robot display device 100 according to the first embodiment of the present invention. As shown in Figure 5, the virtual robot display method M100 includes steps S101 to S111, each step being executed by the virtual robot display device 100. Specifically, each step is executed by the processor 11 of the MR device 10.

[0060] In step S101, the reception unit 110 receives a start command from the user. Specifically, if the user performs an operation to start the playback of the virtual robot, and the reception unit 110 receives the instruction to start ("Yes" in step S101), it proceeds to the process in step S102.

[0061] In step S102, the control unit 130 acquires a work program for operating the welding robot. Specifically, the control unit 130 acquires a work program stored in the memory unit 120.

[0062] For example, if multiple work programs are stored in the storage unit 120, the user may be asked to select a work program in step S101, and in step S102, the control unit 130 may retrieve the work program selected by the user from the storage unit 120.

[0063] In step S103, the trajectory generation unit 132 generates welding robot (TCP) trajectories at T-second intervals. Specifically, the trajectory generation unit 132 obtains the coordinates of the TCP and the movement speed between coordinates based on teaching information, including teaching points, registered in the work program acquired in step S102, and generates the trajectory of the TCP. Note that the trajectory of the TCP may be generated while performing linear interpolation between each teaching point.

[0064] In step S104, the attitude generation unit 133 generates the attitude of the welding robot at T-second intervals. Specifically, the attitude generation unit 133 generates the attitude of the welding robot at T-second intervals based on the welding robot (TCP) trajectory at T-second intervals generated in step S103.

[0065] In step S105, the inverse kinematics calculation unit 134 generates the axis angles of the welding robot at T-second intervals. Specifically, the inverse kinematics calculation unit 134 calculates the axis angles of the welding robot at T-second intervals using inverse kinematics, based on the posture of the welding robot at T-second intervals generated in step S104.

[0066] In this way, axis angle information is generated, which shows the axis angles corresponding to each time interval (T seconds) of the welding robot operating according to the work program, as shown in Figure 3.

[0067] In step S106, the time calculation unit 135 sets the current time as the start time. Specifically, in order to associate subsequent times with the times in the axis angle information shown in Figure 3, the time calculation unit 135 obtains the current time and stores this current time in memory as the start time.

[0068] In step S107, the time calculation unit 135 obtains the current time again.

[0069] In step S108, the time calculation unit 135 calculates the data acquisition time. Specifically, the time calculation unit 135 calculates the difference between the current time obtained in step S107 and the start time stored in memory in step S106. This difference is the elapsed time from the start time, and this is taken as the data acquisition time.

[0070] In step S109, the control unit 130 determines whether the data acquisition time calculated in step S108 exceeds the time for each axis angle information shown in Figure 3. If it exceeds the time ("Yes" in step S109), the process ends; otherwise, if it does not exceed the time ("No" in step S109), the process proceeds to step S110.

[0071] In step S110, the axis angle selection unit 136 acquires the axis angles corresponding to the time closest to the data acquisition time calculated in step S108 among the times in the axis angle information shown in Figure 3. Specifically, the axis angle selection unit 136 selects the time closest to the data acquisition time from among the times t0 to ten in the axis angle information shown in Figure 3, and acquires the axis state (one of R0 to Ren) corresponding to that time.

[0072] In step S111, the display unit 140 displays a virtual robot. Specifically, the display unit 140 displays a virtual robot, which is an image showing the posture of the welding robot based on the axis states (axis angles) acquired in step S110, on an optically semi-transparent head-mounted display.

[0073] Then, the process returns to step S107 and repeats steps S107 to S111.

[0074] As described above, according to the virtual robot display device 100 and virtual robot display method M100 of the first embodiment of the present invention, the axis angle information generation unit 131 generates axis angle information indicating the axis angles corresponding to each time interval of T seconds for a welding robot operating according to a work program, based on the work program stored in the storage unit 120. The time calculation unit 135 calculates the data acquisition time by the difference between the current time and the start time, and the axis angle selection unit 136 selects the axis angles corresponding to the time closest to the data acquisition time among the time intervals in the axis angle information. Then, the display unit 140 sequentially displays virtual robots, which are images showing the posture of the welding robot based on the selected axis angles. As a result, the virtual robot displayed using the MR device 10 can be displayed appropriately so as not to deviate from the operating speed of the actual welding robot.

[0075] In this embodiment, the work program is stored in the storage unit 120, and the axis angle information generation unit 131 generates axis angle information based on the work program. However, the axis angle information generated by the axis angle information generation unit 131 may be stored in the storage unit 120. If the axis angle information is stored in the storage unit 120, the axis angle information generation unit 131 does not need to generate the axis angle information each time a start operation is received from the user, and the axis angle information stored in the storage unit 120 may be used. In other words, when the start operation is received from the user by the reception unit 110, if the axis angle information is not stored in the storage unit 120, the axis angle information generation unit 131 generates the axis angle information based on the work program and stores it in the storage unit 120.

[0076] Furthermore, the axis angle information is not limited to being generated by the axis angle information generation unit 131 in the virtual robot display device 100; for example, it may be generated in advance and stored in the storage unit 120. If the axis angle information is stored in the storage unit 120 in advance, the virtual robot display device 100 does not need to have the axis angle information generation unit 131, and in this case, the work program for generating the axis angle information does not need to be stored in the storage unit 120.

[0077] In this embodiment, the reception unit 110 accepts start operations from the user, but it may also accept setting operations for predetermined intervals (T seconds) in the axis angle information shown in Figure 3. Each axis angle information contains the axis angles at T-second intervals. For example, the T-second interval may be set to an interval of 1ms to 10ms. Increasing the T-second interval reduces the calculation processing for each axis angle, and also reduces the size of the axis angle information stored in the storage unit 120, thus avoiding memory congestion. On the other hand, decreasing the T-second interval reduces the difference between the time closest to the data acquisition time among the time points in each axis angle information and the data acquisition time. As a result, the virtual robot displayed by the display unit 140 can be displayed with less deviation from the actual welding robot's posture and with higher accuracy.

[0078] Furthermore, in this embodiment, as shown in Figure 5, the current time was acquired in step S107, and a virtual robot based on the time and axis angle in the axis angle information shown in Figure 3 corresponding to that time was generated (displayed) in step S111. However, for example, there is a time lag between acquiring the current time in step S107 and generating (displaying) the virtual robot in step S111. The time in the axis angle information shown in Figure 3 may be associated with the current time acquired in step S107, taking this time lag into consideration. As a result, the virtual robot displayed by the display unit 140 can be displayed with less deviation from the posture of the actual welding robot and with high accuracy.

[0079] Furthermore, this time lag may be predicted, for example, by an AI-based regression model, based on CPU usage.

[0080] <Second Embodiment> Next, in the second embodiment of the present invention, a virtual robot display device that takes into account the upper limit speed of the rotational speed of each axis of the welding robot in addition to the virtual robot display device described in the first embodiment will be described. In this embodiment, the same reference numerals are used for components that are the same as in the first embodiment of the present invention, and their detailed descriptions are omitted or simplified. The description will mainly focus on the different components.

[0081] Figure 6 is a functional block diagram showing the functions of the virtual robot display device 200 according to the second embodiment of the present invention. As shown in Figure 6, the virtual robot display device 200 includes a control unit 230 that includes a speed determination unit 231 and a speed conversion unit 232, compared to the virtual robot display device 100 shown in Figure 2.

[0082] The speed determination unit 231 calculates the rotational speed of each axis based on the axis angle corresponding to each time in the axis angle information generated by the axis angle information generation unit 131, and determines whether the calculated rotational speed of each axis exceeds the upper limit speed for each axis.

[0083] In actual welding robots, each axis has an upper limit set for its rotational speed. The speed determination unit 231 calculates the rotational speed of each axis based on the axis angle information shown in Figure 3. The speed determination unit 231 then determines whether the rotational speed of each axis exceeds the upper limit for each axis at each time interval.

[0084] For example, the speed determination unit 231 determines whether there are any axes whose rotational speed exceeds the upper limit speed of that axis, based on the time t0 to ten of the axis angle information shown in Figure 3.

[0085] If the speed determination unit 231 determines that there is an over-speed axis among the axes that exceeds the upper limit speed, the speed conversion unit 232 converts the speed of the over-speed axis to a speed below the upper limit speed. For example, if, during any of the time periods t0 to ten shown in Figure 3 for the angle information of each axis, the speed conversion unit 232 determines that the rotational speed of any of the axes exceeds the upper limit speed of that axis, the speed conversion unit 232 converts the rotational speed of that axis (over-speed axis). The speed conversion unit 232 may convert the rotational speed of that axis (over-speed axis) to, for example, the upper limit speed.

[0086] Furthermore, the speed conversion unit 232 may convert the speeds of each shaft other than the speed-over-speed shaft in accordance with converting the speed of the speed-over-speed shaft to below the upper limit speed. For example, the speed conversion unit 232 converts the rotational speeds of each other shaft at a similar conversion (reduction) rate in accordance with converting (reducing) the speed of the speed-over-speed shaft to the upper limit speed.

[0087] Then, as described above, the angle information for each axis shown in Figure 3 is updated based on the rotational speed of each axis converted by the speed conversion unit 232.

[0088] Figure 7 is a flowchart showing the processing flow of the virtual robot display method M200 executed by the virtual robot display device 200 according to the second embodiment of the present invention. As shown in Figure 7, the virtual robot display method M200 includes steps S201 to S203 compared to the virtual robot display method M100 shown in Figure 5.

[0089] In step S201, the speed determination unit 231 calculates the rotational speed of each axis based on the axis angles corresponding to each time in the axis angle information generated in step S105. Specifically, the speed determination unit 231 calculates the rotational speed of each axis between each time interval of the axis angle information as shown in Figure 3.

[0090] In step S202, the speed determination unit 231 determines whether or not there are any axes that are exceeding the upper speed limit. Specifically, the speed determination unit 231 compares the rotational speed of each axis calculated in step S201 with the upper speed limit of each axis and determines whether or not there are any axes that are exceeding the upper speed limit. If there are any axes that are exceeding the speed limit ("Yes" in step S202), the process proceeds to step S203; if there are no axes that are exceeding the speed limit ("No" in step S202), the process proceeds to step S106.

[0091] In step S203, the speed conversion unit 232 converts the rotational speed of the over-speed shaft to a speed below the upper limit. Specifically, the speed conversion unit 232 converts the rotational speed of the shaft determined to be an over-speed shaft in step S202 to a speed below the upper limit of that shaft, and updates the shaft angle information shown in Figure 3 accordingly. Furthermore, in response to the conversion of the rotational speed of the over-speed shaft, the speed conversion unit 232 may similarly (for example, with a similar reduction ratio) convert the rotational speed of the other shafts as well, and update the shaft angle information shown in Figure 3 accordingly.

[0092] As described above, according to the virtual robot display device 200 and virtual robot display method M200 of the second embodiment of the present invention, in addition to the virtual robot display device 100 and virtual robot display method M100, the speed determination unit 231 and speed conversion unit 232 convert the rotational speed of each axis of the welding robot, taking into consideration the upper limit speed of the rotational speed of each axis of the welding robot. Therefore, the virtual robot displayed using the MR device 10 can be displayed more appropriately so as not to deviate from the actual operating speed of the welding robot.

[0093] In each embodiment, an MR device was used as an example of a virtual robot display device. However, the virtual robot display device is not limited to MR, and can be applied to technologies that connect real space and virtual space, such as VR, AR, and SR, or to display devices that display images corresponding to real machines. Furthermore, the MR device may be non-transparent and can be applied to devices that capture images of the real world and composite those images onto images of the virtual world, such as MetaQuest® and Vision Pro®.

[0094] The embodiments described above are provided to facilitate understanding of the present invention and are not intended to limit its interpretation. The elements, arrangement, materials, conditions, shapes, and sizes of the embodiments are not limited to those exemplified and can be modified as appropriate. Furthermore, it is possible to partially substitute or combine the configurations shown in different embodiments. [Explanation of Symbols]

[0095] 10...MR device, 11...Processor, 12...Bus, 13...Work memory, 14...Storage memory, 15...Driver, 16...Reception unit, 17...Display unit, 18...Shooting unit, 100...Virtual robot display device, 110...Reception unit, 120...Storage unit, 130...Control unit, 131...Axis angle information generation unit, 132...Trajectory generation unit, 133...Attitude generation unit, 134...Inverse kinematics calculation unit, 135...Time calculation unit, 136...Axis angle selection unit, 140...Display unit, 200...Virtual robot display device, 230...Control unit, 231...Velocity determination unit, 232...Velocity conversion unit, M100, M200...Virtual robot display method, S101~S111, S201~S203...Each step of virtual robot display method M100 and M200, R...Virtual robot, W...Work

Claims

1. A virtual robot display device that visualizes a robot virtually in real space, A storage unit that stores information on each axis angle, which indicates the axis angle corresponding to each time point at a predetermined interval, of the robot operating according to a work program, A control unit that generates a virtual robot, which is an image showing the posture of the robot corresponding to one of the time intervals, based on the aforementioned axis angle information, The system includes a display unit that sequentially displays the virtual robots generated by the control unit, The control unit, A time calculation unit that calculates the time in each of the aforementioned axis angle information corresponding to the current time, Includes an axis angle selection unit that selects each axis angle corresponding to the time in the axis angle information calculated by the time calculation unit, Virtual robot display device.

2. The aforementioned storage unit stores the aforementioned work program. The control unit further includes an axis angle information generation unit that generates the axis angle information for each axis based on the work program and stores it in the storage unit. The virtual robot display device according to claim 1.

3. Each of the aforementioned axis angle information generation units is: A trajectory generation unit generates a trajectory for the robot based on the position of the robot operating according to the aforementioned work program, A posture generation unit generates the posture of the robot based on the trajectory of the robot generated by the trajectory generation unit, Includes an inverse kinematics calculation unit that calculates each axis angle of the robot corresponding to each time interval based on the posture of the robot generated by the posture generation unit, The virtual robot display device according to claim 2.

4. It also includes a reception unit that accepts startup commands from the user, When the reception unit receives a start operation from the user, if the axis angle information is not stored in the storage unit, the axis angle information generation unit generates the axis angle information based on the work program and stores it in the storage unit. The virtual robot display device according to claim 2 or 3.

5. The system further includes a reception unit that receives setting operations for the predetermined interval from the user, When the receiving unit receives a setting operation for the predetermined interval from the user, each axis angle information generation unit generates axis angle information indicating the axis angle corresponding to each time point in the received predetermined interval based on the work program and stores it in the storage unit. The virtual robot display device according to claim 2 or 3.

6. The control unit includes a speed determination unit that calculates the rotational speed of each axis based on the axis angles corresponding to each time in the axis angle information, and determines whether the calculated rotational speed of each axis exceeds the upper limit speed for each axis. If the speed determination unit determines that there is an over-speeding shaft among the shafts that exceeds the upper limit speed, the speed conversion unit further includes a speed conversion unit that converts the speed of the over-speeding shaft to a speed below the upper limit speed. The virtual robot display device according to claim 1.

7. The speed conversion unit converts the speed of each of the other axes in accordance with converting the speed of the speed-over-speed axis to a speed below the upper limit speed. The virtual robot display device according to claim 6.

8. A virtual robot display method performed by a virtual robot display device that virtually visualizes a robot in real space, A control step of generating a virtual robot, which is an image showing the posture of the robot corresponding to any of the time intervals, based on axis angle information that indicates the axis angles of the robot corresponding to each time interval of the robot, which operates according to a work program and is stored in a memory unit, The control step includes a display step that sequentially displays the virtual robots generated in the control step, The control step is, A time calculation step that calculates the time in each of the aforementioned axis angle information corresponding to the current time, The process includes a step of selecting each axis angle corresponding to the time in the axis angle information calculated in the time calculation step, Method for displaying virtual robots.

Images