Augmented reality glasses device and display program

Abstract

Augmented reality glasses device (1) comprising: a translucent display section (11), wherein the augmented reality glasses device (1) is suitable for displaying a path of a tool (101) in a machine tool (100) on the display section (11), wherein the augmented reality glasses device (1) comprises: a block call section (12) that calls a program block that causes the tool (101) to move and operate; a path determination section (13) which determines a path and a direction of movement of the tool (101) in a workpiece coordinate system based on several of the retrieved successive time series program blocks; a display control section (14) which causes the display section (11) to display the specific path and direction of movement of the tool (101) stereographically; an imaging section (16) suitable for capturing images, each passing through the display section (11) and each containing the tool (101), as captured images; a position identification section (17) that identifies a position of a tip of the tool (101) contained in each of the recorded images, wherein the display control section (14) causes the display section (11) to display the path and direction of movement of the tool (101) at the position of the tip of the tool (101) contained in each of the recorded images; and a usage information retrieval section (20) which retrieves as usage information whether cutting fluid is used in the machine tool (100), wherein The display control section (14) causes the display section (11) to display the path and direction of movement of the tool (101) at the position of the tip of the tool (101) contained in each of the recorded images, if the usage information indicates that no cutting fluid is used.

Description

BACKGROUND OF THE INVENTION

[0001] The present invention relates to an augmented reality glasses device and a display program.

[0002] A numerical control device that controls a machine tool and displays a tool trajectory on a screen of the device is known. Using this device, it is possible to check the machining status of a workpiece using the tool trajectory. A device that can repeat the execution and interruption of each individual machining program when troubleshooting machining programs is also known.

[0003] By displaying the trajectory of a tool in a single machining program, such a device makes it possible to execute a single machining program while simultaneously checking the progress of the machining program. However, it is more advantageous to display not only the trajectory of a work object but also the movement direction of the work object. A device that can also display the movement direction of a robot that is the subject of a work object has been proposed (see, for example, Patent Document 1).

[0004] Patent Document 1: Unexamined Japanese Patent Application, Publication No. JP 2014 - 180 707 A

[0005] DE 10 2017 221 678 A1 describes how to properly perform a simulation using augmented reality technology. An augmented reality simulation device includes: augmentation information display means for displaying a virtual object so as to be superimposed on a real machine tool; relation information acquisition means for acquiring first relation information, which is information specifying a relationship between the virtual object and control axes of a numerical control system that controls the machine tool, and second relation information, which is information specifying settings of the control axes themselves and the relationship between the control axes;a conversion means for converting a movement amount in a first coordinate system, which is a coordinate system of the control axis, into movement information in a second coordinate system, which is a coordinate system for allowing the extension information display means to display the virtual object, based on the first relational information and the second relational information; and an extension information calculation means for calculating a display position and a display angle of the virtual object based on the movement information in the second coordinate system after the conversion. The extension information display means displays the virtual object based on a calculation result of the calculation means.

[0006] DE 10 2013 114 372 A1 describes a load display device for a machine tool that acquires machine coordinate values ​​and load values ​​for axes of the machine tool for each predetermined time from an axis control circuit that controls the machine tool. Three-dimensional coordinate values ​​of a tool center point are calculated to obtain a tool trajectory from machine coordinate values ​​for the axes acquired for each predetermined time and from information about the machine tool structure. When displaying the tool trajectory on a display device, vectors of the load values ​​for the axes stored for each predetermined time are displayed.

[0007] DE 10 2012 103 075 B4 describes a tool path display device for a machine tool, which displays a three-dimensional path of a movable part of the machine tool, which is designed to control the positions and attitudes of a tool and a workpiece by means of a plurality of drive axes and a numerical controller, wherein the tool path display device comprises: a data acquisition unit designed to simultaneously acquire and store actual position data about the drive axes at each time point as time series data;a movable part path calculation unit configured to calculate the three-dimensional coordinate values ​​of the movable part as viewed from a coordinate system fixedly connected to the workpiece, based on the actual position data and data on the mechanical structure of the machine tool stored in the data acquisition unit, thereby determining the three-dimensional path of the movable part; a stereo image generation unit configured to generate stereo image data for the left and right eyes based on the three-dimensional path of the movable part determined by the movable part path calculation unit;and a stereo image display unit configured to display the left and right eye stereo image data obtained by the stereo image generation unit so that the image data can be viewed from the corresponding left and right eyes, respectively, wherein a point corresponding to a specific time point is highlighted in the stereo image data generated by the stereo image generation unit, and the highlighted point moves along a path in the display as the tool moves over time.;

[0008] DE 10 2009 029 062 A1 describes that a device for machining state monitoring is provided on a machine tool and comprises: a plurality of actual CCD cameras for imaging the tool and the workpiece from different viewpoints and for generating actual two-dimensional image data thereof; a virtual image generation section with a plurality of virtual CCD cameras corresponding to the actual CCD cameras, in which the tool and the workpiece of a three-dimensional model are imaged by the virtual CCD cameras from the individual viewpoints and virtual two-dimensional image data are generated therefrom;and a display control section for selecting a virtual CCD camera corresponding to the virtual two-dimensional image data from the generated two-dimensional image data in which the distal end of the tool is not obscured by the workpiece, and for displaying the actual two-dimensional image data generated by the actual CCD camera corresponding to the selected virtual CCD camera on a display device.;

[0009] EP 3 171 592 A1 describes that, in a stereoscopic image display device, an image acquisition section acquires an image to be displayed. An image display section displays a virtual image of the image obtained by the image acquisition section to a field of view of a user of the stereoscopic image display device. The image display section is configured to change the position of the virtual image displayed to the user and changes the position at which the virtual image is displayed based on depth information of an object appearing in the image.

[0010] US 2018 / 0 130 376 A1 describes that a system is provided that assists a user in at least one of the following operations: welding, cutting, joining, and plating. The system comprises a welding tool and a welding helmet with a face-mounted display. The system also includes a spatial tracker configured to track the welding helmet and welding tool in 3D space relative to an object to be processed. A processor-based subsystem is configured to generate virtual objects based on objects found in a welding environment and to transfer the virtual objects to a predetermined position on the front display. SUMMARY OF THE INVENTION

[0011] At the same time, in some machine tools, a machining position is moved due to the movement not only of the tip of a tool but also of a table on which a workpiece is mounted. For example, in a rectangular coordinate system represented by an X-axis, a Y-axis, and a Z-axis, the tip of the tool can only move in the Z-axis direction. At the same time, the table can move in both the X-axis and Y-axis directions. Even in a case where multiple items of a work move with respect to each other in this way, it is advantageous if the relative position of the tip of the tool in the workpiece coordinate system can be intuitively displayed.

[0012] (1) One aspect of the disclosure relates to an augmented reality glasses device. The augmented reality glasses device, which includes a light-transmitting display section and is capable of displaying a trajectory of a tool in a machine tool on the display section, includes a block retrieval section that retrieves a program block that causes the tool to move and operate, a trajectory determination section that determines a trajectory and a moving direction of the tool in a workpiece coordinate system based on a plurality of the retrieved consecutive time-series program blocks, and a display control section that causes the display section to stereographically display the determined trajectory and moving direction of the tool.

[0013] (2) One aspect of the disclosure relates to a display program. The display program causes a computer to function as an augmented reality glasses device. The augmented reality glasses device includes a light-transmitting display section and is adapted to display a trajectory of a tool in a machine tool on the display section. The display program causes the computer to function as a block retrieval section that retrieves a program block that causes the tool to move and operate, a trajectory determination section that determines a trajectory and a moving direction of the tool in a workpiece coordinate system based on a plurality of the retrieved consecutive time-series program blocks, and a display control section that causes the display section to stereographically display the determined trajectory and moving direction of the tool.

[0014] According to the disclosure, an augmented reality glasses device and a display program are provided which are suitable for intuitively displaying a position of a tip of a tool in a workpiece coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic configuration diagram showing the overall configuration of an augmented reality glasses device and a display program according to a first embodiment. Fig. 2 is a block diagram showing the configuration of the augmented reality glasses device according to the first embodiment. Fig. 3 is a conceptual view showing a trajectory and a moving direction of a tip of a tool displayed by a pair of display sections in the augmented reality glasses device and the display program according to the first embodiment. Fig. 4 is a flowchart showing the flow of a process in the augmented reality glasses device and the display program according to the first embodiment. Fig. 5 is a flowchart showing the flow of a process in the augmented reality glasses device and the display program according to the first embodiment. Fig. 6 is a conceptual view showing trajectories and moving directions superimposed on the tip of the tool and on a table, which are displayed by the display section in the augmented reality glasses device and the display program according to the first embodiment. Fig. 7 is a conceptual view showing trajectories and moving directions superimposed on the tip of the tool and on the table, which are displayed by the display section in the augmented reality glasses device and the display program according to the first embodiment. Fig. 8 is a block diagram showing the configuration of an augmented reality glasses device according to a second embodiment. Fig. 9 is a flowchart showing the flow of a process in the augmented reality glasses device and a display program according to the second embodiment. Fig. 10 is a conceptual view showing trajectories and moving directions superimposed on the tip of the tool and on the table, which are displayed by a display section in the augmented reality glasses device and the display program according to the second embodiment. Fig. 11 is a conceptual view showing trajectories and moving directions of a tip of a tool displayed by a display section in an augmented reality glasses device according to a modification. Fig. 12 is a conceptual view showing trajectories and moving directions of a tip of a tool displayed by a display section in an augmented reality glasses device according to a modification. DETAILED DESCRIPTION OF THE INVENTION

[0015] Below, with reference to Fig. 1 to Fig. 12, an augmented reality glasses device 1 and a display program according to each embodiment of the disclosure are described. Before describing the augmented reality glasses device 1 and the display program according to each embodiment, a machine tool 100 and a numerical control device 200 associated with the augmented reality glasses device 1 will be described. In the following description, a "workpiece coordinate system" refers to a coordinate system determined, for example, by manual control of a machine tool, and a coordinate system related to a workpiece, which is an object to be machined.

[0016] As in Fig. 1, the machine tool 100 is, for example, a machining device connected to the numerical control device 200 as shown in Fig. 1. The machine tool 100 operates according to an operation signal output from the numerical control device 200. The machine tool 100 includes a table 102 for mounting a workpiece W and a tool 101 arranged above the table 102. The machine tool 100 machines the workpiece W, for example, by operating both the table 102 and the tool 101 in accordance with the operation signal. The machine tool 100 operates either under automatic control in accordance with a command from the numerical control device 200 or under manual control using a manual pulse generator (not shown). The automatic control is used, for example, when the workpiece W is actually machined with the tool 101. The manual control is used, for example, when debugging, detecting the workpiece W, or the like is performed.

[0017] The numerical control device 200 is a device capable of outputting an operation signal to the machine tool 100 according to program blocks of a preset machining program. In the present embodiment, the numerical control device 200 is installed as a unit with the single machine tool 100. The numerical control device 200 controls the operation of the machine tool 100 by sequentially executing a sequence of program blocks included in the machining program according to a sequence of operation signals.

[0018] In the machine tool 100 and the numerical control device 200 described above, the machine tool 100 is manually controlled and then automatically controlled. First, the machine tool 100 is manually controlled, for example, using the pulse generator to move the tip of the tool 101 or the table 102 in each specified axis direction of the X, Y, and Z axes. The position, size, and the like of the workpiece W are detected, with the tip of a measuring probe (not shown) attached instead of the tool 101.

[0019] Subsequently, the machine tool 100 is automatically controlled by the program blocks executed by the numerical control device 200. The machine tool 100 actually removes the workpiece W using the tool 101, for example, using a cutting fluid (not shown).

[0020] Next, with reference to Fig. 1 to Fig. 7 describes the augmented reality glasses device 1 and the display program according to the first embodiment. The augmented reality glasses device 1 according to the present embodiment is, for example, a device that displays the trajectories and movement directions of the tip of the tool 101 in the workpiece coordinate system. More specifically, the augmented reality glasses device 1 is a glasses-type display device suitable for checking the machining status of the machine tool 100. The augmented reality glasses device 1 is worn by an operator H who manages the numerical control device 200 and the machine tool 100. The augmented reality glasses device 1 is associated with, for example, a unit consisting of the numerical control device 200 and the machine tool 100. The augmented reality glasses device 1 is connected to the numerical control device 200 to communicate with it wirelessly. As shown in FIG. Fig. 2, the augmented reality glasses device 1 includes a display section 11, a block retrieval section 12, a trajectory determination section 13, a display control section 14, a retrieval number limiting section 15, an imaging section 16, a position identification section 17, a manual operation identification section 18, and a movement direction retrieval section 19.

[0021] The display section 11 is, for example, a translucent display. The display sections 11 are arranged in pairs in front of the eyes of the employee H. Each display section 11 is configured to display different individual pieces of information in addition to an image transmitted through them. For example, the display section 11 transmits the images of the tool 101 and the table 102 of the machine tool 100 and displays different individual pieces of information superimposed on the images of the tool 101 and the table 102.

[0022] The block retrieval section 12 is implemented, for example, by the operation of a CPU. The block retrieval section 12 retrieves program blocks that cause the tool 101 to move and operate. For example, the block retrieval section 12 retrieves program blocks loaded during or immediately before the execution of a machining program preset in the numerical control device 200. More specifically, the block retrieval section 12 retrieves a predetermined number of a sequence of program blocks of the machining program during or immediately before its use to operate the tool 101. In the present embodiment, the block retrieval section 12 retrieves a predetermined number of program blocks, including an active program block or a program block to be executed next, and one or more program blocks to be subsequently executed by the numerical control device 200.The block retrieval section 12 retrieves, for example, information identifying an active program and information indicating the position of the tool 101 in the workpiece coordinate system from the numerical control device 200.

[0023] The path determination section 13 is implemented, for example, by the operation of the CPU. The path determination section 13 identifies the paths and movement directions of the tool 101 in the workpiece coordinate system based on several consecutive time-series program blocks. More specifically, the path determination section 13 determines movement paths and movement directions using an active program block or a program block to be executed next and one or more program blocks to be executed subsequently from a sequence of program blocks. The path determination section 13 determines the active program block and the position of the tool 101 based on the retrieved information indicating the position of the tool 101 in the workpiece coordinate system and the information identifying the active program block.

[0024] The display control section 14 is implemented, for example, by the operation of the CPU. The display control section 14 causes the display section 11 to stereographically display the determined paths and movement directions of the tool 101. As shown in Fig. 3, for example, the display control section 14 causes the pair of right and left display sections 11 to each display a parallax image representing the trajectories and movement directions of the tip of the tool 101 in the workpiece coordinate system. In other words, the display control section 14 causes the display section 11 to display the trajectories and movement directions of the tip of the tool 101 for a current operation and a subsequent operation as stereo images. Thus, the display control section 14 provides the employee H with a three-dimensional image (a 3D image). The display control section 14 displays markers (in Fig. 3 N10, N11 and N12) for identifying program blocks. In the present embodiment, the display control section 14 causes the display section 11 to Fig. 3, for displaying the paths and directions of movement of the tool 101 along with the coordinate axes of the workpiece coordinate system. The display control section 14 causes the display section 11 to display the specific active program blocks and the specific position of the tool 101.

[0025] The fetch number limiting section 15 is implemented, for example, by the operation of the CPU. The fetch number limiting section 15 limits the number of program blocks to be fetched by the block fetching section 12 according to the specified number of program blocks to be fetched or according to a movement distance of the tool 101 to be traveled as a result of the execution of the program blocks. The fetch number limiting section 15 limits the number of program blocks to be fetched by the block fetching section 12, for example, to three. Thus, the fetch number limiting section 15 limits the trajectories and movement directions of the tip of the tool 101 determined by the trajectory determination section 13 to the trajectories and movement directions over a sequence of three program blocks. The fetch number limiting section 15 limits the number of program blocks to be fetched by the block fetching section 12, for example, by limiting the movement distance of the tool 101.As a result, the call number limiting section 15 limits the number of a sequence of program blocks to be used for determination by the path determining section 13 so that the movement distance of the tool 101 to be traversed according to a sequence of program blocks is within a predetermined range. Specifically, the call number limiting section 15 sums the movement distances of the tool 101 in a sequence of program blocks based on an active program block or a program block to be executed next. The call number limiting section 15 limits the number of program blocks to be called to the number of program blocks for which the movement distance of the tool 101 corresponds to the longest distance and a predetermined distance is not exceeded.

[0026] The imaging section 16 is, for example, a device that captures images. The imaging section 16 is configured to retrieve, as captured images, images each passing through the display section 11 and each including the tool 101. In other words, the imaging section 16 is configured to retrieve the images of the tool 101 of the machine tool 100 as captured images via the display section 11. The images of the tool 101 of the machine tool 100 are visually recognized by the worker H. The imaging section 16 retrieves, for example, one captured image for each pair of the right and left display sections 11. The imaging section 16 captures images each including the table 102. The imaging section 16 captures, for example, images each including the tip of the tool 101 and the table 102.

[0027] The position identification section 17 is implemented, for example, by the operation of the CPU. The position identification section 17 identifies the position of the tip of the tool 101 included in each of the captured images. The position identification section 17 identifies the relative position of the tip of the tool 101 in the workpiece coordinate system, for example, by detecting a position detection mark attached to the tip of the tool 101 in each of the captured images. The position identification section 17 identifies the position of the table 102 included in each of the captured images.

[0028] The manual operation identification section 18 is implemented, for example, by the operation of the CPU. The manual operation identification section 18 identifies a switch from the automatic operation of the machine tool 100 to the manual operation of the machine tool 100. The manual operation identification section 18 identifies, for example, a switch from a mode (automatic control) in which the machine tool 100 is controlled by the numerical control device 200 to a mode (manual control) in which the machine tool 100 is operated using the pulse generator.

[0029] The movement direction retrieval section 19 is implemented, for example, by the operation of the CPU. Upon identifying a switch to manual operation, the movement direction retrieval section 19 retrieves a manually specified movement direction of at least one of the tool 101 and the table 102. The movement direction retrieval section 19 retrieves, for example, the movement direction of at least one of the tool 101 and the table 102 from the numerical control device 200 to act according to a movement command on the axis specified by the pulse generator.

[0030] Next, referring to the flowchart in Fig. 4 describes a workflow using the augmented reality glasses device 1. In connection with this workflow, an example is described in which the machine tool 100 is controlled manually and then automatically.

[0031] First, the manual operation identification section 18 determines whether the control for the operation of the machine tool 100 is manual control. The manual operation identification section 18 determines whether the control is manual control, for example, by retrieving the operation mode from the numerical control device 200. If the control is manual control (YES in step S1), the process proceeds to step S2. If the control is automatic control (NO in step S2), the process proceeds to step S11 according to Fig. 5 continued.

[0032] In step S2, the imaging section 16 retrieves captured images. For example, the imaging section 16 retrieves images that pass through the pair of display sections 11, respectively, as captured images.

[0033] Next, the position identification section 17 identifies the position of the tip of the tool 101 and the position of the table 102 included in each of the captured images (step S3). The position identification section 17 identifies the position of the tip of the tool 101 and the position of the table 102 using, for example, the markers attached to the tip of the tool 101 and the table 102.

[0034] Next, the movement direction retrieval section 19 retrieves the movement direction along each axis set using the pulse generator (step S4). The movement direction retrieval section 19 retrieves, for example, information about the axes (operation axes) specified by the pulse generator via the numerical control device 200. The movement direction retrieval section 19 retrieves an operation direction in which the tool 101 or the table 102, or both, are actually driven by the pulse generator. The movement direction retrieval section 19 retrieves information about the movement direction (a plus direction or a minus direction) on an operation axis by retrieving, for example, the rotation direction of a rotary knob pulse generator.

[0035] Subsequently, the display control section 14 causes the display section 11 to display the retrieved operation axis and movement direction (step S5). The display control section 14 causes the display section 11 to display the operation axis and movement direction superimposed on the tool 101 at the tip position or on the table 102. As shown in Fig. 6, the display control section 14 causes the display section 11 to display, for example, the position of the table 102 included in each of the captured images, such that the table 102 is operated in the movement direction set to the plus direction along the operation axis set to the X-axis. Fig. As shown in Figure 7, the display control section 14 causes the display section 11 to display the tip position of the tool 101 included in each of the captured images, indicating that the tip of the tool 101 is operated along the operation axis set to the Z axis in the movement direction set to the plus direction. For example, the display control section 14 may cause the display section 11 to display superimposed images as stereo images.

[0036] It is then determined whether a subsequent movement driven by the pulse generator is planned (step S6). If a subsequent movement driven by the pulse generator is planned (YES in step S6), the process returns to step S4. On the other hand, if no further movements driven by the pulse generator are planned (NO in step S6), the process continues with step S7.

[0037] In step S7, the manual operation identification section 18 determines whether the operation mode should be switched from manual operation to automatic operation. If the operation mode is switched (YES in step S7), the process proceeds to step S11 according to Fig. 5. On the other hand, if the operation mode is not switched (NO in step S7), the process ends.

[0038] As in Fig. As shown in Figure 5, the fetch number limiting section 15 limits the number of program blocks to be fetched from the numerical control device 200 (step S11). Subsequently, the block fetching section 12 retrieves the limited number of program blocks from the numerical control device 200 (step S12).

[0039] The path determination section 13 determines the relative paths and movement directions of the tip of the tool 101 in the workpiece coordinate system based on the retrieved program blocks (step S13). The path determination section 13 determines the active program block and the position of the tool 101. As shown in Fig. As shown in Figure 3, the display control section 14 causes the display section 11 to display the determined trajectories and movement directions as stereo images (step S14). The display control section 14 causes the display section 11 to display the determined active program block and the determined position of the tool 101.

[0040] Subsequently, it is determined whether a subsequent program block exists (step S15). If a subsequent program block exists (YES in step S15), the process returns to step S11; whereas if no subsequent program block exists (NO in step S15), the process ends in this flow.

[0041] Next, the display program according to the present embodiment will be described. The components included in the augmented reality glasses device 1 are implemented by hardware or software, or by a combination of hardware and software. Here, the components implemented by software are components implemented by a computer that loads the display program.

[0042] The display program can be stored on various types of non-transitory computer-readable media and made accessible to a computer. The non-transitory computer-readable medium includes various types of specific storage media. Examples of the non-transitory computer-readable medium include a magnetic storage medium (e.g., a floppy disk, a magnetic tape, or a hard disk drive), a magneto-optical storage medium (e.g., a magneto-optical disk), a CD-ROM (compact disc read-only memory), a CD-R, a CD-R / W, and semiconductor memories (e.g., a mask ROM, a PROM (programmable ROM), an EPROM (rewritable PROM), a flash ROM, or a RAM (random access memory)). The display program can be made accessible to a computer via various types of volatile computer-readable media.Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable media allows the display program to be accessible to a computer via a wired communication channel, such as an electrical cable and an optical fiber, or via a wireless communication channel.

[0043] The augmented reality glasses device 1 and the display program according to the first embodiment achieve the following advantageous results.

[0044] (1) The augmented reality glasses device 1, which includes the light-transmitting display section 11 and is suitable for displaying the trajectory of the tool 101 of the machine tool 100 with the tool 101 on the display sections 11, includes the block retrieval section 12, which retrieves program blocks that cause the tool 101 to move and operate, the trajectory determination section 13, which determines the trajectory and the movement direction of the tool 101 in the workpiece coordinate system based on a plurality of the retrieved consecutive time-series program blocks, and the

[0045] Display control section 14, which causes the display section 11 to stereographically display the determined trajectory and movement direction of the tool 101. This allows the worker H wearing the augmented reality glasses device 1 to recognize the trajectory and movement direction of the tool 101 as a three-dimensional image (3D image). This makes it possible for the worker H to intuitively recognize the relative trajectory and movement direction of the tip of the tool 101.

[0046] (2) The display control section 14 causes the display section 11 to display the trajectory and movement direction of the tool 101 along with the coordinate axes. This shows the movement direction of the tip of the tool 101 when viewing the workpiece coordinate system from the position of the operator H relative to the coordinate axes. This allows the operator H to further intuitively recognize the relative trajectory and movement direction of the tip of the tool 101.

[0047] (3) The augmented reality glasses device 1 further includes the retrieval number limiting section 15, which limits the number of program blocks to be retrieved by the block retrieval section 12 according to the set number of program blocks to be retrieved or a movement distance of the tool 101 to be moved as a result of executing the program blocks to be retrieved. Therefore, the amount of information of the program blocks held by the augmented reality glasses device 1 is limited. This saves the memory capacity used by the augmented reality glasses device 1.

[0048] (4) The augmented reality glasses device 1 further includes the imaging section 16, which is capable of capturing images passing through the display section 11 and each including the tool 101 as captured images, and the position identification section 17, which identifies the position of the tip of the tool 101 included in each of the captured images. The display control section 14 causes the display section 11 to display the trajectory and the moving direction of the tool 101 superimposed on the position of the tip of the tool 101 included in each of the captured images. It is possible to display the trajectory and the moving direction of the tool 101 superimposed on the position of the tip of the actual tool 101, thereby making it possible to provide the device in which it is even easier to intuitively grasp the trajectory and the moving direction of the tip of the tool 101.When the position of the tip of the tool 101 of the machine tool 100 is not visually recognized from the outside, the trajectory and movement direction of the tool 101 are shown in the workpiece coordinate system. On the other hand, when the position of the tip of the tool 101 of the machine tool 100 is visually recognized from the outside, the trajectory and movement direction of the tool 101 are shown as the trajectory of the actual tool 101. Therefore, the augmented reality glasses device 1 is a device suitable for displaying the trajectory and movement direction of the tool 101 even when the position of the tip of the tool 101 is not visually recognized. The augmented reality glasses device 1 is a device suitable for intuitively displaying a trajectory and movement direction superimposed on the position of the tip of the tool 101 when the position of the tip of the tool 101 is visually recognized.

[0049] (5) The imaging section 16 captures images each including the table 102. The position identification section 17 identifies the position of the table 102. The trajectory determination section 13 identifies the trajectory and movement direction of the table 102 in the workpiece coordinate system based on a plurality of consecutive time-series program blocks and the position of the table 102. The display control section 14 causes the display section 11 to display the trajectory and movement direction of the table 102 superimposed at the position of the table 102 included in each of the captured images. This makes it possible to separately display the trajectory and movement direction of the tip of the tool 101 and the table 102, thereby enabling the operator H to intuitively recognize the operation of the actual machine.

[0050] (6) The augmented reality glasses device 1 further includes the manual operation identification section 18 that identifies a switch from the automatic operation of the machine tool 100 to the manual operation of the machine tool 100, and the movement direction retrieval section 19 that retrieves a manually set movement direction of at least one of the tool 101 and the table 102 upon identification of a switch to the manual operation. The display control section 14 superimposes the retrieved movement direction on at least one of the tool 101 and the table 102. This makes it possible to switch the display mode between a manual control mode in which the tip of the actual tool 101 is visually recognized and an automatic control mode that operates during a machining operation, providing an even more user-friendly device.

[0051] (7) The display program causes a computer to function as an augmented reality glasses device 1, which includes the light-transmitting display section 11 and is capable of displaying the trajectory of the tool 101 in the machine tool 100 on the display section 11. The display program causes the computer to function as a block retrieval section 12, which retrieves program blocks that cause the tool 101 to move and operate, a trajectory determination section 13, which determines the trajectory and movement direction of the tool 101 in the workpiece coordinate system based on a plurality of the consecutive time-series program blocks, and a display control section 14, which causes the display section 11 to stereographically display the determined trajectory and movement direction of the tool 101.This allows the employee H wearing the augmented reality glasses device 1 to recognize the trajectory and direction of movement of the tool 101 as a three-dimensional image (3D image). This allows the employee H to intuitively recognize the relative trajectory and direction of movement of the tip of the tool 101.

[0052] Next, with reference to Fig. 8 to Fig. 10, the augmented reality glasses device 1 and the display program according to a second embodiment are described. As in Fig. As shown in Fig. 8, the augmented reality glasses device 1 and the display program according to the second embodiment differ from those according to the first embodiment in that a usage information retrieval section 20 is further provided that retrieves, as usage information, whether cutting fluid is used in the machine tool 100. Furthermore, the augmented reality glasses device 1 and the display program according to the second embodiment differ from those according to the first embodiment in that the display control section 14 causes the display section 11 to display the trajectory and the movement direction of the tool 101 superimposed at the tip position of the tool 101 included in each of the captured images when the usage information indicates that cutting fluid is not used.

[0053] The usage information retrieval section 20 is implemented, for example, by the operation of the CPU. The usage information retrieval section 20 retrieves, as usage information, whether cutting fluid is used in the machine tool 100. In other words, the usage information retrieval section 20 retrieves, as usage information, whether the tip of the tool 101 is hidden due to the use of cutting fluid.

[0054] Next, with reference to Fig. 9, the operation sequence of the augmented reality glasses device 1 according to the present embodiment is described. This sequence is described as a subroutine of step S14 according to Fig. 5. As in Fig. As shown in Figure 9, the usage information retrieval section 20 retrieves the usage information from the numerical control device 200 (step S21). It is then determined whether cutting fluid is being used (step S22). If cutting fluid is not being used (NO in step S22), the process proceeds to step S23. On the other hand, if cutting fluid is being used (YES in step S22), the display control section 14 causes the pair of right and left display sections 11 to display, for example, stereo images showing the trajectory and movement direction of the tip of the tool 101 in the workpiece coordinate system, as shown in Figure 9. Fig. 3 (step S24). This ends the process.

[0055] In step S23, the display control section 14 causes the display sections 11 to display the trajectory and the moving direction of the tip of the tool 101 superimposed at the position of the tip of the actual tool 101 included in each of the captured images. As shown in Fig. 10, the display control section 14 causes the display section 11 to separately display, for example, the movement of the tool 101 and the movement of the table 102. This ends the process.

[0056] The augmented reality glasses device 1 and the display program according to the second embodiment achieve the following advantageous results. (8) The augmented reality glasses device 1 further includes the usage information retrieval section 20, which retrieves, as usage information, whether cutting fluid is used in the machine tool 100. If the usage information indicates that cutting fluid is not used, the display control section 14 causes the display section 11 to display the trajectory and movement direction of the tool 101 superimposed on the tip position of the tool 101 included in each of the captured images.This makes it possible to show the trajectory and movement direction relative to the tip position of the tool 101 in the workpiece coordinate system at the tip position of the actual tool 101 when no cutting fluid is required during troubleshooting or the like. This also makes it possible to intuitively show the trajectory and movement direction of the tool 101 to the operator H.

[0057] The embodiments of the augmented reality glasses device and the display program according to the disclosure have been described; however, the disclosure is not limited to the above-described embodiments and can be modified according to requirements.

[0058] In the embodiments described above, for example, when the angle of the display section 11 changes with respect to the position of the tip of the tool 101, the angle of the trajectory can be changed according to the change in the angle. The augmented reality glasses device 1 can further include, for example, a displacement retrieval section (not shown) that retrieves a displacement of the angle with respect to the actual position of the tool 101. The display control section 14 changes the angle of the workpiece coordinate system displayed on the display section 11 according to a displacement of the angle retrieved by the displacement retrieval section. The display control section 14 can, for example, change the angle of the trajectory and the moving direction of the tip of the tool 101 to be displayed on the display section 11, as shown in Fig. 12, compared to an image in which the X, Y and Z axes are fixed, as in Fig.11, and display the changed image. The display control section 14 can receive, via an input section (not shown), the input of a position that is the reference at a shift in angle, and change the view according to a shift from the reference position. For example, the display control section 14 can receive the input of a trajectory and a movement direction to be recorded from a predetermined angle as a reference position, and record a trajectory and a movement direction whose angle has been changed according to a shift from the reference position.

[0059] In the second embodiment described above, the usage information retrieval section 20 retrieves usage information to determine whether cutting fluid is being used; however, the configuration is not limited to this. Whether cutting fluid is being used can be determined, for example, depending on whether the position identification section 17 can identify an image (a mark) at the position of the tip of the tool 101 included in each of the images captured by the imaging section 16. If the position identification section 17 can identify an image (a mark) at the position of the tip of the tool 101, the display control section 14 can display the moving direction of the tip of the tool 101 at the position of the tip of the actual tool 101. The display control section 14 can display the moving direction of the table 102 at the position of the actual table 102.

[0060] In the above-described embodiments, the display control section 14 may further cause the display section 11 to display the coordinates of each axis. The display control section 14 may further display a movement distance (a remaining movement distance) of the tool 101 resulting from the execution of the program blocks following the active program block. In this case, the call number limiting section 15 may calculate the movement distance of the tool 101 to be moved as a result of the execution of the remaining program block(s) among the program blocks set in the numerical controller 200.

[0061] In the above-described embodiments, the retrieval number limiting section 15 is configured to limit the number of program blocks to be retrieved according to the set number of program blocks or a movement distance of the tool 101 to be traveled as a result of executing the program blocks; however, the configuration is not limited thereto. The retrieval number limiting section 15 may limit the number of program blocks to be retrieved, for example, according to the capacity of a memory of the augmented reality glasses device 1 or the set number of program blocks manually input in advance.

[0062] In the embodiments described above, the block retrieval section 12 is configured to retrieve information about the position of the tool 101 in the workpiece coordinate system; however, the configuration is not limited thereto. For example, the block retrieval section 12 may retrieve information about the position of the tool 101 only for a program block(s) with a block length longer than a predetermined length. In this case, the path determination section 13 only needs to determine the active program block instead of the position of the tool 101.

[0063] In the embodiments described above, the path determining section 13 is configured to determine the active program block and the position of the tool based on the retrieved information indicating the position of the tool in the workpiece coordinate system and the information identifying the active program block; however, the configuration is not limited to this. For example, the path determining section 13 may determine the position of the tool by determining whether there is a delay in communication from the numerical control device 200 to the block retrieval section 12. In this case, the path determining section 13 may determine the position of the tool 101 calculated with a delay taken into account based on information indicating the position of the tool 101 and the amount of deviation of the tool 101.

[0064] In the above-described embodiments, when there are limitations on the communication speed between the numerical control device 200 and the augmented reality glasses device 1, the block retrieval section 12 does not need to retrieve information indicating the position of the tool 101. In this case, the trajectory determination section 13 does not need to determine the position of the tool 101. The display control section 14 can cause the display section 11 to display a trajectory in which the tool 101 is present as the position of the tool 101 by causing the display section 11 to display the active program block with emphasis. EXPLANATION OF REFERENCE SYMBOLS 1 augmented reality glasses device 11 Display section 12 Block call section 13 Orbit determination section 14 Display control section 15 Number of hits limit section 16 Imaging section 17 Position identification section 18 Manual operation identification section 19 Movement direction retrieval section 20 Usage Information Retrieval Section 100 machine tools 101 Tools 102 Table 200 numerical control device W workpiece

Claims

[1] Augmented reality glasses device (1) comprising: a light-transmitting display section (11), wherein the augmented reality glasses device (1) is suitable for displaying a path of a tool (101) in a machine tool (100) on the display section (11), wherein the augmented reality glasses device (1) comprises: a block retrieval section (12) that retrieves a program block that causes the tool (101) to move and operate; a path determination section (13) which determines a path and a direction of movement of the tool (101) in a workpiece coordinate system based on a plurality of the retrieved successive time series program blocks; a display control section (14) which causes the display section (11) to stereographically display the determined path and direction of movement of the tool (101); an imaging section (16) adapted to capture images each passing through the display section (11) and each including the tool (101) as captured images; a position identification section (17) which identifies a position of a tip of the tool (101) included in each of the captured images, wherein the display control section (14) causes the display section (11) to display the path and the direction of movement of the tool (101) superimposed at the position of the tip of the tool (101) included in each of the captured images; and a usage information retrieval section (20) which retrieves as usage information whether cutting fluid is used in the machine tool (100), wherein the display control section (14) causes the display section (11) to display the trajectory and the direction of movement of the tool (101) superimposed at the position of the tip of the tool (101) included in each of the captured images when the usage information indicates that no cutting fluid is being used. [2] The augmented reality glasses device (1) according to claim 1, wherein the display control section (14) causes the display section (11) to display the trajectory and the moving direction of the tool (101) together with coordinate axes. [3] The augmented reality glasses device (1) according to claim 1 or 2, further comprising a retrieval number limiting section (15) that limits a number of program blocks to be retrieved by the block retrieval section (12) according to a predetermined number of program blocks to be retrieved or a moving distance of the tool (101) to be traveled as a result of executing the program blocks to be retrieved. [4] Augmented reality glasses device (1) according to one of claims 1 to 3, wherein the imaging section (16) captures images each including a table (102) of the machine tool (100) for mounting a workpiece as captured images, the position identification section (17) identifies a position of the table (102), the path determination section (13) identifies a path and a direction of movement of the table (102) in the workpiece coordinate system based on several of the successive time series program blocks and the position of the table (102), and the display control section (14) causes the display section (11) to display the trajectory and the direction of movement of the table (102) superimposed at the position of the table (102) included in each of the captured images. [5] Augmented reality glasses device (1) according to one of claims 1 to 4, further comprising: a manual operation identification section (18) that identifies a switch from an automatic operation of the machine tool (100) to a manual operation of the machine tool (100); and a movement direction retrieval section (19) which, upon identification of a switch to manual operation, retrieves a manually set movement direction of at least the tool (101) and / or the table (102), wherein the display control section (14) displays the retrieved direction of movement on at least the tool (101) and / or the table (102). [6] A display program that causes a computer to function as an augmented reality glasses device (1), the augmented reality glasses device (1) comprising: a light-transmitting display section (11), the augmented reality glasses device (1) being suitable for displaying a trajectory of a tool (101) in a machine tool (100) on the display section (11), the display program causing the computer to function as: Block retrieval section (12) that retrieves a program block that causes the tool (101) to move and work; Path determination section (13) which determines a path and a direction of movement of the tool (101) in a workpiece coordinate system based on a plurality of the successive time series program blocks; Display control section (14) which causes the display section (11) to stereographically display the determined path and direction of movement of the tool (101); An imaging section (16) adapted to capture images each passing through the display section (11) and each including the tool (101) as captured images; A position identification section (17) which identifies a position of a tip of the tool (101) included in each of the captured images, wherein the display control section (14) causes the display section (11) to display the path and the direction of movement of the tool (101) superimposed at the position of the tip of the tool (101) included in each of the captured images; and A usage information retrieval section (20) which retrieves as usage information whether cutting fluid is used in the machine tool (100), wherein the display control section (14) causes the display section (11) to display the trajectory and the direction of movement of the tool (101) superimposed at the position of the tip of the tool (101) included in each of the captured images when the usage information indicates that no cutting fluid is being used.

Images