Display devices for industrial machinery

Abstract

Provided is a display device for industrial machinery capable of automatically switching between augmented reality display and three-dimensional computer graphics display. The robot display device (2) comprises: a camera (2b); a display unit (20) that displays a first display and a second display in a switchable manner, wherein the first display is a display based on three-dimensional computer graphics of a robot (3) or three-dimensional computer graphics of the robot (3) and its peripheral equipment (4), and the second display is a display using augmented reality of the robot (3) and its peripheral equipment (4) captured by the camera (2b); and a selection unit (21) that enables the first display and displays the first display on the display unit (20) when the camera (2b) is not facing the robot (3) and its peripheral equipment (4), and enables the second display and displays the second display on the display unit (20) when the camera (2b) is facing the robot (3) and its peripheral equipment (4).

Description

Technical Field

[0001] The present disclosure relates to a display device for industrial machinery. Background Art

[0002] It can display industrial machinery and its peripheral equipment on the screen as virtual three-dimensional computer graphics, and can also display various industrial machinery information such as program teaching points, TCP (Tool Center Point) trajectory, coordinate system, etc. on the screen.

[0003] Such displays using three-dimensional computer graphics are virtual and therefore require comparison with actual industrial machinery and its peripheral equipment, making intuitive understanding difficult in some cases.

[0004] In contrast, by using a display device such as a tablet computer with a camera to display various information together on an image of an actual robot and its peripheral devices using augmented reality (AR), intuitive understanding is possible (for example, see Patent Document 1).

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-107379 Summary of the Invention

[0008] Problems to be solved by the invention

[0009] However, if the camera is not facing the industrial machinery such as robots and their peripheral equipment, the industrial machinery and their peripheral equipment will not be displayed, which is meaningless. In this case, it is desirable to display the aforementioned virtual three-dimensional computer graphics, but manually switching between the augmented reality display and the three-dimensional computer graphics display is complicated.

[0010] The present disclosure has been made in view of the above-mentioned problems, and provides a display device for industrial machinery that can automatically switch between augmented reality display and three-dimensional computer graphics display.

[0011] Solutions for solving problems

[0012] One embodiment of the present disclosure is a display device for industrial machinery, comprising: a camera; a display unit, which displays a first display and a second display in a switchable manner, wherein the first display is a display based on three-dimensional computer graphics of the industrial machinery or three-dimensional computer graphics of the industrial machinery and its peripheral equipment, and the second display is a display using augmented reality for the industrial machinery and its peripheral equipment captured by the camera; and a selection unit, which enables the first display and displays the first display on the display unit when the camera is not facing the industrial machinery and its peripheral equipment, and enables the second display and displays the second display on the display unit when the camera is facing the industrial machinery and its peripheral equipment.

[0013] Effects of the Invention

[0014] According to one embodiment of the present disclosure, it is possible to provide a display device for industrial machinery that can automatically switch between augmented reality display and three-dimensional computer graphics display. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of a robot system including a robot display device according to one embodiment of the present disclosure.

[0016] Figure 2 This is a functional block diagram of a robot display device according to one embodiment of the present disclosure.

[0017] Figure 3A This is a schematic diagram of a robot display device according to one embodiment of the present disclosure, showing a state in which a first display is displayed.

[0018] Figure 3B This is a schematic diagram of a robot display device according to one embodiment of the present disclosure, showing a state in which a second display is displayed.

[0019] Figure 4 This is a flowchart illustrating the operation of the robot display device according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] Hereinafter, a robot system 1 including a robot display device 2 according to an embodiment of the present disclosure will be described with reference to the drawings.

[0021] First, use Figure 1 and Figure 2 The configuration of a robot system 1 including a robot display device 2 according to this embodiment will be described. Figure 1 1 is a schematic diagram of a robot system 1 including a robot display device 2 according to the present embodiment. Figure 2This is a functional block diagram of the robot display device 2 according to this embodiment.

[0022] Figure 1 and Figure 2 The robot system 1 shown displays information about the robot 3 and its peripheral equipment 4 on a display screen 2a via a display unit 20 of a robot display device 2. Specifically, the robot system 1 includes the robot display device 2, the robot 3, the peripheral equipment 4, and a control device 5.

[0023] The robot display device 2 is a display device that applies the display device of the industrial machinery disclosed in the present invention to a robot. The robot display device 2 is a portable communication terminal such as a tablet terminal. Specifically, the robot display device 2 includes a camera 2b, a display unit 20, an input unit (not shown) that uses the display unit 20 as a touch panel to input information, a communication unit (not shown) that communicates with the control device 5, an arithmetic processing unit such as a CPU (Central Processing Unit) (not shown), auxiliary storage units such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive) that store various programs (not shown), and a main storage unit such as a RAM (Random Access Memory) that stores data temporarily required when the arithmetic processing unit executes programs (not shown).

[0024] The display unit 20 displays the first display (see Figure 3A ) and the second display (refer to Figure 3B ) are displayed on the display screen 2a of the robot display device 2 in a switchable manner, the first display is based on the three-dimensional computer graphics of the robot 3 or the three-dimensional computer graphics of the robot 3 and its peripheral equipment 4, and the second display is an augmented reality (AR) display of the robot 3 and its peripheral equipment 4 captured by the camera 2b.

[0025] Here, in augmented reality, it is possible to superimpose and display augmented information such as a virtual three-dimensional object on an object in the real space by extracting specific feature points, such as a marker, from an image captured by a camera and aligning their positions. In this embodiment, the marker can also be installed on the robot 3, etc., and an originally existing part can also be used as a marker. For example, the robot 3 itself can also be used as a marker. In addition, the position alignment marker can also be set in the background, and is an object that can provide a fixed point as a reference for position and scale. By capturing such a position alignment marker with a camera, the position and orientation (posture) of the camera can be calculated, and augmented information such as a virtual three-dimensional object can be naturally superimposed on an object in the real space.

[0026] When displaying the first display, the display unit 20 can, for example, display at least one of the tool tip point of the robot 3, the teaching point of the program, the coordinate system of the robot 3, the motion range of each axis of the robot 3, the jog method of the robot 3, the trajectory of the robot 3, the position register, the information of the sensors possessed by the robot 3, and the reachable area of ​​the robot 3 as information related to the robot 3 and its peripheral equipment 4.

[0027] Similarly, when displaying the second display, the display unit 20 can display, for example, the tool tip point of the robot 3, the teaching point of the program, the coordinate system of the robot 3, the motion range of each axis of the robot 3, the jog method of the robot 3, the trajectory of the robot 3, the position register, the information of the sensors possessed by the robot 3, and at least one of the reachable area of ​​the robot 3 as information related to the robot 3 and its peripheral equipment.

[0028] Information related to the three-dimensional computer graphics of the robot 3 used in the first display is pre-acquired from the control device 5 and stored in the robot display device 2. Information related to the three-dimensional computer graphics of the peripheral equipment 4 used in the first display is pre-input into the robot display device 2 by, for example, acquiring from the control device 5, and stored in the robot display device 2. Information on the characteristic points (markers, shapes) of the robot 3 and its peripheral equipment 4 used in the second display is pre-input into the robot display device 2 by, for example, acquiring from the control device 5, and stored in the robot display device 2. Information displayed with the first display or the second display is pre-acquired from the control device 5 and stored in the robot display device 2.

[0029] like Figure 2 As shown, a CPU (not shown) included in the robot display device 2 realizes various functions such as the display unit 20 and the selection unit 21 by executing various programs.

[0030] When the camera 2b is not facing the robot 3 and its peripherals 4, the selection unit 21 enables the first display and displays the first display on the display screen 2a via the display unit 20. If the camera 2b is facing the robot 3 and its peripherals 4, the selection unit 21 enables the second display and displays the second display on the display screen 2a via the display unit 20. Whether the camera 2b is facing the robot 3 and its peripherals 4 is determined based on whether the robot 3 and its peripherals 4 in real space are detected from the image capture results of the camera 2b.

[0031] Return to Figure 1The robot 3 and its peripherals 4 are displayed on the display screen 2a via the display unit 20 of the robot display device 2. The robot 3 operates under the control of the control device 5. The peripherals 4 are devices attached to the robot 3, such as conveyors. The control device 5 controls the robot 3 and its peripherals 4 and communicates with the robot display device 2.

[0032] Next, use Figure 3A A case where the robot display device 2 displays the first display on the display screen 2 a via the display unit 20 will be described. Figure 3A 1 is a schematic diagram of the robot display device 2 according to the present embodiment, in a state where the first display is displayed.

[0033] like Figure 3A As shown, when the camera 2b is not facing the robot 3 and its peripheral equipment 4, the selection unit 21 enables the first display based on three-dimensional computer graphics and displays the first display on the display screen 2a via the display unit 20. At this time, as described above, the display unit 20 displays at least one of the following information: the tool tip point of the robot 3, the program teaching point, the coordinate system of the robot 3, the motion range of each axis of the robot 3, the jog method of the robot 3, the trajectory of the robot 3, the position register, the information of the sensors possessed by the robot 3, and the reachable area of ​​the robot 3.

[0034] Next, use Figure 3B A case where the robot display device 2 according to the present embodiment displays the second display on the display screen 2 a via the display unit 20 will be described. Figure 3B 1 is a schematic diagram of the robot display device 2 according to the present embodiment, showing a state in which the second display is displayed.

[0035] like Figure 3B As shown, when the camera 2b is directed toward the robot 3 and its peripheral equipment 4, the selection unit 21 enables the second display using augmented reality and displays the second display on the display screen 2a through the display unit 20. At this time, as described above, the tool tip point of the robot 3, the program teaching point, the coordinate system of the robot 3, the motion range of each axis of the robot 3, the jog method of the robot 3, the trajectory of the robot 3, the position register, the information of the sensor possessed by the robot 3, and the reachable area of ​​the robot 3 are all displayed on the display unit 20. Figure 3B In the example shown, the user coordinate system and the motion range of each axis are displayed.

[0036] Next, use Figure 4 The operation of the robot display device 2 according to this embodiment will be described. Figure 4 This is a flowchart illustrating the operation of the robot display device 2 .

[0037] like Figure 4 As shown, first, the selection unit 21 determines whether the camera 2b is directed toward the robot 3 and its peripheral equipment 4 (step S10).

[0038] When the camera 2b is directed toward the robot 3 and its peripheral equipment 4 (if the determination in step S10 is "YES"), the selection unit 21 enables the second display based on augmented reality and displays the second display on the display screen 2a via the display unit 20 (step S20).

[0039] If the camera 2b is not facing the robot 3 and its peripheral equipment 4 (if the determination in step S10 is "No"), the selection unit 21 enables the first display based on virtual three-dimensional computer graphics and displays the first display on the display screen 2a via the display unit 20 (step S30). The above steps complete the present process.

[0040] According to this embodiment, the following effects are achieved.

[0041] The robot display device 2 according to this embodiment is provided with a display unit 20 that displays a first display and a second display in a switchable manner. The first display is a display based on three-dimensional computer graphics of the robot 3 or three-dimensional computer graphics of the robot 3 and its peripherals 4, and the second display is a display using augmented reality of the robot 3 and its peripherals 4 captured by the camera 2b. Furthermore, a selection unit 21 is provided that enables the first display and displays it on the display unit 20 when the camera 2b is not facing the robot 3 and its peripherals 4, and enables the second display and displays it on the display unit 20 when the camera 2b is facing the robot 3 and its peripherals 4.

[0042] Thus, when camera 2b is not facing the robot 3 and its peripherals 4, a first display based on three-dimensional computer graphics of the robot 3 or three-dimensional computer graphics of the robot 3 and its peripherals 4 can be performed. Furthermore, when camera 2b is facing the robot 3 and its peripherals 4, a second display using augmented reality of the robot 3 and its peripherals 4 captured by camera 2b can be performed. In other words, a robot display device capable of automatically switching between augmented reality display and three-dimensional computer graphics display can be provided.

[0043] Furthermore, the robot display device 2 according to this embodiment is configured to display at least one of the tool tip point of the robot 3, the program teaching point, the coordinate system of the robot 3, the motion range of each axis of the robot 3, the jog method of the robot 3, the trajectory of the robot 3, the position register, the information of the sensors possessed by the robot 3, and the reachable area of ​​the robot 3 as information related to the robot 3 and its peripheral devices 4 on the first display and the second display. This enables intuitive understanding and confirmation.

[0044] The present disclosure is not limited to the above-described embodiment, and various changes and modifications can be made.

[0045] For example, in the above embodiment, the display device for industrial machinery of the present disclosure is applied to the robot display device 2 , but the present disclosure is not limited thereto and can be applied to display devices for industrial machinery such as various machine tools in addition to the robot 3 .

[0046] Description of Reference Numerals

[0047] 1: Robot system; 2: Robot display device (display device of industrial machinery); 2a: Display screen; 3: Robot (industrial machinery); 4: Peripheral equipment; 5: Control device; 20: Display unit; 21: Selection unit.

Claims

1. A display device for industrial machinery, comprising: Camera; a display unit that switches between a first display and a second display, wherein the first display is a display based on three-dimensional computer graphics of the industrial machine or three-dimensional computer graphics of the industrial machine and its peripheral equipment, and the second display is a display using augmented reality of the industrial machine and its peripheral equipment captured by the camera; and a selection unit, wherein when the camera is not facing the industrial machine and its peripheral equipment, the selection unit enables the first display and displays the first display on the display unit, and when the camera is facing the industrial machine and its peripheral equipment, the selection unit enables the second display and displays the second display on the display unit, in, By extracting specific feature points from the image captured by the camera and aligning their positions, augmented information of a three-dimensional object is superimposed and displayed on an object in a real space.

2. The display device for industrial machinery according to claim 1, wherein: The display unit displays at least one of the tool tip point, the program teaching point, the coordinate system, the motion range of each axis, the jog method, the trajectory, the position register, the information of the sensor possessed by the industrial machinery, and the reachable area of ​​the industrial machinery as information related to the industrial machinery and its peripheral equipment in the first display and the second display.

Images