Information display apparatus

Abstract

To provide an information display apparatus which displays visual information at a position outside a visual field of a wearer, does not block the visual field of the wearer, and prevents the work of the wearer from being hindered.SOLUTION: A display apparatus includes: a display body 10 having a transparent display 11 capable of displaying visual information in a visual field of a wearer in a state of being mounted on a head of the wearer; a line-of-sight detection sensor 31 capable of detecting a line of sight of the wearer; and an arithmetic unit 22 for determining a position where the visual information is displayed on the transparent display 11 based on the detected line of sight. The calculation unit 22 determines a visual field of the wearer based on the line of sight, and determines a position outside the field of view and inside the field of view of the wearer as a position at which the visual information is displayed on the transparent display 11.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a user-wearable information display device. [Background technology]

[0002] 2. Description of the Related Art There is known an information display device in which a display main body displays various necessary information while the display main body is worn by a user on the head.

[0003] Patent document 1 discloses a program that generates a virtual space, identifies a user's reference line of sight, identifies a main screen from among multiple display objects, displays it in the virtual space in association with the reference line of sight, and outputs it to a head-mounted display.

[0004] Patent Document 2 discloses a display control device that sets one of a plurality of display modes in which the display direction of visual information according to the display position on the display is defined, and displays the visual information on the display in a direction according to the position information acquired by the set display mode. Specifically, a spherical image is displayed three-dimensionally on the display, and the position is adjusted. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-45294 [Patent Document 2] International Publication No. 2022 / 190735 Summary of the Invention [Problem to be solved by the invention]

[0006] Information display devices worn by a user are sometimes used to display information that assists the wearer in performing some task. That is, such information display devices are used in so-called augmented reality, where information is superimposed on real space. However, in conventional technologies such as those described in the above-mentioned patent documents, when superimposing information on real space, visual information is displayed in the field of view of the user who is viewing the work object, which may cause the visual information to obscure the work object and interfere with the work.

[0007] The present invention has been made in consideration of the above-mentioned circumstances, and its purpose is to provide an information display device that can prevent the work of a wearer wearing the display main body from being interfered with. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, an information display device according to the present invention has the following features. a display device main body having a display unit capable of displaying visual information in the wearer's field of vision when worn on the wearer's head; a gaze detection unit capable of detecting the gaze of the wearer; a calculation unit that determines a position at which the visual information is to be displayed on the display unit based on the detected line of sight, The calculation unit determines a field of view of the wearer based on the line of sight, and determines a position outside the field of view but within the field of view of the wearer as a position at which the visual information is to be displayed on the display unit. Information display device. [Effects of the Invention]

[0009] According to the present invention, visual information is displayed at a position outside the field of view of the wearer, so the field of view of the wearer is not obstructed and the wearer's work is not hindered.

[0010] The present invention has been briefly described above. The details of the present invention will become clearer by reading the following detailed description of the invention (hereinafter referred to as "embodiments") with reference to the accompanying drawings. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating a general configuration of a wearable device that is an information display device according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing a schematic configuration of a wearable device according to an embodiment. [Figure 3] FIG. 3 is a schematic diagram showing the position of visual information displayed on the transparent display. DETAILED DESCRIPTION OF THE INVENTION

[0012] Specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0013] 1 is a schematic diagram showing the general configuration of a wearable device 1, which is an information display device according to an embodiment. The wearable device 1 of this embodiment is a smart glass that is worn on the head of a human body and displays various information in the wearer's field of vision, thereby realizing an information display device. The wearable device 1 of this embodiment has a gaze detection function, a voice recognition function, an information display function, etc., and detects the wearer's gaze and, based on the gaze, displays visual information such as images, text, figures, videos, etc. on the surface of the smart glass that corresponds to the wearer's field of vision.

[0014] Specifically, as shown in FIG. 1, the wearable device 1 includes a display main body 10, a processing unit 20, a gaze detection sensor 31, an acceleration sensor 32, a microphone 33, a moving object detection sensor 34, and a communication unit 35.

[0015] The display main body 10 is a head-mounted display (HMD) type eyeglass-type display device that can be worn on the head of a human body. When worn on the head of a human body, the display main body 10 can display information in the field of view of the wearer. Specifically, the display main body 10 includes a transparent display 11 and a frame 12.

[0016] The transparent display 11 is capable of displaying visual information and constitutes a visual information display unit in the wearable device 1. The transparent display 11 has such high transparency that the background can be seen through the transparent display 11. The transparent display 11 is constituted by, for example, a liquid crystal display, an organic EL display, or the like. Here, the transparent display 11 is constituted by a right display 11R corresponding to the wearer's right eyeball, and a left display 11L corresponding to the wearer's left eyeball. The transparent display 11 may be translucent or colored.

[0017] The frame 12 is a structure that is worn on the head of the human body and supports the transparent display 11 in the wearer's field of vision, and is configured including strings, a frame, etc. The frame 12 supports the right display 11R in a position facing the wearer's right eyeball, and supports the left display 11L in a position facing the wearer's left eyeball. The frame 12 is equipped with various components that constitute the wearable device 1, such as a processing unit 20, a gaze detection sensor 31, an acceleration sensor 32, a microphone 33, and a moving object detection sensor 34.

[0018] With the above configuration, the display main body 10 can display visual information on the transparent display 11 in a state where the frame 12 is worn on the head of a human body, by superimposing the visual information on the transparent display 11 on a background through the transparent display 11, thereby displaying the visual information in the field of view of the wearer. The visual information displayed in the field of view of the wearer by the transparent display 11 includes, for example, images, characters, text, etc. Note that the transparent display 11 is not limited to the above and may be configured as, for example, a retinal projection display or the like.

[0019] The processing unit 20 is a processing circuit that realizes various processing functions in the wearable device 1. The processing unit 20 is realized by a processor, which is an arithmetic processing device such as a CPU (Central Processing Unit). Details of the processing unit 20 will be described later.

[0020] The gaze detection sensor 31 constitutes a gaze detection unit capable of detecting the gaze of the wearer. The gaze detection sensor 31 is provided, for example, on the central string portion of the frame 12. The gaze detection sensor 31 is configured, for example, with a camera capable of capturing images toward the wearer, Lidar (Light Detection And Ranging), or the like.

[0021] Gaze detection sensor 31 can detect the gaze of the wearer by, for example, detecting the pupils of the wearer's eyes and the Purkinje image (light reflected on the surface of the cornea) using the pupil-corneal reflex method. Gaze detection sensor 31 may also detect the direction of the pupils of both of the wearer's eyes and calculate the intersection of the directions of the two pupils as the gaze. Gaze detection sensor 31 may also be configured as an electrooculography sensor built into the central chord of frame 12, which detects the wearer's gaze by detecting the potential difference around the eyes caused by eye movement. Note that when wearable device 1 is used in a limited space indoors, gaze detection sensor 31 may be installed indoors instead of on frame 12.

[0022] The acceleration sensor 32 constitutes an acceleration detection unit that detects acceleration acting on the display device main body 10 in accordance with the movement of the display device main body 10. In this example, the acceleration sensor 32 mainly detects acceleration that occurs in accordance with the movement of the display device main body 10 linked to the wearer changing the orientation of their head.

[0023] The microphone 33 constitutes a voice input unit that can input the voice of the wearer or the like. The microphone 33 is a directional microphone that can change the sound collection direction and directional angle. Here, the directional angle represents the angular range in which sound can be collected, with a smaller angle representing a narrower angle range and a larger angle representing a wider angle range. The microphone 33 can be, for example, a two-channel speaker-tracking microphone array.

[0024] The moving object detection sensor 34 constitutes a moving object detection unit capable of detecting the movement of an object. In this example, the object includes the wearer's finger, an instrument, a tool, etc., which are used to instruct changes in the position or size of visual information, and detects changes in a specific shape, direction, color, light, movement time, and distance.

[0025] The communication unit 35 is configured to be able to receive data from an external communication terminal 40. The communication unit 35 may also transmit data to the communication terminal 40. The communication terminal 40 may be a general electronic device such as a mobile phone, smartphone, tablet, or personal computer, and may have an application installed thereon that enables operation of the wearable device 1. The communication terminal 40 may also be a controller dedicated to the wearable device 1.

[0026] The camera device 36 constitutes a video / image input device, and inputs video or images that overlap the wearer's line of sight, including the wearer's field of vision and field of view. The calculation unit 22 may adjust at least one of the display position and size of the visual information based on the information input to the camera device 36.

[0027] Next, the processing unit 20 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing a schematic configuration of the wearable device 1 according to the embodiment. As shown in Fig. 2, the processing unit 20 includes a control unit 21, a calculation unit 22, and a storage unit 23. The control unit 21 is a main processing circuit in the processing unit 20 that realizes various processing functions of the wearable device 1. The calculation unit 22 is a calculation device that determines the position at which visual information is displayed on the transparent display 11 based on the line of sight of the wearer detected by the line of sight detection sensor 31. The function of the calculation unit 22 may be realized by the control unit 21. Details of the calculation unit 22 will be described later.

[0028] The storage unit 23 is a memory circuit that stores various data (information). The storage unit 23 stores, for example, programs that enable the wearable device 1 to realize various functions. The storage unit 23 also stores various data such as data required for various processes in the processing unit 20 and data to be displayed on the transparent display 11. The various data in the storage unit 23 is read out as needed by the control unit 21, etc. The storage unit 23 may be realized by a cloud server, etc., connected to the wearable device 1 via a network.

[0029] 3 is a schematic diagram showing the position of visual information to be displayed on the transparent display 11. In this embodiment, the calculation unit 22 determines the field of view of the wearer based on the line of sight detected by the line of sight detection sensor 31, and determines a position outside the field of view but within the field of view of the wearer as the position where the visual information is to be displayed on the transparent display 11. The field of view is the position (range) to which the wearer directs their gaze, and the field of view is the range that the wearer can see. In other words, the field of view is wider than the visual field.

[0030] The range of the field of view based on the line of sight can be set, for example, to a predetermined angular range from the detected line of sight, or may be determined by other methods.

[0031] 3, the calculation unit 22 determines a position outside the field of view but within the field of view of the wearer as the position where the visual information is to be displayed, and the control unit 21 displays the visual information based on that position. As a result, the visual information is displayed at a position that does not overlap the field of view of the wearer.

[0032] The wearer performs various tasks while wearing the display device main body 10. Here, the wearer performs the tasks while gazing at the task object that exists in the real space through the transparent display 11, and the transparent display 11 displays visual information that assists the task, thereby improving the efficiency of the task. The visual information that assists the task includes the processing procedure of the task object, a pass / fail judgment of the task procedure, etc.

[0033] If visual information were displayed within the field of view, it could block the wearer's line of sight from the processing target, hindering their work. With wearable device 1 of this embodiment, the visual information is displayed at a position outside the wearer's field of view but within the field of view, so it does not block the wearer's field of view and prevents the wearer's work from being hindered.

[0034] When the visual information is displayed outside the field of view, the visual information may not be detailed text information, but may be simple figures, colors, etc. For example, wearable device 1 may be linked to a device that verifies assembly procedures based on input from a video / image input device, and may display a blue mark if the assembly procedure for the work object is correct, or a red mark if it is incorrect.

[0035] When the gaze detection sensor 31 detects a movement of the gaze, the calculation unit 22 may determine at least one of the position and size of the visual information, as shown by the arrow in Fig. 3. Since a movement of the gaze causes a movement of the field of view, the calculation unit 22 basically re-determines the position of the visual information and changes the position of the visual information. Furthermore, when the wearer intentionally moves the gaze extremely upward or downward, the calculation unit 22 may re-determine the size of the visual information and change the size of the visual information.

[0036] For example, if the wearer shifts their gaze significantly upward, the visual information may be enlarged, and if they shift their gaze significantly downward, the visual information may be reduced. Also, if the wearer shifts their gaze significantly to the left or right, the visual information may be moved from the edge of the field of view to an arbitrary position closer to the center that does not obstruct the field of view. This allows the wearer to use the movement of their gaze to change at least one of the position and size of the visual information according to the situation.

[0037] The calculation unit 22 may determine at least one of the position and size of the visual information based on the movement of the display device main body 10 detected by the acceleration sensor 32. Specifically, the calculation unit 22 can determine at least one of the position and size of the visual information based on the movement of the display device main body 10 linked to a change in the orientation of the wearer's head.

[0038] For example, when the wearer tilts their head upward, the visual information may be enlarged, and when they tilt their head downward, the visual information may be reduced. Furthermore, when the wearer significantly moves their head left or right, the visual information may be moved from the left or right edge of the field of view to any position closer to the center that does not obstruct the field of view. Furthermore, when the wearer significantly moves their head up or down, the visual information may be moved from the top or bottom edge of the field of view to any position closer to the center that does not obstruct the field of view. In this way, the wearer can use the movement of their head to change at least one of the position and size of the visual information according to the situation.

[0039] The calculation unit 22 may determine at least one of the position and size of the visual information based on voice input to the microphone 33. For example, when the wearer or the like inputs voice such as "left," "right," "up," or "down," the calculation unit 22 re-determines the position of the visual information and changes the position of the visual information. For example, when the wearer or the like inputs voice such as "enlarge," "reduce," "bigger," or "smaller," the calculation unit 22 re-determines the size of the visual information and changes the size of the visual information. In this way, the wearer can use their own or another voice to change at least one of the position and size of the visual information according to the situation.

[0040] The calculation unit 22 may determine at least one of the position and size of the visual information based on the movement of an object detected by the moving object detection sensor 34. For example, when the moving object detection sensor 34 detects an object such as the wearer's finger that instructs a change in the position or size of the visual information, the calculation unit 22 can re-determine at least one of the position and size of the visual information. This allows the wearer to use their finger or other indicator to change at least one of the position and size of the visual information according to the situation.

[0041] The calculation unit 22 may determine at least one of the position and size of the visual information based on data received from the communication terminal 40. For example, when the operator inputs an operation to change the position or size of the visual information into the communication terminal 40, the communication unit 35 receives data corresponding to this operation input, and the calculation unit 22 can re-determine at least one of the position and size of the visual information. This allows the wearer to use the communication terminal 40 to change at least one of the position and size of the visual information according to the situation.

[0042] The gaze detection sensor 31 may be configured to be able to detect the gaze distance. In this case, the calculation unit 22 may determine at least one of the position and size of the visual information based on the gaze distance detected by the gaze detection sensor 31. This allows the wearer to change at least one of the position and size of the visual information in accordance with the gaze distance. If the gaze detection sensor 31 is a LiDAR, the gaze distance can be easily detected.

[0043] The processing unit 20 does not necessarily have to be provided in the display device main body 10, but may be provided in an external device such as the communication terminal 40. When the communication terminal 40 has the processing unit 20, the communication terminal 40 may have a function to manipulate the visual information. For example, if the visual information is not displayed in an appropriate position as in Fig. 3, the wearer may operate the communication terminal 40 to change the position, size, etc. of the visual information.

[0044] Furthermore, the calculation unit 22 may display visual information within the field of view of the wearer. For example, the calculation unit 22 may display visual information so as to cover the field of view when the wearer is not performing any work or when a predetermined operation is input.

[0045] Two or more pieces of visual information may be displayed on the transparent display 11. In this case, one piece of visual information may display buttons for zooming in, zooming out, etc., so that the button can be selected by the line of sight, and the size of the other piece of visual information may be adjusted.

[0046] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0047] Here, the features of the above-described information display device according to the present invention will be briefly summarized and listed below in [1] to [8].

[0048] [1] A display device main body (10) having a display unit (transparent display 11) capable of displaying visual information in the wearer's field of vision when worn on the wearer's head; a gaze detection unit (gaze detection sensor 31) capable of detecting the wearer's gaze; a calculation unit (22) that determines a position at which the visual information is to be displayed on the display unit based on the detected line of sight, The calculation unit determines a field of view of the wearer based on the line of sight, and determines a position outside the field of view but within the field of view of the wearer as a position at which the visual information is to be displayed on the display unit. Information display device.

[0049] According to the information display device having the configuration [1] above, visual information is displayed at a position outside the field of view of the wearer, so the field of view of the wearer is not obstructed and the wearer's work is not hindered.

[0050] [2] When the gaze detection unit detects a movement of the gaze, the calculation unit determines at least one of a position and a size of the visual information. The information display device according to [1] above.

[0051] According to the information display device having the above configuration [2], the wearer can use the movement of his / her line of sight to change at least one of the position and size of the visual information according to the situation.

[0052] [3] Further comprising an acceleration detection unit (acceleration sensor 32) capable of detecting the movement of the display device main body, the calculation unit determines at least one of a position and a size of the visual information based on the movement of the display device body detected by the acceleration detection unit. The information display device according to [1] above.

[0053] According to the information display device having the configuration [3] above, the wearer can change at least one of the position and size of the visual information according to the situation by, for example, moving his or her head.

[0054] [4] Further provided with a voice input unit (microphone 33) capable of inputting voice, the calculation unit determines at least one of a position and a size of the visual information based on the voice input to the voice input unit. The information display device according to [1] above.

[0055] According to the information display device configured as described above in [4], the wearer can use his or her own voice or another voice to change at least one of the position and size of the visual information according to the situation.

[0056] [5] Further comprising a moving object detection unit (moving object detection sensor 34) capable of detecting the movement of an object; the calculation unit determines at least one of a position and a size of the visual information based on the movement of the object detected by the moving object detection unit. The information display device according to [1] above.

[0057] According to the information display device configured as described above in [5], the wearer can use his / her finger or other indicator to change at least one of the position and size of the visual information according to the situation.

[0058] [6] Further comprising a communication unit (35) capable of receiving data from an external communication terminal; the calculation unit determines at least one of a position and a size of the visual information based on data received from the communication terminal. The information display device according to [1] above.

[0059] According to the information display device having the configuration [6] above, the wearer can use the communication terminal to change at least one of the position and size of the visual information according to the situation.

[0060] [7] The line-of-sight detection unit is capable of detecting a line-of-sight distance, the calculation unit determines at least one of a position and a size of the visual information based on the line-of-sight distance detected by the line-of-sight detection unit. The information display device according to [1] above.

[0061] According to the information display device having the configuration [7] above, the wearer can change at least one of the position and size of the visual information in accordance with the line-of-sight distance.

[0062] [8] A video / image input device (camera device 36) is provided for inputting a video or image that overlaps the wearer's line of sight, including the wearer's field of vision and field of view, The calculation unit (22) adjusts at least one of the display position and size of the visual information based on the information input to the video / image input device. The information display device according to [1] above. [Explanation of symbols]

[0063] 1. Wearable devices (information display devices) 10 Display unit 11 Transparent display (display section) 20 Processing section 21 Control section 22 Arithmetic section 23 Memory section 31 Gaze detection sensor (gaze detection unit) 32 Acceleration sensor (acceleration detection unit) 33 Microphone (audio input section) 34 Moving object detection unit (moving object detection sensor) 35 Communications Department 36 Camera equipment (video and image input devices) 40 Communication terminal

Claims

1. a display device main body having a display unit capable of displaying visual information in the wearer's field of vision when worn on the head of a human body; a gaze detection unit capable of detecting the gaze of the wearer; a calculation unit that determines a position at which the visual information is to be displayed on the display unit based on the detected line of sight, The calculation unit determines a field of view of the wearer based on the line of sight, and determines a position outside the field of view but within the field of view of the wearer as a position at which the visual information is to be displayed on the display unit. Information display device.

2. the calculation unit determines at least one of a position and a size of the visual information when the gaze detection unit detects a movement of the gaze. The information display device according to claim 1 .

3. further comprising an acceleration detection unit capable of detecting the movement of the display device main body; the calculation unit determines at least one of a position and a size of the visual information based on the movement of the display device body detected by the acceleration detection unit. The information display device according to claim 1 .

4. further comprising a voice input unit capable of inputting voice; the calculation unit determines at least one of a position and a size of the visual information based on the voice input to the voice input unit. The information display device according to claim 1 .

5. Further comprising a moving object detection unit capable of detecting the movement of an object, the calculation unit determines at least one of a position and a size of the visual information based on the movement of the object detected by the moving object detection unit. The information display device according to claim 1 .

6. further comprising a communication unit capable of receiving data from an external communication terminal; the calculation unit determines at least one of a position and a size of the visual information based on data received from the communication terminal. The information display device according to claim 1 .

7. the line-of-sight detection unit is capable of detecting a line-of-sight distance, the calculation unit determines at least one of a position and a size of the visual information based on the line-of-sight distance detected by the line-of-sight detection unit. The information display device according to claim 1 .

8. a video / image input device for inputting a video or image that overlaps the wearer's line of sight, including the wearer's field of vision and field of view; the calculation unit adjusts at least one of the display position and size of the visual information based on information input to the video / image input device. The information display device according to claim 1 .

Images