YesUp Rearview AR

Abstract

The present invention provides a head-mounted display device that naturally and safely presents the user's rearward field of vision in accordance with their eye movements. [Solution] A head-mounted display device is equipped with a rear-view camera at the end of the temple, and the acquired image is overlaid on the user's field of view. The display is automatically controlled according to the direction of gaze, and the image of the user's rear field of view is presented only when the user looks upwards, without obstructing the forward view. Furthermore, if the user is checking AR information displayed in a position that they look up at, the display of unnecessary rear images can be avoided. The display position and transparency of the rear field of view image can be adjusted in the settings, and it can be used day or night or in low-light environments, allowing the user to quickly and intuitively check the status of their rear field of view when needed.

Description

Technical Field

[0003] ,

[0002] , , , , , , , , , , ,

[0001] The present invention relates to a head-mounted display device, and particularly to a technology for naturally and safely presenting the situation in the rear view area of a user according to the line-of-sight movement. In this specification, the "head-mounted display device" means a device that is mounted on the head of a user and can display video information in the user's field of view. It includes smart glasses mounted on a spectacle frame, AR glasses, VR goggles, helmet-integrated display devices, and the like.

Background Art

[0002] Conventional smart glasses and AR devices are known to include a camera for acquiring surrounding video and presenting it to the user. Also, a configuration for imaging the rear view area has been proposed. <0000​​​​​​​​​​​​​​​​​​​​​​​​Another challenge is to enable control that prevents unnecessary rearward image display when the user is viewing AR information displayed at a position where they are looking upwards. Means for solving the problem: The present invention provides a head-mounted display device that allows the user to check the situation in the rear-view area without obstructing the user's forward view, and enables the user to grasp images of the rear-view area with only natural eye movements. A head-mounted display device according to one aspect of the present invention comprises at least the following components. • Imaging means for capturing images of the user's rearward field of view. • Eye-line detection means for detecting the direction of the user's gaze • Display means for superimposing images onto the real field of view • Control means for controlling the presentation state of the image based on the direction of gaze. The control means displays the image of the rearward view area captured by the imaging means to the display means only when the gaze detection means determines that the gaze is directed upward at a predetermined angle or more. If the user's gaze is directed in any direction other than upwards, the display can be promptly hidden, or its position, size, and transparency can be adjusted to ensure it does not obstruct the user's forward field of vision. Specific examples include moving the rear view to the edge of the screen or outside the frame, shrinking it to an extremely small size like a dot, or increasing its transparency to present it in a way that makes it almost invisible. Various known technologies can be applied as means for detecting gaze in a head-mounted display device, including a camera system that detects infrared light sources and eyeball reflections, a corneal reflection pattern analysis system, an image analysis system for the pupil center position, and an optical sensor system. For imaging, a camera mounted on the rear end of the temple can be used. The display method can utilize optical see-through display elements or the like, and does not obstruct the actual field of view. Examples of embodiments of the present invention: Preferred embodiments of the present invention will be described below. However, the present invention is not limited to these embodiments. (1) Head-mounted body of head-mounted display device The head-mounted display device has a spectacle-type frame and can be worn while ensuring normal vision. Other head-mounted structures, such as those integrated with a helmet, may also be used. (2) Imaging means of head-mounted display device The imaging means (camera, etc.) for capturing images of the rearward field of view is positioned facing backward on the rear end of the temple of the frame of the head-mounted display device, and acquires images of the user's rearward field of view. It can be configured as a monocular or compound (stereo) system, and wide-angle imaging using a fisheye lens is also possible. (3) Eye-gaze detection means of head-mounted display device A head-mounted display device has gaze detection means positioned around the lens to detect the user's gaze direction. Examples of such methods include the following: (a) Infrared light illumination + reflection detection camera method (b) Angle estimation method using sensing lenses (c) Electrooculography (EOG) detection method These methods may be used in combination to improve detection accuracy. (4) Display means for head-mounted display device The head-mounted display device incorporates optical see-through display elements and other components into the lens section. The image from the user's rearward field of view is superimposed at the top or edge of the field of view. This can be applied to both monocular and binocular displays. (5) Control means for head-mounted display device The control means for the head-mounted display device displays the image of the rearward field of view captured by the imaging means on the display means only when the gaze detection means directs the gaze upward at a predetermined angle or greater. Furthermore, if the gaze is not directed upwards, the presentation can be controlled to avoid obstructing the forward field of vision. (6) Example of operation of a head-mounted display device The rear view is displayed at the top of the field of view only when the user looks up, and disappears immediately when they look back. Furthermore, if the viewer is looking in any direction other than upwards, the image can be presented in a way that does not obstruct the forward view, such as by completely hiding it, moving it to the edge of the screen, shrinking it to a tiny size, or increasing its transparency to make it almost invisible. This improves safety and convenience even in situations where forward attention is required. Effects of the present invention: According to the present invention, the following effects are achieved by linking gaze detection and image presentation control. (1) Because the image of the user's rearward field of view can be displayed only when necessary, it does not impair the actual visibility of what is in front, and thus enhances safety. (2) The user can grasp the situation in the rear field of view by simply turning their gaze slightly upward, eliminating the need for head movements or manual operation, thus reducing the burden on the user. (3) Even when the user is looking up at the AR information displayed, unnecessary rear view display can be avoided, thus ensuring safety while preventing discomfort or confusion. (4) Because the user can intuitively check the situation in the rear field of view when needed, it is useful in environments where surrounding surveillance is required, such as cycling, walking assistance, security, and work sites. Industrial applicability: The head-mounted display device of the present invention can present images of the rearward field of view according to the user's gaze direction, making it applicable to a variety of industrial fields where it is necessary to safely and efficiently grasp the surrounding situation. For example, it is useful in situations where it is necessary to quickly grasp the situation in the user's rearward field of vision, such as rearward monitoring while cycling or walking, vehicle driving assistance, monitoring of following vehicles and transport equipment in logistics sites, surrounding area monitoring in construction sites and factories, and expanding the field of vision for security guards and disaster relief workers. Furthermore, it can be applied to consumer products that require enhanced vision, such as those for sports training, gaming and entertainment, tourist information, and disability assistance. Therefore, the present invention can be used in a wide range of fields in industry and daily life, and has high industrial value.

Claims

1. A head-mounted display device worn by the user, An imaging means for capturing images of the user's rearward field of view. A gaze detection means for detecting the direction of the user's gaze, A display means that superimposes the captured image onto the user's field of view. Control means for controlling the presentation state of the captured image based on the aforementioned line of sight direction, Equipped with, The control means is If the user's gaze is determined to be outside the primary area of ​​forward vision and intended to check the rearward vision area, A head-mounted display device characterized by displaying the captured image within the user's field of vision.

2. The control means is When the control means determines that the gaze state is a gaze state intended for checking the rearward field of view, based on the fact that the gaze direction is directed upward at a predetermined angle or more, The head-mounted display device according to claim 1, characterized in that the captured image is displayed in the upper region within the user's field of vision.

3. The control means is If it is determined that the aforementioned gaze state is not a gaze state intended for checking the rearward field of view, Stop or hide the display of the captured image. Or, The captured image is controlled by controlling at least one of the display position, display size, transparency, display brightness, or display update. The head-mounted display device according to claim 1 or 2, characterized in that it displays in a manner that does not obstruct the forward view.