Smart Zoom VR / AR Glasses

Abstract

We provide glasses-type displays with eye-tracking and pressure sensor-based zoom control. [Solution] The glasses-type display of the present invention combines a pressure sensor and a touch sensor located on the frame with a gaze detection means to realize zoom operation based on the direction of gaze. It enables magnification adjustment according to the strength of the pressure, zoom out by swiping, zoom lock by maintaining contact, and reset by releasing the finger, and also has a function to prevent malfunctions.

Description

[Technical Field]

[0001] The present invention relates to a glasses-type display device that enables intuitive zoom operation in the technical fields such as VR (virtual reality), AR (augmented reality), and MR (mixed reality). [Background Art]

[0002] In conventional glasses-type displays, intuitive zoom operation that combines touch sensors, pressure sensors, and line-of-sight detection means to perform continuous or stepwise zoom adjustment according to pressure intensity or zoom adjustment via a swipe motion has not been realized. Description of the Invention

[0003] The glasses-type display device of the present invention can be considered to have the following configuration as an example. However, the present invention is not limited thereto, and other combinations of sensors and algorithms are also possible. The glasses-type display device of the present invention comprises touch sensors and pressure sensors built in or disposed on the surface on both the upper side and lower side of the frame portion (rim portion and / or temple portion) when the glasses are worn, and realizes zoom-in and zoom-out when a user performs a pinching motion on the frame or a swipe operation. Furthermore, by combining a line-of-sight detection means, intuitive zoom control based on the direction of the user's line of sight is enabled. The glasses-type display device of the present invention is characterized in that it uses touch sensors and pressure sensors built in or disposed on the surface on both the upper side and lower side of the frame portion (rim portion and / or temple portion) when the glasses are worn, and a line-of-sight detection means for detecting a user's line of sight, and performs zoom control on display content in the line-of-sight direction based on the finger contact state and pressure intensity detected by the aforementioned touch sensors and pressure sensors. In addition, fine zoom operation (fine adjustment mode), large zoom operation (quick zoom mode), and the like can be realized according to the magnitude of the pinching pressure. Furthermore, by utilizing the aforementioned touch sensor, zoom-out via a swipe operation, zoom fixing by maintaining contact, and resetting by releasing a finger are enabled. Example Embodiments

[0004] The eyeglass-type display device of the present invention can be thought of in the following embodiments. (1) Basic configuration • Built-in or exposed touch sensors and pressure sensors are placed on both the upper and lower sides of the frame (rim and / or temple) when the glasses are worn to detect user operation. • Incorporates gaze detection means (eye-tracking sensor, infrared camera, pupil tracking sensor, etc.) to detect the user's gaze direction. • The aforementioned touch sensor and pressure sensor are linked with the gaze detection means to control the zoom of the displayed content in the direction of the user's gaze. (2) Operation flow of zoom operation • Zoom in (magnification): When the user pinches both the upper and lower parts of the frame while wearing the glasses, the pressure sensor detects the strength of the pinch and magnifies the object in the line of sight. The stronger the pressure, the greater the zoom magnification. • Zoom out (reduce): When the touch sensor detects that the user's finger is in contact with either the upper or lower part of the frame when the glasses are being worn, and also detects a horizontal swipe operation along the frame (or left-right if located on the temple), it reduces the zoom magnification in accordance with the swipe operation. • Zoom lock: The touch sensor detects that the user's finger is in contact with either the upper or lower part of the frame when the glasses are worn, and if this contact continues for a predetermined time without any swiping operation, the zoom state is locked. • Zoom Reset: When the touch sensor and pressure sensor detect that finger contact has ceased, the zoom state is reset. (3) Detailed configuration of the sensor The aforementioned pressure sensor may employ, but is not limited to, capacitive, piezoresistive, piezorelectric, optical, or MEMS methods. The aforementioned touch sensor can employ, but is not limited to, capacitive, optical, pressure-sensing (resistive), ultrasonic, or proximity sensor technologies. The aforementioned gaze detection means can employ, but is not limited to, an infrared camera system, a pupil tracking system, a gaze estimation system utilizing machine learning, an electrophysiological gaze detection (EOG) system, or a laser-based gaze tracking system. (4) Detailed instructions on how to use the device The user touches the touch sensors and pressure sensors, which are built into or exposed on both the upper and lower parts of the frame (rim and / or temple) when wearing the glasses, with their fingers and applies pressure to the frame in an upward and downward direction. The aforementioned pressure sensor detects this operation and adjusts the zoom magnification according to the strength of the pressure. The aforementioned touch sensor detects when the user's finger is in contact with either the upper or lower part of the frame when the glasses are worn, and by detecting a swipe operation or the user holding the glasses without a swipe operation, it becomes possible to perform zoom out and zoom lock operations. • Because it uses the user's line of sight as a reference, zooming in and out is possible while maintaining a natural field of view. • If the system detects that the user has not touched the touch sensor or pressure sensor for a predetermined period of time, the zoom state is reset. To prevent malfunctions, for example, it is possible to immediately perform a zoom reset when the touch sensor and pressure sensor detect the loss of contact. However, to prevent malfunctions, it is conceivable that the system should not perform a reset for short periods of contact loss, for example, so that the touch sensor and pressure sensor do not mistakenly interpret temporary loss of contact (accidental loss of finger contact). A reset is performed if contact is lost continuously for a predetermined period of time (e.g., 500 milliseconds or more). Other embodiments are also conceivable. This predetermined time can be changed according to the user's operating environment and settings. Effects of this embodiment

[0005] According to this embodiment, the following effects can be obtained. (1) Intuitive and highly responsive zoom operation This invention features an operating system that combines gaze detection, a pressure sensor, a touch sensor, and swipe gestures. Based on the user's gaze, it identifies the object of focus and smoothly adjusts the zoom magnification according to the pressure intensity. This configuration enables continuous and intuitive zoom operation that responds to the user's intentions, unlike conventional switch-type or step-by-step zoom operations. (2) Analog zoom control according to pressure intensity In this invention, by utilizing the "pressure level" detected by a pressure sensor as input information for the user interface, flexible operation becomes possible, such as fine adjustment with light pressure and rapid zoom with strong pressure. This allows for a more natural operating experience without compromising the user experience, even in immersive environments such as VR / AR. (3) Integrated operating system using multiple input sensors By complementaryly utilizing multiple sensors—gaze input, touch, pressure, and swipe gestures—it is possible to reduce errors, improve the accuracy of understanding user intent, and enhance the immediacy of operations. In particular, by identifying objects through gaze input, controlling the amount of operation through touch and pressure, and adding swipe gestures, a more sophisticated control system can be provided. (4) Stable user interface (UI) and improved UX Features such as zoom lock and reset functions can be implemented to prevent accidental operation, providing users with a stable and stress-free operating experience. Furthermore, compared to conventional stepped zoom buttons, it delivers a superior user experience in terms of operability, flexibility, and responsiveness.

Claims

1. (Basic configuration) A glasses-type display device, - Touch sensors and pressure sensors are built into or placed on the surface of both the upper and lower parts of the frame (rim and / or temple). - Eye-tracking means for detecting the direction of the user's gaze (eye-tracking sensor, infrared camera, pupil tracking sensor, etc.) - A control unit that changes the zoom magnification based on information from the touch sensor, pressure sensor, and gaze detection means. Equipped with, A spectacle-type display device characterized in that the zoom magnification is adjusted based on the pressure intensity detected by the aforementioned pressure sensor, thereby enlarging or shrinking the display object in the line of sight.

2. (Changes in zoom due to pressure level) In the eyeglass-type display device according to claim 1, - When fingers make contact with both the upper and lower parts of the frame (rim and / or temple), and both upper and lower pressure sensors simultaneously detect pressure, the zoom magnification is adjusted. - The zoom magnification can be adjusted continuously or in steps according to the pressure intensity detected by the pressure sensor. - If the pressure is weak, make small adjustments by reducing the change in zoom magnification (fine-tuning mode). - When pressure is strong, the zoom magnification change is increased, enabling faster zooming in and out (quick zoom mode). A spectacle-type display device characterized by the following features.

3. (Zoom out) In the eyeglass-type display device according to claim 1, - The touch sensor detects that the user's finger is in contact with either the upper or lower side of the frame portion. - A glasses-type display device characterized in that, when the user swipes left or right, the zoom magnification of the displayed object in the direction of the user's gaze is reduced in accordance with the swipe operation.

4. (Zoom fixed) In the eyeglass-type display device according to claim 1, - The touch sensor is characterized by detecting when the user's finger is in contact with either the upper or lower side of the frame, and when this contact continues for a predetermined time without any swipe operation, it performs control to fix the zoom magnification. Subsequently, by performing the swipe operation described in claim 3 again, or the pressure operation described in claim 2 again, the zoom magnification lock is released, and the system returns to a state where the zoom magnification can be changed.

5. (Zoom Reset) In the eyeglass-type display device according to claim 1, A spectacle-type display device characterized by resetting the zoom state when it is detected that contact with the aforementioned touch sensor and pressure sensor has completely ceased.