Smart zoom VR / AR glasses

Abstract

We provide a glasses-type display with zoom control function using eye tracking and pressure sensors. The glasses-type display of the present invention combines a pressure sensor and a touch sensor arranged in the frame with a gaze detection means to realize zooming based on the gaze direction. It enables magnification adjustment according to the strength of pressure, zooming out by swiping, zoom fixing by maintaining contact, and resetting by removing the finger, and also has a function to prevent malfunction.

Description

[Technical field]

[0001] The present invention relates to a glasses-type display device that enables intuitive zoom operations in technical fields such as VR (virtual reality), AR (augmented reality), and MR (mixed reality). [Background technology]

[0002] In conventional glasses-type displays, a touch sensor, a pressure sensor, and a gaze detection means have not been combined to realize an intuitive zoom operation such as continuous or stepwise zoom adjustment according to the strength of pressure or zoom adjustment by a swipe motion.

[0003] The eyeglass-type display device of the present invention may have the following configuration as an example: However, the present invention is not limited to this, and other sensors and algorithms may also be combined. The eyeglass-type display device of the present invention is equipped with touch sensors and pressure sensors built into or placed on the upper and lower sides of the frame (rim and / or temple) when the eyeglasses are worn, and allows the user to zoom in and out by pinching or swiping the frame. In addition, by combining it with a gaze detection means, intuitive zoom control based on the user's gaze direction is possible. The eyeglass-type display device of the present invention is characterized by using touch sensors and pressure sensors built into or arranged on the surface of both the upper and lower sides of the frame portion (rim portion and / or temple portion) when the eyeglasses are worn, and gaze detection means for detecting the user's gaze, and zooming and controlling the displayed content in the gaze direction based on the finger contact state and pressure strength detected by the touch sensors and pressure sensors. Also, depending on the strength of pinching pressure, fine zooming (fine adjustment mode), large zooming (quick zoom mode), and the like can be realized. Furthermore, by utilizing the touch sensors, it is possible to zoom out by swiping, fix the zoom by maintaining contact, and reset by releasing the finger. Example embodiments

[0004] The eyeglass-type display device of the present invention may be embodied in the following manner. (1) Basic configuration - Built-in or exposed touch sensors and pressure sensors are placed on both the top and bottom of the frame (rim and / or temple) when the glasses are worn to detect user operations. -Incorporate gaze detection means (eye tracking sensor, infrared camera, pupil tracking sensor, etc.) to detect the user's gaze direction. The touch sensor and pressure sensor are linked to the gaze detection means to control the zoom of the displayed content in the direction of the user's gaze. (2) Zoom operation flow Zoom in (magnification): When a user pinches both the top and bottom of the frame while wearing the glasses, the pressure sensor detects the strength of the pressure and magnifies the object in the line of sight. The stronger the pressure, the greater the zoom magnification. Zoom out (reduction): When the touch sensor detects that the user's finger is touching either the top or bottom of the frame when the glasses are being worn, and detects a horizontal swipe operation along the frame (left and right if the touch sensor is located on the temple), the zoom magnification is reduced in accordance with the swipe operation. Zoom lock: When the touch sensor detects that the user's finger is touching either the top or bottom of the frame when the glasses are being worn, and the touch continues for a specified period of time without any swiping operations, the zoom state is locked. · Zoom Reset: Resets the zoom state when the touch and pressure sensors detect loss of finger contact. (3) Detailed configuration of the sensor The pressure sensor may be of a capacitance type, a piezoresistance type, a piezoelectric type, an optical type, or a MEMS type, but is not limited thereto. The touch sensor may be a capacitive type, an optical type, a pressure sensing type (resistive type), an ultrasonic type, a proximity sensor type, or the like, but is not limited thereto. The gaze detection means may be, but is not limited to, an infrared camera method, a pupil tracking method, a gaze estimation method using machine learning, an electrophysiological gaze detection (EOG method), a laser-based gaze tracking method, etc. (4) Operation details When wearing the glasses, the user touches the touch sensors and pressure sensors built into or exposed on both the upper and lower sides of the frame (rim and / or temple) with their fingers and applies pressure to pinch the frame in the vertical direction. The pressure sensor detects this movement and adjusts the zoom factor depending on the strength of the pressure. The touch sensor detects that the user's finger is touching either the top or bottom of the frame when the glasses are being worn, and detects whether the user is holding the frame with a swipe or without a swipe, making it possible to perform operations such as zooming out and locking the zoom. -Since it is based on the user's line of sight, it is possible to zoom in and out while maintaining a natural field of view even during zoom operations. When it is detected that the user has not touched the touch sensor or pressure sensor for a predetermined period of time, the zoom state is reset. To prevent malfunction, for example, when the touch sensor and pressure sensor detect a loss of contact, it is possible to immediately perform a zoom reset. However, in order to prevent malfunction, for example, a design may be considered in which the touch sensor and pressure sensor do not perform a reset in the event of a short loss of contact, so that they do not mistakenly recognize a temporary loss of contact (accidental loss of finger contact). A reset is performed if contact is lost continuously for a specified period of time (e.g., 500 ms or more). The predetermined time period can be changed according to the user's environment and settings.

[0005] According to this embodiment, the following effects can be obtained. (1) Intuitive and highly responsive zoom operation The present invention has an operation system that combines gaze detection, a pressure sensor, and a touch sensor and swipe operation, and can identify the gaze target based on the user's gaze and smoothly adjust the zoom magnification according to the strength of the pressure. This configuration realizes continuous and intuitive zoom operation according to the user's intention, unlike conventional switch-type or step-by-step zoom operations. (2) Analog zoom control according to pressure strength In this invention, the "pressure" detected by the pressure sensor is used as input information for the user interface, enabling flexible operations such as fine adjustment with light pressure and rapid zoom with strong pressure. This allows for a more natural feel to the user without compromising the UX, even in immersive environments such as VR / AR. (3) Integrated operation system using multiple input sensors By utilizing multiple sensors, such as gaze, touch, pressure, and swipe motion, in a complementary manner, it is possible to reduce operational errors, accurately understand the user's intention, and increase the immediacy of operations. In particular, by identifying the target object through gaze input, controlling the amount of operation through touch and pressure, and adding a swipe motion, it is possible to provide a more advanced operation system. (4) Stable user interface (UI) and improved UX It is also possible to implement a zoom lock function to prevent erroneous operation, a reset function, etc., providing a stable operating experience with less stress for users. Also, compared to conventional step-by-step zoom buttons, it provides a superior user experience in terms of operability, flexibility, and responsiveness.

Claims

1. (Basic configuration) A glasses-type display device, Touch sensors and pressure sensors built into or disposed on the surface of both the upper and lower sides of the frame (rim and / or temple) when the glasses are worn; - Line of sight detection means for detecting the direction of a user's line of sight; A control unit is provided that changes a zoom magnification based on information from the touch sensor, the pressure sensor, and the line of sight detection means, A glasses-type display device, characterized in that the zoom magnification is adjusted based on the strength of the pressure detected by the pressure sensor, and the display object in the line of sight is enlarged or reduced.

2. (Zoom changes with pressure level) 2. The eyeglass-type display device according to claim 1, When the user puts on the eyeglasses, the lens adjusts the zoom magnification when the user's fingers touch both the upper and lower sides of the frame (rim and / or temple) and both the upper and lower pressure sensors detect pressure simultaneously. The zoom magnification can be adjusted continuously or stepwise according to the strength of the pressure detected by the pressure sensor. If the pressure detected by the pressure sensor is weak, the zoom magnification change is reduced to perform fine adjustment (fine adjustment mode); A glasses-type display device characterized in that, when the pressure detected by the pressure sensor is strong, the change in zoom magnification is increased, enabling more rapid enlargement and reduction (quick zoom mode).

3. (Zoom out) 2. The eyeglass-type display device according to claim 1, The touch sensor detects that a user's finger is in contact with either the upper side or the lower side of the frame portion when the eyeglasses are worn, - In this state, when a swipe operation is performed horizontally along the frame portion (left and right if positioned on the temple portion), the zoom magnification of the displayed object in the line of sight is reduced in accordance with the swipe operation.

4. (Zoom fixed) 2. The eyeglass-type display device according to claim 1, The touch sensor detects that the user's finger is in contact with either the upper side or the lower side of the frame when the eyeglasses are worn, and performs control to fix the zoom magnification when the touch continues for a predetermined period of time without a swipe operation, etc. - Thereafter, by performing the swipe operation described in claim 3 again or the pressure operation described in claim 2 again, the zoom magnification is released from the fixed state and the zoom magnification is returned to a state where it can be changed.