Eye movement interaction method, head-mounted device and computer readable medium

Abstract

The present invention provides an eye movement interaction method for a virtual reality and / or augmented reality head-mounted device, a head-mounted device, and a computer readable medium, the eye movement interaction method comprising: displaying a virtual reality scene on a screen without displaying a menu; tracking the sight of the user; judging whether the user performs a menu triggering eventthrough eye movement or not according to the tracked sight line; and under the condition that the user performs the menu triggering event, displaying the menu. The eye movement interaction method canalso comprise the following steps: judging whether a user performs a menu selection event or not according to the tracked sight line under the condition of displaying a menu; and under the conditionof determining that the user performs the menu selection event, executing a command corresponding to the selected menu item, and hiding the menu. The menu in the eye movement interaction method of thehead-mounted device is invisible at ordinary times, the view of the user is not occupied, the menu appears only when the user wants to call out the menu, and the effectiveness, accuracy and robustness of the menu triggering technology and the selection technology are verified through experiments.

Description

technical field [0001] The present invention generally relates to virtual reality and / or augmented reality headsets, and more particularly to eye tracking based interaction methods and headsets. Background technique [0002] With the development of virtual reality (VR) and augmented reality (AR) technologies, these head-mounted devices that can render virtual objects have entered the lives of the masses, are used by a wide range of user groups, and are applied in various fields, such as games , education, medical care, special training, etc. Examples of virtual reality devices are e.g. Figure 9 The virtual reality helmet shown and Figure 10 Augmented reality helmet shown. [0003] Eye trackers can be deployed in existing head-mounted displays to track user eye movement and gaze position in VR and AR. However, the scenarios and functions that eye trackers can use are very limited, mainly due to the following points: 1. The tracking accuracy of eye trackers is not very hi...

AI Technical Summary

Problems solved by technology

However, the scenes and functions that the eye tracker can use are very limited, mainly due to the following points: 1. The tracking accuracy of the eye tracker is not very high, generally there will be an error of 0.5-2°, and the user is using the eye tracker to perform The movement of the eyeball is not so stable during the interaction, which leads to the unreliable signal obtained by the eye tracker, which is often accompanied by irregular beating and a lot of noise; 2. The user's eyes are always open in the use of VR and AR. Yes, how to judge the user's eyes to perform an intentional interaction at a certain moment is a relatively big challenge

[0004] At present, eye movement interaction in VR and AR is very limited: 1. Eye movement is usually used as an implicit input for user interaction, such as using eye movement to determine which area the user is paying attention to, so that specific modifications to that area make the user You can use your hands or controllers to input better; 2. You can use your eyes to point or track objects in the interface to select menus or select objects, but this kind of interaction cannot be popularized in practical scenarios because it is impossible to judge whether users use VR and Whether the natural eye movement of AR is selecting the target, leading to the problem of false triggering; 3. Using the eyes to stay or draw a specific trajectory, these operations are not easy to be falsely triggered in the user's natural eye movement, but the input efficiency of these interactions is low

Images