Eye movement interaction method, head mounted device and computer readable medium

Abstract

Eye movement interaction method for virtual reality and / or augmented reality head-mounted device, head-mounted device, computer readable medium, eye movement interaction method comprising: displaying a virtual reality scene on a screen without displaying a menu; tracking a user line of sight; according to the tracked line of sight, determine whether the user has performed a menu trigger event through eye movement; if the user has performed a menu trigger event, display the menu. The eye movement interaction method may also include: in the case of displaying the menu, judging whether the user has performed a menu selection event according to the tracked line of sight; in the case of determining that the user has performed a menu selection event, executing a command corresponding to the selected menu item, and Hide menu. The menu in the eye movement interaction method of the head-mounted device is usually invisible, that is, it will not occupy the user's field of vision, and it will only appear when the user wants to call out the menu. Experiments have verified the effectiveness of the menu trigger technology and selection technology. performance, accuracy and robustness.

Description

technical field [0001] The present invention generally relates to a virtual reality and / or augmented reality head-mounted device, and more particularly relates to an eye-tracking-based interaction method and a head-mounted device. Background technique [0002] With the development of virtual reality (VR) and augmented reality (AR) technologies, these head-mounted devices that can render virtual objects have entered the lives of the public, used by a large number of user groups, and applied in various fields, such as games , education, medical care, specialized training, etc. Examples of virtual reality devices are for example Figure 9 The virtual reality headset shown and Figure 10 The augmented reality headset shown. [0003] Eye trackers can be deployed in existing head-mounted displays to track the movement and gaze of users in VR and AR. However, the scenes and functions that the eye tracker can use are very limited, mainly due to the following points: 1. The tracki...

AI Technical Summary

Problems solved by technology

However, the scenes and functions that the eye tracker can use are very limited, mainly due to the following points: 1. The tracking accuracy of the eye tracker is not very high, generally there will be an error of 0.5-2°, and the user is using the eye tracker to perform The movement of the eyeball is not so stable during the interaction, which leads to the unreliable signal obtained by the eye tracker, which is often accompanied by irregular beating and a lot of noise; 2. The user's eyes are always open in the use of VR and AR. Yes, how to judge the user's eyes to perform an intentional interaction at a certain moment is a relatively big challenge

[0004] At present, eye movement interaction in VR and AR is very limited: 1. Eye movement is usually used as an implicit input for user interaction, such as using eye movement to determine which area the user is paying attention to, so that specific modifications to that area make the user You can use your hands or controllers to input better; 2. You can use your eyes to point or track objects in the interface to select menus or select objects, but this kind of interaction cannot be popularized in practical scenarios because it is impossible to judge whether users use VR and Whether the natural eye movement of AR is selecting the target, leading to the problem of false triggering; 3. Using the eyes to stay or draw a specific trajectory, these operations are not easy to be falsely triggered in the user's natural eye movement, but the input efficiency of these interactions is low

Images