Eye tracking human-machine interaction method and apparatus

Abstract

The invention relates to a view line tracking man-machine interactive method, including: view line tracking information is collected and a view line focus position is obtained according to the view line tracking information; facial image information is collected and facial action is recognized according to the facial image information; a control command corresponding to the facial action is output according to the view line focus position. the invention also relates to a view line tracking man-machine interactive device which comprises a view line tracing processing unit which is used for the view line tracking operation, a facial action recognition unit for collecting the facial image information and recognizing the facial action according to the facial image information and a control command output unit which is respectively connected with the view line tracking processing unit and the facial action recognition unit and is used for outputting the control command corresponding to the facial action according to the view line focus position. the embodiment of the invention provides a non-contact view line tracking man-machine interactive method and a device with interaction protocols.

Description

technical field [0001] The invention relates to a human-computer interaction method and device, in particular to a sight-tracking human-computer interaction method and device, and belongs to the field of human-computer interaction. Background technique [0002] Devices such as keyboards, mice, and joysticks provide the communication interface between the machine and the operator, realizing human-computer interaction. In recent years, with the development of eye-tracking technology, mouse simulation and keyboard simulation based on eye-tracking have been researched and applied in the field of human-computer interaction, such as assisting disabled people to complete man-machine communication, assisting aviation pilots to operate computers, etc. On a personal computer, the result of gaze tracking can control the movement of the mouse pointer, thereby replacing the mouse to realize the function of human-computer interaction. In some industrial control, aviation application scen...

AI Technical Summary

Problems solved by technology

In this way, it cannot be applied to special occasions such as aviation driving where the hand cannot be used for operation, and some disabled people who cannot move their hands cannot use the system.

[0005] (2) When the system is used in public facilities, due to the need to touch the mouse device, the possibility of infectious diseases among the crowd is increased

[0007] It is difficult to choose a more suitable value for the gaze dwell time. If you choose a smaller value, it is easy to cause misoperation, because when the line of sight pauses for a while, a pop-up menu will appear, and the operator does not want to do any action. In this case Especially in some specific actions (such as reading articles, watching videos, etc.)

If you choose a larger value, it will prolong the execution time of the operation command and greatly reduce the efficiency of the interaction

In addition, the pop-up menu will also affect the operator's observation of the screen content in many occasions

In short, such an interaction protocol will lead to the interaction process is not humanized enough, the efficiency is also low, and it is easy to cause misoperation

Images