Method and device for operating browser based on eye-movement tracking

Abstract

The invention discloses a method and a device for operating a browser based on eye-movement tracking. The method comprises the steps of shooting a facial image of a user through a camera at every set time and describing the facial image to a Canvas, describing eyeball images to the Canvas after the eyeball images are extracted from the facial image, and acquiring the pixel data of the eyeball images from the Canvas; acquiring the eyeball data shot every time according to the pixel data and a set eyeball catching algorithm, and determining the user to be focused on a special content in a browser page when the difference valve of every adjacent two-time eyeball positions is less than a preset threshold valve and the retention time of eyeballs is more than the set time of the eyeballs; if the semi-diameters of pupils are detected to be amplified, looking for corresponding focal coordinates of the pupils in a webpage, and determining dom element label objects corresponding to the focal coordinates; and executing corresponding browsing operation according to relative relations between the preset dom element label subjects and performed actions. The execution of the browsing relative operation can be realized based on the eyeball movement by using the method.

Description

technical field [0001] The present invention relates to the technical field of mobile communication, and more specifically, to a method and device for operating a browser based on eye tracking. [0002] Background technique [0003] At present, most of the browsing operations of mobile terminal browsers are performed through touch operations. For example, when viewing web pages, clicks and manipulations are required. However, if the user is inconvenient to operate the browser with both hands, for example, the user is taking a bus, and there are too many people on the bus, the user has to hold the handrail with one hand and hold the mobile terminal device (such as mobile terminal device with a large screen), it is difficult for him to perform related browsing operations; or the user is a disabled person who only holds a mobile terminal device in one hand, so he can only use a mobile terminal device with a small screen. This makes it impossible to feel the user experience b...

AI Technical Summary

Problems solved by technology

However, if the user is inconvenient to operate the browser with both hands, for example, the user is taking a bus, and there are too many people on the bus, the user has to hold the handrail with one hand and hold the mobile terminal device (such as mobile terminal device with a large screen), it is difficult for him to perform related browsing operations; or the user is a disabled person who only holds a mobile terminal device in one hand, so he can only use a mobile terminal device with a small screen. This makes it impossible to feel the user experience brought by large-screen mobile terminal equipment.

[0004] Therefore, the prior art only performs browsing operations through touch operations, which is inconvenient for users in some scenarios

Images