A user interaction method and device

Abstract

The present invention provides a user interaction method and equipment. The method mainly includes: using an infrared camera to obtain the first operation data of the user on the user interface through an infrared grating; using a depth camera to obtain the second operation data of the user on the user interface The first operation data and the second operation data are fused to obtain user operation data; based on the user operation, the display content of the projection unit on the user operation interface is updated. Its judgment on the user's actions is more accurate, thereby realizing more precise control; the present invention uses the obtained depth image to obtain rich gestures based on the advanced depth judgment model, and based on the user's gestures, more and richer interaction methods can be realized , there will be expansion of follow-up functions; through the combination of depth camera and infrared signal to determine the gesture action, the recognition accuracy of gesture action is further improved, and in addition, the automatic correction of the projector can also be realized.

Description

technical field [0001] The invention relates to the technical field of human-computer interaction, in particular to a user interaction method and device. Background technique [0002] Human-computer interaction is a study of the interactive relationship between the system and the user. A system can be a variety of machines, or it can be computerized systems and software. The human-computer interface usually refers to the part visible to the user. The user communicates with the system through the human-computer interaction interface and performs operations. It can be as small as the play button on a radio, or as large as the dashboard of an airplane or the control room of a power plant. The design of the human-computer interaction interface should include the user's understanding of the system (that is, the mental model), which is for the usability or user-friendliness of the system. [0003] In the prior art, the touch control scheme used by interactive projectors is bas...

AI Technical Summary

Problems solved by technology

The disadvantages of this solution are: the emitter must be placed on the interaction plane, and the shape of the projector is limited; it cannot handle occlusion in the horizontal direction, and cannot handle objects with height; any object may be touched by mistake; the accuracy needs to be improved; projection The interface of the instrument cannot be automatically calibrated

[0004] In addition, in the prior art, no matter whether it is infrared or video-based acquisition of user actions, the action is captured based on the current frame, and the recognition accuracy is low, and the prior art is based on single-modal acquisition of user actions, which cannot be simultaneously Capturing user actions based on two or more signals, while the attitude accuracy of user actions acquired by a single signal is low. Therefore, how to improve the recognition accuracy of user actions is an important and difficult point of human-computer interaction

Images