User interaction method and device

Abstract

The invention provides a user interaction method and device. The method mainly comprises the steps of: using an infrared camera for obtaining first operation data of a user on a user interface throughan infrared grating; acquiring second operation data of the user on the user interface by using a depth camera, and fusing the first operation data and the second operation data to obtain user operation data; and updating the display content of the projection unit on the user operation interface based on the user operation. The judgment on the user action is more accurate, so that more accurate control is realized, the acquired depth image is adopted to acquire rich gesture actions based on the advanced depth judgment model, and more and richer interaction methods can be realized based on thegestures of the user, so that subsequent function extension is realized; the depth camera is combined with the infrared signal to determine the gesture action, so that the gesture action recognitionprecision is further improved, and in addition, the automatic correction of the projector can be realized.

Description

technical field [0001] The invention relates to the technical field of human-computer interaction, in particular to a user interaction method and device. Background technique [0002] Human-computer interaction is a study of the interactive relationship between the system and the user. A system can be a variety of machines, or it can be computerized systems and software. The human-computer interface usually refers to the part visible to the user. The user communicates with the system through the human-computer interaction interface and performs operations. It can be as small as the play button on a radio, or as large as the dashboard of an airplane or the control room of a power plant. The design of the human-computer interaction interface should include the user's understanding of the system (that is, the mental model), which is for the usability or user-friendliness of the system. [0003] In the prior art, the touch control scheme used by interactive projectors is bas...

AI Technical Summary

Problems solved by technology

The disadvantages of this solution are: the emitter must be placed on the interaction plane, and the shape of the projector is limited; it cannot handle occlusion in the horizontal direction, and cannot handle objects with height; any object may be touched by mistake; the accuracy needs to be improved; projection The interface of the instrument cannot be automatically calibrated

[0004] In addition, in the prior art, no matter whether it is infrared or video-based acquisition of user actions, the action is captured based on the current frame, and the recognition accuracy is low, and the prior art is based on single-modal acquisition of user actions, which cannot be simultaneously Capturing user actions based on two or more signals, while the attitude accuracy of user actions acquired by a single signal is low. Therefore, how to improve the recognition accuracy of user actions is an important and difficult point of human-computer interaction

Images