AR-oriented navigation type interactive paradigm system

Abstract

The invention provides an AR-oriented navigation type interactive paradigm system. The system comprises an input and perception module, a fusion module and an application expression module; the inputand sensing module obtains hand skeleton depth information through kinect to obtain a gesture depth map, and gesture recognition is completed; t experimental keywords are classified through voice keyword recognition and extraction, andthe similarity probability is obtained through similarity calculation to complete voice recognition; the fusion module classifies user intentions by calculating signal perception of different states in different modes according to gesture recognition, voice recognition and force feedback expression, determines multi-mode intentions and sets an intention expert knowledge base; and the expression module is used for carrying out voice navigation and visual presentation on the virtual chemical experiment and carrying out perceptual force feedback expression. Gesture and voice multi-modes are fused by adopting a natural interaction technology, and a natural interaction mode is explored, so that the user load is reduced, and different user experiences are brought.

Description

technical field [0001] The invention belongs to the technical field of experimental interaction, in particular to an AR-oriented navigational interaction paradigm system. Background technique [0002] In current middle school chemistry, many chemical experiments have dangerous experimental phenomena such as explosion and corrosion, which makes it impossible for students to operate the experiment. It is not easy for students to observe in experiments with gas generation. Middle school students only observe experimental phenomena through textbooks or videos. Therefore, it is necessary to The new experimental method not only meets the requirements of the middle school chemistry experiment syllabus, but also solves the problems raised. It enables students to operate experiments at any time, saves chemical reagent resources, and allows students to truly explore and discover the truth and mechanism of experiments by themselves. [0003] With the development of human-computer inte...

AI Technical Summary

Problems solved by technology

According to research, traditional education and teaching cannot meet the requirements and goals of teaching effects in the new era. Traditional chemical experiments have dangerous phenomena such as explosion and corrosion. Therefore, more and more education and teaching focus on new virtual experiment teaching. To a certain extent, it improves students' thinking ability and enhances their ability to imagine the virtual space, but most of them only use the model in a single virtual scene to complete the experimental operation, which lacks the sense of reality of the real environment, causing students to be unable to integrate into the virtual experimental teaching.

Faced with the current augmented reality technology, most of them use tagged card recognition to complete the three-dimensional reproduction of the virtual model, and a few are based on wearable devices such as data gloves or use gesture recognition sensors to complete the fusion of the real world and the virtual scene. Unable to fully immerse in the experiment of virtual-real interaction

Images