System and method for assisting pairing training of autistic child by applying fine gesture recognition apparatus

Abstract

The invention discloses a system and a method for assisting pairing training of an autistic child by applying a fine gesture recognition apparatus. The system comprises a computer an the fine gesture recognition apparatus; a pairing training system is operated in the computer; the pairing training system comprises an input module, a data processing and visual module, an action capturing module, an output module and a model base; and when the action capturing model performs fine gesture recognition on the captured actions by the recognition apparatus, positions and actions of hands are captured and mapped to a virtual three-dimensional model of hands in real time to represent in corresponding positions in a virtual scene. The system and the method are used for assisting a trainer to arrange training process and record training data in a training process so as to reduce burden of the trainer; through gesture recognition, the actual and natural actions of the trainer are reserved, so as to improve the training effect; and the system is applied to the pairing training of the autistic child, and the equipment is low in cost and the system can be popularized easily.

Description

technical field [0001] The invention belongs to the field of basic skill training for children with autism in special education, and specifically relates to a system and method for assisting pairing training of children with autism by using a fine gesture recognition device. Background technique [0002] Human-computer Nature Interaction technology, referred to as Natural Interaction (Human-computer Nature Interaction, HCNI; or Human-machine Nature Interaction, HMNI). Natural interaction is now an umbrella term used to refer to various ways of interacting with a system using gestures or other bodily movements as input. (See O'HARA K, HARPER R, MENTIS H, et al. On the naturalness of touchless: Putting the “ interaction & rdquo; back into NUI [J]. ACM Transactions on Computer-Human Interaction, 2013, 20 (1): 526- 36.) [0003] In the "Horizon Report" released by the New Media Consortium (NMC) from 2010 to 2016, technologies such as gesture computing, wearable devices, ...

AI Technical Summary

Problems solved by technology

[0017] The technical problem to be solved by the present invention is: to overcome the shortcomings of the difficulty in collecting and presenting physical models for pairing training, the large workload of teachers, the disadvantages of canceling hand grasping actions in mouse-click pairing training, and the disadvantages of simply presenting abstract content in round-based plane games , to provide a pairing tr...

Images