System and method for assisting pairing training of autistic child by applying fine gesture recognition apparatus

Abstract

The invention discloses a system and a method for assisting pairing training of an autistic child by applying a fine gesture recognition apparatus. The system comprises a computer an the fine gesture recognition apparatus; a pairing training system is operated in the computer; the pairing training system comprises an input module, a data processing and visual module, an action capturing module, an output module and a model base; and when the action capturing model performs fine gesture recognition on the captured actions by the recognition apparatus, positions and actions of hands are captured and mapped to a virtual three-dimensional model of hands in real time to represent in corresponding positions in a virtual scene. The system and the method are used for assisting a trainer to arrange training process and record training data in a training process so as to reduce burden of the trainer; through gesture recognition, the actual and natural actions of the trainer are reserved, so as to improve the training effect; and the system is applied to the pairing training of the autistic child, and the equipment is low in cost and the system can be popularized easily.

Description

technical field [0001] The invention belongs to the field of basic skill training for children with autism in special education, and specifically relates to a system and method for assisting pairing training of children with autism by using a fine gesture recognition device. Background technique [0002] Human-computer Nature Interaction technology, referred to as Natural Interaction (Human-computer Nature Interaction, HCNI; or Human-machine Nature Interaction, HMNI). Natural interaction is now an umbrella term used to refer to various ways of interacting with a system using gestures or other bodily movements as input. (See O'HARA K, HARPER R, MENTIS H, et al. On the naturalness of touchless: Putting the “ interaction & rdquo; back into NUI [J]. ACM Transactions on Computer-Human Interaction, 2013, 20 (1): 526- 36.) [0003] In the "Horizon Report" released by the New Media Consortium (NMC) from 2010 to 2016, technologies such as gesture computing, wearable devices, ...

AI Technical Summary

Problems solved by technology

[0017] The technical problem to be solved by the present invention is: to overcome the shortcomings of the difficulty in collecting and presenting physical models for pairing training, the large workload of teachers, the disadvantages of canceling hand grasping actions in mouse-click pairing training, and the disadvantages of simply presenting abstract content in round-based plane games , to provide a pairing training system for autistic children using a fine gesture recognition device. On the basis of retaining complete cognitive and motor training, it reduces the work of the trainer to arrange the training process and record data, so that the trainer can focus more For the guidance of children with autism, the training equipment is simple and easy to operate, suitable for daily basic skills training of children with autism

Images