Human-machine interaction multi-mode early intervention system for improving social interaction capacity of autistic children

Abstract

The invention discloses a human-machine interaction multi-mode early intervention system for improving the social interaction capacity of autistic children. The system comprises a multi-point touch screen, a computer and three cameras respectively mounted at the left and right sides of the touch screen and above the middle part of the touch screen, wherein each camera is provided with a microphone and connected with the computer through a USB (Universal Serial Bus) interface; and the system is provided with six basic modules, namely a visual signal processing module, a voice signal processing module, a physical interactive interface module, a multi-mode fusion module, an intelligent control console module and a real scene simulation module, wherein the modules are combined with computer vision, voice recognition, behavior identification, intelligent agent and virtual reality technologies so as to support and improve the social interaction capacity of the autistic children. Development and change of several children in the learning environment are tracked for half year, wherein the social interaction capacity of most children is improved obviously, and other children also make some progress in the aspect of interaction capacity.

Description

technical field [0001] The invention relates to a human-computer interaction multimodal intelligent learning system of computer science, cognitive science and pedagogy, in particular to a human-computer interaction multimodal early intervention system for improving the social interaction ability of children with autism. Background technique [0002] Childhood autism, also known as autism, is a developmental disorder that occurs in early childhood due to neurological disorders, and its symptoms include abnormal social skills, communication skills, interests and behavior patterns. Although autism was discovered in 1943, the research and treatment of autism did not receive real attention until the 1980s. For a long time, the cause of autism in children has remained a mystery. Many researchers suspect that autism is controlled by genes and then controlled by genes. Triggered by environmental factors, there is currently no effective drug for treatment, and only long-term educatio...

AI Technical Summary

Problems solved by technology

Relevant research abroad mainly uses various virtual reality technologies to assist children with autism in learning, creating a safe and controllable learning environment for children, but has not yet established a set of methods to improve the social interaction ability of children with autism and adapt to different individual children. multi-modal human-computer interaction system

Images