Family cooperative training system and method for early intervention of autism

Abstract

The invention relates to the technical field of intelligent rehabilitation and special education, in particular to a family cooperative training system and method for autism early intervention, and the method is characterized in that the method comprises the following steps: 1, a parent fills in an autism early intervention scale through a mobile terminal App, the App uploads questionnaire data and a family environment video to a cloud AI evaluation service, and the AI evaluation service is a cloud AI evaluation service; a child personalized intervention file and a training target are generated, and evaluation and filing are carried out; 2, the cloud AI evaluation service automatically generates a standardized multi-dimensional training courseware and a family training plan according to a training target, and pushes the standardized multi-dimensional training courseware and the family training plan to the family end device through the edge gateway to complete scheme issuing; 3, at the home terminal, parents use the touch terminal to play training courseware according to the training plan, and meanwhile, a wearable sensor collects movement and physiological data of children, the personalized matching of training, the participation degree of the parents and the overall intervention effect are improved, and quantifiable, traceable and efficient cooperation of early intervention is achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent rehabilitation and special education technology, specifically to a family-based collaborative training system and method for early intervention in autism. Background Technology

[0002] In the current field of early intervention, most training for children with autism relies primarily on face-to-face teaching at professional rehabilitation centers or hospitals, or remote tutoring based on a single mobile app, specifically including: Centralized manual intervention program: Therapists conduct structured training for children in rehabilitation centers, with parents passively cooperating; Single-modal remote tutoring tools: Some online platforms offer video demonstrations or scale assessment functions to record videos or questionnaires uploaded by parents; Wearable / camera-assisted data acquisition: Existing research has used wearable sensors or cameras for behavioral monitoring, but these are limited to laboratory environments or single data streams; However, the lack of a family collaborative training system that integrates edge-cloud collaboration, real-time multimodal data acquisition and processing, and closed-loop iteration makes it difficult to accurately deliver personalized interventions, provide timely training feedback, and guarantee parental participation and intervention effectiveness. Therefore, this paper proposes a family collaborative training system and method for early intervention of autism to address the above problems. Summary of the Invention

[0003] The purpose of this invention is to provide a family collaborative training system and method for early intervention in autism, in order to solve the problems of the lack of a family collaborative training system with end-to-edge-cloud collaboration, real-time acquisition and processing of multimodal data, and closed-loop iteration, which makes it difficult to accurately deliver personalized interventions, provide timely training feedback, and guarantee parental participation and intervention effectiveness.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A family-based collaborative training system and method for early intervention in autism, including the following steps: Step 1: Parents fill out the Early Intervention Scale for Autism via a mobile app. The app uploads the questionnaire data and home environment video to the cloud AI assessment service to generate a personalized intervention file and training goals for the child, and conducts assessment and record building. Step 2: The cloud-based AI assessment service automatically generates standardized multi-dimensional training courseware and home training plans based on the training objectives, and pushes them to home devices through the edge gateway to complete the solution distribution; Step 3: At home, parents use a touch-screen device to play training materials according to the training plan. At the same time, wearable sensors collect children's movement and physiological data, and environmental cameras collect visual and behavioral data to carry out training and data collection. Step 4: The home gateway receives raw sensor and video data, uses an embedded AI model to extract key features such as gaze duration, motion standardization, and heart rate changes, and caches the processing results to complete edge preprocessing; Step 5: The edge gateway synchronizes the processed feature data to the cloud AI engine. The cloud AI engine evaluates the training effect based on the preset model, generates periodic intervention reports, and pushes them to the parent app and the supervisor's terminal to complete cloud analysis and feedback. Step 6: The therapist views the intervention report and training video through the supervision terminal, modifies the training plan online or resets the goals. The modifications are synchronized to the cloud AI assessment service and distributed to the family terminal to realize the intervention closed loop and complete the plan iteration and remote supervision.

[0005] As a further optimization of the present invention, in step 2, the solution distribution further includes: S21: Goal Mapping: Mapping personalized intervention goals for children to a predefined set of training dimensions. ; S22: Courseware Selection: Retrieve multimedia courseware corresponding to each dimension from the content management service; S23: Schedule: Automatically generate daily / weekly training schedules based on mapping results.

[0006] As a further optimization of this invention, in S21, a multi-dimensional target mapping algorithm is used in the target mapping stage, and the mathematical expression of the multi-dimensional target mapping algorithm is: ; In the formula, This represents the overall score of the m-th training dimension. This represents the standardized score of the nth assessment item. This represents the weight of the nth evaluation item corresponding to the mth training dimension, where N is the total number of evaluation items. The courseware for each dimension is sorted from highest to lowest score, and training plans are formed according to priority.

[0007] As a further optimization of the present invention, step 3, training execution and data acquisition further includes the following steps: S31: Parents initiate training on the App, and the touch terminal loads locally cached courseware resources and plays the demonstration. S32: Wearable sensors transmit acceleration, gyroscope, heart rate, and skin conductance signals in real time via BLE; S33: The ambient camera captures video at a preset angle and pushes it to the home gateway.

[0008] As a further optimization of this invention, in step 4, the edge preprocessing stage, the deviation distance of the kth key point is calculated using an action completion scoring algorithm. The deviation distance of the kth key point The calculation formula is: ; In the formula, This represents the target coordinates of the k-th keypoint in the preset training action. This represents the actual coordinates of the k-th key point detected by the camera, where k is the key point index. The system evaluates the child's performance by calculating the average or weighted sum of the deviation distances of all key points and provides real-time feedback.

[0009] As a further optimization of the present invention, in step 4, the home gateway performs the following sub-steps: S41: Time synchronization: NTP calibration timestamps are uniformly applied to wearable data and video frames; S42: Gaze duration calculation: Based on facial landmark tracking, gaze events are detected and accumulated; S43: Feature caching: Cache the extracted gaze duration, action completion rate, and physiological signal features in a local queue.

[0010] As a further optimization of the present invention, in S43, the formula for calculating the gaze duration is: ; In the formula, M is the total number of detected gaze events. Indicates the start time of the i-th gaze event. The duration of the gaze indicates the end time of the i-th gaze event, and the gaze duration is used to assess the child's visual concentration in the training materials.

[0011] As a further optimization of the present invention, it includes: a parent terminal App, a cloud AI assessment service module, a home gateway, wearable sensors and environmental cameras, and a supervisory terminal; The parent terminal app is used to collect and fill in the early intervention scale for autism, upload videos of the home environment, and receive personalized training courseware and periodic intervention reports sent from the cloud. The cloud-based AI assessment service module is used to receive scale data and environmental videos from the parent's terminal app, generate personalized intervention files and training goals for children based on the scales and videos, and perform multi-dimensional goal mapping based on the multi-dimensional goal mapping algorithm. It automatically generates standardized multi-dimensional training courseware and family training plans, and distributes them to the family gateway; The home gateway is used to receive and locally cache courseware bundles downloaded from the cloud, aggregate motion and physiological data from wearable sensors, and video streams from environmental cameras. During the edge preprocessing stage, it utilizes the deviation distance of the k-th keypoint. The formula for calculating fixation duration is: , The system calculates children's action completion rate and children's gaze duration characteristics, and then caches the extracted features and uploads them to the cloud AI engine. The wearable sensor and environmental camera are used to collect physiological signals such as children's movement, heart rate, and skin conductance, as well as visual behavior data, and push them to the home gateway via BLE or RTSP. The supervisory terminal is used to receive periodic intervention reports and training videos generated by the cloud AI engine, and supports therapists to modify training plans online to achieve closed-loop iteration of training.

[0012] As a further optimization of the present invention, the system further includes a memory, a processor, and an electronic program stored in the memory and capable of running on the processor, wherein the processor, when running the electronic program, is capable of implementing the steps of the method according to any one of claims 1-7.

[0013] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs a home-based collaborative training system that integrates a parent-side app, a home gateway, and a cloud-based AI assessment engine. This system allows parents to easily execute training tasks and simultaneously collect motion, visual, and physiological signals on their touchscreen devices. Real-time preprocessing and feature extraction are also performed on the home gateway, ensuring low latency and reliable uploading of multimodal data. The cloud-based AI engine generates periodic reports based on this data, enabling therapists to remotely adjust training plans and distribute them with a single click, forming a complete closed-loop iteration. This significantly improves personalized training matching, parental participation, and overall intervention effectiveness, achieving quantifiable, traceable, and highly collaborative early intervention. Attached Figure Description

[0014] Figure 1 This is a flowchart of the family-based collaborative training method for early intervention of autism according to the present invention; Figure 2 This is a system block diagram of the family-based collaborative training system for early intervention in autism according to the present invention. Detailed Implementation

[0015] Please see Figures 1-2 The present invention provides a technical solution: The family-based collaborative training system and methods for early intervention in autism achieve integrated management of the entire process of assessment, implementation, and feedback through multi-party collaboration and closed-loop iteration mechanism involving families, schools, and therapists. This not only improves the personalization and accuracy of training programs but also enhances parental participation and the traceability of intervention effects. Specifically, Step 1: Parents fill out the early intervention questionnaire for autism through a mobile app. The app uploads the questionnaire data and a video of the family environment to the cloud AI assessment service to generate a personalized intervention file and training goals for the child, and conducts assessment and record-keeping. Step 2: The cloud-based AI assessment service automatically generates standardized multi-dimensional training courseware and home training plans based on the training objectives, and pushes them to home devices through the edge gateway to complete the solution distribution; Step 3: At home, parents use a touch-screen device to play training materials according to the training plan. At the same time, wearable sensors collect children's movement and physiological data, and environmental cameras collect visual and behavioral data to carry out training and data collection. Step 4: The home gateway receives raw sensor and video data, uses an embedded AI model to extract key features such as gaze duration, motion standardization, and heart rate changes, and caches the processing results to complete edge preprocessing; Step 5: The edge gateway synchronizes the processed feature data to the cloud AI engine. The cloud AI engine evaluates the training effect based on the preset model, generates periodic intervention reports, and pushes them to the parent app and the supervisor's terminal to complete cloud analysis and feedback. Step 6: Therapists can view intervention reports and training videos through the supervision terminal, modify the training plan online or reset the goals, and synchronize the modifications to the cloud AI assessment service and distribute them to the family terminal to realize the intervention closed loop and complete the plan iteration and remote supervision.

[0016] As a further technical solution for implementing this solution, step 2, the distribution of the solution, further includes: S21: Goal Mapping: Mapping personalized intervention goals for children to a predefined set of training dimensions. ; S22: Courseware Selection: Retrieve multimedia courseware corresponding to each dimension from the content management service; S23: Schedule: Based on the mapping results, daily / weekly training schedules are automatically generated. The detailed target mapping, courseware selection and schedule scheduling process ensures a high degree of fit between the training plan in terms of content and pace, so that the intervention plan not only meets the individual needs of children, but also has operability and feasibility. As a further technical implementation of this scheme, in S21, a multi-dimensional target mapping algorithm is adopted in the target mapping stage. The mathematical expression of the multi-dimensional target mapping algorithm is: ; In the formula, This represents the overall score of the m-th training dimension. This represents the standardized score of the nth assessment item. This represents the weight of the nth evaluation item corresponding to the mth training dimension, where N is the total number of evaluation items. The courseware for each dimension is sorted from high to low according to the score, and a training plan is formed according to the priority. The multi-source evaluation data is scientifically integrated to quantify the importance of different training dimensions, which provides mathematical support for the accurate selection and priority ranking of training courseware and improves the quantifiability and evaluability of the solution. As a further implementation of this solution, step 3, training execution and data acquisition, further includes the following steps: S31: Parents initiate training on the App, and the touch terminal loads locally cached courseware resources and plays the demonstration. S32: Wearable sensors transmit acceleration, gyroscope, heart rate, and skin conductance signals in real time via BLE; S33: The environmental camera collects video at preset angles and pushes it to the home gateway. Through a structured training execution process and multimodal data collection, it comprehensively and in real time records the child's behavior and physiological reactions throughout the training process, providing rich and accurate data support for subsequent effect evaluation and program optimization. As a further technical solution to this scheme, in the edge preprocessing stage of step 4, the deviation distance of the kth key point is calculated using an action completion scoring algorithm. The deviation distance of the kth key point The calculation formula is: ; In the formula, This represents the target coordinates of the k-th keypoint in the preset training action. This represents the actual coordinates of the k-th key point detected by the camera, where k is the key point index. By calculating the average or weighted sum of the deviation distances of all key points, the child's performance is evaluated and feedback is provided in real time. Using key point deviation as a metric, the child's performance is objectively and quantitatively evaluated, improving the real-time nature and accuracy of training feedback. As a further implementation of this solution, in step 4, the home gateway performs the following sub-steps: S41: Time synchronization: NTP calibration timestamps are uniformly applied to wearable data and video frames; S42: Gaze duration calculation: Based on facial landmark tracking, gaze events are detected and accumulated; S43: Feature caching: The extracted gaze duration, action completion rate, and physiological signal features are cached in a local queue. The unified timestamp and multi-feature parallel caching ensure the accurate temporal alignment and complete preservation of multimodal data, preventing data loss due to network fluctuations. As a further technical implementation of this solution, in S43, the formula for calculating gaze duration is: ; In the formula, M is the total number of detected gaze events. Indicates the start time of the i-th gaze event. This indicates the end time of the i-th gaze event. The gaze duration is used to assess the child's visual concentration in the training materials, quantify the child's attention distribution during the training process, and provide a basis for adjusting the difficulty and designing the interaction of subsequent training content. As a further implementation of this solution, the technical solutions include a parent terminal app, a cloud-based AI assessment service module, a home gateway, wearable sensors and environmental cameras, and a supervisory terminal. The parent terminal app is used to collect and fill out the early intervention scale for autism, upload videos of the home environment, and receive personalized training courseware and periodic intervention reports sent from the cloud. The cloud-based AI assessment service module receives scale data and environmental videos from the parent's terminal app, generates personalized intervention files and training goals for children based on the scales and videos, and uses a multi-dimensional goal mapping algorithm. It automatically generates standardized multi-dimensional training courseware and family training plans, and distributes them to the family gateway; The home gateway is used to receive and locally cache courseware bundles downloaded from the cloud, aggregate motion and physiological data from wearable sensors, and video streams from environmental cameras. During edge preprocessing, it utilizes the deviation distance of the k-th keypoint. The formula for calculating fixation duration is: , The system calculates children's action completion rate and children's gaze duration characteristics, and then caches the extracted features and uploads them to the cloud AI engine. Wearable sensors and environmental cameras are used to collect physiological signals such as children's movement, heart rate, and skin conductance, as well as visual behavior data, and push them to the home gateway via BLE or RTSP. The supervisory end is used to receive periodic intervention reports and training videos generated by the cloud AI engine, and supports therapists to modify training plans online to achieve closed-loop iteration of training. The system's hardware and software modules are highly integrated, and an edge-cloud collaborative architecture is constructed from data collection and preprocessing to cloud analysis and supervisory feedback. It has scalability, real-time performance and ease of use. As a further implementation of this solution, the system also includes a memory, a processor, and an electronic program stored in the memory and capable of running on the processor. When the processor runs the electronic program, it can implement the steps of any one of claims 1-7. By deploying the executable electronic program on a general-purpose hardware platform, the method is automated, standardized, and reusable, further reducing system deployment costs and improving maintenance efficiency.

[0017] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be noted that due to the limitations of textual expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A family-based collaborative training method for early intervention in autism, characterized by: Includes the following steps: Step 1: Parents fill out the Early Intervention Scale for Autism via a mobile app. The app uploads the questionnaire data and home environment video to the cloud AI assessment service to generate a personalized intervention file and training goals for the child, and conducts assessment and record building. Step 2: The cloud-based AI assessment service automatically generates standardized multi-dimensional training courseware and home training plans based on the training objectives, and pushes them to home devices through the edge gateway to complete the solution distribution; Step 3: At home, parents use a touch-screen device to play training materials according to the training plan. At the same time, wearable sensors collect children's movement and physiological data, and environmental cameras collect visual and behavioral data to carry out training and data collection. Step 4: The home gateway receives raw sensor and video data, uses an embedded AI model to extract key features such as gaze duration, motion standardization, and heart rate changes, and caches the processing results to complete edge preprocessing; Step 5: The edge gateway synchronizes the processed feature data to the cloud AI engine. The cloud AI engine evaluates the training effect based on the preset model, generates periodic intervention reports, and pushes them to the parent app and the supervisor's terminal to complete cloud analysis and feedback. Step 6: The therapist views the intervention report and training video through the supervision terminal, modifies the training plan online or resets the goals. The modifications are synchronized to the cloud AI assessment service and distributed to the family terminal to realize the intervention closed loop and complete the plan iteration and remote supervision.

2. The family-based collaborative training method for early intervention in autism according to claim 1, characterized in that: In step 2, the distribution of the plan further includes: S21: Goal Mapping: Mapping personalized intervention goals for children to a predefined set of training dimensions. ; S22: Courseware Selection: Retrieve multimedia courseware corresponding to each dimension from the content management service; S23: Schedule: Automatically generate daily / weekly training schedules based on mapping results.

3. The family-based collaborative training method for early intervention in autism according to claim 2, characterized in that: In S21, the target mapping stage employs a multi-dimensional target mapping algorithm, the mathematical expression of which is: ； In the formula, This represents the overall score of the m-th training dimension. This represents the standardized score of the nth assessment item. This represents the weight of the nth evaluation item corresponding to the mth training dimension, where N is the total number of evaluation items. The courseware for each dimension is sorted from highest to lowest score, and training plans are formed according to priority.

4. The family-based collaborative training method for early intervention in autism according to claim 1, characterized in that: In step 3, training execution and data acquisition further include the following steps: S31: Parents initiate training on the App, and the touch terminal loads locally cached courseware resources and plays the demonstration. S32: Wearable sensors transmit acceleration, gyroscope, heart rate, and skin conductance signals in real time via BLE; S33: The ambient camera captures video at a preset angle and pushes it to the home gateway.

5. The family-based collaborative training method for early intervention in autism according to claim 1, characterized in that: In step 4, the edge preprocessing stage, the deviation distance of the k-th key point is calculated using the action completion scoring algorithm. The deviation distance of the kth key point The calculation formula is: ； In the formula, This represents the target coordinates of the k-th keypoint in the preset training action. This represents the actual coordinates of the k-th key point detected by the camera, where k is the key point index. The system evaluates the child's performance by calculating the average or weighted sum of the deviation distances of all key points and provides real-time feedback.

6. The family-based collaborative training method for early intervention in autism according to claim 1, characterized in that: In step 4, the home gateway performs the following sub-steps: S41: Time synchronization: NTP calibration timestamps are uniformly applied to wearable data and video frames; S42: Gaze duration calculation: Based on facial landmark tracking, gaze events are detected and accumulated; S43: Feature caching: Cache the extracted gaze duration, action completion rate, and physiological signal features in a local queue.

7. The family-based collaborative training method for early intervention in autism according to claim 1, characterized in that: In S43, the formula for calculating the gaze duration is: ； In the formula, M is the total number of detected gaze events. Indicates the start time of the i-th gaze event. The duration of the gaze indicates the end time of the i-th gaze event, and the gaze duration is used to assess the child's visual concentration in the training materials.

8. The family-based collaborative training system for early intervention in autism according to claim 1, characterized in that: This includes a parent terminal app, a cloud-based AI assessment service module, a home gateway, wearable sensors and environmental cameras, and a monitoring terminal; The parent terminal app is used to collect and fill in the early intervention scale for autism, upload videos of the home environment, and receive personalized training courseware and periodic intervention reports sent from the cloud. The cloud-based AI assessment service module is used to receive scale data and environmental videos from the parent's terminal app, generate personalized intervention files and training goals for children based on the scales and videos, and perform multi-dimensional goal mapping based on the multi-dimensional goal mapping algorithm. It automatically generates standardized multi-dimensional training courseware and family training plans, and distributes them to the family gateway; The home gateway is used to receive and locally cache courseware bundles downloaded from the cloud, aggregate motion and physiological data from wearable sensors, and video streams from environmental cameras. During the edge preprocessing stage, it utilizes the deviation distance of the k-th keypoint. The formula for calculating fixation duration is: , The system calculates children's action completion rate and children's gaze duration characteristics, and then caches the extracted features and uploads them to the cloud AI engine. The wearable sensor and environmental camera are used to collect physiological signals such as children's movement, heart rate, and skin conductance, as well as visual behavior data, and push them to the home gateway via BLE or RTSP. The supervisory terminal is used to receive periodic intervention reports and training videos generated by the cloud AI engine, and supports therapists to modify training plans online to achieve closed-loop iteration of training.

9. The family-based collaborative training system for early intervention in autism according to claim 8, characterized in that, The system further includes a memory, a processor, and an electronic program stored in the memory and capable of running on the processor, wherein the processor, when running the electronic program, is capable of implementing the steps of the method according to any one of claims 1-7.

Images