A somatosensory interaction fault-tolerant method suitable for ASD children

Abstract

The application discloses a somatosensory interaction fault-tolerant method suitable for ASD children, relates to the technical field of children's mental health, and comprises the following steps: S1, constructing an ASD child motion feature library; S2, gradientized motion fault-tolerant determination; S3, real-time emotion self-adaptive adjustment; S4, interactive rhythm dynamic adaptation; and S5, positive feedback reinforcement. The somatosensory interaction fault-tolerant method suitable for ASD children accurately adapts to the core development characteristics of ASD children, such as small motion amplitude, poor continuity, weak limb control, sensory sensitivity and short attention maintenance, greatly reduces the somatosensory interaction triggering and completion threshold, reduces the frustration, anxiety and emotional resistance of children from the root, significantly improves the interaction compliance and active participation willingness of ASD children, realizes stress-free, sustainable and personalized somatosensory interaction rehabilitation intervention, and effectively solves the technical pain points that existing somatosensory solutions are not suitable for ASD children and cannot be stably used.

Description

Technical Field

[0001] This invention relates to the field of children's mental health technology, specifically to a motion-sensing interaction fault-tolerant method adapted for children with ASD. Background Technology

[0002] With the development of wearable motion-sensing interaction technology, it has been gradually applied to various fields such as children's education and rehabilitation training. However, the motion judgment logic and interaction rules of existing motion-sensing interaction solutions are mainly designed for healthy people, and have serious adaptation defects for children with autism spectrum disorder (ASD), failing to meet the usage needs of children with ASD. Specifically, the following technical defects exist: Children with ASD generally exhibit developmental characteristics such as small range of motion, poor motor coherence, short attention span, and weak limb control. Existing motion-sensing interaction solutions have high trigger thresholds and strict judgment standards, making it difficult for children with ASD to complete standard interaction tasks. This can easily lead to feelings of frustration, resulting in extremely poor user compliance and hindering continuous rehabilitation training and intervention. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a motion-sensing interaction fault-tolerant method adapted for children with ASD, solving the problems mentioned in the background section.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a motion-sensing interaction fault-tolerant method adapted for children with ASD, comprising the following steps: S1. Construction of ASD Children's Action Feature Database: Collect action samples of children with ASD of different severity, construct an ASD children's action feature database including small-amplitude movements, discontinuous movements, stereotyped movements, and partially occluded movements, label the effective judgment thresholds, fault tolerance windows and triggering conditions corresponding to each type of movement, and set three basic fault tolerance levels according to mild, moderate and severe ASD. S2. Gradient Action Fault Tolerance Judgment: Based on the ASD children's action feature library, a gradient action fault tolerance judgment mechanism is constructed, which includes amplitude fault tolerance, time fault tolerance, trajectory fault tolerance, and passive action fault tolerance. The action effective judgment rules are dynamically adjusted according to the selected basic fault tolerance level. S3. Real-time Adaptive Emotion Adjustment: Through voice emotion recognition and body movement feature recognition, the emotional state of children with ASD is obtained in real time, and the fault tolerance level and scene sensory stimulation parameters are automatically adjusted according to the emotional state to achieve adaptive switching of interaction mode. S4. Dynamic Adaptation of Interactive Rhythm: Based on the speed at which children with ASD complete their actions and their reaction time, dynamically adjust the countdown time of interactive tasks, the animation playback speed, and the level of operation guidance to adapt to the child's cognitive and motor rhythm. S5, Positive Feedback Reinforcement: For every effective interaction by a child with ASD, personalized positive visual and auditory feedback is triggered, with no negative failure prompts throughout the process.

[0005] Furthermore, in step S2, the specific rule for amplitude tolerance is as follows: using 100% of the standard movement amplitude of healthy people as the baseline trigger threshold, and according to the selected basic tolerance level, the effective trigger threshold is lowered to 30%-80%, and the child's completion of the corresponding amplitude movement is judged as an effective movement.

[0006] Furthermore, in step S2, the specific rules for time tolerance are as follows: based on the standard action judgment window of 1s for healthy people, the effective judgment window is extended to 3s-5s according to the selected basic tolerance level, allowing children to complete the target action step by step and in a non-continuous manner.

[0007] Furthermore, in step S2, the specific rule for trajectory tolerance is: to broaden the scope of motion trajectory judgment, to judge only the core features of the motion, and to ignore unnecessary motion trajectory detail deviations.

[0008] Furthermore, in step S2, the specific rules for passive action tolerance are as follows: for children with severe ASD, the active action triggering rules are adjusted to be that the duration of the child's eye contact with the screen interaction area is ≥0.5s, or the passive limb action matches the target action, which is then judged as an effective interaction.

[0009] Furthermore, in step S3, the real-time emotion adaptive adjustment specifically involves: when the child is identified as exhibiting irritability, anxiety, or resistance, the error tolerance level is automatically increased, the screen flickering effect is turned off, the sound effect volume is reduced, and a low sensory stimulation mode is switched; when the child's emotions are identified as stable and the continuous action completion rate meets the standard, the error tolerance level is gradually reduced within a preset range, and the action judgment requirements are increased.

[0010] Furthermore, in step S4, the dynamic adaptation of the interaction rhythm also includes: supporting the pause of the interaction process at any time, step-by-step guidance, disabling the automatic process advancement mechanism, and allowing children to control the interaction rhythm independently.

[0011] Furthermore, in step S5, the positive feedback reinforcement specifically involves: customizing feedback content based on the child's interests and preferences, setting the feedback intensity in a gradient manner according to the degree of action completion, outputting only positive incentive content throughout the process, and having no error prompts or failure penalty mechanisms.

[0012] Furthermore, the following steps are also included: S6, Social Intent Recognition and Enhanced Fault Tolerance for Social Interaction S6-1, Social Scene Recognition and Subject Segmentation: Detects dual and multi-person human sensor signals and switches to social interaction mode. The image is segmented through the motion sensor camera to distinguish between the ASD child and social companions, and locks in limb, facial and relative distance information. S6-2, Real-time recognition of multimodal social intent: Integrates visual and speech recognition, collects social eye contact, social approach, social body movements, and social vocal signals, and determines the existence of social intent if any signal is met; S6-3, Five-dimensional social action gradient fault tolerance judgment: In addition to amplitude fault tolerance, time fault tolerance, trajectory fault tolerance and passive action fault tolerance, social collaboration fault tolerance is added. The amplitude, time and collaboration judgment threshold of social actions are adjusted according to the mild, moderate and severe ASD levels. The collaborative interaction is still judged to be effective if the response delay is ≤3s. S6-4, Social Emotion and Rhythm Adaptive Adjustment: Adjusts the fault tolerance level and sensory stimulation parameters according to the social emotional state, cancels automatic countdown, supports pausing and repeating social processes, and allows children to control the rhythm of social interaction independently. S6-5. Social-Specific Positive Feedback and Data Recording: Output graded positive feedback based on social intent and action completion, record social interaction data, and generate a social skills growth curve.

[0013] This invention provides a motion-sensing interaction fault-tolerant method adapted for children with ASD, which has the following beneficial effects: 1. This motion-sensory interaction error-tolerant method adapted for children with ASD constructs a motion feature library specifically for children with ASD and a multi-dimensional gradient motion error-tolerant judgment mechanism. Combined with real-time emotion adaptive adjustment, dynamic adaptation of interaction rhythm, and full-process positive feedback reinforcement, it accurately adapts to the core developmental characteristics of children with ASD, such as small movement range, poor continuity, weak limb control, sensory sensitivity, and short attention span. It significantly reduces the threshold for triggering and completing motion-sensory interaction, fundamentally reducing children's frustration, anxiety, and emotional resistance. It significantly improves children with ASD's interaction compliance and willingness to actively participate, achieving stress-free, sustainable, and personalized motion-sensory interaction rehabilitation intervention. It effectively solves the technical pain points of existing motion-sensory solutions being poorly adapted to children with ASD and unable to be used stably.

[0014] 2. This motion-sensory interaction fault-tolerant method adapted for children with ASD significantly reduces the difficulty of two-person and group social interactions for children with ASD by capturing multimodal social intentions at low thresholds, providing fault tolerance for five-dimensional social action gradients, and adaptively adjusting social rhythm. It eliminates social interaction pressure and frustration, effectively stimulating children's social initiative. At the same time, it can quantitatively record social interaction data and generate ability growth curves, providing an objective assessment basis for social rehabilitation training of children with ASD. It expands motion-sensory interaction technology from single-person intervention to social training scenarios, greatly improving the application scope and rehabilitation intervention value of the solution. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating steps S1-S5 of a motion-sensing interaction fault-tolerant method adapted for children with ASD according to the present invention. Figure 2 This is a flowchart illustrating step S6 of the present invention, which is a method for adapting motion-sensing interaction to children with ASD. Detailed Implementation

[0016] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0017] like Figures 1-2 As shown, this invention provides a technical solution: a motion-sensing interaction fault-tolerant method adapted for children with ASD. This embodiment is based on a 43-inch motion-sensing smart screen hardware terminal equipped with a domestically produced RK3588 main control chip, adapted to the Octonauts geography science motion-sensing intervention course developed with the UE5 engine, and implements the motion-sensing interaction fault-tolerant method adapted for children with ASD as described in this invention for children aged 3-12. The specific implementation steps are as follows: Step 1: Construction of the ASD Children's Action Feature Database Using motion capture and annotation technology, motion samples from children with ASD of varying severity are collected to construct a motion feature library for children with ASD, including small-amplitude movements, disjointed movements, stereotyped movements, and partially occluded movements. Effective judgment thresholds, tolerance windows, and triggering conditions for each type of movement are annotated, and three basic tolerance levels are set according to mild, moderate, and severe ASD. This achieves the function of establishing a specific motion benchmark for ASD, thus adapting to the motion characteristics of children with different degrees of severity. Specifically, action samples were collected from children aged 3-12 with mild, moderate, and severe ASD, accumulating over 1000 hours of valid action data to construct a dedicated action feature library for children with ASD. This feature library includes 12 typical action features of children with ASD, such as small-amplitude jumps, disjointed hand waving, stereotyped body movements, and movements obscured by the body. For each type of action, corresponding effective judgment thresholds, tolerance windows, and triggering conditions were labeled. At the same time, three basic tolerance levels were set according to the severity of ASD: Level 1 tolerance for mild ASD, Level 2 tolerance for moderate ASD, and Level 3 tolerance for severe ASD. The higher the tolerance level, the lower the threshold for effective triggering of the action. Step 2: Configure Gradient Action Fault Tolerance Employing multi-dimensional threshold matching technology, and based on the ASD children's action feature library, a gradient action fault tolerance judgment mechanism is constructed, including amplitude fault tolerance, time fault tolerance, trajectory fault tolerance, and passive action fault tolerance. The action effective judgment rules are dynamically adjusted according to the selected basic fault tolerance level, thereby realizing the flexible adaptation function of action judgment and reducing the threshold for interaction triggering. Specifically, in the motion-sensing interaction determination of the UE5 engine, a gradient-based action fault-tolerance determination mechanism is embedded, and the specific configuration rules for the four dimensions are as follows: (1) Amplitude tolerance: The amplitude threshold corresponding to the first level of tolerance is 80% of the standard amplitude of healthy people, the second level of tolerance is 50%, and the third level of tolerance is 30%. Children are judged to be valid actions when they complete the corresponding amplitude of jumping, waving, and grabbing the target. (2) Time tolerance: The effective judgment window for the first level of tolerance is 2s, the second level of tolerance is 3s, and the third level of tolerance is 5s, allowing children to complete the action step by step within the extended judgment window without having to complete the standard action in one continuous motion. (3) Trajectory tolerance: The allowable range of motion trajectory deviation corresponding to Level 1 tolerance is 20% of the standard trajectory, Level 2 tolerance is 40%, and Level 3 tolerance is 60%. Only the core direction and end position of the motion are judged, and unnecessary detail deviations of the intermediate trajectory are ignored. (4) Passive action tolerance: In the three-level tolerance mode, the passive action judgment rule is enabled. If the child's eyes are in contact with the screen interaction area for ≥0.5s, or if the passive limb action is matched with the target action with the assistance of the parents, it is judged as an effective interaction, which is suitable for the interaction needs of children with severe ASD. Step 3: Real-time adaptive emotion adjustment Employing multimodal emotion recognition technology, this system acquires the emotional state of children with ASD in real time through voice emotion recognition and body movement feature recognition. Based on the emotional state, it automatically adjusts the fault tolerance level and scene sensory stimulation parameters to achieve adaptive switching of interaction modes. This enables the function of emotion-fault tolerance linkage regulation, thereby reducing children's emotional resistance. Specifically, the device collects children's voice data through the 4-array microphones of the motion-sensing smart screen, collects children's body movements and facial micro-expression data through the binocular camera, and deploys a conventional multimodal emotion recognition model on the terminal side to identify children's six emotional states in real time: happy, calm, depressed, irritable, anxious, and fearful. When the system detects that a child is irritable, anxious, or resistant, it automatically raises the fault tolerance level by one level. At the same time, it turns off the flashing particle effects in the scene, reduces the background sound volume by 30%, lowers the screen saturation to 0.8, and switches to a low sensory stimulation mode. When the system detects that the child has successfully completed the action more than three times in a row and is in a stable and positive mood, it gradually lowers the fault tolerance level within a preset safety range and raises the action judgment requirements to achieve step-by-step motor ability training. Step 4: Dynamically Adapting the Interaction Rhythm Employing rhythm perception and dynamic scheduling technology, the countdown timer, animation playback speed, and operation guidance level of interactive tasks are dynamically adjusted according to the speed of action completion and reaction time of children with ASD, adapting to the child's cognitive and motor rhythm; realizing the personalized matching function of interactive rhythm, thereby improving children's operation compliance; Specifically, the system tracks children's average reaction time and task completion time in real time and dynamically adjusts the interaction rhythm: when a child's average reaction time is greater than 2 seconds, the task countdown time is automatically doubled, the animation playback speed is reduced by 50%, and step-by-step voice guidance is enabled, with each step accompanied by clear voice prompts and visual guidance; at the same time, the system allows children to pause the interaction process at any time, with no automatic timeout failure mechanism, allowing children to fully control the interaction rhythm and adapting to the cognitive and motor rhythms of different children; Step 5: Positive Feedback Reinforcement Employing multimedia incentive output technology, it triggers personalized positive visual and auditory feedback for every effective interaction by children with ASD, with no negative failure prompts throughout the process; thus achieving a positive incentive guidance function, protecting children's self-confidence and enhancing their willingness to interact proactively; Specifically, for each effective interaction by the child, regardless of the degree of completion, corresponding positive feedback is triggered: basic encouragement feedback is triggered for 30%-50% completion, moderate incentive feedback is triggered for 50%-80% completion, and high-intensity reward feedback is triggered for completion above 80%. The feedback content can be customized according to the child's interests and preferences. In this embodiment, positive interactive voice and animation of IP characters in the game scene are triggered first. There are no prompts or punishment mechanisms related to "error" or "failure" throughout the process. Only guided voice prompts are used to prompt the child to adjust the action, continuously strengthening the child's willingness to actively interact. This embodiment has been clinically piloted and verified. Among the ASD children participating in the pilot program, 87.5% were able to complete the entire somatosensory intervention course independently without any obvious resistance. Children who participated for more than 4 consecutive weeks showed significant improvements in their ability to complete actions and their willingness to actively interact, thus verifying the feasibility and beneficial effects of the technical solution of this invention.

[0018] Step 6: Implementation of Social Intent Recognition and Social Interaction Fault Tolerance Enhancement When the system detects a two-person motion sensing signal (ASD child + companion or therapist), it automatically switches to a social interaction motion sensing scene, and uses a binocular camera to complete limb segmentation and lock the limb and facial information of the ASD child and companion. Social intent recognition uses a low threshold for determination. If a child looks at a peer's face for ≥0.3 seconds, reaches out to a peer, or makes a social sound, it is determined that there is a social intent and the social interaction process is triggered. Social gesture tolerance is adapted according to ASD level: For children with mild ASD, social waving and handshaking gestures with an amplitude of 60% and completed within 3 seconds are considered valid; for children with moderate ASD, gestures with an amplitude of 30% and completed within 5 seconds are considered valid; for children with severe ASD, social gestures are considered valid as long as the limbs face the peer or are completed within 8 seconds or with assistance, and the peer response delay is ≤3 seconds, which is still considered to be effective. When the system detects that a child is experiencing social anxiety, it immediately increases the social tolerance level, turns off scene flashing effects, lowers the sound volume, and uses voice prompts to guide the companion to "wait for the child," thus extending the social waiting time. Based on children's social intentions and actions, positive feedback is output in stages: when a social intention is recognized, the program plays "You want to interact with your friends, great job!"; when a social action is completed, the program plays "You successfully shook hands with your friend!"; and when a standard social action is completed, a reward animation for two people on the same screen is triggered. The system records the number of times children intend to socialize and the rate of their independent socialization in real time, generating a daily social skills growth curve to provide objective intervention data for rehabilitation therapists. In summary, this motion-sensory interaction error-tolerant method adapted for children with ASD constructs a motion feature library specifically for children with ASD and a multi-dimensional gradient motion error-tolerant judgment mechanism. Combined with real-time emotion adaptive adjustment, dynamic adaptation of interaction rhythm, and full-process positive feedback reinforcement, it accurately adapts to the core developmental characteristics of children with ASD, such as small motion amplitude, poor continuity, weak limb control, sensory sensitivity, and short attention span. It significantly reduces the threshold for triggering and completing motion-sensory interaction, fundamentally reducing children's frustration, anxiety, and emotional resistance. It significantly improves children with ASD's interaction compliance and willingness to actively participate, achieving stress-free, sustainable, and personalized motion-sensory interaction rehabilitation intervention. It effectively solves the technical pain points of existing motion-sensory solutions being poorly adapted to children with ASD and unable to be used stably. By capturing multimodal social intentions at low thresholds, allowing for five-dimensional social action gradient fault tolerance, and adaptive adjustment of social rhythm, the difficulty of two-person and small-group social interactions for children with ASD is significantly reduced, eliminating social interaction pressure and frustration, and effectively stimulating children's social initiative. At the same time, social interaction data can be quantitatively recorded and ability growth curves can be generated, providing an objective assessment basis for social rehabilitation training for children with ASD. This expands the somatosensory interaction technology from single-person intervention to social training scenarios, greatly enhancing the application scope and rehabilitation intervention value of the solution.

[0019] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A motion-sensing interaction fault-tolerant method adapted for children with ASD, characterized in that: Includes the following steps: S1. Construction of ASD Children's Action Feature Database: Collect action samples of children with ASD of different severity, construct an ASD children's action feature database including small-amplitude movements, discontinuous movements, stereotyped movements, and partially occluded movements, label the effective judgment thresholds, fault tolerance windows and triggering conditions corresponding to each type of movement, and set three basic fault tolerance levels according to mild, moderate and severe ASD. S2. Gradient Action Fault Tolerance Judgment: Based on the ASD children's action feature library, a gradient action fault tolerance judgment mechanism is constructed, which includes amplitude fault tolerance, time fault tolerance, trajectory fault tolerance, and passive action fault tolerance. The action effective judgment rules are dynamically adjusted according to the selected basic fault tolerance level. S3. Real-time Adaptive Emotion Adjustment: Through voice emotion recognition and body movement feature recognition, the emotional state of children with ASD is obtained in real time, and the fault tolerance level and scene sensory stimulation parameters are automatically adjusted according to the emotional state to achieve adaptive switching of interaction mode. S4. Dynamic Adaptation of Interactive Rhythm: Based on the speed at which children with ASD complete their actions and their reaction time, dynamically adjust the countdown time of interactive tasks, the animation playback speed, and the level of operation guidance to adapt to the child's cognitive and motor rhythm. S5. Positive Feedback Reinforcement: For every effective interaction by a child with ASD, personalized positive visual and auditory feedback is triggered, with no negative failure prompts throughout the process.

2. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S2, the specific rule for amplitude tolerance is as follows: the trigger threshold is based on 100% of the standard movement amplitude of healthy people. According to the selected basic tolerance level, the effective trigger threshold is lowered to 30%-80%. When a child completes a movement of the corresponding amplitude, it is judged as an effective movement.

3. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S2, the specific rules for time tolerance are as follows: based on the standard action judgment window of 1s for healthy people, the effective judgment window is extended to 3s-5s according to the selected basic tolerance level, allowing children to complete the target action step by step and in a non-continuous manner.

4. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S2, the specific rule for trajectory tolerance is to broaden the scope of motion trajectory judgment, judge only the core features of the motion, and ignore unnecessary motion trajectory detail deviations.

5. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S2, the specific rules for passive action tolerance are as follows: for children with severe ASD, the active action triggering rules are adjusted to be that the duration of the child's eye contact with the screen interaction area is ≥0.5s, or the passive limb action matches the target action, which is then judged as an effective interaction.

6. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S3, the real-time emotion adaptive adjustment specifically involves: when the child is identified as being irritable, anxious, or resistant, the error tolerance level is automatically increased, the screen flickering effect is turned off, the sound effect volume is reduced, and the mode is switched to low sensory stimulation; when the child's emotions are identified as stable and the continuous action completion rate is met, the error tolerance level is gradually reduced within a preset range, and the action judgment requirements are increased.

7. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S4, the dynamic adaptation of the interaction rhythm also includes: supporting the pause of the interaction process at any time, step-by-step guidance, turning off the automatic process advancement mechanism, and allowing children to control the interaction rhythm independently.

8. The motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: In step S5, positive feedback reinforcement specifically means: customizing feedback content according to the child's interests and preferences, setting the feedback intensity in a gradient according to the degree of action completion, outputting only positive incentive content throughout the process, and having no error prompts or failure punishment mechanisms.

9. A motion-sensing interaction fault-tolerant method adapted for children with ASD according to claim 1, characterized in that: It also includes the following steps: S6, Social Intent Recognition and Enhanced Fault Tolerance for Social Interaction S6-1, Social Scene Recognition and Subject Segmentation: Detects dual and multi-person human sensor signals and switches to social interaction mode. The image is segmented through the motion sensor camera to distinguish between the ASD child and social companions, and locks in limb, facial and relative distance information. S6-2, Real-time recognition of multimodal social intent: Integrates visual and speech recognition, collects social eye contact, social approach, social body movements, and social vocal signals, and determines the existence of social intent if any signal is met; S6-3, Five-dimensional social action gradient fault tolerance judgment: In addition to amplitude fault tolerance, time fault tolerance, trajectory fault tolerance and passive action fault tolerance, social collaboration fault tolerance is added. The amplitude, time and collaboration judgment threshold of social actions are adjusted according to the mild, moderate and severe ASD levels. The collaborative interaction is still judged to be effective if the response delay is ≤3s. S6-4, Social Emotion and Rhythm Adaptive Adjustment: Adjusts the fault tolerance level and sensory stimulation parameters according to the social emotional state, cancels automatic countdown, supports pausing and repeating social processes, and allows children to control the rhythm of social interaction independently. S6-5. Social-Specific Positive Feedback and Data Recording: Output graded positive feedback based on social intent and action completion, record social interaction data, and generate a social skills growth curve.

Images