Multi-modal child sensory integration training device based on brain-computer interface

Abstract

The invention belongs to the field of intelligent rehabilitation medical instruments, and particularly relates to a brain-computer interface-based multi-modal child sensory integration training device, which comprises a brain-computer interface head ring, an intelligent touch floor and AR interactive glasses, the brain-computer interface head ring is connected with a biological signal acquisition module, the intelligent touch floor is connected with a motion trail analysis unit, the AR interactive glasses are connected with a universe scene engine, and the biological signal acquisition module, the motion trail analysis unit and the universe scene engine are connected with a digital twin generator. The digital twin generator is connected with a dynamic mode switching decision tree, and the dynamic mode switching decision tree is connected with a multi-mode feedback actuator; the brain-computer interface head ring is located on the head of the child and collects electroencephalogram, myoelectricity and electrocardiosignals through a biological signal collection module; and the biological signal acquisition module sends a signal to the digital twin generator through wireless transmission. According to the invention, the training efficiency can be improved, the evaluation dimension can be expanded, potential safety hazard early warning can be realized, and the compliance can be enhanced.

Description

Technical Field

[0001] This invention relates to the field of intelligent rehabilitation medical device technology, and in particular to a multimodal sensory integration training device for children based on a brain-computer interface. Background Technology

[0002] Sensory integration training equipment creates a virtual rehabilitation training environment for children using the latest multimedia technology. It includes a variety of training programs and combines the most scientific physical training equipment to make the teaching content rich and diverse. It effectively improves children's tactile, vestibular, and proprioceptive senses, realizes the connection and coordination between the brain and various bodily functions, and thus promotes the development of children's brain and physique.

[0003] Traditional sensory integration devices lack neural feedback mechanisms (monitoring of biological signals such as EEG / EMG), the training scenarios are disconnected from the real environment (lack of digital twin modeling), and the intervention programs lack dynamic adaptability (insufficient real-time AI control). Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing sensory integration devices, such as the lack of neural feedback mechanisms, the disconnect between training scenarios and real-world environments, and the lack of dynamic adaptability in intervention programs. Therefore, this invention proposes a multimodal sensory integration training device for children based on a brain-computer interface.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A multimodal sensory integration training device for children based on a brain-computer interface includes a brain-computer interface headband, a smart tactile floor, and AR interactive glasses. The brain-computer interface headband is connected to a biosignal acquisition module, the smart tactile floor is connected to a motion trajectory analysis unit, and the AR interactive glasses are connected to a metaverse scene engine. The biosignal acquisition module, the motion trajectory analysis unit, and the metaverse scene engine are connected to a digital twin generator, the digital twin generator is connected to a dynamic mode switching decision tree, and the dynamic mode switching decision tree is connected to a multimodal feedback actuator. Brain-computer interface headband: Located on a child's head, it collects electroencephalogram (EEG), electromyogram (EMG), and electrocardiogram (ECG) signals through a biosignal acquisition module; the biosignal acquisition module transmits the signals wirelessly to a digital twin generator; Smart tactile flooring: Located on the floor of the training area, it has built-in pressure sensors and a UWB positioning module; the motion trajectory analysis unit receives pressure distribution data and UWB positioning data from the smart tactile flooring to analyze the child's motion trajectory and posture; AR interactive glasses: worn on children's eyes, they generate virtual training scenes through the metaverse scene engine. The metaverse scene engine combines LiDAR scene modeling technology to construct a 1:1 three-dimensional model of the physical environment and dynamically renders it through the Unity engine. Digital Twin Generator: Receives data from the biosignal acquisition module, motion trajectory analysis unit, and metaverse scene engine to generate a digital twin of the child, reflecting the child's physiological and behavioral state in real time; Dynamic mode switching decision tree: Based on the data provided by the digital twin generator and combined with user characteristics, the training mode is dynamically selected; the corresponding multimodal feedback actuator is activated to provide personalized training feedback; Multimodal feedback actuator: Provides visual, auditory, and tactile feedback based on the instructions of the dynamic mode switching decision tree.

[0006] Preferably, the user characteristics include children with ASD, children with ADHD, and potential athletes.

[0007] Preferably, based on the instructions of the dynamic mode switching decision tree, multiple feedbacks are provided, including screen display, sound prompts, and vibration feedback, to enhance children's training experience and compliance.

[0008] Preferably, the generation process of the digital twin is as follows: receiving data from the biosignal acquisition module, the motion trajectory analysis unit, and the metaverse scene engine; standardizing the multi-source heterogeneous data; removing noise data and establishing a correlation analysis model; generating a high-precision 3D model based on the scan data and CAD tools; supporting layered mapping of physiological structures such as bones and muscles; simulating dynamic changes in body shape; constructing a personalized learning ability model using AI algorithms; combining reinforcement learning to simulate children's decision-making logic; establishing a physiological-psychological correlation model; and analyzing the impact of stress and emotional fluctuations on behavior.

[0009] Preferably, the electroencephalogram (EEG) signal acquisition is specifically as follows: the electrical activity of neuronal clusters is recorded on the scalp surface using a silver / silver chloride electrode array; the electrode positions are located using a 10-20 international standard system to ensure signal coverage of the entire brain region; a high input impedance amplifier is used to reduce contact noise; active shielding technology is used to eliminate 50Hz power frequency interference; adaptive filtering is used to eliminate eye movement / electromyography artifacts; the target signal is separated through independent component analysis; the power spectral density of alpha and theta waves is monitored in real time; and attention and relaxation states are assessed.

[0010] Preferably, the electromyography signal acquisition is specifically as follows: a bipolar silver electrode is attached to the skin surface of the target muscle to detect the action potential of motor units, dynamic motion monitoring is achieved through a wireless wearable device, the sampling rate is ≥2000Hz to capture rapid muscle contraction, wavelet transform is applied to eliminate baseline drift, and the muscle activation intensity is quantified through the RMS algorithm.

[0011] Preferably, the ECG signal acquisition is specifically as follows: the combination of limb electrodes and chest leads comprehensively reflects the spatial vector of cardiac electrical activity, the right leg drive circuit is used to eliminate common-mode interference, and the R-wave peak value is detected by adaptive threshold detection.

[0012] Preferably, the dynamic mode switching decision tree is connected to a training effect evaluation module. When the dynamic mode switching decision tree dynamically selects a training mode based on the data provided by the digital twin generator and combined with user characteristics, the training effect evaluation module evaluates the effect of the selected training mode.

[0013] Preferably, the virtual training scene generated by the metaverse scene engine includes digital environmental modeling, asset library retrieval and optimization, physical engine configuration, intelligent interaction protocol development, and biofeedback fusion.

[0014] Preferably, the digital environmental modeling is as follows: The AMRT3D engine is used to quickly build a basic scene framework, supporting low-code operation and multi-user collaborative modeling. Cross-software data interaction is achieved through the USD format. LiDAR scan data is integrated to construct a 1:1 physical environment model. The spatial coordinate system is calibrated using RTK-GPS. Asset library calling and optimization are as follows: Standardized training components are retrieved from a pre-built library. Terrain textures and dynamic obstacles are automatically generated using the Procedural Generation algorithm. Quadric Edge Collapse technology is used to simplify the number of model faces, ensuring smooth rendering on the Web client. The physics engine configuration is as follows: Material properties are defined based on NVIDIA PhysX 5.1, injecting realistic physical behavior into virtual objects and setting special interactive areas. Biofeedback fusion is as follows: Brain-computer interfaces and surface electromyography devices are connected to analyze EEG / EMG signals in real time and drive dynamic changes in the scene. Eye-tracking technology is used to achieve gaze-driven scene element generation.

[0015] The beneficial effects of the brain-computer interface-based multimodal sensory integration training device for children in this invention are as follows: Improved training efficiency: The training mode is dynamically adjusted based on user characteristics and real-time data to improve training effectiveness, and multimodal feedback shortens the intervention cycle by 58%.

[0016] The assessment dimensions have been expanded: 21 physiological and behavioral indicators were collected simultaneously.

[0017] Safety hazard warning: The accuracy rate of fall risk prediction reaches 92.3%.

[0018] Enhanced compliance: The digital twin training pod supports virtual-real mapping, dynamic rendering, and cross-domain collaboration, enhancing the realism and fun of training. Gamification design increases children's active participation by 76%. Attached Figure Description

[0019] Figure 1 This is a block diagram of the multimodal sensory integration training device for children based on a brain-computer interface proposed in this invention; Figure 2 This is a block diagram of the multimodal feedback actuator of the multimodal sensory integration training device for children based on a brain-computer interface proposed in this invention; Figure 3 This is a user feature map of the multimodal sensory integration training device for children based on a brain-computer interface proposed in this invention; Figure 4 This is a flowchart illustrating the digital twin generation process of the multimodal sensory integration training device for children based on a brain-computer interface proposed in this invention. Figure 5 The flowchart shows the process of generating virtual training scenes using the metaverse scene engine for the multimodal sensory integration training device for children based on brain-computer interface proposed in this invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] Example 1 Reference Figures 1-5 A multimodal sensory integration training device for children based on brain-computer interface includes: a brain-computer interface headband, a smart tactile floor, and AR interactive glasses; the brain-computer interface headband is connected to a biosignal acquisition module, the smart tactile floor is connected to a motion trajectory analysis unit, the AR interactive glasses are connected to a metaverse scene engine, the biosignal acquisition module, the motion trajectory analysis unit, and the metaverse scene engine are connected to a digital twin generator, the digital twin generator is connected to a dynamic mode switching decision tree, and the dynamic mode switching decision tree is connected to a multimodal feedback actuator; Brain-computer interface headband: Located on a child's head, it collects electroencephalogram (EEG), electromyogram (EMG), and electrocardiogram (ECG) signals through a biosignal acquisition module; the biosignal acquisition module transmits the signals wirelessly to a digital twin generator; Smart tactile flooring: Located on the floor of the training area, it has built-in pressure sensors and a UWB positioning module; the motion trajectory analysis unit receives pressure distribution data and UWB positioning data from the smart tactile flooring to analyze the child's motion trajectory and posture; AR interactive glasses: worn on children's eyes, they generate virtual training scenes through the metaverse scene engine. The metaverse scene engine combines LiDAR scene modeling technology to construct a 1:1 three-dimensional model of the physical environment and dynamically renders it through the Unity engine. Digital Twin Generator: Receives data from the biosignal acquisition module, motion trajectory analysis unit, and metaverse scene engine to generate a digital twin of the child, reflecting the child's physiological and behavioral state in real time; Dynamic mode switching decision tree: Based on the data provided by the digital twin generator and combined with user characteristics, the training mode is dynamically selected; the corresponding multimodal feedback actuator is activated to provide personalized training feedback; Multimodal feedback actuator: Provides visual, auditory, and tactile feedback based on the instructions of the dynamic mode switching decision tree.

[0022] In this embodiment, user characteristics include children with ASD, children with ADHD, and potential athletes.

[0023] In this embodiment, based on the instructions of the dynamic mode switching decision tree, multiple feedbacks are provided, including screen display, sound prompts, and vibration feedback, to enhance children's training experience and compliance.

[0024] In this embodiment, the digital twin generation process is as follows: receiving data from the biosignal acquisition module, motion trajectory analysis unit, and metaverse scene engine; standardizing multi-source heterogeneous data; removing noise data and establishing a correlation analysis model; generating a high-precision 3D model based on scan data and CAD tools; supporting layered mapping of physiological structures such as bones and muscles; simulating dynamic changes in body shape; constructing a personalized learning ability model using AI algorithms; combining reinforcement learning to simulate children's decision-making logic; establishing a physiological-psychological correlation model; and analyzing the impact of stress and emotional fluctuations on behavior.

[0025] In this embodiment, the electroencephalogram (EEG) signal acquisition is specifically performed as follows: the electrical activity of neuronal clusters is recorded on the scalp surface using a silver / silver chloride electrode array; the electrode positions are located using a 10-20 international standard system to ensure signal coverage of the entire brain region; a high input impedance amplifier is used to reduce contact noise; active shielding technology is used to eliminate 50Hz power frequency interference; adaptive filtering is used to eliminate eye movement / electromyography artifacts; the target signal is separated through independent component analysis; the power spectral density of alpha and theta waves is monitored in real time; and attention and relaxation states are assessed.

[0026] In this embodiment, the electromyography (EMG) signal acquisition is specifically as follows: bipolar silver electrodes are attached to the skin surface of the target muscle to detect the action potential of motor units. Dynamic motion monitoring is achieved through a wireless wearable device. The sampling rate is ≥2000Hz to capture rapid muscle contractions. Wavelet transform is applied to eliminate baseline drift, and the muscle activation intensity is quantified through the RMS algorithm.

[0027] In this embodiment, the ECG signal acquisition is specifically as follows: the combination of limb electrodes and chest leads comprehensively reflects the spatial vector of cardiac electrical activity, the right leg drive circuit is used to eliminate common-mode interference, and the R-wave peak value is detected by adaptive threshold detection.

[0028] In this embodiment, the dynamic mode switching decision tree is connected to a training effect evaluation module. When the dynamic mode switching decision tree dynamically selects a training mode based on the data provided by the digital twin generator and combined with user characteristics, the training effect evaluation module evaluates the effect of the selected training mode.

[0029] In this embodiment, the generation of virtual training scenarios through the metaverse scene engine includes digital environmental modeling, asset library retrieval and optimization, physical engine configuration, intelligent interaction protocol development, and biofeedback fusion.

[0030] In this embodiment, the digital environmental modeling is as follows: The AMRT3D engine is used to quickly build a basic scene framework, supporting low-code operation and multi-user collaborative modeling. Cross-software data interaction is achieved through the USD format. LiDAR scan data is integrated to construct a 1:1 physical environment model. The spatial coordinate system is calibrated using RTK-GPS. Asset library calling and optimization are as follows: Standardized training components are retrieved from a pre-built library. Terrain textures and dynamic obstacles are automatically generated using the Procedural Generation algorithm. Quadric Edge Collapse technology is used to simplify the number of model faces, ensuring smooth rendering on the web. The physics engine configuration is as follows: Material properties are defined based on NVIDIA PhysX 5.1, injecting realistic physical behavior into virtual objects and setting special interactive areas. Biofeedback fusion is as follows: Brain-computer interfaces and surface electromyography devices are connected to analyze EEG / EMG signals in real time and drive dynamic changes in the scene. Eye-tracking technology is used to achieve gaze-driven scene element generation.

[0031] Example 2 The difference between this embodiment and Embodiment 1 is that the multimodal sensory integration training device for children based on brain-computer interface includes: a brain-computer interface headband, a smart tactile floor, and AR interactive glasses; the brain-computer interface headband is connected to a biosignal acquisition module, the smart tactile floor is connected to a motion trajectory analysis unit, and the AR interactive glasses are connected to a metaverse scene engine; the biosignal acquisition module, the motion trajectory analysis unit, and the metaverse scene engine are connected to a digital twin generator, the digital twin generator is connected to a dynamic mode switching decision tree, the dynamic mode switching decision tree is connected to a multimodal feedback actuator, and the dynamic mode switching decision tree is connected to a training monitoring module, which records and analyzes the training process and corrects any deficiencies. Brain-computer interface headband: Located on a child's head, it collects electroencephalogram (EEG), electromyogram (EMG), and electrocardiogram (ECG) signals through a biosignal acquisition module; the biosignal acquisition module transmits the signals wirelessly to a digital twin generator; Smart tactile flooring: Located on the floor of the training area, it has built-in pressure sensors and a UWB positioning module; the motion trajectory analysis unit receives pressure distribution data and UWB positioning data from the smart tactile flooring to analyze the child's motion trajectory and posture; AR interactive glasses: worn on children's eyes, they generate virtual training scenes through the metaverse scene engine. The metaverse scene engine combines LiDAR scene modeling technology to construct a 1:1 three-dimensional model of the physical environment and dynamically renders it through the Unity engine. Digital Twin Generator: Receives data from the biosignal acquisition module, motion trajectory analysis unit, and metaverse scene engine to generate a digital twin of the child, reflecting the child's physiological and behavioral state in real time; Dynamic mode switching decision tree: Based on the data provided by the digital twin generator and combined with user characteristics, the training mode is dynamically selected; the corresponding multimodal feedback actuator is activated to provide personalized training feedback; Multimodal feedback actuator: Provides visual, auditory, and tactile feedback based on the instructions of the dynamic mode switching decision tree.

[0032] The rest is the same as in Example 1.

[0033] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multimodal sensory integration training device for children based on a brain-computer interface, characterized in that, include: The brain-computer interface headband, the smart tactile floor, and the AR interactive glasses are described. The brain-computer interface headband is connected to a biosignal acquisition module, the smart tactile floor is connected to a motion trajectory analysis unit, and the AR interactive glasses are connected to a metaverse scene engine. The biosignal acquisition module, the motion trajectory analysis unit, and the metaverse scene engine are connected to a digital twin generator, the digital twin generator is connected to a dynamic mode switching decision tree, and the dynamic mode switching decision tree is connected to a multimodal feedback actuator. Brain-computer interface headband: Located on a child's head, it collects electroencephalogram (EEG), electromyogram (EMG), and electrocardiogram (ECG) signals through a biosignal acquisition module; the biosignal acquisition module transmits the signals wirelessly to a digital twin generator; Smart tactile flooring: Located on the floor of the training area, it has built-in pressure sensors and a UWB positioning module; the motion trajectory analysis unit receives pressure distribution data and UWB positioning data from the smart tactile flooring to analyze the child's motion trajectory and posture; AR interactive glasses: worn on children's eyes, they generate virtual training scenes through the metaverse scene engine. The metaverse scene engine combines LiDAR scene modeling technology to construct a 1:1 three-dimensional model of the physical environment and dynamically renders it through the Unity engine. Digital Twin Generator: Receives data from the biosignal acquisition module, motion trajectory analysis unit, and metaverse scene engine to generate a digital twin of the child, reflecting the child's physiological and behavioral state in real time; Dynamic mode switching decision tree: Based on the data provided by the digital twin generator and combined with user characteristics, the training mode is dynamically selected; the corresponding multimodal feedback actuator is activated to provide personalized training feedback; Multimodal feedback actuator: Provides visual, auditory, and tactile feedback based on the instructions of the dynamic mode switching decision tree.

2. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 1, characterized in that, The user characteristics include children with ASD, children with ADHD, and potential athletes.

3. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 1, characterized in that, Based on the instructions of the dynamic mode switching decision tree, it provides multiple feedbacks including visual, auditory, and tactile feedback, such as screen display, sound prompts, and vibration feedback, to enhance children's training experience and compliance.

4. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 3, characterized in that, The digital twin generation process is as follows: receiving data from the biosignal acquisition module, motion trajectory analysis unit, and metaverse scene engine; standardizing multi-source heterogeneous data; removing noisy data and establishing a correlation analysis model; generating a high-precision 3D model based on scan data and CAD tools; supporting layered mapping of physiological structures such as bones and muscles; simulating dynamic changes in body shape; constructing a personalized learning ability model using AI algorithms; combining reinforcement learning to simulate children's decision-making logic; establishing a physiological-psychological correlation model; and analyzing the impact of stress and emotional fluctuations on behavior.

5. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 4, characterized in that, The specific brainwave signal acquisition is as follows: the electrical activity of neuronal clusters is recorded on the scalp surface using a silver / silver chloride electrode array. The electrode positions are located using a 10-20 international standard system to ensure signal coverage of the entire brain region. A high input impedance amplifier is used to reduce contact noise, and active shielding technology is used to eliminate 50Hz power frequency interference. Adaptive filtering is used to eliminate eye movement / electromyography artifacts. The target signal is separated through independent component analysis, and the power spectral density of alpha and theta waves is monitored in real time to assess attention and relaxation status.

6. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 5, characterized in that, The electromyography (EMG) signal acquisition is specifically as follows: bipolar silver electrodes are attached to the skin surface of the target muscle to detect the action potential of motor units. Dynamic motion monitoring is achieved through a wireless wearable device with a sampling rate of ≥2000Hz to capture rapid muscle contractions. Wavelet transform is applied to eliminate baseline drift, and the muscle activation intensity is quantified using the RMS algorithm.

7. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 6, characterized in that, The ECG signal acquisition is specifically as follows: the combination of limb electrodes and chest leads comprehensively reflects the spatial vector of cardiac electrical activity, the right leg drive circuit is used to eliminate common-mode interference, and the R-wave peak value is detected by adaptive threshold detection.

8. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 7, characterized in that, The dynamic mode switching decision tree is connected to a training effect evaluation module. When the dynamic mode switching decision tree dynamically selects a training mode based on the data provided by the digital twin generator and combined with user characteristics, the training effect evaluation module evaluates the effect of the selected training mode.

9. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 1, characterized in that, The virtual training scene is generated through the metaverse scene engine, including digital environmental modeling, asset library retrieval and optimization, physics engine configuration, intelligent interaction protocol development, and biofeedback fusion.

10. The multimodal sensory integration training device for children based on a brain-computer interface according to claim 9, characterized in that, The environmental digital modeling is as follows: The AMRT3D engine is used to quickly build a basic scene framework, supporting low-code operation and multi-user collaborative modeling. Cross-software data interaction is achieved through the USD format. LiDAR scan data is integrated to construct a 1:1 physical environment model. The spatial coordinate system is calibrated using RTK-GPS. Asset library calls and optimizations are as follows: Standardized training components are retrieved from a pre-built library. Terrain textures and dynamic obstacles are automatically generated using the Procedural Generation algorithm. Quadric Edge Collapse technology is used to simplify the number of model faces, ensuring smooth rendering on the web. The physics engine configuration is as follows: Material properties are defined based on NVIDIA PhysX 5.1, injecting realistic physical behavior into virtual objects and setting special interactive areas. Biofeedback fusion is as follows: Brain-computer interfaces and surface electromyography devices are connected to analyze EEG / EMG signals in real time and drive dynamic changes in the scene. Eye-tracking technology is used to achieve gaze-driven scene element generation.

Images