Multi-mode interactive intelligent wearable walking learning assisting system for visually impaired children based on gait analysis algorithm
By integrating multimodal interactive intelligent wearable toddler assistance system for visually impaired children, the problems of abnormal gait and insufficient feedback during the toddler process of visually impaired children are solved, real-time and high-precision gait monitoring and personalized feedback are achieved, and the safety and efficiency of toddlers are improved.
Patent Information
- Application Number
- CN202510789484.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-08-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Visually impaired children have abnormal gait patterns formed due to the lack of visual information during the toddler process. The existing equipment is expensive and cannot achieve real-time high-precision gait analysis. They lack multimodal interactive feedback and data management capabilities, making it difficult to popularize in a home environment.
A multimodal interactive intelligent wearable toddler assist system for visually impaired children based on gait analysis algorithm is designed, integrating gait detection, multi-sensory assisted interaction, navigation and obstacle avoidance, data fusion and cloud synchronization, learning progress evaluation and update and maintenance modules, adopting lightweight models and edge computing, combining tactile, auditory and visual feedback to achieve real-time monitoring and personalized guidance.
It significantly improves the safety and efficiency of visually impaired children toddlers. Through multimodal interactive feedback and data analysis, personalized training suggestions are provided, which improves the accuracy and real-time gait detection to meet family usage needs.
Smart Images
Figure CN120478113A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of intelligent wearable devices, and in particular provides a walking assistance system for visually impaired children. Background Art
[0002] Visually impaired children face numerous challenges when learning to walk. Due to a lack of visual information, they struggle to observe and imitate normal gait patterns, leading to abnormal gait patterns such as turning their feet inward, turning their feet outward, or dragging their feet. These abnormal gaits not only affect walking efficiency but can also lead to long-term bone and muscle damage. Existing gait analysis technologies primarily rely on specialized equipment and laboratory environments. For example, gait detection solutions based on high-precision motion capture systems are expensive and complex to operate, making them difficult to adopt for home or daily use.
[0003] At the same time, although traditional wearable devices can provide certain gait monitoring functions, due to limited computing resources, they are often unable to achieve real-time, high-precision gait analysis, especially in dynamic environments. The ability to track joint motion trajectories and identify abnormal gaits is insufficient. In addition, existing technologies also have limitations in interactive feedback. There is a lack of multimodal auxiliary interactive systems designed for the characteristics of visually impaired children, such as the organic combination of voice prompts, tactile feedback, and auditory navigation, which results in users being unable to obtain timely and intuitive guidance when learning to walk. At the same time, the lack of data management and analysis capabilities also limits parents and rehabilitation therapists' comprehensive understanding and scientific evaluation of children's gait training progress. Therefore, there is an urgent need for an intelligent wearable walking assistance system that can monitor gait in real time, accurately identify abnormal patterns, and provide personalized multi-sensory feedback to meet the special needs of visually impaired children in the process of learning to walk, and provide parents and rehabilitation institutions with scientific data support and training suggestions. Summary of the Invention
[0004] This paper addresses the problems of visually impaired children experiencing gait abnormalities, difficulty perceiving the environment, and a lack of real-time feedback due to visual impairment during their learning to walk. The paper proposes a multimodal, interactive, intelligent, wearable walking assistance system for visually impaired children based on a gait analysis algorithm. By integrating a gait detection module, a multisensory interaction module, a navigation and obstacle avoidance module, a data fusion and cloud synchronization module, a learning progress assessment module, and an update and maintenance module, the system enables real-time monitoring of gait parameters, abnormal pattern recognition, interactive feedback, and long-term data analysis for visually impaired children.
[0005] The present invention provides a multimodal interactive intelligent wearable walking assistance system for visually impaired children based on a gait analysis algorithm, which includes a gait detection module, a multi-sensory assistance interaction module, a navigation and obstacle avoidance module, a data fusion and cloud synchronization module, a learning progress assessment module, and an update and maintenance module. The gait detection module is used to capture the user's lower limb movements and extract gait parameters; the multi-sensory assistance interaction module is used to provide real-time guidance to the user through tactile, auditory, and visual feedback; the navigation and obstacle avoidance module is used to analyze environmental information in real time and plan a safe walking path; the data fusion and cloud synchronization module is used to store and process user data; the learning progress assessment module is used to record and analyze historical gait parameters to generate personalized training recommendations; and the update and maintenance module is used to support system firmware upgrades and function expansions.
[0006] Furthermore, the gait detection module uses a camera or RGB sensor to capture the user's lower limb movements, and extracts the coordinates of key joints such as the hip, knee, and ankle based on the Medi aPipe or OpenPose algorithm. In particular, to ensure real-time performance, the BlazePose lightweight human posture estimation model is used for real-time tracking, with a frame rate of not less than 30FPS. Combined with an inertial measurement unit (IMU) such as a gyroscope to correct jitter errors in visual data, dynamic tracking and three-dimensional posture estimation are supported. In the analysis of key joint position changes between consecutive frames, a gait cycle curve is generated, including the swing phase and the support phase. Spatiotemporal parameters such as step length, cadence, and speed, as well as dynamic parameters such as joint angle changes and muscle pressure distribution are extracted, and muscle pressure distribution data is obtained through a flexible pressure sensor.
[0007] Furthermore, the current gait data is compared with a baseline of normal gait trained with standard gait models, such as LSTM or Transformer models, to identify abnormal patterns such as pigeon-toed, pigeon-toed, and dragging. To address limb occlusion, time series prediction algorithms such as Kalman filtering are used to supplement missing joint information and ensure data integrity. To meet the low-power, real-time processing requirements of edge computing chips, model pruning and quantization are used to reduce computational complexity, emphasizing a design strategy that combines low-power, real-time processing with high-precision offline analysis.
[0008] Furthermore, the multi-sensory auxiliary interaction module integrates a tactile sensor array, which encodes corresponding instructions through different vibration patterns, such as reminders to correct gait abnormalities or prompts of obstacle distances. Auditory feedback uses directional audio feedback technology, combined with speech synthesis and noise reduction technology, to provide clear voice interaction instructions in noisy environments. Natural language prompts are generated through the RNN / TTS algorithm to enhance user participation. The parent-side App provides a visual interface that displays joint motion trajectory heat maps, gait parameter analysis and time series curves, and supports abnormal alarms such as pop-ups and SMS notifications, as well as the generation of personalized training suggestions.
[0009] Specifically, the navigation and obstacle avoidance module uses a camera or RGB sensor to analyze obstacle distances in real time, combined with Bluetooth low-power beacon positioning technology to plan a safe walking path. It also integrates IMU sensor data, such as acceleration and angular velocity, to improve gait phase detection accuracy. Based on gait parameters and environmental perception results, it assesses the user's gait stability in real time to prevent falls.
[0010] Furthermore, the data fusion and cloud synchronization module stores user data via Firebase or AWS IoT cloud platforms, supporting multi-device access and historical record queries. The visualization engine uses WebGL or Unity to render 3D gait models and replay gait animations generated based on MediaPipe keypoint data. It dynamically adjusts detection confidence thresholds and optimizes gait analysis algorithms based on user behavior. Federated learning technology is used to protect user privacy while enabling data collaboration across multiple devices.
[0011] Specifically, the Learning Progress Assessment module records and analyzes historical gait parameters, including stride length, cadence, joint angle change curves, and the Limb Symmetry Index (LSI). It generates daily, weekly, and monthly gait reports, comparing pre- and post-training data to display progress. Based on abnormal gait patterns, it generates targeted training plans and guides users toward gradual improvement through tactile and auditory feedback.
[0012] Furthermore, the update and maintenance module supports online system firmware upgrades, fixes potential vulnerabilities, and introduces new features. It also provides custom settings options, allowing users to adjust feedback intensity and interaction methods according to actual needs.
[0013] The technical solution of the present invention has the following technical effects:
[0014] By combining camera or RGB sensor data with IMU sensor data, the accuracy of gait detection and anomaly identification can be significantly improved; through lightweight models and edge computing chips, low-power, high-efficiency local real-time processing can be ensured; through multi-sensory interaction design such as tactile, auditory and visual feedback, user participation is enhanced and the toddler experience is improved; through data fusion and cloud synchronization modules, cross-platform compatibility and remote monitoring are achieved, making it easier for parents to analyze user data and develop personalized training plans. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 This is a schematic diagram of the overall process of the multimodal interactive intelligent wearable walking assistance system for visually impaired children based on the gait analysis algorithm of the present invention;
[0016] Figure 2 It is a detailed structural diagram of the gait detection system and multi-sensory auxiliary interaction module of the present invention.
[0017] The accompanying drawings are numbered as follows:
[0018] Camera or RGB sensor; 2. Hip joint; 3. Knee joint; 4. Ankle joint; 5. Analysis of joint position changes between consecutive frames; 6. Gait cycle curve; 7. Spatiotemporal parameters (step length, cadence, and speed); 8. Dynamic parameters (joint angle changes, muscle pressure distribution); 9. Flexible pressure sensor; 10. IMU sensor (gyroscope); 11. Standard gait model; 12. Pigeon-toe abnormal pattern; 13. Pigeon-toe abnormal pattern; 14. Drag-step abnormal pattern; 15. Kalman filter; 16. Lightweight model (B lazePose); 17. Model pruning (Pruning); 18. Model quantization (Quantization); 19. Low-power edge computing chip; 20. Tactile sensor array; 21. Auditory feedback (directional audio feedback technology); 22. Speech recognition algorithm (RNN / TTS); 23. Parent-side app; 24. Data cloud synchronization (Firebase or AWS) IoT); 25. WebGL or Unity visualization engine; 26. 3D gait model playback; 27. Federated learning; 28. Path planning algorithm; 29. Real-time obstacle distance analysis; 30. Historical gait parameter recording; 31. Daily / weekly / monthly gait report; 32. Joint motion trajectory heat map; 33. Abnormal alarm (pop-up window, SMS notification); 34. Personalized training suggestion generation; 35. Update and maintenance module. DETAILED DESCRIPTION
[0019] The multimodal interactive intelligent wearable walking assistance system for visually impaired children based on gait analysis algorithm of the present invention realizes real-time monitoring of gait parameters of visually impaired children, abnormal pattern recognition and interactive feedback by integrating multiple functional modules. Figure 1 and attached Figure 2 , describe the specific implementation of the system in detail.
[0020] In practical applications, when a visually impaired child wears the walking aid, a camera or RGB sensor 1 first captures key information about the user's lower limb movements. The coordinates of the hip joint 2, knee joint 3, and ankle joint 4 are extracted using MediaPipe or OpenPose algorithms. The position data of these key joints is transmitted to the joint position change analysis module 5 between consecutive frames, generating a gait cycle curve 6 that includes complete information about the swing and stance phases. Spatiotemporal parameters 7 such as stride length, cadence, and speed are extracted for further analysis. Dynamic parameters 8 such as joint angle changes and muscle pressure distribution are also acquired using a flexible pressure sensor 9. To ensure high-precision pose estimation, an IMU sensor 10 combines gyroscope data to correct for jitter errors in the visual data, achieving three-dimensional pose estimation. These data together form the basis of a standard gait model 11, which is used to compare current gait data to identify abnormal pigeon-toed patterns 12, pigeon-toed patterns 13, and dragging patterns 14. To address limb occlusion, a Kalman filter 15 algorithm supplements missing joint information to ensure data integrity. In order to meet the low power consumption requirements of the edge computing chip 19, the lightweight model 16 is optimized using BlazePose, while the amount of computation is reduced through model pruning 17 and model quantization 18, enabling the system to achieve real-time processing locally.
[0021] As an important component of the multi-sensory auxiliary interaction module, the tactile sensor array 20 provides command feedback to the user through different vibration patterns. For example, when an abnormal pigeon-toed pattern 12 is detected, the tactile sensor array 20 will emit a vibration of a specific frequency to remind the user to correct the gait. Auditory feedback uses directional audio feedback technology 21 to provide clear voice prompts in noisy environments through speech synthesis and noise reduction technology. The RNN / TTS speech recognition algorithm 22 generates natural language prompts to enhance the user's sense of participation. The parent-side App 23 provides a visual interface that displays a heat map 32 of joint motion trajectories, gait parameter analysis, and time series curves, and supports abnormal alarms 33 in the form of pop-up windows or SMS notifications. In addition, the parent-side App 23 can also generate personalized training suggestions 34 and formulate targeted training plans based on the user's abnormal gait patterns.
[0022] The navigation and obstacle avoidance module uses a camera or RGB sensor 1 to analyze obstacle distances in real time and, combined with Bluetooth low-power beacon positioning technology, plans a safe walking path. The path planning algorithm 28 dynamically adjusts the walking route based on environmental perception to avoid potential hazards. Data from the IMU sensor 10, such as acceleration and angular velocity, is integrated to improve gait phase detection accuracy. This data, combined with real-time analysis of gait stability, helps prevent falls. Through real-time obstacle distance analysis 29, the system can predict and alert users to avoid obstacles.
[0023] The data fusion and cloud synchronization module stores user data via Firebase or the AWS IoT cloud platform24, supporting multi-device access and historical record queries. WebGL or Unity visualization engines25 render 3D gait model playback26, generating complete gait animations based on MediaPipe keypoint data. Federated learning27 technology protects user privacy while enabling data collaboration across multiple devices. Dynamic adjustment of detection confidence thresholds allows the system to optimize gait analysis algorithms based on user behavior. The historical gait parameter recording30 module records multiple metrics, including stride length, cadence, joint angle change curves, and the limb symmetry index (LSI). Daily / weekly / monthly gait reports31 compare pre- and post-training data to display progress and help parents understand their child's toddler progress.
[0024] The Update and Maintenance module regularly checks the system firmware version and supports online upgrades to fix potential vulnerabilities and introduce new features. Custom settings allow users to adjust feedback intensity and interaction methods based on their needs. For example, the vibration intensity and frequency of haptic feedback can be adjusted based on user preferences to suit children of different ages.
[0025] During operation, the system begins by collecting data from a camera or RGB sensor 1. After extracting the coordinates of the hip joint 2, knee joint 3, and ankle joint 4, it enters the inter-frame joint position change analysis 5. By generating a gait cycle curve 6, spatiotemporal parameters 7 and kinetic parameters 8 are extracted, forming a complete gait dataset. This data is input into a standard gait model 11 for abnormal pattern recognition. If an abnormal pigeon-toed pattern 12 or pigeon-toed pattern 13 is detected, the tactile sensor array 20 and directional audio feedback technology 21 are triggered for real-time feedback. Simultaneously, data from the flexible pressure sensor 9 and IMU sensor 10 further enhance the accuracy of gait analysis. A Kalman filter 15 algorithm completes missing joint information in the event of limb occlusion, ensuring the continuity and reliability of data analysis. After optimization through model pruning 17 and model quantization 18, the lightweight model 16 is deployed on a low-power edge computing chip 19 to meet real-time requirements. Throughout the entire process, the data fusion and cloud synchronization module uploads user data to the cloud via Firebase or the AWS IoT cloud platform24. The parent app23 uses the WebGL or Unity visualization engine25 to view 3D gait model playback26 and receive anomaly alerts33 and personalized training recommendations34. Federated learning27 technology protects user privacy while facilitating multi-device collaboration. Historical gait parameter records30 and daily, weekly, and monthly gait reports31 provide a basis for long-term learning progress assessment. Finally, the update and maintenance module ensures continuous system optimization and enhances the user experience.
[0026] The above process not only demonstrates the overall operating principles of the system but also demonstrates the close collaboration between its various modules. By integrating modules such as gait detection, multi-sensory interaction, navigation and obstacle avoidance, data fusion and cloud synchronization, learning progress assessment, and updates and maintenance, this invention successfully addresses the multiple challenges faced by visually impaired children during their learning to walk, significantly improving both efficiency and safety.
Claims
1. A multimodal interactive intelligent wearable walking assistance system for visually impaired children based on gait analysis algorithm, characterized by The system comprises a gait detection module (1), a multi-sensory auxiliary interaction module (20), a navigation and obstacle avoidance module (28), a data fusion and cloud synchronization module (24), a learning progress evaluation module (30), and an update and maintenance module (35), wherein the gait detection module (1) is used to capture the user's lower limb movement and extract gait parameters, the multi-sensory auxiliary interaction module (20) provides real-time guidance to the user through tactile, auditory and visual feedback, the navigation and obstacle avoidance module (28) is used to analyze environmental information in real time and plan a safe walking path, the data fusion and cloud synchronization module (24) is used to store and process user data, the learning progress evaluation module (30) is used to record and analyze historical gait parameters to generate personalized training suggestions, and the update and maintenance module (35) is used to support system firmware upgrades and function expansions.
2. The system according to claim 1, characterized in that The gait detection module (1) uses a camera or RGB sensor (1) to capture the user's lower limb movements and extracts the key joint coordinates of the hip joint (2), knee joint (3), and ankle joint (4) based on the MediaPipe or OpenPose algorithm.
3. The system according to claim 2, characterized in that The BlazePose lightweight human pose estimation model (16) is further used for real-time tracking with a frame rate of no less than 30 frames per second.
4. The system according to claim 1, characterized in that The multi-sensory auxiliary interaction module (20) integrates a tactile sensor array (20), encodes corresponding instructions through different vibration patterns, and uses directional audio feedback technology (21) combined with speech synthesis and noise reduction technology to provide clear voice prompts.
5. The system according to claim 4, characterized in that Furthermore, natural language prompts are generated through the RNN / TTS algorithm (22), and the joint motion trajectory heat map (32) and gait parameter analysis and timing curves are displayed in the parent-side App (23).
6. The system according to claim 1, characterized in that The navigation and obstacle avoidance module (28) uses a camera or RGB sensor (1) to analyze the distance to obstacles in real time and plans a safe walking path in combination with low-power Bluetooth beacon positioning technology.
7. The system according to claim 6, characterized in that Further integration of IMU sensor (10) data such as acceleration and angular velocity can improve the accuracy of gait phase detection.
8. The system according to claim 1, characterized in that The data fusion and cloud synchronization module (24) stores user data through Firebase or AWS IoT cloud platform (24) and uses federated learning technology (27) to protect user privacy.
9. The system according to claim 1, characterized in that The learning progress assessment module (30) records and analyzes historical gait parameters, including step length, step frequency, joint angle change curve, and limb symmetry index (LSI), and generates daily / weekly / monthly gait reports (31).