Intelligent blind person navigation system

Abstract

The invention provides an intelligent blind person navigation system, and relates to the technical field of intelligent wearable blind person navigation, the intelligent blind person navigation system comprises a multi-mode sensing unit, an A I decision center, a bidirectional feedback module and an emergency response system.According to the intelligent blind person navigation system, through a three-layer sensing architecture of a dual-light fusion camera, a laser radar and an ultrasonic array, 94% of obstacle detection rate is still kept under 5lux low illumination, and the blind person navigation efficiency is improved. And the effective detection distance reaches 8 meters. The semantic segmentation algorithm enables the detection rate of the suspended obstacle to rise from 12% to 92%, and the recognition error of the glass door in the underground garage scene is only 2.1 cm. And a multi-sensor space-time alignment technology is adopted, so that the false alarm rate in a complex scene is reduced from 25% to 3.2%, and all-weather and full-height accurate sensing of obstacles is realized. A biological characteristic + environmental risk multi-stage triggering mechanism is adopted, automatic alarming and shared positioning are performed within 2.1 seconds when a user falls down, and the success rate of rescue for falling down of the underground garage is increased from 62% to 100%. An environment video with positioning is automatically recorded without operation timeout, and a'prevention-early warning-rescue 'closed-loop safety net is constructed.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent wearable blind navigation, and more specifically, particularly relates to an intelligent blind navigation system. Background Art

[0002] Existing blind navigation systems have problems such as single sensor dependence (e.g., the positioning error of lidar is 37% in 65dB noise, and binocular vision fails 82% at <10lux), one-way and inefficient feedback mechanisms (the tactile guide cane can only detect within 1.2 meters, and the vibration direction error is ±15°), lagging dynamic response (the response to sudden obstacles is >1.2 seconds, and the false alarm rate is 41%), and failure in extreme scenarios (the failure probability in rainy and foggy days is 65%, and the misjudgment rate of stairs is 53%). As a result, visually impaired users can only travel independently about 0.8 kilometers per day on average and have 4.7 collisions per week.

[0003] More critically, existing solutions (such as CN117224370A) cannot meet the all-weather safety requirements due to defects in spatio-temporal alignment algorithms (IOU < 0.3), emergency dependence on manual triggering (the fall response is 15 seconds), and bulky wearables (the battery life is <6 hours, and the cost is >4800 yuan). The present invention addresses the three major industry pain points of multi-modal perception failure, lack of dynamic adaptation, and lagging emergency response. Through spatio-temporal alignment multi-modal fusion (IOU = 0.85), gait adaptive strategies (the false alarm rate is reduced by 81.7%), and biometric feature-triggered emergencies (automatic alarm in 2.1 seconds), it breaks through the reliability bottleneck of traditional devices in complex environments and provides an all-weather and highly reliable navigation solution for visually impaired users. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an intelligent blind navigation system to address the above issues.

[0005] An intelligent blind navigation system includes:

[0006] A multi-modal perception unit: integrating a high-definition camera, lidar, ultrasonic array, and inertial sensors to collect environmental videos, obstacle distances, user poses, and gait data in real time;

[0007] An AI decision-making center: built-in with spatio-temporal alignment algorithms and multi-task large models to fuse and process multi-source data and output the three-dimensional coordinates of obstacles, risk levels, and dynamic paths;

[0008] A two-way feedback module: including bone conduction headphones and a vibrating bracelet, dynamically switching the feedback mode according to the risk level (e.g., a risk level of L3 triggers "a branch 2 meters ahead on the left, vibrating 3 times + voice prompt");

[0009] Emergency Response System: It monitors the abnormal heart rate or inactivity timeout of the user through a biosensor, and automatically triggers a three-level alarm: playing a distress audio, sending an environmental video with location to the emergency contact, and linking nearby intelligent devices (such as street lights and cameras) to broadcast a rescue signal.

[0010] Preferably, the multi-modal perception unit adopts a hierarchical fusion architecture:

[0011] Bottom layer: The lidar and ultrasonic sensors are spatially calibrated to construct the obstacle contour.

[0012] Middle layer: The camera identifies suspended obstacles (such as banners and steps) through semantic segmentation, and compensates for motion blur by combining IMU data.

[0013] Top layer: The historical trajectory is fused to predict the movement trend of the obstacle, and a dynamic risk heat map is output.

[0014] Preferably, the AI decision center includes a user adaptation module:

[0015] Gait recognition model: Analyze the step length and step frequency through an acceleration sensor to generate a personalized walking map.

[0016] Dynamic prompt strategy: Adjust the voice interval according to the real-time walking speed. For example, when the user accelerates, the turning instruction is broadcast 0.5 s in advance.

[0017] Historical behavior learning: Based on reinforcement learning, record the user's response time to the prompt and optimize the lead of path planning.

[0018] Preferably, the two-way feedback module realizes multi-dimensional information encoding:

[0019] Voice channel: Adopt emotional synthesis technology to transmit the risk level through intonation changes (such as using a rapid high pitch for L5 level: "Emergency! Stop immediately!").

[0020] Tactile channel: The vibrating bracelet sets tactile points, and encodes the direction through the combination of vibration duration and position.

[0021] Redundant design: When the environmental noise > 75 dB, automatically enhance the tactile feedback intensity and shorten the voice interval.

[0022] Preferably, the emergency response system includes a multi-level trigger mechanism:

[0023] First-level warning: When it is detected that the user enters a dangerous area (such as within < 0.3 m from the edge of the stairs), trigger a high-frequency vibration + voice countdown ("Reaching the edge in 3 seconds").

[0024] Second-level alarm: If the prompt is not responded to continuously 3 times (such as not turning), start the camera to record the environmental video and send it to the emergency contact synchronously.

[0025] Level 3 rescue: If a user falls or their heart rate drops suddenly, the system will automatically call emergency services and share their real-time location via Beidou positioning.

[0026] Preferably, the high-definition camera integrates low-light enhancement technology:

[0027] Dual-light fusion sensor: collects visible light and infrared light simultaneously, and synthesizes clear images through generative adversarial networks (GAN);

[0028] Dynamic aperture adjustment: automatically switches according to the ambient illumination, starts the defogging algorithm (dark channel prior) in rainy and foggy days, and improves the detection distance.

[0029] Preferably, the AI decision center supports offline incremental learning:

[0030] The edge stores historical data (including more than 200,000 obstacle samples) and automatically generates pseudo labels when new types of obstacles (such as mobile charging piles) are detected;

[0031] The cloud model is updated monthly: by integrating user data through federated learning, new obstacle recognition categories (such as "escalator apron") are added, and the false alarm rate is reduced from 8.7% to 3.2%.

[0032] Preferably, the smart wearable device comprises a modular interface:

[0033] Tactile gloves: connected to the host through a magnetic interface, support fingertip vibration, and are used for fine obstacle prompts (such as "there is a cable 0.5m ahead, and the fingertips vibrate continuously");

[0034] Exoskeleton assistance: Optional knee joint assist device automatically adjusts support force according to the slope of the path, reducing walking energy consumption by 22%;

[0035] Low power consumption design: the sensor wakes up in time-sharing mode, and the whole device can last for ≥8 hours (under typical working conditions).

[0036] Preferably, the voice interaction module supports scenario-based dialogue:

[0037] Fuzzy input of destination: When the user says "go to the nearest toilet", the system combines POI data and real-time traffic conditions to recommend barrier-free routes (such as detours to avoid stairs);

[0038] Environmental query: Use the camera to identify surrounding signs and answer "The sign on the left says 'Emergency Exit'";

[0039] Emotional interaction: Identify the anxiety in the user's voice and proactively prompt "A safer route has been planned for you."

[0040] Preferably, the system is verified by digital twin:

[0041] Offline training phase: Simulate the movement of blind people in a virtual city (including more than 1000 complex scenarios) to optimize the prompting strategy (for example, the leading distance in the staircase scenario is increased from 1.2 m to 1.8 m).

[0042] Real-time verification: Compare the inertial sensor data with the digital twin model to detect positioning drift, and the continuous positioning accuracy in complex environments is maintained at ≥92%.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] In the present invention, through a three-layer perception architecture of a dual-light fusion camera + lidar + ultrasonic array, the obstacle detection rate remains 94% even under low illuminance of 5 lux, and the effective detection distance reaches 8 meters (extended by 60% compared with traditional lidar in rainy and foggy days). The semantic segmentation algorithm increases the detection rate of suspended obstacles (such as banners and tree branches) from 12% to 92%, and the recognition error of glass doors in the underground garage scenario is only 2.1 cm. The multi-sensor spatio-temporal alignment technology reduces the false alarm rate from 25% to 3.2% in complex scenarios, realizing precise perception of all-weather and all-height obstacles.

[0045] In the present invention, based on the dynamic strategy of user gait map + reinforcement learning, the system automatically adjusts the voice interval according to the walking speed. The alarm leading distance in the staircase scenario is increased from 0.5 m to 1.8 m, and the path coincidence degree in a complex shopping mall reaches 92%. The five-level risk coding (L1-L5) combines tactile positioning and emotional voice, and still maintains a feedback recognition rate of 92% in 75 dB noise. The number of false alarms is reduced from 17.5 times per kilometer to 3.2 times, and the user error correction operation is reduced by 76%, truly realizing an intelligent navigation that "understands user habits".

[0046] In the present invention, the unique biometric + environmental risk multi-level triggering mechanism automatically alarms and shares the position within 2.1 seconds when the user falls, and the success rate of fall rescue in the underground garage is increased from 62% to 100%. The environmental video with positioning is automatically recorded when there is no operation timeout, the response time of the emergency contact is shortened from 12 minutes to 3 minutes, and the risk of staying at traffic intersections is reduced by 83%. The L5-level emergency alarm (buzzer + high-frequency vibration) enables the stop response time in dangerous scenarios to be only 0.4 seconds, and the collision force is attenuated by 60%, building a closed-loop safety net of "prevention - early warning - rescue".

[0047] In the present invention, the modular wearable design (tactile gloves + exoskeleton assistance) reduces the walking energy consumption by 22% and has a battery life of 8.2 hours. The natural voice interaction (emotional synthesis + scenario-based dialogue) has a high satisfaction rate. The average daily independent travel distance of visually impaired users is increased from 0.8 km to 3.2 km, and the social participation degree is improved. Under large-scale production, the cost per set is reduced, and the reliability of IP65 waterproof + no failure after a 1.5-meter drop promotes the transformation of blind navigation from "laboratory" to "daily use". Brief Description of the Drawings

[0048] Figure 1 is a schematic diagram of the system flow of the present invention;

[0049] Figure 2 is a schematic diagram of the system composition of the present invention;

[0050] Figure 3 is a schematic diagram of the integrated content of the multi-modal perception unit in the present invention;

[0051] Figure 4 is a schematic diagram of the built-in content of the AI decision-making center in the present invention;

[0052] Figure 5 is a schematic diagram of the content of the two-way feedback module in the present invention;

[0053] Figure 6 is a schematic diagram of the principle of the emergency response system in the present invention;

[0054] Figure 7 is a hierarchical schematic diagram of the multi-modal perception unit in the present invention;

[0055] Figure 8 is a schematic diagram of the adaptive module of the AI decision-making center in the present invention;

[0056] Figure 9 is a schematic diagram of the two-way feedback module in the present invention;

[0057] Figure 10 is a three-level early warning schematic diagram of the emergency response system in the present invention. Detailed Embodiment

[0058] The following further describes the embodiments of the present invention in detail in conjunction with the drawings and embodiments. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0059] Please refer to Figures 1 - 10 , the present invention provides an intelligent blind navigation system, including:

[0060] Multi-modal perception unit: integrating a high-definition camera (resolution ≥ 4K), lidar (detection range ≥ 20m), ultrasonic array (accuracy ± 2mm), and inertial sensor (nine-axis IMU), and collecting environmental video, obstacle distance, user pose, and gait data in real time;

[0061] AI decision-making center: built-in spatio-temporal alignment algorithm (delay ≤ 5ms) and multi-task large model (integrating YOLOv8 object detection and Transformer time series analysis), performing fusion processing on multi-source data, and outputting three-dimensional coordinates of obstacles (error ≤ 5cm), risk levels (L1-L5), and dynamic paths;

[0062] Bidirectional feedback module: includes bone conduction headphones (voice delay ≤ 150ms) and a vibration bracelet (tactile positioning accuracy ±5°), dynamically switching feedback modes according to risk levels;

[0063] Emergency response system: Through biosensors, it monitors the user's abnormal heart rate (>120bpm) or no operation timeout (>30s), and automatically triggers a three-level alarm: plays a 105dB distress audio, sends a location-based environmental video to emergency contacts, and links nearby smart devices to broadcast rescue signals.

[0064] Device Example 1: Multimodal fusion navigation device (indoor scene):

[0065] Hardware configuration:

[0066]

[0067] Software Algorithm:

[0068] Spatiotemporal alignment algorithm: The lidar and camera are calibrated by external parameters (rotation matrix R = ±0.5°, translation vector T = ±1cm), and the point cloud and image matching IOU = 0.85 (rainy and foggy days);

[0069] Risk level classification: L1 (safety, >2m), L2 (caution, 1-2m), L3 (warning, 0.5-1m), L4 (danger, 0.3-0.5m), L5 (emergency, <0.3m);

[0070] Voice strategy: Level L3 triggers "There is a step 1 meter in front of the left, vibrates 3 times + voice: 'Stairs on the upper left, be careful to lift your feet'".

[0071] Measured data (compared with traditional lidar solutions)

[0072]

[0073]

[0074] Device Example 2: Emergency Response System (Outdoor Complex Environment):

[0075] Hardware Enhancements:

[0076] Biosensor: integrated PPG heart rate monitoring (sampling rate 25Hz, accuracy ±2bpm);

[0077] Beidou positioning module: UBLOXNEO-M8U (positioning accuracy ±1m, cold start time ≤30s); high-decibel speaker: 105dB buzzer (3000Hz pulse, transmission distance 50m).

[0078] Trigger Logic:

[0079]

[0080] Simulation Experiment (n = 50 visually impaired users):

[0081]

[0082] Method Example 1: Personalized Gait Adaptation (Dynamic Path Planning):

[0083] Data Collection:

[0084] Gait Database: Collect data of 500 visually impaired users (step length 0.4 - 0.8 m, step frequency 0.8 - 1.6 Hz);

[0085] Reinforcement Learning Model: Based on the PPO algorithm, reward function = (path length / shortest path) × (reciprocal of response time).

[0086] Policy Optimization:

[0087] Walking speed of 0.5 m / s (slow walking): Voice interval of 2.5 s, turning reminder 1.2 m in advance;

[0088] Walking speed of 1.2 m / s (fast walking): Voice interval of 0.8 s, turning reminder 0.6 m in advance;

[0089] Historical Habit: For a certain user, the average turning response is 0.8 s, and the path planning lead is +0.3 s.

[0090] Comparative Experiment (comparing with traditional static planning):

[0091]

[0092] Method Example 2: Multimodal Feedback Coding (noisy environment): Tactile Coding Scheme:

[0093]

[0094] Noise Environment Test (75 dB mall):

[0095]

[0096] Full-scenario Verification Experiment Example:

[0097] Experiment Example 1: Low Illumination Environment Comparison (below 10 lux):

[0098] Test Scheme:

[0099] Scenario: Underground garage (illumination 5 lux, including columns, fire hydrants, slopes); Indicators: Obstacle detection rate, positioning error, user satisfaction (on a 5-point scale). Test Results:

[0100]

[0101]

[0102] Conclusion:

[0103] The present invention synthesizes images through GAN, with PSNR = 28 dB at 5 lux and an effective detection distance of 8 m, solving the problem that 82% of traditional vision solutions fail at <10 lux.

[0104] Experimental Example 2: Adaptability to complex terrain (stairs / ramp):

[0105] Test parameters:

[0106] Stairs: 15 steps, step height 18 cm, depth 25 cm;

[0107] Ramp: slope 15°, surface slippery (friction coefficient 0.3);

[0108] Indicators: alarm lead time, user passing rate, number of false alarms.

[0109] Result comparison:

[0110]

[0111]

[0112] Experimental Example 3: Emergency response timeliness (n = 100 simulations):

[0113] Test process:

[0114] Trigger conditions: user enters the edge of the stairs (distance 0.2 m), falls (acceleration 25 m / s 2 ); Indicators: alarm delay, rescue arrival time, degree of user injury (virtual score).

[0115] Data table:

[0116]

[0117] Comparative example (core technology breakthrough):

[0118] Comparative example 1: Multimodal fusion vs single sensor (obstacle recognition):

[0119] sensor sunny day detection rate rainy day detection rate hanging obstacle detection rate typical failure scenario the present invention 97％ 89％ 92% (banner / branch) none (within 5m) single lidar 91％ 68％ 41% (height < 2m) rainy day point cloud noise single ultrasonic wave 85％ 72％ 12% (no height data) undetected hanging pipeline

[0120] Comparative example 2: Dynamic adaptation vs static planning (path optimization):

[0121] index the present invention (dynamic) traditional (static) measured case (mall) path length +5% (safe detour) 0％ avoid suddenly emerging cleaning robot response delay 0.4s 1.2s steering instruction 0.8s in advance number of user error corrections 0.3 times / km 2.1 times / km reduce 76% repeated operations

[0122] Mass production example (cost and reliability):

[0123] Hardware cost (per set):

[0124] component the present invention (scaled - up) traditional solution cost reduction multi - modal sensor ￥1200 ￥1800 (lidar single configuration) 33％ main control chip Cambricon MLU220 (￥500) NVIDIA Jetson (￥1500) 67％ wearable device ￥800 (modular) ￥1500 (customized) 47％

[0125] Reliability test (5000 - hour aging):

[0126]

[0127]

[0128] User empirical study (n = 200 visually - impaired users, 3 - month follow - up):

[0129] Subjective experience:

[0130] dimension rating of the present invention (5 points) rating of traditional device key feedback safety 4.7 3.2 "Dare to cross the intersection alone for the first time” ease of use 4.5 2.8 "The voice is like a friend, not mechanical” environmental adaptability 4.6 2.5 "Finally not afraid of falling in rainy days”

[0131] Objective data:

[0132] index the present invention baseline value (without device) improvement amplitude average daily independent travel distance 3.2 km (city) 0.8 km 300％ number of obstacle collisions 0.1 times / week 4.7 times / week ↓98％ social participation degree (questionnaire) 78% (actively go out) 32％ ↑144％

[0133] From the above embodiments, it can be concluded that:

[0134] Environmental perception: Multimodal fusion improves the detection rate of complex scenes to 94% (compared with 68% of the traditional method), and the effective distance in low - illumination environment breaks through to 8m;

[0135] Dynamic response: Gait adaptation reduces the path - planning delay from 1.2s to 0.4s, and the false - alarm rate from 17.5% to 3.2%;

[0136] Safety guarantee: The emergency response time is shortened by 76.7%, and the success rate of fall rescue is increased from 62% to 100%;

[0137] User experience: The tactile positioning accuracy is ±3° (±15° in the traditional case), the fatigue degree is reduced by 38%, and the daily travel distance is increased by 300%.

[0138] The embodiments of the present invention are given for purposes of illustration and description, and are not exhaustive or limit the invention to the disclosed form. Many modifications and variations are obvious to those of ordinary skill in the art. The embodiments are chosen and described in order to better illustrate the principles of the invention and its practical application, and to enable those of ordinary skill in the art to understand the invention so as to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. An intelligent blind navigation system, characterized in that, include: Multimodal perception unit: Integrates high-definition cameras, lidar, ultrasonic arrays, and inertial sensors to collect real-time environmental video, obstacle distance, user posture, and gait data; AI Decision Center: Built-in spatiotemporal alignment algorithm and multi-task large model to integrate multi-source data and output obstacle 3D coordinates, risk level and dynamic path; Bidirectional feedback module: includes bone conduction headphones and a vibration bracelet, which dynamically switches feedback modes according to risk levels; Emergency response system: Through biosensors, it monitors abnormal user heart rate or inactivity timeout, and automatically triggers a three-level alarm: plays a distress audio, sends a location-based environmental video to emergency contacts, and links nearby smart devices to broadcast rescue signals.

2. The intelligent blind navigation system according to claim 1, wherein, The multimodal perception unit adopts a layered fusion architecture: Bottom layer: LiDAR and ultrasonic sensors are spatially calibrated to construct obstacle contours; Middle layer: The camera identifies suspended obstacles through semantic segmentation and compensates for motion blur in combination with IMU data; Top layer: Fusion historical trajectories to predict obstacle movement trends and output dynamic risk heat maps.

3. The intelligent blind navigation system according to claim 1, characterized in that, The AI decision center includes a user adaptation module: Gait recognition model: Analyze step length and frequency through acceleration sensors to generate personalized walking maps; Dynamic prompt strategy: adjust the voice interval according to the real-time pace; Historical behavior learning: Based on reinforcement learning, the user’s response time to prompts is recorded to optimize the advance amount of path planning.

4. The intelligent blind navigation system according to claim 1, characterized in that, The bidirectional feedback module realizes multi-dimensional information encoding: Voice channel: Using emotion synthesis technology to convey risk levels through changes in tone; Tactile channel: The vibrating bracelet sets the tactile points and encodes the direction through the combination of vibration duration and position; Redundant design: When the ambient noise is greater than 75dB, the tactile feedback intensity is automatically enhanced and the voice interval is shortened.

5. The intelligent blind navigation system according to claim 1, characterized in that, The emergency response system includes a multi-level trigger mechanism: Level 1 warning: When the user enters a dangerous area, high-frequency vibration + voice countdown is triggered; Level 2 alarm: If there is no response to the prompt for three consecutive times, the camera will start recording the surrounding video and send it to the emergency contact at the same time; Level 3 rescue: If a user falls or their heart rate drops suddenly, the system will automatically call emergency services and share their real-time location via Beidou positioning.

6. The intelligent blind navigation system according to claim 1, characterized in that, The high-definition camera integrates low-light enhancement technology: Dual-light fusion sensor: collects visible light and infrared light simultaneously, and synthesizes clear images through generative adversarial networks (GAN); Dynamic aperture adjustment: automatically switches according to the ambient illumination, activates the defogging algorithm in rainy and foggy days, and improves the detection distance.

7. The intelligent blind navigation system according to claim 1, characterized in that, The AI Decision Center supports offline incremental learning: The edge stores historical data and automatically generates pseudo labels when new types of obstacles are detected; The cloud model is updated monthly: through federated learning to integrate user data, new obstacle recognition categories are added, and the false alarm rate is reduced from 8.7% to 3.2%.

8. The intelligent blind navigation system according to claim 1, wherein, The smart wearable device includes a modular interface: Haptic gloves: connected to the host through a magnetic interface, support fingertip vibration for fine obstacle prompts; Exoskeleton assistance: Optional knee joint assist device automatically adjusts support force according to the slope of the path, reducing walking energy consumption by 22%; Low power consumption design: the sensor wakes up in time-sharing mode, and the whole device can last for ≥ 8 hours.

9. The intelligent blind navigation system according to claim 1, wherein The voice interaction module supports scenario-based dialogue: Destination fuzzy input: When the user mentions a destination, the system combines POI data and real-time traffic conditions to recommend barrier-free routes; Environmental query: Answer by identifying surrounding signs through a camera; Emotional interaction: Identify the anxiety in the user's voice and give proactive prompts.

10. The intelligent blind navigation system according to claim 1, characterized in that, The system is verified through digital twin: Offline training stage: Simulate the movement of blind people in a virtual city (including more than 1000 complex scenarios) to optimize the prompting strategy; Real-time verification: Compare inertial sensor data with the digital twin model to detect positioning drift, and maintain a continuous positioning accuracy of ≥92% in complex environments.

Images