Artificial intelligence walking stick for the blind

Abstract

This invention relates to a smart cane for blind workers, comprising a cane body, a sensor module, a processing module, an interaction module, a power module, and a communication module disposed inside the cane body; and an integrated omnidirectional obstacle detection system, a navigation system specifically for the blind, a scene recognition and voice description system, an emergency call system, and a voice interaction system. This invention achieves omnidirectional obstacle detection through the omnidirectional obstacle detection system, with a wide range and accurate and reliable detection results; simultaneously, it realizes four major functions of the cane: navigation, recognition, interaction, and emergency call through the navigation system, scene recognition and voice description system, emergency call system, and voice interaction system. Each functional system utilizes cloud-configured and locally deployed model algorithms, resulting in fast response speed and a high degree of intelligence.

Description

Technical Field

[0001] This invention relates to the field of assistive devices for the blind, specifically to an artificial intelligence cane designed for blind and visually impaired individuals. It is particularly suitable for assisting totally blind, low-vision, and other visually impaired people with independent travel and daily living needs. Background Technology

[0002] Currently, the main difficulties faced by blind and visually impaired people when traveling include: 1. Inability to perceive obstacles in front, making them prone to collisions or falls; 2. Inability to recognize traffic lights and road signs; 3. Inability to independently plan routes and navigate; 4. Inability to obtain information about the surrounding environment (shops, facilities, etc.); 5. Inability to seek help in a timely manner in emergencies.

[0003] The following shortcomings exist in the currently available canes for the blind: 1. Traditional white canes use aluminum alloy or carbon fiber rods with rubber tips at the bottom to detect obstacles through tactile feedback. However, their detection range is limited (they can only detect objects that can be touched), and they cannot detect obstacles above or provide environmental information.

[0004] 2. Electronic canes add ultrasonic sensors to the basic white cane, using vibration feedback to indicate the distance to obstacles. However, they are limited in function, only able to detect obstacles, unable to identify object types, and unable to provide navigation.

[0005] 3. Smart canes (such as WeWalk) integrate ultrasonic sensors and mobile apps, providing basic navigation functions, but they rely on mobile phones, are complex to operate, require a learning curve, and have short battery life (about 8 hours).

[0006] 4. AI vision assistance products (such as Envision Glasses) use cameras and AI recognition to provide object recognition and text reading functions, but they require wearing glasses, are expensive ($3000+), have short battery life (about 4 hours), and require an internet connection to use. Summary of the Invention

[0007] The present invention aims to solve the various problems mentioned above, and thus provide an artificial intelligence cane for the blind with a wide detection range (all-round obstacle detection), comprehensive functions (navigation, recognition, interaction, and emergency call), zero learning cost (out-of-the-box use), and long battery life (enough for 16 hours of use per day).

[0008] To solve the above problems, the technical solution adopted by the present invention is as follows: A smart cane for blind workers includes a cane body, a sensor module, a processing module, an interaction module, a power module, and a communication module placed inside the cane body; and an all-around obstacle detection system, a navigation system for the blind, a scene recognition and voice description system, an emergency call system, and a voice interaction system integrated inside the cane body. The omnidirectional obstacle detection system achieves corresponding detection functions based on ultrasonic sensors, infrared sensors, and a multi-sensor fusion visual recognition model. The navigation system for the blind is based on the Beidou / GPS dual-mode positioning module on the top of the cane, cloud-based pedestrian navigation API, and an improved tactile paving priority path planning algorithm to achieve voice navigation and danger warning functions for the blind. The scene recognition and voice description system is based on a visual recognition model that integrates multiple sensors, including a camera configured on the front of the cane and an all-around obstacle detection system. The emergency call system is based on a gyroscope, a one-button SOS call, an automatic recording module, and fall detection control logic configured in the handle of the cane, and realizes the dual functions of automatic and manual emergency call when falling. The voice interaction system is based on a proprietary speech recognition model, a natural language understanding module, and a speech synthesis model, and its core function is to meet the convenient interaction needs of blind people who do not rely on vision.

[0009] Furthermore, the multi-sensor fusion visual recognition model of the omnidirectional obstacle detection system is based on an improved version of the YOLOv5 real-time target detection algorithm. The specific training and adaptation process is as follows: Step 1: Dataset Preprocessing (1) Screening and adaptation of public datasets: Select effective samples related to blind people’s travel from public datasets; perform data augmentation on the screened samples by size normalization, grayscale correction, random flipping and Gaussian blur to adapt to the light change scene from the perspective of blind people; (2) Collection and annotation of self-collected datasets: self-collected images from the perspective of blind people; fine annotation of self-collected images, including: obstacle type, obstacle distance, obstacle position coordinates, and scene type; (3) Data set fusion and partitioning: The preprocessed public samples and self-sampled samples are fused together and divided into training set, validation set and test set according to the proportion. At the same time, the corresponding labels of ultrasonic sensor detection distance are added to the dataset for subsequent multi-sensor data fusion training.

[0010] Step 2: Model Training The training platform was NVIDIA V100 × 4, the training time was 72 hours, and the specific training parameters were set as follows: (1) Initial parameters: learning rate 0.001, batch size 32, number of iterations 100, weight decay coefficient 0.0005, momentum 0.937; (2) Loss function: CIoU loss function + cross-entropy loss function is adopted, where the category recognition loss weight is set to 1.2 and the bounding box regression loss weight is set to 0.8 to ensure the accuracy of obstacle category recognition and distance localization; (3) Improvement and optimization: The attention mechanism SE-Net is added to the PANet structure in the Neck layer to enhance the feature extraction capability of near obstacles; one convolutional layer is reduced in the CSPDarknet53 in the Backbone layer to reduce the number of model parameters and adapt to the computing power of embedded AI chips. (4) Training process control: Cosine annealing learning rate decay is used for the first 50 epochs of training, and a fixed learning rate is used for the next 50 epochs; validation is performed every 10 epochs. If the accuracy of the validation set does not improve for 3 consecutive times, training is stopped and the optimal model weights are saved.

[0011] Step 3: Model Quantization and Deployment Adaptation (1) Quantization process: The trained model is quantized. During the quantization process, a self-collected test set is used for calibration to ensure that the quantization is <2%. After quantization, the model volume is compressed to 1 / 4 of the original volume to meet the storage requirements of the embedded chip. (2) Inference engine adaptation: The quantized model is deployed to the Rockchip RKNN inference engine. The model inference process is optimized for the NPU computing power of the RK3566 chip. Redundant feature extraction channels are closed to ensure that the inference speed reaches 30FPS to meet the real-time detection requirements. (3) Multi-sensor fusion adaptation: After the model is deployed, a data fusion interface is added to realize the real-time synchronization of visual recognition results with ultrasonic sensor and infrared sensor data. The Kalman filter + weighted average fusion method is used to output the final obstacle detection result.

[0012] Furthermore, the implementation logic of the navigation system specifically for the blind includes the following steps: The BeiDou / GPS dual-mode positioning module collects the current location information of blind people in real time, and the positioning accuracy meets the navigation needs of blind people, ensuring that the location information is accurate and reliable. The communication module establishes communication with the cloud-based pedestrian navigation API via 4G / 5G to obtain the basic data required for navigation. The processing module runs an improved tactile paving priority path planning algorithm. Based on the acquired data, it prioritizes the selection of travel routes containing tactile paving, actively avoids identified dangerous road sections, and optimizes the navigation logic in combination with the walking characteristics of blind people to avoid complex turns. The interactive module broadcasts clear and concise navigation prompts and danger warnings to blind people through a full-range speaker, and provides tactile feedback from a vibration motor to assist in directional guidance, ensuring convenient and safe navigation and adapting to usage scenarios where blind people do not rely on vision.

[0013] Furthermore, the improved tactile paving priority path planning algorithm of the navigation system for the blind is based on the classic A* algorithm. The core improvements that distinguish it from conventional path planning algorithms include three points: First, it introduces a tactile paving weight coefficient and a graded penalty mechanism for dangerous road sections. By allocating weights, it increases the selection priority of tactile paving paths, and by imposing graded penalties, it reduces the selection probability of dangerous road sections, thus achieving the core requirement of "tactile paving priority and danger avoidance." Second, it improves the heuristic function by combining it with the walking characteristics of blind people, abandoning the conventional design that only considers spatial distance, and adapting it to the walking speed and travel habits of blind people, making the navigation path more in line with the actual travel scenarios of blind people. Third, it sets up a real-time path update mechanism, linking the real-time data of the obstacle detection system and the positioning module, dynamically adapting to scenarios such as temporary obstacles and changes in road conditions, and avoiding path rigidity.

[0014] Furthermore, the fall detection control logic of the emergency call system includes the following process: Step 1: Signal Acquisition and Preprocessing: The gyroscope acquires the acceleration and attitude angle data of the cane in real time, with the sampling frequency set to 10Hz. The processing module filters the acquired raw data to obtain stable acceleration amplitude and attitude angle. Step 2, Fall Detection Trigger Conditions: When the processing module detects that the following two conditions are met simultaneously, a preliminary fall detection is triggered: ① A sudden change in acceleration amplitude, and acc_magnitude > 2.5g; ② A drastic change in the cane posture angle, and angle > 60°; Step 3, Fall Confirmation Logic: After a preliminary fall is determined, the processing module starts a 3-second countdown confirmation mechanism. During this period, gyroscope data is continuously collected. If no posture recovery action is detected within 3 seconds and no manual cancellation command is received from the user, the fall is officially confirmed. If posture recovery or user cancellation command is detected within 3 seconds, the fall determination process is terminated and an SOS call is not triggered. Step 4, SOS Call Trigger Control: After a fall is officially confirmed, the processing module immediately triggers the SOS distress call command and simultaneously initiates the following linked operations: ① Controls the speaker of the interaction module to play a voice confirmation prompt, and at the same time controls the vibration motor to vibrate continuously to remind the user that a fall distress call has been triggered; ② Starts the automatic recording module to begin recording ambient sounds; ③ Controls the communication module to initiate the distress call process; The complete SOS emergency call process is as follows: Fall detection (gyroscope acquisition + filtering) → Voice confirmation (speaker plays "Fall detected, call for help? Automatically call for help in 10 seconds," vibration motor vibrates synchronously as a prompt) → User confirmation / timeout (user can cancel the call for help with one button on the handle; if no user action is taken within 10 seconds, it will automatically proceed to the next step) → Send SMS / phone (communication module sends an SOS SMS containing the current location link and fall time to 3 preset emergency contacts in sequence, and dials the emergency contacts' phone numbers in preset order; if no one answers, it automatically switches to the next one until all 3 contacts are reached) → Continuous recording (records the entire scene for emergency contacts to understand the situation; the recording file can be uploaded to the cloud for backup via the communication module) → Waiting for rescue (every 5 minutes, a message is broadcast through the speaker: "Rescue is on its way, please wait patiently," while continuously synchronizing location information to emergency contacts until the user manually stops the call for help or the rescuers confirm their arrival). In addition to automatic fall triggering, the emergency call system can also be manually triggered by the user through the one-button SOS call button on the handle. After pressing the button, the fall detection and confirmation steps are skipped directly and the voice confirmation step is entered. The subsequent process is the same as automatic triggering, which is suitable for users who have not fallen but encounter emergency situations (such as getting lost or sudden illness). The emergency call system also includes an anomaly handling mechanism: if the communication module detects a network anomaly (no 4G / 5G or WiFi signal), it will immediately initiate a local voice prompt "Network anomaly, unable to send SOS message, please move to an area with signal", and continuously attempt to connect to the network until the network is restored, automatically resend the self-help SMS and make an SOS call; if the automatic recording module malfunctions, it will only trigger the SMS / phone call for help, without affecting the implementation of the core SOS function.

[0015] Furthermore, the voice interaction system is implemented collaboratively based on an Automatic Speech Recognition (ASR) model, a Natural Language Understanding (NLU) module, and an improved Text-to-Speech (TTS) model: the user issues voice commands through a microphone, the ASR model converts the speech into text, the NLU module parses the text intent and generates control commands and feedback text, the control commands trigger the corresponding functional system to run, and the feedback text is converted into speech by the improved Text-to-Speech (TTS) model and broadcast to the user through a speaker. Simultaneously, a vibration motor can be combined to provide tactile assistance, completing the full voice interaction. The three components operate independently yet seamlessly collaborate without any technical conflicts. The speech recognition model (ASR) is a TDNN model developed based on the Kaldi toolkit. It adopts a TDNN and LSTM combined architecture. Its core function is to convert user voice commands into text, providing support for interactive command recognition. Its feature extraction uses 80-dimensional Fbank features and CTC+Attention decoding method to adapt to the characteristics of blind people's spoken language and environmental noise scenarios, ensuring that the accuracy of daily voice command recognition is ≥98%, meeting the needs of real-time interaction. It can be implemented by combining the conventional experience of using the Kaldi toolkit.

[0016] The Natural Language Understanding (NLU) module, as an intermediate link module for voice interaction, is a well-known technology in the field. Its core function is to parse the text commands output by the speech recognition model, understand the user's intent (such as "switch mode", "turn up the volume", "navigate to a certain place", etc.), and convert the parsed intent into control commands that the processing module can recognize, and pass them to the corresponding functional system (such as the navigation system, the interaction module). At the same time, it generates feedback text and passes it to the speech synthesis model.

[0017] The improved speech synthesis model (TTS) is based on the VITS end-to-end model and optimized for adaptation. Its core function is to convert the feedback text, navigation prompts, and warning information output by the natural language understanding module into natural speech and broadcast it to the user through a speaker. Its sampling rate is 22050Hz and its MOS score is 4.2 out of 5, which can ensure that the synthesized speech is clear and natural. The core improvement is to optimize the synthesis speed of commonly used spoken commands for the blind (delay ≤100ms) to meet the real-time interaction needs of the blind. Those skilled in the art can implement this improved model based on the conventional optimization experience of the VITS model and the improvement direction of this disclosure.

[0018] Furthermore, the sensor module is used to replace the blind person's vision to perceive the surrounding environment, including four ultrasonic sensors arranged around the top of the cane for detecting obstacles, two infrared sensors arranged at the bottom front of the cane for ground detection to prevent falls, one camera arranged at the top front of the cane for AI visual recognition to replace the blind person's eyes, one Beidou / GPS dual-mode positioning module installed at the top of the cane for positioning and navigation, and a gyroscope installed inside the cane handle for attitude detection to prevent falls.

[0019] Furthermore, the processing module is used to process environmental information in real time and adopts an embedded AI chip; its main chip adopts Rockchip RK3566 quad-core ARM Cortex-A55 (1.8GHz), and AI acceleration is achieved through a built-in 0.8TOPS NPU (neural network processing unit).

[0020] Furthermore, the interaction module includes a voice interaction unit and a haptic feedback unit. The voice interaction unit includes a microphone and a speaker, and the haptic feedback unit includes a vibration motor.

[0021] Furthermore, the power module uses four 18650 lithium-ion batteries connected in series, with a capacity of 3200mAh × 4 = 12800mAh (47.36Wh).

[0022] Furthermore, the communication module adopts the 4G / 5G Quectel EC200U 4G Cat.1 full network compatible module, including a WiFi module and a Bluetooth module integrated into the RK3566 chip.

[0023] The present invention provides an artificial intelligence cane for the blind, which achieves all-round obstacle detection through an all-round obstacle detection system, with a wide range and accurate and reliable detection results; at the same time, it realizes four major functions of navigation, recognition, interaction and emergency call through a navigation system for the blind, a scene recognition and voice description system, an emergency call system and a voice interaction system. Each functional system has fast response speed and high degree of intelligence through cloud-configured and locally deployed model algorithms. Attached Figure Description

[0024] Figure 1 The system composition block diagram of this invention illustrates the connection relationship between the modules.

[0025] Figure 2 The control principle flowchart of this invention illustrates the data processing and control logic. Detailed Implementation

[0026] The invention will now be described in detail with reference to the accompanying drawings.

[0027] Example 1 An artificial intelligence cane for the blind includes a cane body, a sensor module, a processing module, an interaction module, a power module, and a communication module placed inside the cane body; and an all-around obstacle detection system, a navigation system for the blind, a scene recognition and voice description system, an emergency call system, and a voice interaction system integrated inside the cane body.

[0028] The following provides a detailed explanation of the composition and function of each functional module and system within the cane body: The sensor module, designed to replace the visual perception of the blind, includes four ultrasonic sensors for obstacle detection, two infrared sensors for ground detection to prevent falls, one camera for AI visual recognition to replace the blind person's eyes, one Beidou / GPS dual-mode positioning module for positioning and navigation, and one gyroscope for attitude detection to prevent falls. The four ultrasonic sensors are improved HC-SR04 ultrasonic sensors with a detection range of 0.02-5 meters and an accuracy of ±3 mm. They are evenly distributed around the sensor bracket at the top of the cane, with each sensor spaced 90° apart. The two infrared sensors are GP2Y0A21YK0F infrared sensors, located at the front of the bottom of the cane, with a detection range of 0.1-0.8 meters. The camera is an 8-megapixel Sony IMX219, located at the front of the sensor bracket at the top of the cane and covered with a waterproof transparent cover, installed at a 15° angle, with a horizontal viewing angle of 78°. The BeiDou / GPS dual-mode positioning module is a BeiDou / GPS dual-mode UBLOX MAX-M8Q, which is installed on the top of the cane with the antenna facing upwards, and has a positioning accuracy of 2.5 meters. The gyroscope is an MPU6050 six-axis gyroscope installed inside the cane handle, with an acceleration of ±16g and an angular velocity of ±2000° / s.

[0029] The processing module uses an embedded AI chip to process environmental information in real time; its main chip is a Rockchip RK3566 quad-core ARM Cortex-A55 (1.8GHz), which accelerates AI through a built-in 0.8TOPS NPU (neural network processing unit).

[0030] The interactive module includes a voice interaction unit and a tactile feedback unit. The voice interaction unit includes a microphone and a speaker, and the tactile feedback unit includes a vibration motor. The installation positions of each component are designed based on the usage habits of blind people and the functional requirements. The models and parameters are all standard industry choices, as detailed below: The microphone model is INMP441, installed on the inside of the cane handle grip (fitting below the palm, avoiding the gripping force point) to ensure clear acquisition of user voice commands while avoiding obstruction during gripping; the speaker is a 3W full-range speaker, installed near the top of the cane handle (symmetrically distributed with the microphone), facing the user's ear to ensure clear and intelligible voice broadcasts while reducing environmental noise interference; the vibration motor is a flat coin motor, one on each side, symmetrically installed inside the cane handle on both sides, corresponding to left and right directional prompts, fitting the gripping area to ensure obvious tactile feedback without affecting grip comfort. The above model selections are all standard choices in the field of embedded intelligent devices and assistive devices. The INMP441 microphone has low power consumption and high sensitivity, making it suitable for embedded scenarios; the 3W full-range speaker balances volume and power consumption, meeting the voice broadcasting requirements of assistive devices for the blind; the flat coin motor is small in size and has moderate vibration, making it suitable for the internal installation space of the handle. None of these require additional modifications, and those skilled in the art can directly select them based on conventional selection experience.

[0031] The power module uses four 18650 lithium-ion batteries connected in series, with a capacity of 3200mAh × 4 = 12800mAh (47.36Wh).

[0032] The communication module adopts the Quectel EC200U 4G Cat.1 full network compatible module, which includes a WiFi module and a Bluetooth module integrated into the RK3566 chip.

[0033] The following is an introduction to the functions and implementation process of each system: I. All-round obstacle detection system The omnidirectional obstacle detection system is based on a visual recognition model that combines four ultrasonic sensors, two infrared sensors, and multi-sensor fusion to achieve the corresponding detection functions.

[0034] Specifically, a front ultrasonic sensor (detection distance set to 0.5-5 meters) + camera visual recognition (recognition range 0-10 meters) is used to detect the front of the cane; a rear ultrasonic sensor (detection distance set to 0.5-3 meters) is used to detect the back of the cane; a left ultrasonic sensor (detection distance set to 0.5-3 meters) is used to detect the left side of the cane; a right ultrasonic sensor (detection distance set to 0.5-3 meters) is used to detect the right side of the cane; the front ultrasonic sensor, installed at a 30° angle upwards (detection distance set to 1.5-3 meters), is also used for upward detection; and a bottom infrared sensor (detection distance set to 0.1-0.8 meters) is used to detect steps and potholes on the ground.

[0035] The multi-sensor fusion visual recognition model is an improvement on the YOLOv5 real-time object detection algorithm. Its model structure is as follows: Input: 640×480×3 → Backbone: CSPDarknet53 → Neck: PANet + Attention → Head: Detection head + Recognition head → Output: Detection box + Class + Confidence. The training data consists of public datasets (COCO, OpenImages) and self-collected data (10,000+ images from the perspective of blind people). The training data uses a combination of public datasets and self-collected datasets. The specific training and adaptation process is as follows: Step 1: Dataset Preprocessing (1) Screening and adaptation of public datasets: Select samples related to blind people’s travel from COCO and Open Images public datasets (pedestrians, vehicles, steps, trash cans, traffic lights, road signs, etc.), remove irrelevant samples (such as objects at high altitudes and objects not in the travel scene), and screen effective samples; perform data augmentation on the screened samples by size normalization (uniformly adjusted to 640×480), grayscale correction, random flipping (horizontal flipping, probability 50%), and Gaussian blur (variance 0.1-0.3) to adapt to the scene of light changes (such as cloudy days and evenings) from the perspective of blind people.

[0036] (2) Collection and annotation of self-collected dataset: Collect 10,000+ images from the perspective of blind people (collection scenes cover high-frequency travel scenes for blind people such as sidewalks, parks, residential areas, and intersections. The collection angle is consistent with the installation angle of the camera of this invention—tilt 15°, horizontal viewing angle 78°, collection distance 0.02-10 meters, covering different lighting and different obstacle types); perform fine annotation on the self-collected images. The annotation content includes: obstacle category (12 core targets such as pedestrians, vehicles, steps, potholes, trash cans, and road signs), obstacle distance (corresponding to the detection distance of ultrasonic sensors), obstacle location coordinates, and scene type (sidewalk, intersection, park, etc.). The annotation tool is LabelImg, and the annotation accuracy is ≤1 pixel.

[0037] (3) Data set fusion and partitioning: The preprocessed public samples and self-sampled samples are fused and divided into training set, validation set and test set in a ratio of 8:1:1. At the same time, the corresponding labels of ultrasonic sensor detection distance are added to the dataset (each image corresponds to a set of ultrasonic detection data, realizing the association labeling of visual data and ultrasonic data) for subsequent multi-sensor data fusion training.

[0038] Step 2: Specific parameters for model training The training platform was NVIDIA V100 × 4, and the training time was 72 hours. The specific training parameters were set as follows to ensure that the model was compatible with the Rockchip RK3566 chip and the travel scenarios for the blind: (1) Initial parameters: learning rate 0.001, batch size 32, number of epochs 100, weight decay coefficient 0.0005, momentum 0.937; (2) Loss function: CIoU loss function (for bounding box regression) + cross-entropy loss function (for category recognition) are adopted, where the category recognition loss weight is set to 1.2 and the bounding box regression loss weight is set to 0.8 to ensure the accuracy of obstacle category recognition and distance localization; (3) Improvement and optimization: An attention mechanism (SE-Net) is added to the PANet structure in the Neck layer to enhance the feature extraction capability for near obstacles (0.5-3 meters, the high-frequency attention range of blind people); in the CSPDarknet53 of the Backbone layer, one convolutional layer is reduced to reduce the number of model parameters and adapt to the computing power of the embedded AI chip (Rockchip RK3566); (4) Training process control: Cosine annealing learning rate decay is used for the first 50 epochs of training, and a fixed learning rate (0.0001) is used for the next 50 epochs; validation is performed every 10 epochs. If the accuracy of the validation set does not improve for 3 consecutive times, training is stopped and the optimal model weights are saved.

[0039] Step 3: Model Quantization and Deployment Adaptation (1) Quantization process: The trained FP32 precision model is quantized using Rockchip RKNN Toolkit. During the quantization process, a self-collected test set is used for calibration to ensure that the quantization is <2%. After quantization, the model volume is compressed to 1 / 4 of the original volume to meet the storage requirements of embedded chips. (2) Inference engine adaptation: The quantized model is deployed to the Rockchip RKNN inference engine. The model inference process is optimized for the NPU computing power (0.8TOPS) of the RK3566 chip. Redundant feature extraction channels are closed to ensure that the inference speed reaches 30FPS (640×480 resolution) to meet the real-time detection requirements (when blind people travel, they need to provide real-time feedback of obstacle information with a delay of ≤30ms). (3) Multi-sensor fusion adaptation: After the model is deployed, a data fusion interface is added to realize the real-time synchronization of visual recognition results with ultrasonic sensor and infrared sensor data (timestamp synchronization error <10ms). The fusion method of Kalman filtering + weighted average (ultrasonic weight 0.4, visual weight 0.6) is adopted to output the final obstacle detection results (detection box, category, confidence, distance).

[0040] II. Navigation System for the Blind The navigation system for the blind is based on the Beidou / GPS dual-mode positioning module on the top of the cane and the cloud-based pedestrian navigation API. Combined with an improved tactile paving priority path planning algorithm, it realizes voice navigation and danger warning functions for the blind. The core is to solve the technical problems that existing navigation cannot adapt to the travel of the blind, lack tactile paving priority and real-time obstacle avoidance, and make up for the defects of existing blind navigation in the absence of tactile paving guidance and inability to dynamically avoid temporary dangers.

[0041] Its core implementation logic is as follows: The Beidou / GPS dual-mode positioning module collects the current location information of the blind person in real time, and the positioning accuracy meets the navigation needs of the blind person, ensuring that the location information is accurate and reliable; the communication module establishes communication with the cloud-based pedestrian navigation API through 4G / 5G to obtain basic data such as road topology, tactile paving distribution, and dangerous road sections (such as construction, steep slopes, and areas with heavy traffic) required for navigation; the processing module (Rockchip RK3566 chip) runs an improved tactile paving priority path planning algorithm, which prioritizes the selection of travel paths containing tactile paving based on the acquired data, actively avoids identified dangerous road sections, and optimizes the navigation logic in combination with the walking characteristics of the blind person to avoid complex turns; the system has real-time path update capability and can synchronously receive temporary obstacle information from the all-round obstacle detection system, dynamically adjusting the path to avoid temporary obstacles; the interaction module broadcasts clear and concise navigation prompts and danger warnings to the blind person through a full-range speaker, and uses tactile feedback from a vibration motor to assist in indicating direction, ensuring convenient and safe navigation, and adapting to the use scenarios of blind people without visual dependence.

[0042] The improved tactile paving priority path planning algorithm is based on the classic A* algorithm. Its core improvements over conventional path planning algorithms include three points: First, it introduces a tactile paving weight coefficient and a graded penalty mechanism for dangerous road sections. Weight allocation increases the priority of tactile paving path selection, while graded penalties reduce the probability of selecting dangerous road sections, achieving the core requirement of "tactile paving priority and danger avoidance." Second, it improves the heuristic function by considering the walking characteristics of blind people, abandoning the conventional design that only considers spatial distance, and adapting to the walking speed and travel habits of blind people, making the navigation path more consistent with their actual travel scenarios. Third, it sets up a real-time path update mechanism, linking real-time data from the obstacle detection system and positioning module to dynamically adapt to temporary obstacles, road condition changes, and other scenarios, avoiding path rigidity.

[0043] This algorithm can achieve effective protection without disclosing specific parameters. Its core invention lies in the improved algorithm logic of "tactile paving priority + blind person walking adaptation + real-time obstacle avoidance", rather than specific parameter settings. Based on the content of this disclosure, combined with conventional A* algorithm design experience and common sense of path planning technology, those skilled in the industry can complete the specific implementation of this algorithm without excessive effort, which fully meets the requirements of full patent disclosure, while avoiding the risk of plagiarism caused by the disclosure of specific technical details.

[0044] III. Scene Recognition and Speech Description System The scene recognition and voice description system includes a camera configured on the front of the cane, which uses its own AI visual recognition model to recognize traffic lights, public places, shop signs and text.

[0045] IV. Emergency Call System The emergency call system, based on a gyroscope, a one-button SOS call, an automatic recording module, and dedicated fall detection control logic, is configured in the handle of the cane. It achieves both automatic and manual SOS call functions upon a fall. The complete control process for automatically triggering an SOS call upon a fall is as follows: Step 1: Signal Acquisition and Preprocessing: The MPU6050 six-axis gyroscope acquires the acceleration and attitude angle data of the cane in real time. The sampling frequency is set to 10Hz. The processing module (Rockchip RK3566) filters the acquired raw data (using Kalman filtering algorithm to eliminate signal noise caused by environmental interference) to obtain stable acceleration amplitude (acc_magnitude) and attitude angle (angle). Step 2, Fall Detection Trigger Conditions: When the processing module detects that the following two conditions are met simultaneously, a preliminary fall detection is triggered: ① A sudden change in acceleration amplitude, and acc_magnitude > 2.5g (g is the acceleration due to gravity. This threshold has been calibrated through multiple sets of fall tests on blind people and can effectively distinguish between normal walking and falling movements, avoiding false triggers); ② A drastic change in the cane posture angle, and angle > 60° (the angle is based on 0° when the cane is standing vertically. An angle of forward tilting, backward tilting, or side tilting exceeding 60° is judged as an abnormal posture). Step 3, Fall Confirmation Logic: After a preliminary fall is determined, the processing module initiates a 3-second countdown confirmation mechanism. During this period, gyroscope data is continuously collected. If no posture recovery action is detected within 3 seconds (i.e., the cane does not return to a vertical or normal grip posture), and no manual cancellation command is received from the user (triggered via a button on the controller), then the fall is officially confirmed. If posture recovery or a user cancellation command is detected within 3 seconds, the fall determination process is terminated, and an SOS call is not triggered. Step 4, SOS Call Trigger Control: Upon formal confirmation of a fall, the processing module immediately triggers an SOS distress call, simultaneously initiating the following linked operations: ① The speaker of the control interaction module plays a voice confirmation prompt, while the vibration motor continuously vibrates to remind the user that a fall distress call has been triggered; ② The automatic recording module is activated to begin recording ambient sounds (MP3 format, 44.1kHz sampling rate, continuous recording until a rescue response is received or the user manually stops); ③ The communication module (4G / 5G full network compatible module) is controlled to initiate the distress call process, the specific process of which is as follows: Complete SOS Call Process: Fall Detection (Gyroscope Data Acquisition + Filtering) → Voice Confirmation (Speaker plays "Fall detected, call for help? Automatically call for help in 10 seconds," vibration motor vibrates synchronously as a notification) → User Confirmation / Timeout (Users can cancel the call with a single button on the controller; if no user action is taken within 10 seconds, it automatically proceeds to the next step) → Send SMS / Phone Call (The communication module sends an SOS SMS containing the current location link and fall time to three preset emergency contacts in sequence, and simultaneously dials the emergency contacts' phones in a preset order; if no one answers, it automatically switches to the next, until all three contacts are reached or the call is completed) → Continuous Recording (Records the entire scene for emergency contacts to understand the situation; the recording file can be synchronously uploaded to the cloud for backup via the communication module) → Waiting for Rescue (Every 5 minutes, a message is broadcast through the speaker: "Rescue is on its way, please wait patiently," while continuously synchronizing location information to emergency contacts, until the user manually stops the call or the rescue personnel confirm their arrival). Step 5, Manual SOS Call Control: In addition to automatic fall triggering, users can manually trigger an emergency call via the one-button SOS call button on the controller. After pressing the button, the fall detection and confirmation steps are skipped directly, and the voice confirmation step is entered. The subsequent process is the same as automatic triggering, which is suitable for users who have not fallen but encounter an emergency (such as getting lost or sudden illness). Step Six, Abnormal Handling Mechanism: If the communication module detects a network abnormality (no 4G / 5G or WiFi signal), it will immediately initiate a local voice prompt: "Network abnormality, unable to send SOS message, please move to an area with signal." At the same time, it will continue to try to connect to the network until the network is restored, and then automatically resend the self-help SMS and make an SOS call. If the automatic recording module malfunctions, it will only trigger the SMS / phone call for help, without affecting the implementation of the core SOS function.

[0046] IV. Voice Interaction System The voice interaction system is based on the collaborative implementation of an Automatic Speech Recognition (ASR) model, a Natural Language Understanding (NLU) module, and an improved Text-to-Speech (TTS) model: The user issues voice commands through a microphone; the ASR model converts the speech into text; the NLU module parses the text intent and generates control commands and feedback text; the control commands trigger the corresponding system functions; and the feedback text is converted into speech by the improved TTS model and broadcast to the user through a speaker. Simultaneously, a vibration motor can be incorporated to provide tactile feedback, completing the full voice interaction. The three components operate independently yet seamlessly collaborate without any technical conflicts. The speech recognition model (ASR) is a TDNN model developed based on the Kaldi toolkit. It adopts a TDNN and LSTM combined architecture. Its core function is to convert user voice commands into text, providing support for interactive command recognition. Its feature extraction uses 80-dimensional Fbank features and CTC+Attention decoding method to adapt to the characteristics of blind people's spoken language and environmental noise scenarios, ensuring that the accuracy of daily voice command recognition is ≥98%, meeting the needs of real-time interaction. It can be implemented by combining the conventional experience of using the Kaldi toolkit.

[0047] The Natural Language Understanding (NLU) module, as an intermediate link module for voice interaction, is a well-known technology in the field. Its core function is to parse the text commands output by the speech recognition model, understand the user's intent (such as "switch mode", "turn up the volume", "navigate to a certain place", etc.), and convert the parsed intent into control commands that the processing module can recognize, and pass them to the corresponding functional system (such as the navigation system, the interaction module). At the same time, it generates feedback text and passes it to the speech synthesis model.

[0048] The improved speech synthesis model (TTS) is based on and optimized from the VITS end-to-end model. Its core function is to convert feedback text, navigation prompts, and warning information output by the natural language understanding module into natural speech, which is then broadcast to the user through a speaker. With a sampling rate of 22050Hz and a MOS score of 4.2 out of 5, the synthesized speech is clear and natural. The core improvement is to optimize the synthesis speed of commonly used spoken commands for the blind (delay ≤ 100ms), adapting to the real-time interaction needs of blind people. Those skilled in the art can implement this improved model based on conventional optimization experience with the VITS model and the improvement direction of this disclosure. The following describes the five working modes of the smart cane in conjunction with the various functional systems of this application.

[0049] Work Mode 1: Daily Walking Mode Application scenario: Zhang San, a blind person, commutes daily from home to work (familiar with the route, approximately 1 kilometer). Implementation process: 1. Power on: Press and hold the power button for 3 seconds until the voice prompt "Welcome" is heard; 2. Automatically enter daily walking mode: The system recognizes the current location as "home" and automatically enters daily walking mode; 3. Basic Obstacle Detection: Based on the core technology of the all-round obstacle detection system of this invention, detection and prompting are achieved through the following technical methods. Among them, multi-sensor fusion and AI visual recognition are the innovative parts of this case, which are explained in detail below: • Trash can detected 2 meters ahead: The ultrasonic sensor (HC-SR04 improved version, detection distance 0.5-5 meters) on the front of the cane detects the distance in real time, and at the same time, it is linked with the front AI camera (Sony IMX219, 8 million pixels) to collect images. The improved YOLOv5 visual recognition model (integrating SE-Net attention mechanism) identifies the object as a trash can. The processing module (Rockchip RK3566) combines the distance data detected by ultrasonic detection with the visual recognition results, and broadcasts a voice prompt "There is an obstacle 2 meters ahead, a trash can" through a 3W full-range speaker. At the same time, it controls the right vibration motor to vibrate slightly to help indicate the location of the obstacle. • Pedestrian detected 1 meter to the left: The ultrasonic sensor on the top left of the cane (detection distance 0.5-3 meters) detects the distance and, combined with the AI ​​visual recognition model, confirms that the object is a pedestrian. The processing module converts the distance and category information into a concise voice prompt "There is someone 1 meter to the left," and simultaneously controls the left vibration motor to vibrate, clearly indicating the location of the pedestrian. The improved accuracy of pedestrian recognition at close range (0.5-3 meters) is due to the SE-Net attention mechanism added to the Neck layer of the visual model. 4. Concise prompts: Only prompts for obstacles within 3 meters to reduce interference; 5. Low power consumption operation: CPU frequency 800MHz, power consumption 2.5W; 6. Arrival at destination: The system recognizes the current location as "Company" and provides a voice prompt saying "You have arrived at your destination".

[0050] Technical results: 15-minute commute time, approximately 6% power consumption, and 95% obstacle detection accuracy.

[0051] Working Mode 2: Navigation Mode Application scenario: Blind man Li Si goes to the train station for the first time (unfamiliar route, about 2 kilometers). Implementation process: 1. Voice command: "I want to go to the train station"; 2. Path planning: Based on the core technology of the navigation system for the blind, it calls the Gaode Map pedestrian navigation API in the cloud to obtain road topology data, and combines it with the real-time location information collected by the Beidou / GPS dual-mode positioning module (UBLOX MAX-M8Q, positioning accuracy 2.5 meters). The processing module runs an improved tactile paving priority path planning algorithm (based on the classic A* algorithm, with the introduction of tactile paving weight and dangerous road section penalty mechanism) to automatically select the optimal route containing tactile paving, and simultaneously avoid dangerous road sections such as construction and heavy traffic fed back by the API and the all-round obstacle detection system to complete the path planning. 3. Detailed Navigation: • "Go straight ahead for 50 meters"; • "Turn left 5 meters ahead"; • "Caution: Steps ahead"; • "The train station entrance is 100 meters ahead"; The detailed navigation is achieved through the interaction module in conjunction with the navigation system and obstacle detection system, with the specific technical support as follows: • "Go straight ahead 50 meters": The Beidou / GPS dual-mode positioning module updates the position in real time. The processing module combines the coordinate data of the planned path to calculate the distance between the current position and the next turning point. The prompt is broadcast through a 3W full-range speaker. At the same time, the real-time position is broadcast synchronously every 10 seconds to ensure accurate obstacle avoidance. • "Turn left 5 meters ahead": The processing module combines the coordinates of the turning node in the path planning with the real-time position of the positioning module to trigger a voice prompt 5 meters in advance, and simultaneously controls the left vibration motor to vibrate continuously for 2 seconds to help blind people judge the turning direction (vibration on the left indicates a left turn, and vibration on the right indicates a right turn), which is suitable for blind people without visual dependence. • "Step ahead, please be careful": The infrared sensor (GP2Y0A21YK0F, detection distance 0.1-0.8 meters) at the front of the bottom of the cane detects the step on the ground. The visual recognition model confirms the type and height of the step. The processing module converts the ground obstacle information into a voice prompt and controls the vibration motors on both sides to vibrate synchronously to remind the blind person to watch their step. • "The train station entrance is 100 meters ahead": The visual recognition model (improved YOLOv5) identifies the train station entrance sign, combines it with the distance data detected by the ultrasonic sensor, and the processing module associates it with the destination information of the navigation path to announce the entrance distance prompt, helping blind people to confirm their arrival progress; 4. Real-time location tracking: The location is announced every 10 seconds; 5. Danger Warning: "Construction ahead, please detour"; The visual recognition model of the all-around obstacle detection system identifies construction signs and construction areas, and synchronously links with the real-time traffic data obtained by the communication module. The processing module determines that the construction area affects the current planned route, immediately triggers a voice warning, and runs an improved route planning algorithm to quickly replan the detour route to ensure navigation continuity. 6. Arrival at the destination: "You have arrived at the train station, which is on your right."

[0052] Technical results: Navigation time 25 minutes, power consumption approximately 15%, route planning accuracy 98%.

[0053] Work Mode 3: Exploration Mode Application scenario: Blind man Wang Wu exploring a new environment in a park. Usage process: 1. Switch Mode: Voice command "Switch to Explorer Mode"; 2. Detailed Environment Description: Utilizing a scene recognition and voice description system and an all-around obstacle detection system, the core technologies are a proprietary AI visual recognition model (improved YOLOv5) and multi-sensor fusion technology. The specific implementation method is as follows: • "There is a tree on the left and a bench on the right": The ultrasonic sensors on the left and right sides of the cane detect the distance to the objects on the left and right sides respectively. The AI ​​camera on the front collects environmental images and identifies the objects as trees and benches through an AI visual recognition model (which optimizes the accuracy of object recognition in public places). The processing module combines the location and distance data detected by the sensors and converts it into a concise and easy-to-understand voice description. At the same time, the vibration motor on the corresponding side vibrates slightly to indicate the location of the objects. • "Trash can 3 meters ahead": The distance (3 meters) is detected by the front ultrasonic sensor, the AI ​​visual recognition model identifies the object as a trash can, the processing module integrates the distance and category information and broadcasts the voice prompt. The improved accuracy of close-range object recognition is due to the SE-Net attention mechanism of the Neck layer of the visual model (the innovation of this case). • "This is the park entrance": The AI ​​visual recognition model identifies the park entrance sign (text and graphics), and combined with the location information from the Beidou / GPS positioning module, confirms that the current scene is the park entrance. The scene information is then broadcast via voice to help blind people perceive their surroundings. 3. Object Recognition: This is achieved through the collaboration of an AI visual recognition model from an omnidirectional obstacle detection system and ultrasonic sensors. The specific technical methods are as follows: • "Pedestrian in 5 meters ahead": The ultrasonic sensor at the front detects the distance (5 meters), and the AI ​​visual recognition model accurately identifies the object as a pedestrian (the dynamic object recognition algorithm has been optimized to adapt to pedestrian movement scenarios). The processing module integrates the data and broadcasts the prompt. At the same time, it continuously tracks the pedestrian's position changes. If the pedestrian gets within 3 meters, it will broadcast the reminder again. • "There is a bicycle 2 meters to the left": The ultrasonic sensor on the left detects the distance (2 meters), the AI ​​visual recognition model identifies the object as a bicycle, the processing module combines the orientation and distance data, broadcasts a voice prompt, and simultaneously controls the vibration motor on the left to vibrate, thus clarifying the object's location.

[0054] Technical effect: 90% accuracy in object recognition, enhancing blind people's confidence in independent travel.

[0055] Work Mode 4: Learning Mode Application scenario: Blind man Zhao Liu learns a new route (from his home to a newly opened supermarket). Usage process: 1. Switch Mode: Voice command "Switch to Learning Mode" 2. Route Learning: Utilizing the storage capabilities of a dedicated navigation system for the blind, the positioning module, and the processing module, the core technology is the path memory algorithm (an innovative optimization in this case). The specific implementation method is as follows: • On the first walk, the system records the complete route: the Beidou / GPS dual-mode positioning module collects the location coordinates in real time during the walk (update frequency 1Hz), the processing module records the time and environmental information of each location point (feedback from the all-around obstacle detection system), and stores the complete route data locally to form an initial route file. • On the second trip, the system prompts "This is your frequently used route": The processing module calls the locally stored route file and combines it with the real-time location of the positioning module. It then prompts "This is your frequently used route" via voice, while comparing the deviation between the real-time location and the file route. If the deviation is ≤2 meters, the original route prompt is maintained. • The third walk involves the system optimizing the route (avoiding construction sections): The processing module, in conjunction with the communication module, obtains real-time road conditions and temporary obstacle information from the all-around obstacle detection system. The route is optimized through a path memory algorithm, automatically avoiding construction and congestion sections. The route optimization logic relies on the real-time update mechanism of the improved tactile paving priority path planning algorithm to improve route adaptability. 3. Personal preference settings: Implemented using a voice interaction system, with the specific technical methods as follows: • "Speed ​​up voice": User voice commands are collected by the microphone (INMP441), converted into text commands by the Kaldi-based TDNN speech recognition model, the natural language understanding module interprets the intent of the command, the processing module controls the speaker to adjust the voice playback speed (supports 3 levels of adjustment), and simultaneously provides voice feedback "Speed ​​up voice". • "Turn up the volume": Similarly, after the voice recognition module recognizes the command, the processing module adjusts the speaker volume (supports 5 levels of adjustment) and provides voice feedback on the adjustment result. No manual operation is required throughout the process, which is suitable for blind people who do not rely on vision.

[0056] Technical effect: After learning 3 times, the system remembers frequently visited locations, and route optimization saves 20% of time.

[0057] Working Mode 5: Emergency Mode Application scenario: Sun Qi, a blind man, falls and needs emergency help. Usage process: 1. Fall Detection: Based on the core technology of the emergency call system, the MPU6050 six-axis gyroscope (acceleration ±16g, angular velocity ±2000° / s) in the handle collects the acceleration and posture angle data of the cane in real time (sampling frequency 10Hz). The processing module performs Kalman filtering on the raw data (to eliminate environmental noise). When the acceleration amplitude is detected to be >2.5g (abrupt change) and the posture angle is detected to be >60°, a preliminary fall judgment is triggered. After calibration by multiple groups of blind people's fall tests, false triggering can be effectively avoided. 2. Voice Confirmation: After the processing module triggers the initial fall detection, it controls the 3W full-range speaker to play a voice prompt: "Fall detected, do you want to call for help? Automatic call for help in 10 seconds." At the same time, it controls the vibration motors on both sides to vibrate continuously to remind the user. The voice playback and vibration feedback linkage technology is adapted to blind people without visual perception scenarios. 3. User Confirmation: The processing module continuously monitors user actions (cancel button on the controller). If no user cancel command is detected within 10 seconds (assuming the user is unconscious or unable to operate), the fall is officially confirmed, and an SOS distress call is triggered. If a cancel button operation is detected, the distress call process is terminated. 4. Automatic Emergency Call: This is achieved through the linkage of the communication module and the processing module. The specific technical methods are as follows: • Send SMS to preset contacts: The processing module calls the communication module (Quectel EC200U 4G Cat.1 full network compatible) to obtain the real-time location of the Beidou / GPS positioning module and generate a positioning link. Combined with the current time, it edits a distress message and sends it to the three preset emergency contacts in sequence to ensure that the distress message is delivered in a timely manner. • Make preset phone calls: The communication module makes phone calls to preset contacts in sequence. If no one answers, it automatically switches to the next one until the call is connected or 3 contacts are reached. At the same time, the automatic recording module (MP3 format, 44.1kHz sampling rate) is activated to record the sound of the scene so that emergency contacts can understand the situation. 5. Continuous recording: After the automatic recording module is started, it continuously records ambient sounds and uploads the recording files to the cloud for backup, avoiding the loss of recordings due to local storage failures. 6. Waiting for rescue: The processing module controls the speaker to broadcast "Rescue is on its way" every 5 minutes, and at the same time continuously synchronizes the real-time location with the emergency contact through the communication module until the user manually stops calling for help or the rescuers confirm their arrival, ensuring that the user knows the progress during the rescue process.

[0058] Technical results: Fall detection accuracy rate of 95%, emergency call response time of less than 30 seconds.

[0059] The technical advantages of this invention's artificial intelligence cane for the blind, compared to existing similar products, are shown in the table below: This invention is not limited to the embodiments discussed above. The above description of specific embodiments is intended to describe and illustrate the technical solutions involved in this invention. Obvious modifications, substitutions, or combinations based on the teachings of this invention should also be considered to fall within the protection scope of this invention. The above specific embodiments are used to disclose the best implementation methods of this invention, so that those skilled in the art can apply various embodiments and alternatives of this invention to achieve the objectives of this invention.

Claims

1. A smart cane for blind workers, comprising a cane body, a sensor module, a processing module, an interaction module, a power module and a communication module disposed inside the cane body; and an all-around obstacle detection system, a navigation system for the blind, a scene recognition and voice description system, an emergency call system and a voice interaction system integrated inside the cane body; The omnidirectional obstacle detection system achieves corresponding detection functions based on ultrasonic sensors, infrared sensors, and a multi-sensor fusion visual recognition model. The navigation system for the blind is based on the Beidou / GPS dual-mode positioning module on the top of the cane, cloud-based pedestrian navigation API, and an improved tactile paving priority path planning algorithm to achieve voice navigation and danger warning functions for the blind. The scene recognition and voice description system is based on a visual recognition model that integrates multiple sensors, including a camera configured on the front of the cane and an all-around obstacle detection system. The emergency call system is based on a gyroscope, a one-button SOS call, an automatic recording module, and fall detection control logic configured in the handle of the cane, and realizes the dual functions of automatic and manual emergency call when falling. The voice interaction system is based on a proprietary speech recognition model, a natural language understanding module, and a speech synthesis model, and its core function is to meet the convenient interaction needs of blind people who do not rely on vision.

2. The intelligent cane for blind workers as described in claim 1, characterized in that, The multi-sensor fusion visual recognition model of the omnidirectional obstacle detection system is based on an improved YOLOv5 real-time target detection algorithm. The specific training and adaptation process is as follows: Step 1: Dataset Preprocessing (1) Screening and adaptation of public datasets: Select effective samples related to blind people’s travel from public datasets; perform data enhancement such as size normalization, grayscale correction, random flipping, and Gaussian blur on the screened samples to adapt to the light change scene from the perspective of blind people; (2) Collection and annotation of self-collected datasets: self-collected images from the perspective of blind people; The self-collected images are finely annotated, including: obstacle type, obstacle distance, obstacle location coordinates, and scene type; (3) Data set fusion and partitioning: The preprocessed public samples and self-sampled samples are fused together and divided into training set, validation set and test set according to the proportion. At the same time, the corresponding labels of ultrasonic sensor detection distance are added to the dataset for subsequent multi-sensor data fusion training. Step 2: Model Training The training platform was NVIDIA V100 × 4, the training time was 72 hours, and the specific training parameters were set as follows: (1) Initial parameters: learning rate 0.001, batch size 32, number of iterations 100, weight decay coefficient 0.0005, momentum 0.937; (2) Loss function: CIoU loss function + cross-entropy loss function is adopted, where the category recognition loss weight is set to 1.2 and the bounding box regression loss weight is set to 0.8 to ensure the accuracy of obstacle category recognition and distance localization; (3) Improvement and optimization: The attention mechanism SE-Net is added to the PANet structure in the Neck layer to enhance the feature extraction capability of near obstacles; one convolutional layer is reduced in the CSPDarknet53 in the Backbone layer to reduce the number of model parameters and adapt to the computing power of embedded AI chips. (4) Training process control: Cosine annealing learning rate decay is used for the first 50 epochs of training, and a fixed learning rate is used for the next 50 epochs; validation is performed every 10 epochs. If the accuracy of the validation set does not improve for 3 consecutive times, training is stopped and the optimal model weights are saved. Step 3: Model Quantization and Deployment Adaptation (1) Quantization process: The trained model is quantized. During the quantization process, a self-collected test set is used for calibration to ensure that the quantization is <2%. After quantization, the model volume is compressed to 1 / 4 of the original volume to meet the storage requirements of the embedded chip. (2) Inference engine adaptation: The quantized model is deployed to the Rockchip RKNN inference engine. The model inference process is optimized for the NPU computing power of the RK3566 chip. Redundant feature extraction channels are closed to ensure that the inference speed reaches 30FPS to meet the real-time detection requirements. (3) Multi-sensor fusion adaptation: After the model is deployed, a data fusion interface is added to realize the real-time synchronization of visual recognition results with ultrasonic sensor and infrared sensor data. The Kalman filter + weighted average fusion method is used to output the final obstacle detection result.

3. The artificial intelligence cane for the blind as described in claim 1, characterized in that, The implementation logic of the navigation system specifically for the blind includes the following steps: S1, the Beidou / GPS dual-mode positioning module collects the current location information of blind people in real time; S2, the communication module establishes communication with the cloud-based pedestrian navigation API via 4G / 5G to obtain the basic data required for navigation; S3. The processing module runs an improved tactile paving priority path planning algorithm. Based on the acquired data, it prioritizes the selection of travel routes containing tactile paving, actively avoids identified dangerous road sections, and optimizes the navigation logic in combination with the walking characteristics of blind people to avoid complex turns. The S4 interactive module broadcasts clear and concise navigation prompts and danger warnings to blind people through a full-range speaker. Combined with the tactile feedback of the vibration motor, it provides directional assistance, ensuring convenient and safe navigation and adapting to the usage scenarios of blind people without visual dependence.

4. The artificial intelligence cane for the blind as described in claim 3, characterized in that, The improved tactile paving priority path planning algorithm of the navigation system for the blind is based on the classic A* algorithm. The core improvements that distinguish it from conventional path planning algorithms include three points: First, it introduces a tactile paving weight coefficient and a graded penalty mechanism for dangerous road sections. By allocating weights, it increases the selection priority of tactile paving paths, and by using graded penalties, it reduces the selection probability of dangerous road sections, thus achieving the core requirement of "tactile paving priority and danger avoidance". Second, it improves the heuristic function by combining it with the walking characteristics of blind people. It abandons the conventional design that only considers spatial distance and adapts to the walking speed and travel habits of blind people, making the navigation path more in line with the actual travel scenarios of blind people. Third, a real-time path update mechanism is set up to link the real-time data of the obstacle detection system and the positioning module, dynamically adapt to scenarios such as temporary obstacles and changes in road conditions, and avoid path rigidity.

5. The artificial intelligence cane for the blind as described in claim 1, characterized in that, The fall detection control logic of the emergency call system includes the following process: Step 1: Signal Acquisition and Preprocessing: The gyroscope acquires the acceleration and attitude angle data of the cane in real time, with the sampling frequency set to 10Hz. The processing module filters the acquired raw data to obtain stable acceleration amplitude and attitude angle. Step 2, Fall Detection Trigger Conditions: When the processing module detects that the following two conditions are met simultaneously, a preliminary fall detection is triggered: ① A sudden change in acceleration amplitude, and acc_magnitude > 2.5g; ② A drastic change in the cane posture angle, and angle > 60°; Step 3, Fall Confirmation Logic: After a preliminary fall is determined, the processing module starts a 3-second countdown confirmation mechanism. During this period, gyroscope data is continuously collected. If no posture recovery action is detected within 3 seconds and no manual cancellation command is received from the user, the fall is officially confirmed. If posture recovery or user cancellation command is detected within 3 seconds, the fall determination process is terminated and an SOS call is not triggered. Step 4, SOS Call Trigger Control: After a fall is officially confirmed, the processing module immediately triggers the SOS distress call command and simultaneously initiates the following linked operations: ① Controls the speaker of the interaction module to play a voice confirmation prompt, and at the same time controls the vibration motor to vibrate continuously to remind the user that a fall distress call has been triggered; ② Starts the automatic recording module to begin recording ambient sounds at the scene; ③ Control the communication module to initiate the distress call process.

6. The artificial intelligence cane for the blind as described in claim 5, characterized in that, The voice interaction system is based on the collaborative implementation of a speech recognition model, a natural language understanding module, and an improved speech synthesis model: the speech recognition model converts speech commands into text, the natural language understanding module parses the text intent and generates control commands and feedback text, the control commands trigger the corresponding functional system to run, and the feedback text is converted into speech by the improved speech synthesis model and broadcast to the user through a speaker. At the same time, it is combined with a vibration motor to realize tactile assistance prompts, thus completing the complete voice interaction.

7. The artificial intelligence cane for the blind as described in claim 1, characterized in that, The sensor module is used to replace the visual perception of blind people to perceive the surrounding environment. It includes four ultrasonic sensors arranged around the top of the cane to detect obstacles, two infrared sensors arranged at the bottom front of the cane for ground detection to prevent falls, one camera arranged at the top front of the cane for AI visual recognition to replace the eyes of blind people, one Beidou / GPS dual-mode positioning module installed at the top of the cane for positioning and navigation, and a gyroscope installed inside the cane handle for attitude detection to prevent falls.

8. The artificial intelligence cane for the blind as described in claim 1, characterized in that, The processing module is used to process environmental information in real time and adopts an embedded AI chip; its main chip is Rockchip RK3566 quad-core ARM Cortex-A55, and AI acceleration is achieved through a built-in 0.8TOPS NPU.

9. The artificial intelligence cane for the blind as described in claim 1, characterized in that, The interaction module includes a voice interaction unit and a haptic feedback unit. The voice interaction unit includes a microphone and a speaker, and the haptic feedback unit includes a vibration motor.

10. The artificial intelligence cane for the blind as described in claim 1, characterized in that, The power module uses four 18650 lithium-ion batteries connected in series, with a capacity of 3200mAh × 4 = 12800mAh. The communication module adopts the Quectel EC200U 4G Cat.1 full network compatible module, which includes a WiFi module and a Bluetooth module integrated into the RK3566 chip.

Images