Intelligent blind guiding system based on Beidou positioning system

Abstract

The invention discloses an intelligent blind guiding system based on a Beidou positioning system. The system is composed of a blind guiding stick, a server and a WeChat applet. The blind guiding stick integrates a main controller STM32F407, a Beidou positioning module, a camera module K230, an ultrasonic sensor, a voice module and a key module, road condition elements are recognized in real time through a visual detection algorithm based on YOLOv8, ultrasonic ranging data are processed in combination with an adaptive Kalman filtering algorithm, and multi-source information fusion and safety level evaluation are achieved. And the server calls the Baidu navigation API to provide path planning service. The WeChat applet displays the user position, surrounding road condition information and navigation information in real time, and contacts with family members or volunteers in time in emergency. The system innovatively adopts a response logic and layered safety broadcast mechanism that emergency obstacle avoidance is prior to path guidance, the use threshold is reduced through key and voice interaction, a closed loop of front-end perception-cloud decision-family affection interconnection is constructed, and the travel safety and convenience of visually impaired people are effectively improved.

Description

Technical Field

[0001] This invention discloses an intelligent guidance system for the blind based on the BeiDou positioning system, belonging to the field of assistive technology for people with disabilities. Background Technology

[0002] According to statistics from the World Health Organization in 2023, at least 2.2 billion people worldwide have visual impairments, of whom approximately 43 million live in a state of complete blindness. my country, in particular, has a large number of visually impaired individuals, accounting for 20% of the global total, and faces especially severe challenges, with their quality of life and social participation long limited. Currently, most navigation products on the market focus on simple positioning or obstacle avoidance functions, neglecting the travel experience of visually impaired individuals. Their functions are relatively limited, and the accuracy and effectiveness of navigation need improvement.

[0003] To improve the travel experience for visually impaired individuals and address their difficulties in traveling and the complexity of operating smart devices, especially considering the shortcomings of existing solutions, this invention proposes an intelligent guidance system for the visually impaired based on the BeiDou Navigation Satellite System. Summary of the Invention

[0004] The purpose of this invention is to propose an intelligent guidance system for the blind based on the BeiDou positioning system, so as to solve the problems of difficult travel for visually impaired people and the complexity of operating intelligent devices to the greatest extent, and further improve the travel experience of blind people.

[0005] Specifically, the present invention includes the following steps:

[0006] The system mainly includes a guide cane, a server, and a WeChat mini-program; the guide cane includes an STM32F407 processor, a data acquisition module, a communication module, and a power module.

[0007] II. The data acquisition module consists of a Beidou positioning module, a K230 camera module, an ultrasonic module, a voice interaction module, and a button module, and is used to collect data information such as location, road conditions, and audio.

[0008] The communication module (Ⅲ) is used for data communication between the STM32F407 processor, the server, and the WeChat mini-program.

[0009] IV. The STM32F407 microcontroller processes the acquired data to determine whether the blind person's road conditions are safe and to detect whether the navigation (emergency) button has been pressed. It also transmits the surrounding road conditions, button detection results, and processed data to the server via the communication module.

[0010] V. Call the Baidu Navigation API on the server to provide high-quality travel route planning services based on BeiDou positioning system data;

[0011] VI. The WeChat mini program retrieves data from the server to view the location data of the white cane, surrounding road conditions, navigation information, and contact information of the blind person's family. When the system detects that the emergency button on the device has been triggered, it forcibly switches to emergency mode and establishes contact between the blind person and their family or volunteers through the WeChat mini program platform.

[0012] 1. The intelligent navigation system for the blind based on the BeiDou positioning system according to claim 1, characterized in that, in step IV, the microcontroller STM32F407 processes the acquired data information, determines whether the road conditions where the blind person is located are safe and detects whether the navigation (emergency) button has been pressed, and transmits the surrounding road condition information, button detection results and processed data to the server through the communication module, and at least further includes the following steps:

[0013] The K230 camera module uses the YOLOv8 object detection algorithm to acquire image data in real time. It extracts multi-scale features through a backbone network composed of C2f modules, and enhances these features through PAN-FPN feature pyramid and path aggregation network to achieve real-time identification and localization of key road condition elements such as traffic lights, zebra crossings, tactile paving, road obstacles, and potholes. By setting confidence thresholds and non-maximum suppression and other post-processing mechanisms, it outputs high-precision object detection boxes and category information of people and objects around the guide cane.

[0014] The STM32F407 microcontroller uses an adaptive Kalman filter algorithm to process obstacle distance data collected by ultrasonic sensors. The purpose is to achieve optimal dynamic estimation of obstacle distance in a noisy environment, thereby improving the accuracy, smoothness and reliability of the ranging data.

[0015] The STM32F407 microcontroller performs fusion analysis on the processed data from the ultrasonic sensor and the K230 camera module. Based on the environmental perception results after fusion analysis, it classifies the safety level and reports road condition prompts to the user through the voice interaction module according to the priority logic of "emergency obstacle avoidance takes precedence over path guidance".

[0016] The STM32F407 microcontroller performs noise reduction, sampling rate and format standardization, frame segmentation and lengthening, and normalization on the acquired audio data.

[0017] The system uses the external interrupt of the STM32F407 microcontroller itself to detect whether the navigation (emergency) button is pressed.

[0018] Finally, the processed data, road safety detection results, and key detection results are transmitted to the server via the communication module.

[0019] 2. The intelligent navigation system for the blind based on the BeiDou positioning system according to claim 1, characterized in that step V, which calls the Baidu Navigation API on the server to provide high-quality travel route planning services based on BeiDou positioning system data, shall at least include the following steps:

[0020] When the server receives a command triggered by the navigation buttons, the smart cane system enters navigation mode. First, it calls the Baidu speech recognition API to convert the received audio data into speech-text data. Then, it inputs the speech-text into the Baidu navigation API processing function, and combines it with the high-precision positioning information of the Beidou system to obtain optimized path planning results. Finally, it sends the path planning results to the STM32F407 microcontroller through the communication module, and the STM32F407 microcontroller drives the speech module to perform voice broadcast.

[0021] 3. The intelligent navigation system for the blind based on the BeiDou positioning system according to claim 1, characterized in that, in step VI, the WeChat mini-program retrieves data from the server to view the location data of the white cane, surrounding road conditions, navigation information, and contact information of the blind person's family; when the system detects that the emergency button on the device is triggered, it forcibly switches to emergency mode and establishes contact between the blind person and their family or volunteers through the WeChat mini-program platform, and should at least include the following steps:

[0022] You can view the location data and surrounding road conditions of blind people in real time on the WeChat mini program;

[0023] When the system detects that the emergency button on the device has been triggered, it immediately interrupts any currently running state and forces a switch to emergency mode.

[0024] In emergency mode, the system simultaneously broadcasts "Emergency assistance detected, notifying emergency contacts" via voice module; and pushes emergency information to the blind person's family members via WeChat mini-program, controlling the phone's ringtone and vibration to alert the family members.

[0025] Establish connections between blind people and their families or volunteers on WeChat mini-programs. Attached Figure Description

[0026] Figure 1 This is a flowchart of the process of this invention;

[0027] Figure 2 This is a data transmission diagram of the present invention;

[0028] Figure 3 This is a flowchart of step IV of the present invention;

[0029] Figure 4 This is a flowchart of the YOLOv8 object detection algorithm;

[0030] Figure 5 This is a flowchart of step V of the present invention;

[0031] Figure 6 This is a flowchart of step VI of the present invention;

[0032] Figure 7 This is a screenshot of the WeChat mini-program based on the present invention; Specific implementation methods

[0033] The present invention will now be described in detail with reference to the accompanying drawings. This description is merely illustrative and explanatory, and should not be construed as limiting the scope of protection of the present invention. Furthermore, those skilled in the art can combine the features in the embodiments described herein and in different embodiments based on the description in this document.

[0034] Figure 1 This is a flowchart of the process of the present invention, and the specific implementation steps are as follows:

[0035] The system mainly includes a guide cane, a server, and a WeChat mini-program; the guide cane includes an STM32F407 processor, a data acquisition module, a communication module, and a power module.

[0036] II. The data acquisition module consists of a Beidou positioning module, a K230 camera module, an ultrasonic module, a voice interaction module, and a button module, and is used to collect data information such as location, road conditions, and audio.

[0037] The communication module (Ⅲ) is used for data communication between the STM32F407 processor, the server, and the WeChat mini-program.

[0038] IV. The STM32F407 microcontroller processes the acquired data to determine whether the blind person's road conditions are safe and to detect whether the navigation (emergency) button has been pressed. It also transmits the surrounding road conditions, button detection results, and processed data to the server via the communication module.

[0039] V. Call the Baidu Navigation API on the server to provide high-quality travel route planning services based on BeiDou positioning system data;

[0040] VI. The WeChat mini program retrieves data from the server to view the location data of the white cane, surrounding road conditions, navigation information, and contact information of the blind person's family. When the system detects that the emergency button on the device has been triggered, it forcibly switches to emergency mode and establishes contact between the blind person and their family or volunteers through the WeChat mini program platform.

[0041] In step IV, the STM32F407 microcontroller processes the acquired data to determine whether the road conditions are safe for the blind person and detect whether the navigation (emergency) button has been pressed. It then transmits the surrounding road condition information, button detection results, and processed data to the server via a communication module. The process also includes at least the following steps:

[0042] The K230 camera module uses the YOLOv8 object detection algorithm to acquire image data in real time. The YOLOv8 object detection algorithm process is as follows: Figure 4 As shown, the process includes image input, preprocessing, C2f backbone network for multi-scale feature extraction, PAN-FPN bidirectional feature fusion, parallel classification and regression, postprocessing, and output of target detection results.

[0043] Step 1: Capture images of the surrounding environment using the K210 camera module;

[0044] Step 2: Preprocess the image, that is, first scale the image to a standard size, then normalize it to linearly transform the pixel value from 0-255 to the range of [0,1] or [-1,1], and finally perform color space conversion and histogram equalization.

[0045] Step 3: Extract multi-scale features through a backbone network composed of C2f modules. Each C2f module adopts a cross-stage partial connection design, splitting the input features into a main path and a shortcut path. The main path undergoes nonlinear transformation and feature deepening through multiple convolutional layers, while the shortcut path retains the original features to avoid gradient vanishing. Then, the features of the two paths are concatenated in the channel dimension to achieve feature reuse and information fusion. Finally, shallow features, mid-level features, and deep features of the input image are extracted. Shallow features capture detailed information such as edges and textures, and are sensitive to small targets. Mid-level features extract local structures and component-level semantics, while deep features contain global context and high-level semantics, and are corresponding to large target recognition.

[0046] Step 4: Feature fusion enhancement is performed via PAN-FPN feature pyramid and path aggregation network. By constructing two complementary information transmission paths, a set of enhanced multi-scale feature pyramids containing rich details and high-level semantics is generated. The top-down path upsamples and fuses the high-level semantic information contained in deep features into the mid-level and shallow features, thereby improving the understanding of the overall category and structure of the target. The bottom-up path downsamples the rich texture and spatial details of shallow features and integrates them into high-level features to recover the accurate localization information that may be lost due to network downsampling.

[0047] Step 5: The enhanced multi-scale features are decoded through a parallel dual-branch structure of classification and regression branches to achieve real-time identification and localization of key road condition elements such as traffic lights, zebra crossings, tactile paving, and potholes. The classification branch predicts the probability distribution of each anchor box belonging to each target category through convolutional layers to achieve semantic recognition, while the regression branch outputs the coordinate offset and confidence score of the bounding box to complete spatial localization.

[0048] Step 6: Set confidence thresholds and perform non-maximum suppression for post-processing;

[0049] Step 7: Output high-precision target detection bounding boxes and category information of people and objects around the guide cane;

[0050] The STM32F407 microcontroller uses an adaptive Kalman filter algorithm to process obstacle distance data collected by ultrasonic sensors. The purpose is to achieve optimal dynamic estimation of obstacle distance in a noisy environment, thereby improving the accuracy, smoothness and reliability of the ranging data.

[0051] The STM32F407 microcontroller fuses and analyzes the processed ultrasonic sensor data and the data obtained from the K230 camera module. Based on the fused environmental perception results, it classifies the safety level into several categories: Safe (no obstacles ahead or obstacles > 2.0 meters away, clear camera path recognition, and traffic signals indicating passage is permitted); Caution (obstacles detected at 1.0–2.0 meters away, or the camera detects zebra crossings, slow-moving vehicles, or minor road unevenness requiring attention); Warning (obstacles at 0.5–1.0 meters away, or the camera detects red lights, pedestrians crossing, interrupted tactile paving, potholes, etc., requiring immediate attention); Danger (obstacles < 0.5 meters away and rapidly approaching, or the camera detects suddenly appearing vehicles, pedestrians, deep potholes, etc., posing an immediate threat).

[0052] Based on the priority logic of "emergency obstacle avoidance takes precedence over path guidance", when the system determines the safety level to be "dangerous", the system will interrupt the ongoing path planning, navigation voice broadcast and other tasks and give priority to broadcasting emergency obstacle avoidance instructions. Only after the emergency obstacle avoidance action is completed and the environmental perception data confirms that the threat has been eliminated will the system return from the interruption and resume or adjust its path guidance task.

[0053] Then, the voice interaction module broadcasts traffic alerts to the user. When the level is "Safe," the voice broadcasts "Road is clear, please proceed safely"; when the level is "Caution," the voice broadcasts "Obstacle ahead, please detour" or "Pay attention to changes in traffic conditions"; when the level is "Warning," the voice broadcasts "Please slow down" or "Danger ahead, it is recommended to turn"; and when the level is "Danger," the voice broadcasts "Stop immediately" or "Turn left immediately to avoid it."

[0054] The STM32F407 microcontroller performs noise reduction, sampling rate and format standardization, frame segmentation and lengthening, and normalization on the acquired audio data.

[0055] The external interrupt of the STM32F407 microcontroller is used to detect whether the navigation (emergency) button is pressed.

[0056] Finally, the processed data, surrounding road condition safety detection results, and key detection results are transmitted to the server via the communication module.

[0057] Step V, which involves calling the Baidu Navigation API on the server to provide high-quality travel route planning services based on BeiDou positioning system data, also includes at least the following steps:

[0058] If the server receives a navigation button command, it will automatically enter navigation mode and send a start signal to the STM32F407 microcontroller, causing the STM32F407 microcontroller to drive the voice interaction module to broadcast the preset prompt voice "Navigation has been started, please provide your destination";

[0059] Subsequently, the system calls the Baidu Speech Recognition Application Programming Interface (API) to recognize and process the received user audio data, converting it into voice-text commands. If speech recognition fails, the server sends a control command to the STM32F407 microcontroller, causing the STM32F407 to drive the voice interaction module to announce "Speech recognition failed, please try again." If the preset termination command keyword "exit navigation mode" is detected in the recognition result, the server exits navigation mode and sends a termination command to the STM32F407 microcontroller, causing the STM32F407 to control the voice interaction module to announce "This navigation has ended," and then stops all navigation-related voice output.

[0060] Then, the successfully recognized voice and text commands are used as input parameters to call the function of Baidu Navigation API and obtain navigation commands. Combined with real-time positioning data of blind people, navigation commands are generated and updated in real time.

[0061] Finally, the system uses TTS technology to synthesize navigation instructions into voice data, converts it into binary audio stream format, and transmits the data to the STM32F407 microcontroller via the communication module. The STM32F407 microcontroller then drives the voice interaction module to read out each voice message.

[0062] It should be noted that: throughout the navigation process, the system continuously monitors voice commands in real time. If a voice command containing the keyword "stop navigation" is detected, the server immediately terminates the API calls and data processing processes related to navigation. The STM32F407 microcontroller controls the voice module to announce the end of the navigation and stop subsequent navigation broadcasts.

[0063] In step VI, the WeChat mini-program retrieves data from the server to view the location data of the white cane, surrounding road conditions, navigation information, and contact information of the blind person's family. When the system detects that the emergency button on the device has been triggered, it forcibly switches to emergency mode and establishes contact between the blind person and their family or volunteers through the WeChat mini-program platform. This step also includes at least the following steps:

[0064] You can view the location data and surrounding road conditions of blind people in real time on the WeChat mini program;

[0065] When the system detects that the emergency button on the device has been triggered, it immediately interrupts any currently running state and forces a switch to emergency mode.

[0066] The system announces "Emergency assistance detected, notifying emergency contacts" via voice module; at the same time, the WeChat mini program sends a strong reminder message to the preset family member or volunteer contact, and controls the phone to vibrate to remind the family member while displaying the blind person's location data and surrounding road conditions.

[0067] A two-way communication link is established on a WeChat mini-program between blind people and their families or volunteers. Families or volunteers send voice-text messages to the server via the WeChat mini-program. After receiving the messages, the server converts them into binary audio streams, which are then sent to the STM32F407 microcontroller via the communication module, and the microcontroller controls the voice module to broadcast the messages. At the same time, the STM32F407 microcontroller controls the voice module on the device to collect the voice information sent by the blind person and upload it to the server. The audio data is then converted into voice-text information via the Baidu speech recognition API and pushed to the WeChat mini-program interface for display by the server.

[0068] Compared with the prior art, the advantages of the above embodiments of the present invention are as follows:

[0069] This invention proposes an intelligent guidance system for the visually impaired based on the BeiDou Navigation Satellite System. By constructing a multimodal system that integrates perception, decision-making, and interaction, it achieves a deep fusion of high-precision environmental perception and user-friendly voice guidance. The system's intelligent guide cane integrates BeiDou positioning, multiple sensors, and an embedded vision module. It employs an adaptive Kalman filter algorithm to optimize ultrasonic ranging data and combines it with a YOLOv8-based visual recognition model to achieve real-time detection of road condition elements such as traffic lights, tactile paving, and potholes. An innovative multi-source data fusion mechanism dynamically classifies safety levels based on environmental perception results and follows a response logic of "emergency obstacle avoidance taking precedence over path guidance," providing layered warnings and navigation guidance through a voice interaction module. The system is equipped with physical navigation / emergency dual-function buttons and supports cloud-based voice recognition and TTS broadcast services, greatly reducing the operational threshold for visually impaired users. A real-time monitoring and emergency communication system is built through a WeChat mini-program: in emergency situations, the system automatically triggers voice alarms, sends strong reminders to family members (location sharing and vibration alerts), and establishes a two-way voice communication channel, significantly improving emergency response efficiency. This system upgrades traditional guide devices into a three-in-one protection system of "safety warning, intelligent navigation, and family connection" through the full-link collaboration of "front-end perception, cloud decision-making, and terminal interaction". It constructs a smart travel ecosystem closed loop of "one person, one cane, one cloud, one terminal, and full escort", providing visually impaired people with a reliable, easy-to-use, and emotionally supportive travel assistance solution.

[0070] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent guidance system for the visually impaired based on the BeiDou positioning system, characterized in that, Includes the following steps: The system mainly includes a guide cane, a server, and a WeChat mini-program; the guide cane includes an STM32F407 processor, a data acquisition module, a communication module, and a power module. II. The data acquisition module consists of a Beidou positioning module, a K230 camera module, an ultrasonic module, a voice interaction module, and a button module, and is used to collect data information such as location, road conditions, and audio. The communication module (Ⅲ) is used for data communication between the STM32F407 processor, the server, and the WeChat mini-program. IV. The STM32F407 microcontroller processes the acquired data to determine whether the blind person's road conditions are safe and to detect whether the navigation (emergency) button has been pressed. It also transmits the surrounding road conditions, button detection results, and processed data to the server via the communication module. V. Call the Baidu Navigation API on the server to provide high-quality travel route planning services based on BeiDou positioning system data; VI. The WeChat mini program retrieves data from the server to view the location data of the white cane, surrounding road conditions, navigation information, and contact information of the blind person's family. When the system detects that the emergency button on the device has been triggered, it forcibly switches to emergency mode and establishes contact between the blind person and their family or volunteers through the WeChat mini program platform.

2. The intelligent guidance system for the blind based on the BeiDou positioning system according to claim 1, characterized in that, Step IV, where the STM32F407 microcontroller processes the acquired data to determine whether the road conditions are safe for the blind person and to detect whether the navigation (emergency) button has been pressed, and transmits the surrounding road condition information, button detection results, and processed data to the server via the communication module, includes at least the following steps: The K230 camera module uses the YOLOv8 object detection algorithm to acquire image data in real time. It extracts multi-scale features through a backbone network composed of C2f modules, and enhances these features through PAN-FPN feature pyramid and path aggregation network to achieve real-time identification and localization of key road condition elements such as traffic lights, zebra crossings, tactile paving, road obstacles, and potholes. By setting confidence thresholds and non-maximum suppression and other post-processing mechanisms, it outputs high-precision object detection boxes and category information of people and objects around the guide cane. The STM32F407 microcontroller uses an adaptive Kalman filter algorithm to process obstacle distance data collected by ultrasonic sensors. The purpose is to achieve optimal dynamic estimation of obstacle distance in a noisy environment, thereby improving the accuracy, smoothness and reliability of the ranging data. The STM32F407 microcontroller performs fusion analysis on the processed data from the ultrasonic sensor and the K230 camera module. Based on the environmental perception results after fusion analysis, it classifies the safety level and reports road condition prompts to the user through the voice interaction module according to the priority logic of "emergency obstacle avoidance takes precedence over path guidance". The STM32F407 microcontroller performs noise reduction, sampling rate and format standardization, frame segmentation and lengthening, and normalization on the acquired audio data. The external interrupt of the STM32F407 microcontroller itself is used to detect whether the navigation (emergency) button is pressed. Finally, the processed data, road safety detection results, and button detection results are transmitted to the server via the communication module.

3. The intelligent guidance system for the blind based on the BeiDou positioning system according to claim 1, characterized in that, Step V, which calls the Baidu Navigation API on the server to provide high-quality travel route planning services based on BeiDou positioning system data, should at least include the following steps: When the server receives a command triggered by the navigation buttons, the smart cane system enters navigation mode. First, it calls the Baidu speech recognition API to convert the received audio data into speech-text data. Then, it inputs the speech-text into the Baidu navigation API processing function, and combines it with the high-precision positioning information of the Beidou system to obtain optimized path planning results. Finally, it sends the path planning results to the STM32F407 microcontroller through the communication module, and the STM32F407 microcontroller drives the speech module to perform voice broadcast.

4. The intelligent guidance system for the blind based on the BeiDou positioning system according to claim 1, characterized in that, Step VI involves the WeChat mini-program retrieving data from the server to view the location data of the white cane, surrounding road conditions, navigation information, and contact information for the blind person's family. When the system detects that the emergency button on the device has been triggered, it forcibly switches to emergency mode and establishes contact between the blind person and their family or volunteers through the WeChat mini-program platform. This step should at least include the following steps: You can view the location data and surrounding road conditions of blind people in real time on the WeChat mini program; When the system detects that the emergency button on the device has been triggered, it immediately interrupts any currently running state and forces a switch to emergency mode. In emergency mode, the system simultaneously broadcasts "Emergency assistance detected, notifying emergency contacts" via voice module; and pushes emergency information to the blind person's family members via WeChat mini-program, controlling the phone's ringtone and vibration to alert the family members; Establish connections between blind people and their families or volunteers on WeChat mini-programs.

Images