Electronic guide system for assisting visually impaired people to go out

Abstract

The invention discloses an electronic guide system for assisting visually impaired people to go out, relates to the technical field of auxiliary equipment for disabled people, and aims to solve the problems that a traditional auxiliary means is incomplete in perception and inaccurate in guide. The system comprises a sensing layer, a processing and decision-making layer, a man-machine interaction layer and a power supply module, wherein the sensing layer acquires environment and motion data through a front / lower camera, an ultrasonic sensor array and an IMU (Inertial Measurement Unit); the processing and decision-making layer takes an ARM Cortex-A series SoC as a main processor, multi-source positioning data is fused to plan a barrier-free path, and a guide instruction is generated in combination with an AI recognition and obstacle avoidance algorithm; the man-machine interaction layer realizes bimodal interaction with a voice system through a vibration motor array; and the power supply module supplies power through dynamic adaptation of the BMS. The equipment can realize full-scene accurate perception and guidance, and ensures safe travel of visually impaired people.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of auxiliary equipment for the disabled, and particularly relates to an electronic guiding system for assisting visually impaired people to travel. BACKGROUND

[0002] The auxiliary system relied on by visually impaired people for travel needs to meet the three core requirements of "environment perception, path guidance, and information interaction". The current existing auxiliary system has significant technical limitations: some systems only integrate a single perception module (such as only containing a camera or only containing an ultrasonic sensor), which cannot cover the "ground-non-ground" and "far-medium-near" full-dimensional environment detection; some systems lack a collaborative processing architecture, with the perception module and the interaction module working independently, resulting in delayed or deviated guidance instructions; and some systems have not solved the problem of indoor and outdoor positioning connection, with positioning failure in indoor scenes such as shopping malls and subway stations, and cannot realize continuous guidance.

[0003] With the application of electronic technology in the field of accessibility, the visually impaired auxiliary system needs to break through the bottleneck of "module isolation, lack of cooperation, and poor scene adaptation", and build a complete system with multi-module deep fusion and full-scene adaptation to meet the "safe, accurate, and continuous" travel needs of visually impaired people. SUMMARY

[0004] The purpose of the application is to provide a test fixture for I-shaped crack propagation between the face core and the core of a sandwich panel.

[0005] To achieve the above purpose, the application provides the following technical solutions: An electronic guiding system for assisting visually impaired people in traveling, comprising a perception layer, a processing and decision-making layer, a human-computer interaction layer and a power module, each layer is connected through a hardware interface and a data protocol to form a complete system closed loop; the perception layer comprises a front camera, a lower camera, an ultrasonic sensor array and an inertial measurement unit, the front camera is connected to the processing and decision-making layer through a MIPICSI interface; the lower camera is connected to the processing and decision-making layer through a MIPI CSI interface; the ultrasonic sensor array is connected to the processing and decision-making layer through a UART interface; the inertial measurement unit is connected to the processing and decision-making layer through an I2C interface; the processing and decision-making layer is the core of the system, comprising a main processor, a positioning module, a navigation engine, an AI recognition engine and an obstacle avoidance algorithm module; the positioning module integrates a GPS chip, a Bluetooth / Wi-Fi module; the navigation engine calls an electronic map API through a software interface; the AI recognition engine is based on a convolutional neural network; the human-computer interaction layer comprises a tactile feedback device, a voice system and an operation component, the tactile feedback device is built-in with a micro vibration motor array, connected to the processing and decision-making layer through a GPIO / PWM interface; the voice system comprises a loudspeaker, a microphone and an audio codec chip, connected to the processing and decision-making layer through an I2S interface; the operation component is connected to the processing and decision-making layer through a GPIO interface; the power module comprises a rechargeable lithium battery and a battery management system, the inertial measurement unit is connected to the main processor through an I2C interface.

[0006] Specifically, the ultrasonic sensor array is composed of 4-6 ultrasonic sensors.

[0007] Specifically, the multi-source data fused by the positioning module includes GPS data, Bluetooth beacon / Wi-Fi data, IMU data and visual odometry data.

[0008] Specifically, the data transmission interface of the perception layer and the processing and decision-making layer further comprises: the IMU transmits motion data through an I2C interface, and the ultrasonic sensor array transmits ranging data through a UART interface; the instruction transmission interface of the processing and decision-making layer and the human-computer interaction layer further comprises: the tactile feedback device receives vibration control instructions through a GPIO / PWM interface, and the voice system receives voice control instructions through an I2S interface.

[0009] Specifically, the electronic map API called by the navigation engine has the functions of high-precision path planning, POI information query and real-time road condition acquisition.

[0010] Specifically, the functions of the BMS include: outputting adaptive voltage according to the power supply requirements of each module; real-time acquisition of current data of each module to calculate the total power consumption of the system.

[0011] Specifically, the main processor is an ARM Cortex-A series SoC, supports multi-interface data parallel processing, and can synchronously schedule the sensing layer data acquisition frequency, the processing layer algorithm operation rhythm and the interactive layer instruction output timing.

[0012] Specifically, the operation component includes a wake-up key, a pause key and an emergency stop key, which are respectively used for waking up the device, pausing the guide flow and triggering an emergency stop instruction, and the key operation signals are transmitted in real time to the main processor through the GPIO interface. Beneficial effects

[0013] The three-layer cooperative architecture of the application realizes non-delay transmission of sensing-processing-interactive data through unified scheduling of the main processor, and the response speed of the guide instruction is better than that of the existing isolated module system, thereby avoiding guide deviation caused by asynchronous modules; the multi-source sensing modules are complementary, the front camera recognizes long-distance scenes and traffic lights, the lower camera detects ground conditions, the ultrasonic sensor captures medium and short distance obstacles, and the IMU tracks device motion, thereby completely solving the problem of "single sensing dimension" of the existing system; the dual-mode interactive system can cope with different environments, the information transmission success rate is higher than that of the single-mode system, and the operation threshold of the visually impaired user is reduced; the adaptive power management system can dynamically adjust the power supply of each module, thereby avoiding invalid power consumption, and compared with the existing fixed power supply system, the endurance is improved by more than 30%; the cross-scene positioning connection system can realize seamless switching of "outdoor road - shopping mall entrance - indoor corridor" positioning, thereby guaranteeing the continuity of the guide. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 The figure is a system architecture schematic diagram of an electronic guide system for assisting visually impaired people to travel. DETAILED DESCRIPTION

[0015] The technical solutions of the application will be described below in detail with reference to the drawings, obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0016] In the description of the application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore it cannot be understood as a limitation on the application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0017] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0018] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0019] See Figure 1 This invention consists of four parts: a perception layer, a processing and decision-making layer, a human-computer interaction layer, and a power module. Each part is connected via hardware interfaces and data protocols, forming a complete closed-loop system of "data acquisition - processing and analysis - command output - power support." The specific architecture is as follows: 1. Perception layer Functional positioning: Responsible for collecting outdoor / indoor environmental data and equipment motion data to provide raw data support for the processing layer; Components and functions: Front-facing camera: Employs a high-definition image sensor connected to the processing layer via a MIPI CSI interface, used to capture images of distant scenes and recognize intersections, pedestrian crossings, traffic lights, store names, and bus stop signs; Lower camera: Installed at the bottom of the device, close to the ground, and connected to the processing layer via MIPI CSI interface, used to capture ground images and identify steps, potholes, and curbs; Ultrasonic sensor array: Composed of 4-6 ultrasonic sensors, connected to the processing layer via a UART interface, with a detection range covering several centimeters to several meters, used to collect data on obstacles above the ground, such as streetlights, trash cans, and low tree branches; IMU (Inertial Measurement Unit): It has a built-in accelerometer and gyroscope, and connects to the processing layer via the I2C interface. It is used to collect device motion data, such as speed, acceleration, and orientation angle, to assist the positioning module in correcting position deviations. Data transmission characteristics: Data collected by each sensing module is transmitted to the processing layer in real time through a dedicated interface. The transmission rate is adapted to the module requirements. For example, the data transmission rate of camera image data is higher than that of ultrasonic sensor ranging data.

[0020] 2. Processing and Decision-Making Level Functional positioning: Receive raw data from the perception layer, process it through algorithms and collaborate with modules to generate guidance instructions and control signals, and transmit them to the human-computer interaction layer; Components and functions: Main processor: ARM Cortex-A series SoC is used as the system core to coordinate the work of various modules, supports parallel data processing of multiple interfaces, and can synchronously schedule data acquisition in the perception layer, algorithm operation in the processing layer, and instruction output in the interaction layer. Positioning module: integrates GPS chip and Bluetooth / Wi-Fi module, connects to the main processor through SPI interface, merges GPS data, Bluetooth beacon / Wi-Fi data, IMU data and visual odometry data, calculates position based on camera image and outputs real-time user location information; Navigation Engine: Through software interface calls electronic map API, it provides high-precision route planning, POI (Point of Interest) information, and real-time traffic conditions. Under the control of the main processor, it plans an accessible route from the starting point to the destination based on the location information output by the positioning module, prioritizing routes with pedestrian crossings and no obstacles. AI Recognition Engine: Based on the Convolutional Neural Network (CNN) algorithm, with the support of the main processor, it analyzes the image data collected by the front camera and the lower camera, and outputs object recognition results, such as pedestrians, vehicles, obstacles, scene recognition results, such as intersections, crosswalks, traffic lights, and text recognition results such as store names and bus stop signs. Obstacle avoidance algorithm module: Integrated into the main processor via software, it fuses ranging data from the ultrasonic sensor array with the recognition results from the AI ​​recognition engine to construct an obstacle map of the surrounding environment, calculate safe avoidance paths, and generate obstacle avoidance control commands; Control Logic: The main processor synchronizes the data acquisition frequency of the perception layer through an internal timing control protocol, such as the camera acquiring 30 frames of images per second, the ultrasonic sensor detecting 10 times per second, the algorithm operation rhythm of the processing layer, such as the AI ​​recognition engine processing time of ≤100ms per frame, and the timing of instruction output of the interaction layer, to ensure that the working timing of each module of the system is consistent.

[0021] 3. Human-Computer Interaction Layer Functional positioning: Receives guidance instructions output from the processing layer and converts them into signals that can be perceived by visually impaired users; at the same time, it receives user operation instructions and transmits them to the processing layer. Components and functions: Haptic feedback device: Employing a wristband or handle-style structure, it integrates 3-5 miniature vibration motors forming a motor array. Connected to the processing layer via a GPIO / PWM interface, it outputs different vibration modes based on instructions from the processing layer. Forward mode: All motors vibrate continuously at a low frequency; Steering mode: vibration of the left motor indicates a left turn, and vibration of the right motor indicates a right turn. The vibration intensity / frequency adjusts according to the speed of the turn, with strong vibration during sharp turns and weak vibration during gentle turns. Warning / Stop Mode: All motors vibrate intensely and synchronously; Voice system: Composed of a speaker, microphone, and audio codec chip, connected to the processing layer via an I2S interface. Voice output: Based on TTS (text-to-speech) technology, the scene information and operation prompts output by the processing layer are converted into audio signals (such as "10 meters ahead to the intersection" and "Battery remaining 20%), which are played through the speaker. Voice input: The system receives user voice commands via microphone, such as "set destination XX supermarket" or "increase vibration intensity", and transmits them to the processing layer after processing by the audio codec chip. Operation components: include 2-3 physical buttons, such as wake-up button, pause button, and emergency stop button, which are connected to the processing layer through GPIO interface for users to manually wake up the device, pause the boot process, and trigger an emergency stop. Interaction logic: The guidance instructions output by the processing layer will first trigger the haptic feedback device and simultaneously trigger the voice system; user operation instructions will first be transmitted to the processing layer, and the processing layer will adjust the system working state according to the instructions (such as pausing the data acquisition of the perception layer and the output of the interaction layer instructions after receiving the "pause guidance" instruction).

[0022] 4. Power Module Functional positioning: Provide stable power to all modules of the perception layer, processing and decision-making layer, and human-computer interaction layer, while monitoring system power consumption and battery status; Components and functions: Rechargeable lithium battery: Uses high-capacity lithium-ion batteries as the power source for the system, supporting both fast and slow charging; Battery Management System (BMS): Connects to the main processor via an I2C interface and has three core functions: Voltage adaptation: Output different voltages according to the power supply requirements of each module (e.g., the main processor and camera require 5V, while the vibration motor and microphone require 3.3V). Power consumption monitoring: Real-time collection of current data from each module, calculation of total system power consumption, and transmission to the voice system via the main processor (e.g., "Current power consumption is high, please reduce unnecessary operations"). Safety protection: When the battery voltage is too low, below 3.0V, or when the current of a certain module is abnormal, such as exceeding 150% of the rated value, the power supply to the corresponding module will be automatically cut off or the system low power mode will be triggered to avoid damage to the equipment. Power supply logic: The BMS prioritizes powering the main processor of the processing layer, and then the perception layer and interaction layer modules; when the system is in standby mode, the BMS reduces the power supply to the perception layer and interaction layer, and only maintains the main processor and positioning module to operate at low power, thus extending the battery life.

[0023] (ii) System Data and Control Flow 1. Data Flow Perception layer → Processing and decision-making layer: Front camera / lower camera image data (MIPI CSI interface), ultrasonic sensor ranging data (UART interface), IMU motion data (I2C interface) → Main processor → Distributed to positioning module, AI recognition engine, obstacle avoidance algorithm module; Processing and decision-making layer → Human-computer interaction layer: Vibration control instructions generated by the main processor (GPIO / PWM interface) → Haptic feedback device, voice control instructions (I2S interface) → Voice system; Human-computer interaction layer → Processing and decision-making layer: Microphone voice commands (I2C interface), physical button operation signals (GPIO interface) → Main processor → Triggers the navigation engine to adjust the path or the power module to adjust the power supply; Power supply module → Each layer: BMS output adaptation voltage → Sensing layer, processing and decision-making layer, and human-computer interaction layer modules.

[0024] 2. Control Flow The main processor is the control core, coordinating system operation through the following logic: Startup phase: The main processor receives the wake-up command → triggers the power module to supply power to each layer → controls the perception layer to start data acquisition → the positioning module obtains the initial position → the navigation engine plans the path; Guidance Phase: The main processor synchronously receives data from the perception layer → schedules the AI ​​recognition engine and obstacle avoidance algorithm module to process the data → generates guidance instructions → outputs them to the human-computer interaction layer; Anomaly handling phase: The main processor receives obstacle signals from the ultrasonic sensor / AI recognition engine → immediately triggers the obstacle avoidance algorithm module to generate warning commands → synchronously controls the haptic feedback device and voice system → suspends navigation engine path planning; Standby / Power-off phase: The main processor receives a pause command or a low battery signal from the BMS → the control perception layer stops data acquisition → the human-machine interaction layer stops command output → the power module switches to low-power supply or cuts off power supply.

[0025] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0026] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An electronic guidance system for assisting visually impaired individuals in traveling, characterized in that, The system comprises a perception layer, a processing and decision-making layer, a human-computer interaction layer, and a power module. Each layer is connected via hardware interfaces and data protocols to form a complete closed-loop system. The perception layer includes a front-facing camera, a lower-facing camera, an ultrasonic sensor array, and an inertial measurement unit. The front-facing camera connects to the processing and decision-making layer via a MIPI CSI interface; the lower-facing camera connects to the processing and decision-making layer via a MIPI CSI interface; the ultrasonic sensor array connects to the processing and decision-making layer via a UART interface; and the inertial measurement unit connects to the processing and decision-making layer via an I2C interface. The processing and decision-making layer is the core of the system, including a main processor, a positioning module, a navigation engine, an AI recognition engine, and an obstacle avoidance algorithm module. The positioning module integrates a GPS chip and a Bluetooth / Wi-Fi module. The navigation engine calls the electronic map API through a software interface; the AI ​​recognition engine is based on a convolutional neural network; the human-computer interaction layer includes a haptic feedback device, a voice system, and operating components. The haptic feedback device has a built-in micro vibration motor array and is connected to the processing and decision-making layer through a GPIO / PWM interface; the voice system includes a speaker, a microphone, and an audio codec chip, and is connected to the processing and decision-making layer through an I2S interface; the operating components are connected to the processing and decision-making layer through a GPIO interface; the power module includes a rechargeable lithium battery and a battery management system; and the inertial measurement unit is connected to the main processor through an I2C interface.

2. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The ultrasonic sensor array consists of 4-6 ultrasonic sensors.

3. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The positioning module integrates multi-source data including GPS data, Bluetooth beacon / Wi-Fi data, IMU data, and visual odometry data.

4. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The data transmission interface between the perception layer and the processing and decision-making layer further includes: the IMU transmitting motion data through the I2C interface, and the ultrasonic sensor array transmitting ranging data through the UART interface; the instruction transmission interface between the processing and decision-making layer and the human-computer interaction layer further includes: the haptic feedback device receiving vibration control instructions through the GPIO / PWM interface, and the voice system receiving voice control instructions through the I2S interface.

5. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The navigation engine calls an electronic map API that has high-precision route planning, POI information query, and real-time traffic information acquisition functions.

6. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The functions of the BMS include: outputting an adaptive voltage according to the power supply requirements of each module; and collecting current data of each module in real time to calculate the total power consumption of the system.

7. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The main processor is an ARM Cortex-A series SoC, which supports parallel data processing across multiple interfaces and can synchronously schedule the data acquisition frequency of the perception layer, the algorithm operation rhythm of the processing layer, and the timing of instruction output of the interaction layer.

8. The electronic guidance system for assisting visually impaired persons in traveling according to claim 1, characterized in that, The operation components include a wake-up button, a pause button, and an emergency stop button, which are used to wake up the device, pause the boot process, and trigger an emergency stop command, respectively. The button operation signals are transmitted to the main processor in real time via the GPIO interface.

Images