Intelligent blind person walking stick based on artificial intelligence

Abstract

The invention discloses an intelligent blind person walking stick based on artificial intelligence, and relates to the field of intelligent walking sticks, and the intelligent blind person walking stick comprises a sensing module used for collecting environmental physical information in the advancing direction of a user in real time, and a processing module used for receiving multi-source environmental information collected by the sensing module, establishing a dynamic three-dimensional space model of the current local environment by taking a landing point at the bottom end of the walking stick as a current origin of coordinates; when the user touches the walking stick to the ground once, a'drop point event 'is triggered once, the three-dimensional space model is updated based on new drop point data, and the space offset between the current drop point and the previous drop point is calculated, including the moving distance and the direction change; compared with a scheme only based on distance perception or fixed path navigation, the method has higher environment understanding capability, navigation adaptive capability and real-time correction capability.

Description

Technical Field

[0001] This invention relates to the field of smart canes, specifically to a smart cane for the blind based on artificial intelligence. Background Technology

[0002] With the development of sensor technology, embedded computing platforms, and artificial intelligence algorithms, mobility assistance devices are gradually evolving towards intelligence. Traditional canes, as the primary assistive tool, are widely used in the daily travel of visually impaired people, possessing advantages such as simple structure and ease of use. Some enhanced canes have incorporated functions such as ultrasonic ranging and voice broadcasting to achieve early warning of obstacles ahead and basic interactive feedback.

[0003] In existing technologies, obstacle distances are typically identified using ultrasonic or infrared sensors, and feedback is provided to the user in conjunction with simple vibration or voice prompts. However, these technologies have limited environmental modeling capabilities, making it difficult to fully represent the surrounding three-dimensional spatial structure. Furthermore, they lack the ability to recognize scene semantics (such as traffic lights, road signs, etc.), making it difficult to adapt to the needs of dynamic path adjustments in complex travel environments.

[0004] Therefore, this paper proposes an AI-based intelligent cane for the blind, which combines multi-source sensing methods, artificial intelligence image recognition and path judgment algorithms to construct a comprehensive system in the cane for the blind with spatial modeling, autonomous path judgment and real-time feedback functions. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent cane for the blind based on artificial intelligence, so as to solve the problems mentioned in the background art.

[0006] The present invention can be achieved through the following technical solution: an intelligent cane for the blind based on artificial intelligence, comprising a cane body, the cane body integrating: a sensing module, a processing module, a human-computer interaction module, and a navigation module; The perception module is used to collect environmental physical information in the user's direction of travel in real time and upload the collected information to the processing module; The processing module receives multi-source environmental information collected by the sensing module and establishes a dynamic three-dimensional spatial model of the current local environment with the landing point of the cane as the current coordinate origin. Each time the user touches the ground with the cane, a "landing point event" is triggered. The processing module updates the three-dimensional space model based on the new landing point data and calculates the spatial offset between the current and previous landing points, including the distance and direction of movement. The navigation module receives the destination information set by the user and generates an initial navigation path by combining it with the current 3D spatial model. Subsequently, each time the user's landing point changes, the processing module will spatially compare the new landing point with the navigation path segment. If the offset exceeds the preset angle threshold or distance threshold, it will be identified as "path deviation" and trigger path correction or replanning. The human-computer interaction module, including a voice input module and a multimodal output module, is used to receive destination input commands from users and provide navigation correction information through voice prompts, vibration prompts, or a graphical interface.

[0007] A further technical improvement of the present invention is that the perception module includes: a space structure detection unit, a ground state recognition unit, an attitude and motion measurement unit, and a semantic image recognition unit; The spatial structure detection unit is used to acquire three-dimensional spatial geometric information of obstacles in front and the surrounding environment, and to provide basic coordinate data for the three-dimensional point cloud model. The ground condition recognition unit is used to collect local terrain information of the area under the user's feet, identify ground height differences, structural undulations, material changes, etc., and is used to determine the safety of the landing point and construct a ground grid model. The posture and motion measurement unit is used to identify the changes in the posture angle and dynamic stability of the cane during movement, in order to assist in establishing a spatial coordinate system and trajectory judgment. The semantic image recognition unit is used to identify scene elements in the user's environment that have navigational significance.

[0008] A further technical improvement of the present invention is that the processing module is provided with a control submodule, which is used to schedule the sampling timing, data logging and unified upload timing of each unit in the sensing module.

[0009] A further technical improvement of the present invention is that the semantic image recognition unit performs the following image recognition steps: A1. The camera captures a stream of images from the front at a rate of 10-15 frames per second; A2. The recognition model performs semantic target detection on each frame of the image to identify traffic facility targets and their spatial locations. A3. Based on the target size, azimuth angle and ranging results of the sensing module in the image, calculate the relative spatial position of the target and convert it into local coordinates; A4. Upload the semantic target category, location coordinates, and confidence information to the processing module and add them to the spatial model as the model's semantic label layer.

[0010] A further technical improvement of the present invention lies in the following: the specific calculation method for the spatial offset is as follows: distance traveled The straight-line distance between two landing points is calculated using the three-dimensional Euclidean distance formula. This value represents the physical step size of each user step, serving as the basis for path density analysis; Change of direction (Travel angle deviation): Construct the direction vector V using the vectors of the two landing points: This direction vector is compared with the target segment direction vector in the current navigation path. Calculate the included angle: Among them, the included angle This represents the deviation in path direction.

[0011] A further technical improvement of the present invention is that the specific process of the path correction is as follows: S1. Compare the distance and directional angle between the current landing point and the target point on the path to confirm the type of path deviation; S2. Call the latest 3D spatial model in the processing module, extract the local model area in front of the current landing point, perform rasterization processing on the area, and transform the spatial model into a 2D cost map. Each grid contains the following fields: passage cost and passable marker. Take the current landing point as the starting point and the target point or target direction area on the original navigation path as the ending point to construct a local passage map structure between the starting point and the ending point.

[0012] S3. The navigation module runs a path search algorithm on the local traffic map. The path cost function is: Where g(n): the cumulative travel cost from the starting point to the current node; h(n): the heuristic function from the current node to the target point; after the search is completed, a locally corrected path segment consisting of a series of continuous coordinate points is generated. Smooth the path. S4. The corrected path segment is spliced ​​to the beginning of the original path to form a new navigation path sequence; the first turning point of the current path segment is extracted, and the directional offset angle between it and the current landing point is calculated; the navigation module generates directional adjustment prompts and prompts the user through the human-computer interaction module in the form of voice or vibration.

[0013] S5. For each subsequent landing point of the cane, deviation judgment and tracking will be based on the corrected path; if the corrected path deviates again, the above path correction process will be repeated.

[0014] Compared with the prior art, the present invention has the following beneficial effects: This invention integrates multiple sensing functions, such as spatial structure detection, ground condition recognition, attitude and motion measurement, and semantic image recognition, into the cane body. By uniformly scheduling the sampling time sequence through the control submodule, it realizes the fusion perception of environmental geometric structure and semantic features, providing data support for constructing dynamic three-dimensional spatial models.

[0015] The processing module uses each "cane landing event" as the modeling trigger point, combining point cloud modeling, attitude calculation, and image semantic analysis to construct a local spatial model centered on the landing point. It then uses a path deviation determination mechanism to determine if the user has deviated from the predetermined path. When a deviation occurs, the navigation module can construct a local route map based on the current model information and use a path search algorithm (such as A*) to reconstruct the path, guiding the user smoothly back to the target route.

[0016] Compared to solutions based solely on distance perception or fixed-path navigation, this invention possesses stronger environmental understanding, navigation adaptability, and real-time correction capabilities, making it particularly suitable for unstructured environments such as pedestrian crossings, obstacle avoidance, or temporary construction sites. Through multimodal feedback provided by the human-computer interaction module, it effectively enhances the safety and autonomy of visually impaired individuals in complex scenarios. Attached Figure Description

[0017] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0018] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0019] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0020] Please see Figure 1 As shown, the present invention provides an intelligent cane for the blind based on artificial intelligence, including a cane body, which integrates a sensing module, a processing module, a human-computer interaction module, and a navigation module. The cane is made of lightweight, high-strength carbon fiber, with a moderate weight and length, making it easy for visually impaired people to use for extended periods. The top of the cane features a non-slip grip with a rubber surface and textured surface for improved stability and comfort. The bottom of the cane has replaceable rubber anti-slip tips to increase friction with the ground and prevent slipping.

[0021] The perception module is used to collect environmental physical information in the user's direction of travel in real time; specifically, it includes: a spatial structure detection unit, a ground state recognition unit, an attitude and motion measurement unit, and a semantic image recognition unit. The processing module includes a control submodule, which is used to schedule the sampling timing, data logging, and unified upload timing of each unit in the sensing module, so as to ensure information synchronization and controllability of fusion modeling.

[0022] The spatial structure detection unit is used to acquire three-dimensional spatial geometric information of obstacles in front and the surrounding environment, providing basic coordinate data for the three-dimensional point cloud model. Specifically, this unit includes several sets of ultrasonic ranging sensors and infrared ranging modules installed at different angles (e.g., center 0°, ±30°, ±60°) on the front of the cane. Each set of sensors is connected to a microcontroller, periodically triggering ranging and receiving echo time or reflection intensity signals. The specific working process is as follows: The control submodule sequentially activates each group of sensors to emit detection signals according to a set time interval (e.g., every 50ms). Each group of sensors calculates the three-dimensional coordinates P(x, y, z) of the obstacle point in the local coordinate system using a polar coordinate transformation algorithm based on its installation angle and the measured distance. The spatial point set composed of multiple points is uploaded to the processing module in real time to generate a local dense point cloud. The detection range covers a fan-shaped area with the cane landing point as the origin, with a radius of about 1 to 2 meters and an angle that can be set to 120°.

[0023] The data generated by this unit serves as the geometric boundary input for spatial entities in the 3D model and supports obstacle avoidance and path construction.

[0024] The ground condition recognition unit is used to collect local terrain information of the area under the user's feet, identify ground height differences, structural undulations, material changes, etc., and to determine the safety of the landing point and construct a ground grid model.

[0025] Specifically, the unit is located at the bottom of the cane and includes a set of downward-sloping infrared near-field sensors and a set of capacitive proximity detectors, as well as a miniature pressure sensor array for identifying the pressure distribution at the point of contact.

[0026] Each time the user lands the cane, the proximity sensor at the bottom initiates a measurement; the infrared sensor measures the vertical distance between the cane tip and the ground to determine if there are height differences such as steps or pits; the pressure sensor array senses whether the landing area is evenly stressed and determines whether the ground material is stable (such as grass or soft soil); if abnormal terrain is detected at multiple consecutive landing points (height change >30mm or unstable pressure), the control submodule automatically marks the area as "dangerous terrain" in the model.

[0027] This unit can be updated synchronously with the structural modeling data in each "landing event", providing accurate ground boundary lines or bottom bonding layers.

[0028] The posture and motion measurement unit is used to identify the changes in the posture angle and dynamic stability of the cane during movement, in order to assist in establishing a spatial coordinate system and determining the trajectory.

[0029] Specifically, the unit includes a three-axis accelerometer and a three-axis gyroscope, integrated into an IMU (Inertial Measurement Unit) chip located beneath the cane handle. Its structure is directly connected to the main control board, supporting high-speed data acquisition (sampling rate ≥ 100Hz). The specific sensing process is as follows: The control submodule periodically reads the raw acceleration data (ax, ay, az) and angular velocity data (ωx, ωy, ωz) output by the IMU sensor; it calculates the cane's attitude angles (pitch angle θp, yaw angle θy, roll angle θr) in world coordinates using attitude fusion algorithms (such as complementary filtering or Kalman filtering); it calculates the continuous attitude difference before and after each "landing event" to determine the change in travel direction Δθ, and compares it with the navigation path direction; the attitude data is also used to adjust the point cloud coordinate projection direction to ensure the consistency of the spatial model orientation.

[0030] The semantic image recognition unit is used to identify scene elements in the user's environment that have navigational significance, such as traffic lights, road signs, obstacle types, and road edges.

[0031] Specifically, the unit includes a camera module installed at the center of the front end of the cane, paired with a low-power image processing chip (the image processing chip can be integrated into the semantic image recognition unit itself, or managed as a control sub-module of the processing module), and integrates a lightweight target recognition model (such as YOLOv5-nano or MobileNet-SSD).

[0032] The image recognition steps are as follows: A1. The camera captures a stream of images from the front at a rate of 10-15 frames per second; A2. The recognition model performs semantic target detection on each frame of the image to identify traffic facility targets and their spatial locations (such as the center pixel of the traffic light in the image + confidence level). A3. Based on the target size, azimuth angle and ranging results of the sensing module in the image, calculate the relative spatial position of the target and convert it into local coordinates; A4. Upload the semantic target category, location coordinates, and confidence information to the processing module and add them to the spatial model as the model's semantic label layer. The data collected by each unit is timestamped and uploaded to the processing module.

[0033] The processing module receives multi-source environmental information collected by the sensing module and establishes a dynamic three-dimensional spatial model of the current local environment with the landing point of the cane as the current coordinate origin. Each time the user touches the ground with their cane, a "landing point event" is triggered. The processing module updates the 3D spatial model based on the new landing point data. The model includes a dense point cloud layer, a semantic tag layer, and a ground grid layer. It calculates the spatial offset between the current and previous landing points, including the distance traveled and changes in direction, for navigation offset determination. In this embodiment, the dynamic 3D spatial model is preferably a voxel grid-based structural model, with an additional semantic tag layer to identify semantic target objects and danger zones.

[0034] The processing module integrates a data fusion subunit, which adopts a multi-source synchronization mechanism based on timestamp alignment (such as a time window fusion strategy) to perform spatiotemporal alignment and feature completion of the measurement results of each sensing unit under the same landing event, forming a unified spatial semantic point cloud structure to enhance the accuracy of the model and the ability to recognize environmental semantics.

[0035] Specifically, the landing event triggering and coordinate recording: Whenever the user touches the cane to the ground (i.e., a "landing event" occurs), the ground state recognition unit confirms the contact is complete; the control submodule immediately records the position coordinates of the bottom contact point of the cane in the local coordinate system established at the current moment, defining it as the current landing position P. curr (X1, Y1, Z1); If this is the first landing point, offset calculation is not performed yet, only path tracking is initialized; if this is not the first landing point, the position P of the previous landing point is taken. prev (X0,Y0,Z0) are involved in the calculation.

[0036] Spatial offset calculation: distance traveled The straight-line distance between two landing points is calculated using the three-dimensional Euclidean distance formula. This value represents the physical step size of each user step, serving as the basis for path density analysis.

[0037] Change of direction (Travel angle deviation): Construct the direction vector V using the vectors of the two landing points: This direction vector is compared with the target segment direction vector in the current navigation path. Calculate the included angle: Among them, the included angle This represents the deviation from the path direction, expressed in radians or degrees (convertible).

[0038] The navigation module is used to receive destination information set by the user (which can be input via voice or App) and generate an initial navigation path by combining it with the current 3D spatial model; After generating the navigation path, the navigation module will first transmit the starting direction information of the path (such as "please proceed forward at a slight left angle of 30 degrees") to the user through the human-computer feedback module to guide the user to complete the initial direction adjustment and prevent the user from deviating from the path due to excessive direction error when landing for the first time.

[0039] Subsequently, each time a user's landing point changes, the processing module will spatially compare the new landing point with the navigation path segment. If the offset exceeds the preset angle threshold or distance threshold, it will be identified as "path deviation" and trigger path correction or replanning.

[0040] Specifically, the offset threshold determination: the system presets a pair of tolerance values: the maximum allowable step size offset distance D. thresh (e.g., 0.5m); Maximum permissible directional angle deviation thresh (e.g., 15°); the judgment rules are as follows: like ≦D thresh and ≦ thresh It is assumed that the user is within the original path range; If any indicator exceeds the threshold, it is judged as "path deviation", triggering the navigation module to start the path correction process.

[0041] The specific correction process is as follows: S1. Compare the distance and directional angle between the current landing point and the target point on the path to determine the path deviation type. If the distance deviation is significant, prioritize generating the shortest path that is "close to the target point". If the angle deviation is large, prioritize generating the "direction correction segment".

[0042] S2. Call the latest 3D spatial model in the processing module, extract the local model area in front of the current landing point, preferably a fan-shaped area (e.g., radius 2 meters, angle 120°); perform rasterization processing on this area, converting the spatial model into a 2D cost map. Each grid contains the following fields: passage cost (determined based on whether there are obstacles, ground flatness, and dangerous terrain); passable marker (True / False); take the current landing point as the starting point and the target point or target direction area on the original navigation path as the ending point, and construct a local passage map structure between the starting point and the ending point.

[0043] S3. The navigation module runs a path search algorithm (preferably A algorithm or D Lite algorithm) on the local traffic map; the path cost function is: Where g(n): the cumulative travel cost from the starting point to the current node; h(n): the heuristic function from the current node to the target point (using Euclidean distance); after the search is completed, a locally corrected path segment consisting of a series of continuous coordinate points is generated. The path is smoothed (e.g., using Bézier curves or Catmull-Rom curves) to improve path feasibility and user walking comfort.

[0044] S4. The corrected path segment is spliced ​​to the beginning of the original path to form a new navigation path sequence; the first turning point of the current path segment is extracted, and the directional offset angle between it and the current landing point is calculated; the navigation module generates a directional adjustment prompt message (such as "Please shift to the right by about 25 degrees"), and prompts the user through the human-computer interaction module in the form of voice or vibration.

[0045] S5. For each subsequent landing point of the cane, deviation judgment and tracking will be based on the corrected path; if the corrected path deviates again, the above path correction process will be repeated.

[0046] The human-computer interaction module, including a voice input module and a multimodal output module, is used to receive destination input commands from users and provide navigation correction information through voice prompts, vibration prompts, or a graphical interface.

[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An artificial intelligence-based intelligent blind stick, characterized by, The walking stick includes a walking stick body, and the walking stick body is internally integrated with a perception module, a processing module, a human-computer interaction module and a navigation module. The perception module is used for collecting environmental physical information of a user's walking direction in real time and uploading the collected information to the processing module. The processing module is used for receiving multi-source environmental information collected by the perception module, and establishing a dynamic three-dimensional space model of a current local environment with a landing point of a bottom end of the walking stick as a current coordinate origin. Every time the user touches the ground once, a "landing point event" is triggered, the processing module updates the three-dimensional space model based on new landing point data, and calculates a space offset between the current landing point and the previous landing point, including a moving distance and a direction change. The navigation module is used for receiving destination information set by the user, and generating an initial navigation path in combination with the current three-dimensional space model. Subsequently, after the user's landing point changes each time, the processing module compares the new landing point with a navigation path segment in space, and if the offset exceeds a preset angle threshold or distance threshold, the offset is determined as a "path deviation", and path correction or re-planning is triggered. The human-computer interaction module includes a voice input module and a multi-modal output module, and is used for receiving a destination input instruction issued by the user, and feeding back navigation correction information through voice prompt, vibration prompt or a graphical interface.

2. The intelligent blind stick based on artificial intelligence according to claim 1, characterized in that, The perception module includes a space structure detection unit, a ground state recognition unit, a posture and motion measurement unit and a semantic image recognition unit. The space structure detection unit is used for acquiring three-dimensional space geometric information of a front obstacle and a surrounding environment, and providing basic coordinate data for a three-dimensional point cloud model. The ground state recognition unit is used for collecting local terrain information of an area under the user's feet, recognizing ground height difference, structure fluctuation, material change and the like, and judging landing point safety and constructing a ground grid model. The posture and motion measurement unit is used for recognizing attitude angle change and dynamic stability of the walking stick in a moving process, and assisting in establishing a space coordinate system and trajectory judgment. The semantic image recognition unit is used for recognizing scene elements with navigation significance in a front environment of the user.

3. The intelligent blind stick based on artificial intelligence according to claim 2, characterized in that, The processing module is provided with a control sub-module for scheduling sampling time sequence, data dotting and unified uploading time of units in the perception module.

4. The intelligent blind stick based on artificial intelligence according to claim 3, characterized in that, The semantic image recognition unit performs image recognition in the following steps: A1, a camera acquires a front image stream at a speed of 10-15 frames per second; A2, a recognition model performs semantic target detection on each frame of image, and recognizes a traffic facility target and a spatial position thereof; A3, according to a target size, an azimuth angle in the image and a ranging result of the perception module, a relative spatial position of the target is calculated and converted into a local coordinate; A4, semantic target categories, position coordinates and confidence information are uniformly uploaded to the processing module, and are added into the space model as a model semantic label layer.

5. The intelligent smart blind stick based on artificial intelligence according to claim 4, characterized in that, The specific calculation method of the space offset is as follows: Moving distance : The straight-line distance between the two landing points is calculated using the three-dimensional Euclidean distance formula: , the value represents the physical step length of each step of the user, as the basis for path density analysis; Direction change (heading deviation): construct a direction vector V from the two previous drop point vectors: , the direction vector is compared to the target segment direction vector in the current navigation path an angle calculation is performed: , wherein the included angle is the path direction deviation amount.

6. The intelligent smart blind stick based on artificial intelligence according to claim 5, characterized in that, The specific process of the path correction is as follows: S1, a distance and a direction included angle between the current landing point and a path target point are compared to confirm a path deviation type; S2, call the latest three-dimensional space model in the processing module, extract the local model area in front of the current landing point, and perform rasterization processing on the area to convert the space model into a two-dimensional cost map. Each grid contains the following fields: passage cost, passable mark. Take the current landing point as the starting point, and the target point or target direction area on the original navigation path as the end point. Construct a local passage graph structure between the starting point and the end point; S3, the navigation module runs a path search algorithm on the local traffic graph, with a path cost function: where g(n): cumulative travel cost from the start point to the current node. h(n): heuristic function from the current node to the target point; After the search is completed, a local correction path segment consisting of a series of consecutive coordinate points is generated ; the path is smoothed S4, splice the corrected path segment to the front end of the original path to form a new navigation path sequence. Extract the first turning point of the current path segment and calculate the direction deviation angle between it and the current landing point. Generate direction adjustment prompt information from the navigation module and prompt the user through the human-computer interaction module in the form of voice or vibration; S5, each subsequent walking stick landing point is based on the corrected path for deviation judgment and tracking; If the corrected path is deviated again, repeat the above path correction process.

Images