Intelligent blind stick based on human flow trajectory perception and working method

Abstract

The application provides an intelligent blind stick based on human flow trajectory perception and a working method, the intelligent blind stick comprising an environment perception module, a calculation processing module and a guidance feedback module; the environment perception module is used for acquiring environment perception data; the calculation processing module is used for determining a real-time position and a motion feature of a dynamic target based on the environment perception data; an optimal obstacle avoidance guidance action is determined according to the real-time position and the motion feature of the dynamic target; and the guidance feedback module is used for converting the optimal obstacle avoidance guidance action into a multi-modal prompt signal. The intelligent blind stick in the embodiment of the application significantly improves the travel safety and traffic efficiency of visually impaired users in a dense human flow scene.

Description

Technical Field

[0001] This application relates to the field of portable accessibility assistive devices, and in particular to a smart white cane based on human flow trajectory perception and its working method. Background Technology

[0002] When visually impaired people travel in public spaces such as schools, shopping malls, subway stations, hospitals, and community roads, in addition to recognizing static obstacles such as steps, walls, curbs, pillars, and vehicles, they also need to deal with a large number of dynamic targets with active movement capabilities, such as oncoming pedestrians, pedestrians crossing laterally, cyclists, people pushing bicycles, and people who suddenly change direction.

[0003] In related technologies, traditional white canes mainly rely on tactile contact to detect nearby obstacles. Their perception range is limited, and they can usually only provide feedback to the user when the obstacle is approaching or has already made contact. It is difficult to reflect the movement trend of dynamic targets in a timely manner and achieve dynamic obstacle avoidance guidance for visually impaired users. Summary of the Invention

[0004] This application provides an intelligent white cane and its working method based on pedestrian flow trajectory perception. An environmental perception module continuously collects surrounding environmental perception data, and a calculation and processing module analyzes the real-time position and motion characteristics of dynamic targets based on this data. This accurately captures the movement trends of moving objects such as pedestrians and cyclists, effectively distinguishing between dynamic targets with collision risks and those moving away from the user. Consequently, it can accurately output continuous and executable optimal obstacle avoidance guidance actions. This enables proactive obstacle avoidance guidance based on pedestrian flow trends and future intersections, as well as pre-judgment of dynamic pedestrian flow conflicts. It overcomes the single-judgment logic of traditional devices that rely solely on instantaneous distance to determine danger, effectively reducing false alarms and missed hazard detections in dynamic scenarios. Finally, a guidance feedback module converts the optimal obstacle avoidance guidance actions into multimodal prompts and feeds them back to the visually impaired user, significantly improving the travel safety and efficiency of visually impaired users in densely populated areas.

[0005] This application provides a smart white cane based on pedestrian flow trajectory perception, comprising: The module consists of an environmental perception module, a computational processing module, and a guidance and feedback module. The environmental perception module is used to acquire environmental perception data; The computational processing module is used to determine the real-time position and motion characteristics of dynamic targets based on environmental perception data; and to determine the optimal obstacle avoidance guidance action based on the real-time position and motion characteristics of dynamic targets. The guidance feedback module is used to convert the optimal obstacle avoidance guidance action into multimodal prompt signals.

[0006] According to the intelligent white cane based on pedestrian trajectory perception provided in this application, the computing module is specifically used for: Based on the real-time location and motion characteristics of each dynamic target, determine the dynamic collision risk between the user and the dynamic target; Based on the static obstacle cost, the deviation cost of travel direction, and the dynamic collision risk between the user and the dynamic target corresponding to each candidate passage position, determine the comprehensive passage cost corresponding to each candidate passage position of the user. The optimal obstacle avoidance guidance action is determined based on the comprehensive passage cost corresponding to each candidate passage location of the user.

[0007] According to the intelligent white cane based on pedestrian trajectory perception provided in this application, the dynamic collision risk between the user and the dynamic target is determined based on the real-time position and motion characteristics of each dynamic target, including: Based on the real-time location and motion characteristics of each dynamic target, determine the estimated time of intersection between the user and the dynamic target; The dynamic collision risk between the user and the dynamic target is determined based on the expected intersection time of the user and the dynamic target, the speed of the dynamic target, the detection confidence of the dynamic target, and the crowd density.

[0008] According to the intelligent white cane based on pedestrian trajectory perception provided in this application, the dynamic collision risk between the user and the dynamic target is determined based on the expected intersection time between the user and the dynamic target, the movement speed of the dynamic target, the detection confidence of the dynamic target, and the pedestrian density, including: = || ‖+ - ) in, R i Indicates user and the The dynamic collision risk of a dynamic target, where σ(·) represents the normalization function. α 1 , α 2 , α 3 , α 4 For weight parameters, Indicates user and the The estimated convergence time of each dynamic target To prevent stable terms with a denominator of zero, Indicates the first The velocity of a dynamic target at time t This represents the local crowd density at time t. The confidence level for detecting dynamic targets.

[0009] According to the intelligent white cane based on pedestrian trajectory perception provided in this application, the comprehensive passage cost corresponding to each candidate passage position of the user is determined based on the static obstacle cost, the deviation cost of the travel direction, and the dynamic collision risk between the user and dynamic targets, including: Where C(q) represents the comprehensive passage cost corresponding to the candidate passage position q. , , Here, S(q) represents the static obstacle cost corresponding to the user's candidate passage position q, and D(q) represents the deviation cost from the travel direction. R i Indicates user and the The dynamic collision risk of a dynamic target. Indicates the first The position of a dynamic target at time t. Indicates user and the The estimated convergence time of each dynamic target Indicates the first The velocity of a dynamic target at time t This indicates the scope of dynamic risk diffusion.

[0010] According to the intelligent white cane based on pedestrian trajectory perception provided in this application, the optimal obstacle avoidance guidance action includes at least one of the following: Go straight, veer left, veer right, slow down, and wait.

[0011] According to the intelligent white cane based on pedestrian trajectory perception provided in this application, the guidance feedback module is also used for: If the optimal obstacle avoidance guidance action remains unchanged over multiple consecutive time periods, the repeated output of multimodal cue signals is suppressed, and the corresponding multimodal cue signals are only output when the optimal obstacle avoidance guidance action changes.

[0012] According to the present application, a smart cane based on pedestrian trajectory perception provides a multimodal prompting signal including vibration signal and voice prompt signal.

[0013] According to the present application, a smart cane based on pedestrian trajectory perception provides different optimal obstacle avoidance guidance actions corresponding to different vibration frequencies, vibration durations or voice prompts.

[0014] According to the intelligent white cane based on pedestrian flow trajectory perception provided in this application, the intelligent white cane also includes: The system includes a power supply module and a communication module. The power supply module provides power to the environmental sensing module, computing processing module, and guidance feedback module. The communication module is used to transmit and interact with mobile phones, earphones, the cloud, or remote monitoring terminals.

[0015] This application also provides a method for operating a smart cane, including: The environmental perception module acquires environmental perception data; The computational processing module determines the real-time position and motion characteristics of dynamic targets based on environmental perception data; and determines the optimal obstacle avoidance guidance action based on the real-time position and motion characteristics of dynamic targets. The guidance feedback module converts the optimal obstacle avoidance guidance action into multimodal prompt signals.

[0016] The intelligent white cane and its working method based on pedestrian flow trajectory perception provided in this application continuously collect surrounding environmental perception data through an environmental perception module. The calculation and processing module analyzes the real-time position and motion characteristics of dynamic targets based on the environmental perception data, accurately capturing the movement trends of moving objects such as pedestrians and cyclists, effectively distinguishing between dynamic targets with collision risks and those far from the user. This allows for the accurate output of continuous and executable optimal obstacle avoidance guidance actions, enabling proactive obstacle avoidance guidance based on pedestrian flow trends and future intersection relationships, as well as advance prediction of dynamic pedestrian flow conflicts. This overcomes the single judgment logic of traditional devices that rely solely on instantaneous distance to determine danger, effectively reducing false alarms and missed dangers in dynamic scenarios. Finally, the guidance feedback module converts the optimal obstacle avoidance guidance actions into multimodal prompt signals and feeds them back to the visually impaired user, significantly improving the travel safety and efficiency of visually impaired users in densely populated areas. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is one of the structural schematic diagrams of the smart cane provided in this application.

[0019] Figure 2 This is a schematic diagram of the environmental perception data processing provided in this application.

[0020] Figure 3 This is a schematic diagram illustrating the determination of dynamic collision risks and overall traffic costs provided in this application.

[0021] Figure 4 This is a schematic diagram of obstacle avoidance guidance actions and multimodal feedback mapping provided in this application.

[0022] Figure 5 This is the second structural schematic diagram of the smart cane provided in this application.

[0023] Figure 6 This is a schematic diagram of the overall process of the dynamic obstacle avoidance guidance method; Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The following is combined Figures 1 to 6 This application describes a smart cane based on pedestrian flow trajectory perception and its working method.

[0026] To facilitate a clearer understanding of the technical solutions of the various embodiments of this application, some technical content related to the various embodiments of this application will be introduced first.

[0027] Existing smart canes typically detect obstacles ahead using ultrasonic waves, infrared sensors, millimeter-wave radar, cameras, or laser ranging modules, and output vibration or voice prompts based on the distance to the obstacle. While these methods have some ability to identify static and near-field obstacles, they still have shortcomings in complex pedestrian environments: First, some methods only determine the current distance, lacking estimations of pedestrian movement direction, speed, and future intersections, easily misjudging pedestrians far from the user as dangerous targets, and potentially missing pedestrians about to cross laterally; Second, some methods only output discrete prompts such as "person ahead" or "obstacle on the left," failing to provide continuous, dynamic, and low-burden detour suggestions based on local pedestrian flow; Third, some methods rely on cloud-based visual models or global maps, making it difficult to guarantee real-time performance and robustness when the network is unstable, lighting changes, or there is significant crowd obstruction.

[0028] Therefore, there is an urgent need for a smart white cane and a dynamic obstacle avoidance guidance method for the actual travel scenarios of visually impaired users. This method should not only identify the presence of obstacles, but also understand the temporal changes in pedestrian flow trajectories, determine whether dynamic targets will conflict with the user's path, and transform complex pedestrian movement trends into simple and executable tactile or voice guidance, thereby improving the safety and travel efficiency of visually impaired users in dense, mobile, and frequently obstructed scenarios.

[0029] Figure 1 This is a schematic diagram of a smart cane based on pedestrian flow trajectory perception provided in this application, including: The module consists of an environmental perception module, a computational processing module, and a guidance and feedback module. The environmental perception module is used to acquire environmental perception data; The computational processing module is used to determine the real-time position and motion characteristics of dynamic targets based on environmental perception data; and to determine the optimal obstacle avoidance guidance action based on the real-time position and motion characteristics of dynamic targets. The guidance feedback module is used to convert the optimal obstacle avoidance guidance action into multimodal prompt signals.

[0030] Specifically, in this embodiment, the smart cane includes an environmental perception module, a computing processing module, and a guidance feedback module. The environmental perception module includes a camera, millimeter-wave radar, a depth sensor, an ultrasonic ranging unit, and an inertial measurement unit. It continuously collects raw scene data from the area in front of and to the sides of the cane, including scene images, target depth and distance, radar point clouds, and the cane's own posture. The collected complete perception data is then transmitted to the computing processing module. Optionally, the computing processing module can perform dynamic target detection and tracking on the received environmental perception data, accurately extract the real-time position and motion characteristics of dynamic targets, identify the movement trends of dynamic targets such as pedestrians and cyclists in advance, and generate optimal obstacle avoidance guidance actions based on obstacle avoidance strategies, achieving proactive obstacle avoidance. Optionally, the guidance feedback module can convert the final optimal obstacle avoidance guidance instructions into multimodal prompt signals, intuitively conveying them to visually impaired users, achieving barrier-free travel guidance, and significantly improving the safety and efficiency of visually impaired users in crowded public scenes such as shopping malls and subway stations.

[0031] For example, the smart cane of this application includes a cane body and an environmental perception module, a computing processing module, and a guidance feedback module mounted on the cane body. The cane body can adopt a foldable structure or an integrated structure. The environmental perception module is located on the upper part of the cane body or near the handle, giving it good forward and lateral forward vision. The environmental perception module can include one or more of the following: a monocular camera, a binocular camera, a depth camera, a millimeter-wave radar, an ultrasonic ranging module, and an inertial measurement unit. The camera is used to acquire images of the front and identify pedestrians and passable areas; the depth or ranging module is used to estimate the distance to obstacles; and the inertial measurement unit is used to estimate the cane's posture and the user's movement state. The computing processing module can employ an embedded processor, a mobile processor, or an edge AI chip to execute pedestrian trajectory perception and obstacle avoidance guidance algorithms. The guidance feedback module can include a left vibration unit, a right vibration unit, a central vibration unit, a bone conduction headphone jack, a speaker, or a buzzer.

[0032] For example, such as Figure 2As shown, the smart cane can continuously collect environmental data in front of it, detect dynamic targets in images or point clouds, and obtain the position sequence of the same target at multiple times through a target tracking algorithm. For the i-th dynamic target, its position in the global or sensor coordinate system is detected at time t, and then converted into local coordinates with the user's current position as the origin and the user's orientation as the forward axis. This local position is represented as: in, p i t This represents the local position of the i-th dynamic target relative to the user at time t. x i t and y i t Indicates the target location. x u t and y u t This represents the user's position. By using position differencing across consecutive frames, the local velocity of the target can be estimated. in, v i t Let Δt represent the local velocity of the i-th target at time t, and Δt represent the time interval between adjacent sensing frames. By summarizing the positions and velocities of all dynamic targets, a local pedestrian flow trajectory field is constructed. This trajectory field includes not only the current target position but also the target's direction of motion and velocity, thus providing a basis for subsequent risk prediction.

[0033] The smart cane described in the above embodiment continuously collects surrounding environmental perception data through an environmental perception module. The calculation and processing module analyzes the real-time position and movement characteristics of dynamic targets based on the environmental perception data, accurately capturing the movement trends of moving objects such as pedestrians and cyclists. It effectively distinguishes between dynamic targets with collision risks and those far from the user, thereby accurately outputting continuous and executable optimal obstacle avoidance guidance actions. This enables proactive obstacle avoidance guidance based on pedestrian movement trends and future intersection relationships, as well as advance prediction of dynamic pedestrian conflicts. It overcomes the single judgment logic of traditional devices that rely solely on instantaneous distance to determine danger, effectively reducing false alarms and missed dangers in dynamic scenarios. Finally, the guidance feedback module converts the optimal obstacle avoidance guidance actions into multimodal prompt signals and feeds them back to the visually impaired user, significantly improving the travel safety and efficiency of visually impaired users in densely populated areas.

[0034] In some embodiments, the computation processing module is specifically used for: Based on the real-time location and motion characteristics of each dynamic target, determine the dynamic collision risk between the user and the dynamic target; Based on the static obstacle cost, the deviation cost of travel direction, and the dynamic collision risk between the user and the dynamic target corresponding to each candidate passage position, determine the comprehensive passage cost corresponding to each candidate passage position of the user. The optimal obstacle avoidance guidance action is determined based on the comprehensive passage cost corresponding to each candidate passage location of the user.

[0035] Specifically, in this embodiment, the computational processing module performs dynamic target detection and tracking on the received environmental perception data. After accurately extracting the real-time position and motion characteristics of the dynamic targets, it can calculate the dynamic collision risk between the user and the dynamic targets based on their real-time positions and motion characteristics. Optionally, in this embodiment, the passable area in front of the cane can be discretized into several candidate passable positions. For each candidate passable position, the static obstacle cost, the deviation cost of the travel direction, and the dynamic collision risk are considered simultaneously, and a weighted calculation is performed to obtain the comprehensive passable cost corresponding to each candidate passable position. Among them, the static obstacle cost can be calculated based on the distance between the candidate passable position and static obstacles such as walls, steps, and pillars; the deviation cost of the travel direction can be calculated based on the degree of deviation of the candidate passable position relative to the user's original straight-line route. Further, the comprehensive passable costs corresponding to each candidate passable position can be compared. The optimal obstacle avoidance guidance action is determined based on the candidate passable position with the lowest comprehensive passable cost. If the comprehensive passable cost of all candidate passable positions is higher than a preset safety threshold, it is determined that the risk of the passage environment ahead is high, and a deceleration or waiting-type obstacle avoidance guidance action is output to avoid potential collision hazards.

[0036] The smart cane described in the above embodiment quantifies dynamic collision risks based on the location and movement characteristics of dynamic targets, predicts the probability of collisions with moving targets such as pedestrians and cyclists in advance, distinguishes between high-risk approaching targets and non-threatening targets moving away, and reduces false alarms and missed alarms in dynamic scenarios. It then integrates the distance to static obstacles, the degree of deviation from the travel route, and the weighted calculation of dynamic collision risks to calculate the comprehensive passage cost, effectively balancing passage safety and the user's original travel route, avoiding the problem of hugging walls and deviating significantly from the original route in order to avoid dynamic targets. Finally, by comparing the comprehensive passage costs corresponding to each candidate passage position, the optimal guidance action is determined, realizing intelligent dynamic obstacle avoidance under multiple constraints, which greatly improves the passage safety and walking smoothness of visually impaired users in complex public scenarios with a mix of pedestrians and static obstacles.

[0037] In some embodiments, determining the dynamic collision risk between the user and the dynamic targets based on the real-time position and motion characteristics of each dynamic target includes: Based on the real-time location and motion characteristics of each dynamic target, determine the estimated time of intersection between the user and the dynamic target; The dynamic collision risk between the user and the dynamic target is determined based on the expected intersection time of the user and the dynamic target, the speed of the dynamic target, the detection confidence of the dynamic target, and the crowd density.

[0038] Specifically, in this embodiment, the calculation and processing module can calculate the estimated intersection time between the user and the dynamic target based on the real-time position and motion characteristics of each dynamic target. Optionally, the calculation and processing module can extract the current coordinates, speed, and direction of motion of the dynamic target, and combine this with the user's own speed and direction of travel to calculate the time required for the two to intersect, thus obtaining the estimated intersection time between the user and the corresponding dynamic target. Further, the calculation and processing module can quantify the dynamic collision risk by combining four parameters: the estimated intersection time between the user and the dynamic target, the speed of the dynamic target, the target detection confidence level, and the local crowd density. Among these factors, the shorter the expected convergence time, the more urgent the conflict is, and the higher the collision risk; the faster the dynamic target moves, the greater the impact damage will be once a collision occurs, and the higher the corresponding risk; the higher the local pedestrian density and the more crowded and complex the surrounding traffic environment, the greater the overall collision risk; the dynamic target detection confidence level is used as a correction term in the risk calculation. For low-confidence targets with blurred outlines and unstable tracking, the risk weight is appropriately lowered, which can compensate for the judgment error caused by sensor perception deviation, making the dynamic collision risk assessment results more objective and accurate, and effectively reducing the probability of false alarms and false negatives in pedestrian scenarios.

[0039] The smart cane described in the above embodiment calculates the estimated intersection time by considering the position and motion parameters of the dynamic target and the user, thus predicting potential path conflicts from a time perspective and overcoming the limitation of judging danger solely by instantaneous distance. Furthermore, it integrates four dimensions—intersection time, target speed, crowd density, and detection confidence—to comprehensively quantify collision risk. This approach takes into account the urgency of the conflict, the severity of the collision damage, and the degree of environmental congestion. It also corrects sensor recognition errors through detection confidence, finely distinguishing between high-risk crossing targets and non-threatening, distant targets. This significantly improves the comprehensiveness and accuracy of dynamic collision risk assessment and reduces false alarms and missed alarms in densely populated areas.

[0040] In some embodiments, the dynamic collision risk between the user and the dynamic target is determined based on the expected encounter time between the user and the dynamic target, the speed of the dynamic target, the detection confidence of the dynamic target, and the crowd density, including: = || ‖+ - ) in, R i Indicates user and the The dynamic collision risk of a dynamic target, where σ(·) represents the normalization function.α 1 , α 2 , α 3 , α 4 For weight parameters, Indicates user and the The estimated convergence time of each dynamic target To prevent stable terms with a denominator of zero, Indicates the first The velocity of a dynamic target at time t This represents the local crowd density at time t. The confidence level for detecting dynamic targets.

[0041] Specifically, in this embodiment of the application, the calculation and processing module can quantify the dynamic collision risk between the user and the dynamic target based on the following method: = || ‖+ - ) in, It indicates the urgency of conflict, unlike the traditional method of judging danger solely by distance, and can identify high-risk targets that are far away but about to be crossed. || The "‖" indicates the risk of target movement speed. The greater the instantaneous speed of a dynamic target, the greater the impact damage after a collision. It accurately matches high-risk scenarios such as cyclists moving quickly through traffic, reflecting the real hidden dangers in the environment. This indicates the risk of crowd density; the higher the crowd density, the larger the value, indicating an increased risk of traffic congestion, particularly in high-risk scenarios such as shopping malls and subways with dense crowds. This indicates the detection confidence error correction term, which automatically lowers the risk level for targets that are blurry or intermittently tracked by cameras and radar, reducing false risk assessments caused by perception noise and lowering the overall false alarm rate.

[0042] For example, such as Figure 3 As shown, the smart cane of this application does not judge whether there is danger based solely on the current distance, but predicts whether an intersection will occur in the future based on the movement trend of the target relative to the user.

[0043] For the i-th target, calculate its estimated meeting time: Here, τi represents the estimated intersection time of the i-th target relative to the user's travel path, and ε is a stabilizing term to prevent the denominator from being zero. When τi is small and the target is moving towards the user's travel direction, it indicates that the target may collide with the user's path in a short period of time.

[0044] To reflect the degree of crowding, local crowd density is also calculated: in, ρ t Let Ω represent the local pedestrian density at time t, Ω represent the preset local area in front of the white cane, and I(·) be the indicator function. I is set to 1 when the target is within this area, and 0 otherwise.

[0045] A dynamic risk score is obtained by combining the estimated convergence time, movement speed, crowd density, and target detection confidence. = || ‖+ - ) in, R i Let σ(·) represent the risk score of the i-th dynamic objective, and let σ(·) represent the normalization function. α 1 , α 2 , α 3 , α 4 For weight parameters, c i This risk score provides confidence in target detection or tracking. It allows for the differentiation between targets that are close but moving away and targets that are far away but about to cut in laterally.

[0046] The smart cane described in the above embodiment quantifies the urgency of conflict by estimating the time of intersection between the user and the dynamic target, breaking the limitation of traditional methods that rely solely on instantaneous distance to determine danger. It can identify high-risk targets that are far away but about to cross the path laterally in advance. By superimposing the target's movement speed and local crowd density risk, it effectively adapts to high-risk travel scenarios such as cyclists moving quickly and crowded shopping malls and subways, comprehensively covering various dynamic traffic hazards. At the same time, it introduces target detection confidence to correct the risk value, compensating for the perception bias caused by blurry recognition and tracking interruptions of sensors such as cameras and radar, effectively reducing false alarms and missed alarms in complex crowd scenarios.

[0047] In some embodiments, the comprehensive passage cost corresponding to each candidate passage location of the user is determined based on the static obstacle cost, the deviation cost of travel direction, and the dynamic collision risk between the user and the dynamic target, including: Where C(q) represents the comprehensive passage cost corresponding to the candidate passage position q. , , Here, S(q) represents the static obstacle cost corresponding to the user's candidate passage position q, and D(q) represents the deviation cost from the travel direction. R i Indicates user and the The dynamic collision risk of a dynamic target. Indicates the first The position of a dynamic target at time t. Indicates user and the The estimated convergence time of each dynamic target Indicates the first The velocity of a dynamic target at time t This indicates the scope of dynamic risk diffusion.

[0048] Specifically, in this embodiment of the application, the calculation and processing module can quantify the comprehensive passage cost corresponding to each candidate passage position of the user based on the following method: Where S(q) represents the static obstacle cost corresponding to the user's candidate passage position q, which can be calculated based on the distance between the candidate passage position and obstacles such as walls, steps, and pillars. The closer the point is to the obstacle, the higher the static obstacle cost. D(q) represents the offset cost of the candidate passage position relative to the user's original straight route. The farther it deviates from the original route, the higher the deviation cost. R i Indicates user and the The dynamic collision risk of dynamic targets. In other words, this application integrates the distance to static obstacles, the degree of deviation from the travel route, and the dynamic collision risk to calculate the comprehensive passage cost, effectively balancing passage safety and the user's original travel route. It avoids the problem of hugging the wall and deviating significantly from the original route in order to avoid dynamic targets, and realizes intelligent dynamic obstacle avoidance under multiple constraints. This greatly improves the passage safety and walking smoothness of visually impaired users in complex public scenes with a mix of pedestrians and static obstacles.

[0049] For example, such as Figure 4 As shown, this application discretizes the area in front of the white cane into multiple candidate locations, and each candidate location obtains a comprehensive passage cost based on static obstacle cost, directional deviation cost, and dynamic risk cost.

[0050] For a candidate position q, its comprehensive travel cost is defined as: Where C(q) represents the comprehensive passage cost of candidate position q, S(q) represents the static obstacle cost, and D(q) represents the deviation cost relative to the user's desired travel direction. Ri This represents the risk score for the i-th dynamic objective. λ s , λ d , λ r For weight parameters, σ r This represents the dynamic risk diffusion range. The formula indicates that the closer a candidate path is to a static obstacle, the further it deviates from its original direction of travel, and the closer it is to the predicted location of a high-risk dynamic target, the higher its travel cost. Optionally, a set of guiding actions A can be predefined, which may include going straight, veering left, veering right, decelerating, stopping, and waiting. Each action corresponds to a short-term predicted trajectory T(a). The action with the lowest overall travel cost is selected as the current optimal guiding action. Where a* represents the optimal guiding action, and T(a) represents the short-term predicted trajectory corresponding to action a. If the cost of going straight is the lowest, the user is prompted to stay straight; if there is a lower-risk path on the left or right, the user is prompted to slightly adjust their direction through vibration or voice in the corresponding direction; if the cost of all candidate directions is higher than the safety threshold, the user is prompted to slow down, stop, or wait.

[0051] The smart cane described in the above embodiment quantifies the comprehensive passage cost of each candidate passage location by weighted fusion of static obstacle cost, deviation cost of travel direction cost, and dynamic collision risk. It simultaneously takes into account multiple constraints such as static obstacle avoidance, user's original travel direction, and dynamic pedestrian collision risk. While avoiding various passage hazards, it tries to follow the user's original travel route as closely as possible, providing an objective and reliable basis for determining the optimal obstacle avoidance guidance action and improving the safety and smoothness of travel for visually impaired users.

[0052] In some embodiments, the guidance feedback module is also used for: If the optimal obstacle avoidance guidance action remains unchanged over multiple consecutive time periods, the repeated output of multimodal cue signals is suppressed, and the corresponding multimodal cue signals are only output when the optimal obstacle avoidance guidance action changes.

[0053] Specifically, in this embodiment, the guidance feedback module can continuously monitor the optimal obstacle avoidance guidance action calculated and output at each moment in real time, and cache and compare the guidance action states at multiple consecutive moments. When it is detected that the optimal obstacle avoidance guidance action remains consistent and there is no state change at multiple consecutive moments, it is determined that the current passage environment is stable and the user does not need to adjust their walking posture, thus suppressing the repeated output of the corresponding multimodal prompt signal; only when the optimal obstacle avoidance guidance action is detected to have changed, i.e., the passage risk or optimal passage strategy has changed, will a new multimodal prompt signal be triggered and output in real time to inform the user of the updated obstacle avoidance passage instructions. This can effectively eliminate high-frequency repeated prompt interference, optimize the user's passage experience, and accurately respond to environmental changes, ensuring that the user can perceive the passage strategy adjustment as soon as possible, avoid sudden passage risks in a timely manner, and reduce device power consumption and operating load.

[0054] The smart cane described in the above embodiment continuously compares the optimal obstacle avoidance guidance actions at multiple times through the guidance feedback module, and triggers a prompt signal only when the passage strategy changes. This not only effectively eliminates continuous repetitive prompts and improves the user experience for visually impaired users, but also ensures timely warnings when environmental risks change, thereby improving the device's battery life.

[0055] In some embodiments, the multimodal cue signal includes a vibration signal and a voice cue signal.

[0056] Specifically, in this embodiment, the multimodal prompting signal output by the guidance feedback module of the smart cane includes vibration signals and voice prompts. Optionally, the cane can rely on its built-in vibration motor to output vibration feedback of different frequencies, durations, and intensities as tactile cues. Optionally, different obstacle avoidance actions correspond to different vibration modes, allowing users to quickly distinguish different passage instructions such as going straight, veering left or right, slowing down, and waiting using only tactile feedback, without relying on hearing. Optionally, a standardized voice message can be broadcast through a built-in speaker to clearly inform users of the type of obstacle ahead, pedestrian risks, and suggested modes of travel, supplementing the vibration prompts and helping users fully understand the surrounding traffic environment and avoidance strategies. Optionally, the two prompting signals can be output separately or simultaneously, adapting to different travel scenarios such as noisy or quiet environments. Even if one modality is difficult to perceive due to environmental interference, the other can still clearly transmit instructions, avoiding the loss of prompt information and improving travel safety.

[0057] In some embodiments, different optimal obstacle avoidance guidance actions correspond to different vibration frequencies, vibration durations, or voice prompts.

[0058] Specifically, in this embodiment, the guidance feedback module can pre-establish the correspondence between optimal obstacle avoidance guidance actions and prompting methods, configuring differentiated vibration parameters and voice content for various guidance actions such as going straight, veering left, veering right, decelerating, and waiting. Optionally, different vibration frequencies and durations can be used for differentiation, such as low-frequency short vibrations for going straight, medium-frequency continuous vibrations for veering left and right, and high-frequency long vibrations for waiting. Users can quickly distinguish between going straight, detouring, decelerating, and waiting simply by feeling the vibration in their hands, without relying on hearing. Even in noisy streets, subways, or other scenarios where voices are difficult to hear, users can still accurately obtain passage strategies. Alternatively, exclusive voice prompts can be matched for each type of obstacle avoidance action: "Straight ahead, you can go straight," "Veering left is safer, please adjust your direction slightly to the left," and "Waiting is crowded ahead, please pause and wait." This not only informs users how to proceed but also explains the environmental risks ahead, helping visually impaired users clearly understand the reasons for avoiding danger and improving their ability to judge road conditions. Optionally, the vibration intensity and warning voice can be set according to the level of risk. High-risk waiting actions use high-frequency long vibration and warning voice, while low-risk straight-through actions use weak short vibration, so that users can intuitively and accurately perceive the current level of traffic hazards and make appropriate obstacle avoidance behaviors.

[0059] For example, this application can map guidance actions to short, stable, and low-intensity feedback signals. When the optimal action is to veer left, the left vibration unit vibrates with short pulses; when the optimal action is to veer right, the right vibration unit vibrates with short pulses; when the optimal action is to stop, both left and right vibration units vibrate continuously simultaneously, accompanied by a voice prompt "Please stop"; when the optimal action is to decelerate, the central vibration unit vibrates in a low-frequency mode; when the optimal action is to wait, a prompt "Crowd crossing ahead, please wait" can be given. To reduce prompt interference, a feedback suppression mechanism can be set. When the guidance actions are consistent across multiple consecutive frames, a new prompt is only output when the action changes or the risk level significantly increases; when the risk is low, no voice is output, only slight vibration or no prompt is given. In this way, the smart cane can provide continuous but not overly disruptive dynamic obstacle avoidance guidance in complex crowd environments.

[0060] The smart cane described in the above embodiment can effectively adapt to various travel scenarios, including noisy and quiet ones, through multimodal prompt signals. It can also configure differentiated vibration frequencies, durations, and voice content for various obstacle avoidance actions, and can set gradient prompt intensity according to risk level, allowing users to intuitively distinguish the severity of potential hazards. This significantly improves the recognition of guidance instructions, the completeness of information, and the reliability of travel guidance, thereby improving the travel experience and safety of visually impaired users.

[0061] In some embodiments, the smart cane further includes: The system includes a power supply module and a communication module. The power supply module provides power to the environmental sensing module, computing processing module, and guidance feedback module. The communication module is used to transmit and interact with mobile phones, earphones, the cloud, or remote monitoring terminals.

[0062] Specifically, in this embodiment, the smart cane also includes a power module and a communication module. The power module is electrically connected to the environmental perception module, the computing and processing module, and the guidance feedback module, providing a continuous and stable power supply to all sensing, computing, and feedback hardware within the entire device. This ensures the cane can collect environmental data in real time, perform dynamic obstacle avoidance algorithm calculations, and output multimodal prompt signals. Optionally, the communication module has multi-terminal data interaction capabilities, allowing it to establish data transmission links with mobile phones, Bluetooth headsets, cloud servers, and remote monitoring terminals, achieving bidirectional data interaction and effectively improving the practicality and safety of the smart cane.

[0063] For example, such as Figure 5 As shown, this application provides a smart cane. This smart cane collects environmental information from the front and sides using multi-source sensors, detects, tracks, and predicts the trajectory of moving targets such as pedestrians, constructs a local pedestrian flow trajectory field centered on the user, and combines static obstacles, passable areas, and the user's desired direction of travel to generate dynamic risk distribution and obstacle avoidance guidance actions. The smart cane includes a cane body, an environmental perception module, a computing processing module, a guidance feedback module, a power module, and a communication module. The environmental perception module collects images, depth, distance, inertial attitude, or radar echo information from the front; the computing processing module performs target detection, target tracking, trajectory coordinate transformation, dynamic risk assessment, passage cost map construction, and guidance action decision-making; the guidance feedback module outputs obstacle avoidance prompts to the user through vibration, voice, buzzer, or a combination thereof; the communication module interacts with mobile phones, headphones, cloud services, or remote monitoring devices. For example, as shown... Figure 6 As shown, the smart cane can continuously collect environmental data around the cane and perform time synchronization and posture compensation on the data. Secondly, it can detect and track dynamic targets such as pedestrians and cyclists to obtain the position sequence of the targets in continuous time. Thirdly, it can transform the target position into a user-centered local coordinate system and calculate the target speed, direction of movement, expected intersection time, and minimum expected distance. Then, it can construct a comprehensive passage cost based on the target trajectory, pedestrian density, static obstacles, and passable areas. Finally, it can determine guidance actions such as going straight, veering left, veering right, decelerating, stopping, or waiting based on the comprehensive passage cost, and convert the actions into feedback signals that conform to the cognitive load of visually impaired users.

[0064] It should be noted that the intelligent cane and dynamic obstacle avoidance guidance method of this application have the following advantages compared with traditional methods: First, this application shifts from a single-point judgment of "whether an obstacle exists" to a trajectory judgment of "whether there will be a conflict in the future," which can distinguish between pedestrians far from the user, parallel moving crowds, and high-risk targets about to enter the user's walking path, thereby reducing false alarms and missed alarms. Second, this application uses a local crowd trajectory field to describe the overall movement trend of dynamic targets, enabling the intelligent cane to select low-risk directions in advance in complex crowd environments, rather than passively reminding users after an obstacle approaches. Third, this application unifies static obstacles, dynamic targets, and passable areas into a comprehensive passage cost map, which can output a stable and consistent guidance strategy in different scenarios, improving the continuity, interpretability, and user-executability of the cane prompts. Fourth, this application can achieve real-time calculation on the edge device and can also work in conjunction with mobile phones or cloud devices, which has good deployment flexibility and application promotion value.

[0065] For example, this application also provides a method for operating a smart cane, including: The environmental perception module acquires environmental perception data; The computational processing module determines the real-time position and motion characteristics of dynamic targets based on environmental perception data; and determines the optimal obstacle avoidance guidance action based on the real-time position and motion characteristics of dynamic targets. The guidance feedback module converts the optimal obstacle avoidance guidance action into multimodal prompt signals.

[0066] The method in this embodiment is similar in its specific implementation process and technical effects to that in the embodiment of the smart cane. For details, please refer to the detailed description in the embodiment of the smart cane, which will not be repeated here.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A smart white cane based on pedestrian flow trajectory perception, characterized in that, include: The module consists of an environmental perception module, a computational processing module, and a guidance and feedback module. The environmental perception module is used to acquire environmental perception data; The computational processing module is used to determine the real-time location and motion characteristics of dynamic targets based on environmental perception data; Determine the optimal obstacle avoidance guidance action based on the real-time position and motion characteristics of the dynamic target; The guidance feedback module is used to convert the optimal obstacle avoidance guidance action into multimodal prompt signals.

2. The intelligent white cane based on pedestrian trajectory perception according to claim 1, characterized in that, The calculation and processing module is specifically used for: Based on the real-time location and motion characteristics of each dynamic target, determine the dynamic collision risk between the user and the dynamic target; Based on the static obstacle cost, the deviation cost of travel direction, and the dynamic collision risk between the user and the dynamic target corresponding to each candidate passage position, determine the comprehensive passage cost corresponding to each candidate passage position of the user. The optimal obstacle avoidance guidance action is determined based on the comprehensive passage cost corresponding to each candidate passage location of the user.

3. The intelligent white cane based on pedestrian trajectory perception according to claim 2, characterized in that, Based on the real-time location and motion characteristics of each dynamic target, determine the dynamic collision risk between the user and the dynamic target, including: Based on the real-time location and motion characteristics of each dynamic target, determine the estimated time of intersection between the user and the dynamic target; The dynamic collision risk between the user and the dynamic target is determined based on the expected intersection time of the user and the dynamic target, the speed of the dynamic target, the detection confidence of the dynamic target, and the crowd density.

4. The intelligent white cane based on pedestrian trajectory perception according to claim 3, characterized in that, Based on the estimated encounter time between the user and the dynamic target, the speed of the dynamic target, the detection confidence level of the dynamic target, and the crowd density, the dynamic collision risk between the user and the dynamic target is determined, including: = ‖ ‖+ - ) in, R i Indicates user and the The dynamic collision risk of a dynamic target, where σ(·) represents the normalization function. α 1 , α 2 , α 3 , α 4 For weight parameters, Indicates user and the The estimated convergence time of each dynamic target To prevent stable terms with a denominator of zero, Indicates the first The velocity of a dynamic target at time t This represents the local crowd density at time t. Confidence level for detecting dynamic targets.

5. The intelligent white cane based on pedestrian trajectory perception according to claim 2, characterized in that, Based on the static obstacle cost, travel direction deviation cost, and dynamic collision risk between the user and dynamic targets corresponding to each candidate passage location, the comprehensive passage cost corresponding to each candidate passage location is determined, including: Where C(q) represents the comprehensive passage cost corresponding to the candidate passage position q. , , Here, S(q) represents the static obstacle cost corresponding to the user's candidate passage position q, and D(q) represents the deviation cost from the travel direction. R i Indicates user and the The dynamic collision risk of a dynamic target. Indicates the first The position of a dynamic target at time t. Indicates user and the The estimated convergence time of each dynamic target Indicates the first The velocity of a dynamic target at time t This indicates the scope of dynamic risk diffusion.

6. The intelligent white cane based on pedestrian trajectory perception according to any one of claims 1-5, characterized in that, The optimal obstacle avoidance guidance action includes at least one of the following: Go straight, veer left, veer right, slow down, and wait.

7. The intelligent white cane based on pedestrian trajectory perception according to any one of claims 1-5, characterized in that, The guided feedback module is also used for: If the optimal obstacle avoidance guidance action remains unchanged over multiple consecutive time periods, the repeated output of multimodal cue signals is suppressed, and the corresponding multimodal cue signals are only output when the optimal obstacle avoidance guidance action changes.

8. The intelligent white cane based on pedestrian trajectory perception according to any one of claims 1-5, characterized in that, Multimodal prompting signals include vibration signals and voice prompting signals.

9. The intelligent white cane based on pedestrian trajectory perception according to claim 8, characterized in that, Different optimal obstacle avoidance guidance actions correspond to different vibration frequencies, vibration durations, or voice prompts.

10. The intelligent white cane based on pedestrian trajectory perception according to claim 1, characterized in that, The smart cane also includes: The system includes a power supply module and a communication module. The power supply module provides power to the environmental sensing module, computing processing module, and guidance feedback module. The communication module is used to transmit and interact with mobile phones, earphones, the cloud, or remote monitoring terminals.

11. A method for operating a smart cane, applied to a smart cane as described in any one of claims 1-10, characterized in that, include: The environmental perception module acquires environmental perception data; The computational processing module determines the real-time location and motion characteristics of dynamic targets based on environmental perception data; Determine the optimal obstacle avoidance guidance action based on the real-time position and motion characteristics of the dynamic target; The guidance feedback module converts the optimal obstacle avoidance guidance action into multimodal prompt signals.

Images