Walking-aid robot obstacle avoidance control method and system based on multiple sensors

Abstract

The invention relates to the technical field of intelligent travel control of walking-aided robots, and discloses a multi-sensor-based obstacle avoidance control method and system for a walking-aided robot. The obstacle avoidance control method for the walking-aid robot is applied to walking-aid robot equipment, and specifically comprises the following steps: S101, completing the installation of a multi-sensor module on the walking-aid robot, and collecting multi-source data obtained based on multiple sensors in real time, unifying the multi-source data to a base coordinate system of the walking aid robot by adopting timestamp synchronization and a coordinate system conversion matrix, and generating a fused dynamic obstacle sequence map; and S102, on the basis of the dynamic obstacle sequence map, a Kalman filtering and LSTM neural network fusion model is adopted to predict the motion trajectory of the dynamic obstacle. According to the method, the environment perception and decision-making architecture of the walking-aid robot is reconstructed, a multi-sensor space-time cooperation mechanism is matched through cross-modal data of laser radar and binocular vision, and ultrasonic sensor low obstacle special detection is combined, so that the step recognition rate is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of travel by walking-assistance robots, and in particular to a multi-sensor based obstacle avoidance control method and system for walking-assistance robots. Background Art

[0002] As a core category of intelligent assistive devices, the demand for walking-assistance robots has increased significantly in an aging society and in the field of rehabilitation medicine. The current mainstream technology uses a SLAM mapping and path planning architecture based on lidar or ultrasonic sensors, such as the "lidar + IMU" combined positioning solution proposed in CN112230675A and the ultrasonic array obstacle avoidance method described in CN113419497A.

[0003] However, existing solutions generally rely on a single sensing modality: LiDAR has a 35% failure rate when identifying low-reflectivity obstacles like mirrors and glass. Ultrasonic sensors, limited by their detection range and angular resolution, struggle to cope with sudden dynamic obstacles. Vision-based solutions are susceptible to interference from lighting, with a false detection rate exceeding 45% at night. Furthermore, traditional control architectures model the environment as a static map, failing to consider the trajectory prediction of moving objects like pedestrians and vehicles. This results in a high obstacle avoidance failure rate in dynamic scenarios like shopping malls and hospitals. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-sensor based obstacle avoidance control method and system for a walking-assist robot. By reconstructing the environmental perception and decision-making architecture of the walking-assist robot, the multi-sensor spatiotemporal coordination mechanism matches cross-modal data of lidar and binocular vision, combined with the special detection of low obstacles by ultrasonic sensors, the step recognition rate is greatly improved, aiming to solve the problems in the existing technology.

[0005] The present invention is implemented as follows: a multi-sensor based obstacle avoidance control method for a walking-assist robot, applied to a walking-assist robot device, specifically comprising the following steps: S101: Complete the installation of the multi-sensor module on the walking robot, collect multi-source data obtained by the multi-sensors in real time, use timestamp synchronization and coordinate system conversion matrix to unify the multi-source data into the base coordinate system of the walking robot, and generate a fused dynamic obstacle sequence map; S102: Based on the dynamic obstacle sequence map, a Kalman filter and LSTM neural network fusion model is used to predict the motion trajectory of dynamic obstacles, and the obstacle velocity vector and acceleration are estimated in real time. The LSTM network takes time-series position data as input and outputs a set of trajectory prediction points within a preset time in the future. A kinematic model is constructed for pedestrian and vehicle obstacles to constrain the predicted direction error. S103: Generate an initial global path with the current position of the walking robot as the starting point and the user-set target point as the end point. When the obstacle in the dynamic obstacle sequence map intrudes into the safety threshold, start local path optimization, introduce the spatiotemporal cost function of the dynamic obstacle prediction trajectory, solve the minimum cost path through the gradient descent method, and plan a continuous spline curve as the obstacle avoidance path in real time; S104: Generate speed instructions for the robot's wheeled chassis based on the planned obstacle avoidance path. Use a force sensor array to detect the direction and magnitude of the user's thrust. When the user applies lateral force, the control model adjusts the movement direction. Simultaneously, vibration tactile feedback is used to convey obstacle avoidance direction prompts to the user, forming a closed-loop human-machine collaborative control system. S105: When a sudden obstacle is detected and the third-level braking mechanism is triggered, the IMU angular velocity deviation Δω is continuously monitored during the braking process. If |Δω| ≥ 0.5 rad / s², the risk of instability is determined, and the nearest safe state point is immediately called to fall back to a stable posture, and a warning message is broadcast through the voice module.

[0006] Furthermore, in S101, a multi-sensor module is installed on the walking-assistance robot, wherein the multi-sensor module includes: LiDAR, used to collect environmental depth information in real time and construct a 2D polar coordinate obstacle map; Binocular vision camera, used to collect RGB images of the environment in real time, generate disparity maps through the SGBM algorithm and convert them into 3D point clouds; Ultrasonic sensor array, used to obtain real-time distance information of close-range obstacles and detect step-like obstacles with a height of less than 20 cm; IMU module, used to obtain robot posture data information in real time.

[0007] Furthermore, timestamp synchronization and coordinate system conversion matrix are used to unify the multi-source data into the base coordinate system of the walking robot, generating a fused dynamic obstacle sequence map, including: Establish a Euclidean distance clustering model for lidar point clouds and segment independent obstacle clusters using the DBSCAN algorithm; The binocular vision point cloud is matched with the center of the lidar cluster through KD-Tree accelerated nearest neighbor search. Successfully matched obstacles are assigned ID tags. When the ultrasonic data detects an obstacle at the same location in three consecutive frames, the map is updated. The sensor confidence weights are introduced into the fusion process, and the final obstacle position estimate is a weighted least squares solution.

[0008] Furthermore, in S102, based on the dynamic obstacle sequence map, a Kalman filter and LSTM neural network fusion model is used to predict the motion trajectory of the dynamic obstacle, and the obstacle velocity vector and acceleration are estimated in real time, including: Based on the dynamic obstacle sequence map generated by the fusion of LiDAR and binocular vision, more than 30 consecutive frames of obstacle position time series data are extracted. Unique kinematic models are matched to different obstacle types. For pedestrians, a social force enhancement model is used, while for vehicles, a constant acceleration constraint model is used. A two-stage dynamic estimation is performed through Kalman filtering. The initial motion state at the current moment is predicted based on the historical trajectory. The latest sensor observations are then integrated for error correction, and high-precision velocity vectors and acceleration values ​​are output in real time. The velocity and acceleration vectors output by the Kalman filter are fused with the relative distance of the robot into a five-dimensional feature vector, which is then input into a two-layer LSTM neural network and the logical relationship data of the prediction results.

[0009] Furthermore, the two-layer LSTM neural network and the logical relationship data of the prediction results are input, including: The long-term temporal dependencies of obstacle motion are learned through the memory gate mechanism, which is particularly optimized for modeling nonlinear motions such as random turns of pedestrians and sudden stops of pets. Output the trajectory prediction position of 5 consecutive time points within the next 1 second, add a probability confidence to each point, and convert the prediction result into a spatiotemporal probability heat map; On the spatiotemporal probability heat map, with the current position of the obstacle as the center, a radial probability distribution cloud map is generated along the predicted direction. High-risk areas are marked with a red gradient and superimposed on the navigation interface.

[0010] Furthermore, in S103, local path optimization is started, a spatiotemporal cost function of the dynamic obstacle prediction trajectory is introduced, and a path with the minimum cost is solved by the gradient descent method, including: When a dynamic obstacle intrudes into the safety zone with a radius of 0.8 meters centered on the robot, the initial global path is immediately interrupted. The spatiotemporal intrusion risk value is calculated based on the predicted obstacle trajectory. Obstacles with a moving speed greater than 1m / s are weighted twice as much risk, and static obstacles are treated as having a basic risk value. This initiates a real-time obstacle avoidance response cycle. A spatiotemporal cost function that includes trajectory prediction probability is constructed, and a path curvature constraint is set. The path point coordinates are iteratively optimized through gradient descent to generate C²-continuous cubic B-spline curves. Each curve contains seven control points, and tangent-continuous transition segments are inserted at the inflection points to ensure that the steering angle change rate of the wheeled chassis is ≤15° / s.

[0011] Furthermore, in S104, the force sensor array detects the direction and magnitude of the user's thrust. When the user applies lateral force, the control model adjusts the movement direction, and simultaneously transmits obstacle avoidance direction prompts to the user through vibration tactile feedback, including: The six-dimensional force sensor array distributed on the left and right armrests detects the user's thrust in real time. When the lateral thrust lasts >15N for more than 0.5 seconds, the impedance control model is activated. The steering angle compensation Δω is calculated based on the thrust direction angle θ. The compensation gain increases linearly from 0.1 to 0.5 as the thrust increases, while the maximum steering angle deviation is limited to ≤30°. Tactile feedback generates differentiated vibration codes based on the obstacle avoidance direction. Left obstacle avoidance triggers three short vibrations, and right obstacle avoidance triggers two long vibrations. The vibration intensity is inversely correlated with the distance to the obstacle. An orange arrow is simultaneously displayed on the navigation interface to indicate the obstacle avoidance direction, forming a visual-tactile dual-channel guidance.

[0012] Furthermore, in S105, when a sudden obstacle is detected and the third-level braking mechanism is triggered, the IMU angular velocity deviation Δω is continuously monitored during the braking process. If |Δω| ≥ 0.5 rad / s², the instability risk is determined, including: The criteria for sudden obstacles are: relative distance ≤ 0.3m or approach speed ≥ 1m / s, triggering three-level braking: Level 1: Motor torque PID control is reduced to 50%; Second level: electromagnetic friction plate energized braking; Level 3: reverse compensation current is injected into the motor; Acceleration gradient control during braking: 0-0.3s to -0.5m / s², 0.3-0.8s to -1.2m / s²; The IMU's three-axis angular velocity deviation is monitored in real time. When |Δω| ≥ 0.5 rad / s² for 100 ms, instability is determined. At this time, a safe state point within the last 5 seconds is extracted from the ring buffer. The safe state point includes posture, velocity, and map data. The robot is controlled to retreat to this point along the original path, with the retreat speed limited to 40% of the normal value.

[0013] Compared with the prior art, the multi-sensor based obstacle avoidance control method and system for a walking-assistance robot provided by the present invention has the following beneficial effects: 1. By reconstructing the environmental perception and decision-making architecture of the walking-assistance robot, the multi-sensor spatiotemporal coordination mechanism significantly improves step recognition through cross-modal data matching between lidar and binocular vision, combined with ultrasonic sensors for specialized low-obstacle detection. The detection distance for biological obstacles in low-light environments has been extended from 1.5 meters to 3.5 meters. Secondly, the dynamic obstacle intelligent prediction system, for the first time, integrates the LSTM long-term learning capability with the real-time correction of the Kalman filter to accurately predict the movement trajectory of pedestrians and vehicles. The prediction results are converted into spatiotemporal probability heat maps to drive path planning, improving the success rate of dynamic obstacle avoidance. Furthermore, the path generation quality has achieved a generational leap. The improved A* algorithm introduces an obstacle density heuristic function, shortening the average global path length. Local obstacle avoidance uses C² continuous B-spline optimization to eliminate the curvature mutation problem of traditional broken line paths. This reduces steering jitter of the wheeled chassis by 90%, and reduces the risk of user dizziness by 60%. 2. Outstanding results have been achieved in the dimensions of human-computer interaction and safety protection. The personalized safety model breaks through the limitations of fixed thresholds and dynamically adjusts the safety radius based on the user's grip strength and electromyographic signals, reducing the risk of collision. Secondly, the human-computer interaction conflict is resolved by optimizing the impedance parameters in real time through reinforcement learning, compressing the deviation between the operating lateral thrust and the planned path to ≤5°, and suppressing the peak human-computer force from 40-60N to below 25N, improving the smoothness of operation. In addition, the multi-level fault-tolerant control architecture rewrites the emergency braking standard. The three-level braking mechanism shortens the emergency stop distance at a speed of 1m / s from 1.2 meters to 0.7 meters. Combined with the IMU's real-time angular velocity monitoring and state rollback function, the risk of users falling during emergency stops is reduced. The night mode maintains the ability to recognize biological obstacles within 3.5 meters in moonlight through infrared and thermal imaging fusion, truly realizing full-scene safety protection.

[0014] A multi-sensor based obstacle avoidance control system for a walking-assist robot, used in the above-mentioned obstacle avoidance control method for a walking-assist robot, the obstacle avoidance control system for the walking-assist robot comprising: The multimodal perception module is used to collect environmental data through lidar, binocular vision camera, ultrasonic sensor array and IMU, and output a fused dynamic obstacle sequence map; The trajectory prediction module is used to estimate the obstacle velocity vector and acceleration in real time based on the Kalman filter and LSTM neural network fusion model, and generate a trajectory probability heat map within the next 1 second; The path planning module is used to generate an initial global path starting from the robot's current position and ending at the user's target point. When an obstacle intrudes into the safety threshold, local optimization is initiated and a C²-continuous cubic B-spline obstacle avoidance path is output. The human-machine collaborative control module generates chassis motion instructions based on the planned path, detects user thrust through force sensors and adjusts direction, and synchronizes the tactile feedback module to deliver obstacle avoidance prompts. The safety fault-tolerant module is used to trigger level 3 braking when encountering sudden obstacles, monitor IMU angular velocity deviations, and return to a safe state point, and broadcast warning information through the voice module.

[0015] Specifically, the trajectory prediction module and the path planning module are both deployed on a heterogeneous computing platform, which includes: FPGA acceleration unit for real-time Kalman filtering and spatiotemporal alignment of sensor data, configured with a pipelined parallel architecture to process lidar point clouds; The NPU inference unit is used to load the quantized LSTM trajectory prediction model and output prediction heat maps at a frame rate of 20Hz or higher. CPU decision unit: used to run efficient heuristic search algorithms and gradient descent path optimization for improved path planning, and implement planning cycles through ROS node scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic flow chart of a multi-sensor based obstacle avoidance control method for a walking-assistance robot proposed in the present invention; Figure 2 This is a schematic block diagram of the process of unifying multi-source data into the base coordinate system of the walking-assist robot using timestamp synchronization and a coordinate system conversion matrix in a multi-sensor based walking-assist robot obstacle avoidance control method proposed by the present invention, thereby generating a fused dynamic obstacle sequence map. Figure 3 This is a schematic diagram of the structure of a multi-sensor based obstacle avoidance control system for a walking-assistance robot proposed in the present invention; Figure 4 This is a structural schematic diagram of a heterogeneous computing platform in a multi-sensor based obstacle avoidance control system for a walking-assist robot proposed in the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] The implementation of the present invention is described in detail below with reference to specific embodiments.

[0019] The same or similar numbers in the drawings of this embodiment correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "up", "down", "left", "right", etc. indicate directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. This is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0020] Reference Figure 1-2 As shown, a multi-sensor based obstacle avoidance control method for a walking-assist robot is applied to a walking-assist robot device, specifically comprising the following steps: S101: Complete the installation of the multi-sensor module on the walking robot, collect multi-source data obtained by the multi-sensors in real time, use timestamp synchronization and coordinate system conversion matrix to unify the multi-source data into the base coordinate system of the walking robot, and generate a fused dynamic obstacle sequence map; The multi-sensor module is installed on the walking robot. The multi-sensor module includes: LiDAR, used to collect environmental depth information in real time and construct a 2D polar coordinate obstacle map; Binocular vision camera, used to collect RGB images of the environment in real time, generate disparity maps through the SGBM algorithm and convert them into 3D point clouds; Ultrasonic sensor array, used to obtain real-time distance information of close-range obstacles and detect step-like obstacles with a height of less than 20 cm; The IMU module is used to obtain real-time robot posture data. By reconstructing the environmental perception and decision-making architecture of the walking-assistant robot, the multi-sensor spatiotemporal coordination mechanism uses cross-modal data matching between lidar and binocular vision, combined with the ultrasonic sensor's specialized low-obstacle detection, to significantly improve step recognition. In low-light environments, the detection distance for biological obstacles is extended from 1.5 meters to 3.5 meters. S102: Based on the dynamic obstacle sequence map, a Kalman filter and LSTM neural network fusion model is used to predict the motion trajectory of dynamic obstacles, and the obstacle velocity vector and acceleration are estimated in real time. The LSTM network takes time-series position data as input and outputs a set of trajectory prediction points within a preset time in the future. A kinematic model is constructed for pedestrian and vehicle obstacles to constrain the predicted direction error. Among them, based on the dynamic obstacle sequence map, the Kalman filter and LSTM neural network fusion model is used to predict the motion trajectory of dynamic obstacles and estimate the obstacle velocity vector and acceleration in real time, including: Based on the dynamic obstacle sequence map generated by the fusion of LiDAR and binocular vision, more than 30 consecutive frames of obstacle position time series data are extracted. Unique kinematic models are matched to different obstacle types. For pedestrians, a social force enhancement model is used, while for vehicles, a constant acceleration constraint model is used. A two-stage dynamic estimation is performed through Kalman filtering. The initial motion state at the current moment is predicted based on the historical trajectory. The latest sensor observations are then integrated for error correction, and high-precision velocity vectors and acceleration values ​​are output in real time. The velocity and acceleration vectors output by the Kalman filter are combined with the robot's relative distance into a five-dimensional feature vector. This is then fed into a two-layer LSTM neural network along with the logical relationship data of the prediction results. The dynamic obstacle intelligent prediction system, for the first time, combines the LSTM's long-term learning capabilities with the Kalman filter's real-time correction capabilities to accurately predict the trajectory of pedestrians and vehicles. The prediction results are converted into a spatiotemporal probability heat map to drive path planning, improving the success rate of dynamic obstacle avoidance. S103: Generate an initial global path with the current position of the walking robot as the starting point and the user-set target point as the end point. When the obstacle in the dynamic obstacle sequence map intrudes into the safety threshold, start local path optimization, introduce the spatiotemporal cost function of the dynamic obstacle prediction trajectory, solve the minimum cost path through the gradient descent method, and plan a continuous spline curve as the obstacle avoidance path in real time; Among them, local path optimization is started, the spatiotemporal cost function of the dynamic obstacle prediction trajectory is introduced, and the path with the minimum cost is solved by the gradient descent method, including: When a dynamic obstacle intrudes into the safety zone with a radius of 0.8 meters centered on the robot, the initial global path is immediately interrupted. The spatiotemporal intrusion risk value is calculated based on the predicted obstacle trajectory. Obstacles with a moving speed greater than 1m / s are weighted twice as much risk, and static obstacles are treated as having a basic risk value. This initiates a real-time obstacle avoidance response cycle. A spatiotemporal cost function including trajectory prediction probability is constructed, path curvature constraints are set, and path point coordinates are iteratively optimized through gradient descent to generate C²-continuous cubic B-spline curves. Each curve contains 7 control points, and tangent continuous transition segments are inserted at the inflection points to ensure that the steering angle change rate of the wheeled chassis is ≤15° / s. S104: Generate speed instructions for the robot's wheeled chassis based on the planned obstacle avoidance path. Use a force sensor array to detect the direction and magnitude of the user's thrust. When the user applies lateral force, the control model adjusts the movement direction. Simultaneously, vibration tactile feedback is used to convey obstacle avoidance direction prompts to the user, forming a closed-loop human-machine collaborative control system. The force sensor array detects the direction and magnitude of the user's thrust. When the user applies lateral force, the control model adjusts the direction of movement. Vibration tactile feedback is used to provide the user with obstacle avoidance direction prompts, including: The six-dimensional force sensor array distributed on the left and right armrests detects the user's thrust in real time. When the lateral thrust lasts >15N for more than 0.5 seconds, the impedance control model is activated. The steering angle compensation Δω is calculated based on the thrust direction angle θ. The compensation gain increases linearly from 0.1 to 0.5 as the thrust increases, while the maximum steering angle deviation is limited to ≤30°. Tactile feedback generates differentiated vibration codes based on the obstacle avoidance direction. Left obstacle avoidance triggers three short vibrations, while right obstacle avoidance triggers two long vibrations. The vibration intensity is inversely correlated with the distance to the obstacle. An orange arrow is simultaneously displayed on the navigation interface to indicate the obstacle avoidance direction, forming a visual-tactile dual-channel guidance system. S105: When a sudden obstacle is detected and the third-level braking mechanism is triggered, the IMU angular velocity deviation Δω is continuously monitored during the braking process. If |Δω| ≥ 0.5 rad / s², an instability risk is determined, and the nearest safe state point is immediately called to fall back to a stable posture, and a warning message is broadcast through the voice module; Among them, when a sudden obstacle is detected, the third-level braking mechanism is triggered. The IMU angular velocity deviation Δω is continuously monitored during the braking process. If |Δω| ≥ 0.5rad / s², the instability risk is determined, including: The criteria for sudden obstacles are: relative distance ≤ 0.3m or approach speed ≥ 1m / s, triggering three-level braking: Level 1: Motor torque PID control is reduced to 50%; Second level: electromagnetic friction plate energized braking; Level 3: reverse compensation current is injected into the motor; Acceleration gradient control during braking: 0-0.3s to -0.5m / s², 0.3-0.8s to -1.2m / s²; The IMU's three-axis angular velocity deviation is monitored in real time. When |Δω| ≥ 0.5 rad / s² for 100ms, instability is determined. At this time, a safe state point within the last 5 seconds is extracted from the circular buffer. The safe state point includes posture, velocity, and map data. The robot is controlled to retreat to this point along the original path. The retreat speed is limited to 40% of the normal value. The path generation quality has achieved a generational leap. The improved A* algorithm introduces an obstacle density heuristic function, shortening the global path length on average. Local obstacle avoidance uses C² continuous B-spline optimization to eliminate the curvature mutation problem of traditional broken line paths. The steering jitter of the wheeled chassis is reduced by 90%, and the risk of user dizziness is reduced by 60%.

[0021] In S101 of this embodiment, timestamp synchronization and a coordinate system conversion matrix are used to unify the multi-source data into the base coordinate system of the walking robot, generating a fused dynamic obstacle sequence map, including: Establish a Euclidean distance clustering model for lidar point clouds and segment independent obstacle clusters using the DBSCAN algorithm; The binocular vision point cloud is matched with the center of the lidar cluster through KD-Tree accelerated nearest neighbor search. Successfully matched obstacles are assigned ID tags. When the ultrasonic data detects an obstacle at the same location in three consecutive frames, the map is updated. The sensor confidence weights are introduced into the fusion process, and the final obstacle position estimate is a weighted least squares solution.

[0022] In S102 of this embodiment, the two-layer LSTM neural network and the prediction result logical relationship data are input, including: The long-term temporal dependencies of obstacle motion are learned through the memory gate mechanism, which is particularly optimized for modeling nonlinear motions such as random turns of pedestrians and sudden stops of pets. Output the trajectory prediction position of 5 consecutive time points within the next 1 second, add a probability confidence to each point, and convert the prediction result into a spatiotemporal probability heat map; On the spatiotemporal probability heat map, with the current position of the obstacle as the center, a radial probability distribution cloud map is generated along the predicted direction. High-risk areas are marked with a red gradient and superimposed on the navigation interface.

[0023] This technical solution has achieved outstanding results in the dimensions of human-computer interaction and safety protection. The personalized safety model breaks through the limitations of fixed thresholds and dynamically adjusts the safety radius based on the user's grip strength and electromyographic signals, reducing the risk of collision. Secondly, the human-computer interaction conflict is resolved by optimizing the impedance parameters in real time through reinforcement learning, compressing the deviation between the operating lateral thrust and the planned path to ≤5°, and suppressing the peak human-computer force from 40-60N to below 25N, thereby improving the smoothness of operation. In addition, the multi-level fault-tolerant control architecture rewrites the emergency braking standard. The three-level braking mechanism shortens the emergency stop distance at a speed of 1m / s from 1.2 meters to 0.7 meters. Combined with the IMU angular velocity real-time monitoring and state rollback function, the risk of users falling during emergency stops is reduced. The night mode maintains the ability to recognize biological obstacles at 3.5 meters in moonlight through infrared-thermal imaging fusion, truly realizing full-scene safety protection.

[0024] Reference Figure 3-4As shown, a multi-sensor based obstacle avoidance control system for a walking-assisting robot is used for the above-mentioned obstacle avoidance control method for a walking-assisting robot. The obstacle avoidance control system for the walking-assisting robot includes: a multimodal perception module for collecting environmental data through lidar, binocular vision camera, ultrasonic sensor array and IMU, and outputting a fused dynamic obstacle sequence map; a trajectory prediction module for estimating the obstacle velocity vector and acceleration in real time based on a Kalman filter and LSTM neural network fusion model, and generating a trajectory probability heat map within the next 1 second; a path planning module for generating an initial global path with the current position of the robot as the starting point and the user's target point as the end point, starting local optimization when the obstacle intrudes into the safety threshold, and outputting a C² continuous cubic B-spline obstacle avoidance path; a human-machine collaborative control module for generating chassis motion instructions according to the planned path, through The force sensor detects the user's thrust and adjusts the direction, and the synchronous tactile feedback module transmits obstacle avoidance prompts; the safety fault-tolerant module is used to trigger the third-level braking in the event of a sudden obstacle, monitor the IMU angular velocity deviation and return to a safe state point, and broadcast warning information through the voice module. By reconstructing the environmental perception and decision-making architecture of the walking-assist robot, the multi-sensor spatiotemporal coordination mechanism greatly improves the step recognition rate through cross-modal data matching of lidar and binocular vision, combined with the special detection of low obstacles by ultrasonic sensors, and the detection distance of biological obstacles in low-light environments is extended from 1.5 meters to 3.5 meters. Secondly, the dynamic obstacle intelligent prediction system integrates the LSTM long-term learning capability with the real-time correction of the Kalman filter for the first time, accurately predicts the running trajectory of pedestrians / vehicles, and converts the prediction results into a spatiotemporal probability heat map to drive path planning, thereby improving the success rate of dynamic obstacle avoidance.

[0025] In this embodiment, the trajectory prediction module and the path planning module are both deployed on a heterogeneous computing platform. The heterogeneous computing platform includes: an FPGA acceleration unit for real-time execution of Kalman filtering and spatiotemporal alignment of sensor data, and a pipeline parallel architecture for processing lidar point clouds; an NPU inference unit for loading the quantized LSTM trajectory prediction model and supporting output of prediction heat maps at a frame rate of more than 20Hz; a CPU decision unit for running an efficient heuristic search algorithm and gradient descent path optimization for improved path planning, and realizing the planning cycle through ROS node scheduling, achieving outstanding results in the dimensions of human-computer interaction and safety protection. The personalized safety model breaks through the limitations of fixed thresholds and dynamically adjusts the safety radius based on the user's grip strength and electromyographic signals, reducing the risk of collision. Secondly, the human-machine force interaction conflict is resolved by optimizing the impedance parameters in real time through reinforcement learning, compressing the deviation between the operating lateral thrust and the planned path to ≤5°, and the peak human-machine force is suppressed from 40-60N to below 25N, improving the smoothness of operation. In addition, the multi-level fault-tolerant control architecture rewrites the emergency braking standard. The three-level braking mechanism shortens the emergency stop distance at a speed of 1m / s from 1.2 meters to 0.7 meters. Combined with the IMU angular velocity real-time monitoring and state rollback function, the risk of users falling during emergency stops is reduced.

[0026] This technical solution achieves a generational leap in path generation quality. The improved A* algorithm introduces an obstacle density heuristic function, shortening the average global path length. Local obstacle avoidance uses C² continuous B-spline optimization to eliminate the curvature mutation problem of traditional broken line paths. The steering jitter of the wheeled chassis is reduced by 90%, and the risk of user dizziness is reduced by 60%. At the same time, the night mode maintains the ability to recognize biological obstacles within 3.5 meters in moonlight environments through infrared and thermal imaging fusion, truly realizing full-scene safety protection.

[0027] In this embodiment, a C²-continuous cubic B-spline obstacle avoidance path is output. A B-spline curve is a parameterized curve consisting of control points and basis functions. Compared with simple broken lines or arcs, B-splines can generate complex and smooth paths, and local adjustments do not affect the global shape.

[0028] The curve is composed of third-order polynomials (the highest degree is 3). Cubic curves are the gold standard for motion control: they can express rich shapes while avoiding high-order computational complexity.

[0029] C² continuous description of the smoothness characteristics of the curve connection is: C 0 Continuous: The curve is continuous at the connection point; C¹ continuous: The curve is continuous in the tangent direction at the connection point; C² continuous: the curvature of the curve is continuous at the connection points; The A* algorithm is an efficient heuristic search algorithm widely used in path planning, especially for global path planning of walking-assist robots. It mainly uses technical principles and innovative applications in obstacle avoidance systems to analyze the cost function: f(n)=g(n)+h(n), where g(n) is the actual cost (distance, time, etc.) from the starting point to the node n, and h(n) is the heuristic estimated cost from the node n to the end point.

[0030] In this embodiment, the entire operation process can be controlled by a computer to provide signal feedback to implement the steps in sequence. These are all conventional knowledge of current automated control and will not be described in detail in this embodiment.

[0031] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-sensor based obstacle avoidance control method for a walking-assist robot, characterized in that: Applied to walking-assist robot equipment, specifically including the following steps: S101: Complete the installation of the multi-sensor module on the walking robot, collect multi-source data obtained by the multi-sensors in real time, use timestamp synchronization and coordinate system conversion matrix to unify the multi-source data into the base coordinate system of the walking robot, and generate a fused dynamic obstacle sequence map; S102: Based on the dynamic obstacle sequence map, a Kalman filter and LSTM neural network fusion model is used to predict the motion trajectory of dynamic obstacles, and the obstacle velocity vector and acceleration are estimated in real time. The LSTM network takes time-series position data as input and outputs a set of trajectory prediction points within a preset time in the future. A kinematic model is constructed for pedestrian and vehicle obstacles to constrain the predicted direction error. S103: Generate an initial global path with the current position of the walking robot as the starting point and the user-set target point as the end point. When the obstacle in the dynamic obstacle sequence map intrudes into the safety threshold, start local path optimization, introduce the spatiotemporal cost function of the dynamic obstacle prediction trajectory, solve the minimum cost path through the gradient descent method, and plan a continuous spline curve as the obstacle avoidance path in real time; S104: Generate speed instructions for the robot's wheeled chassis based on the planned obstacle avoidance path. Use a force sensor array to detect the direction and magnitude of the user's thrust. When the user applies lateral force, the control model adjusts the movement direction. Simultaneously, vibration tactile feedback is used to convey obstacle avoidance direction prompts to the user, forming a closed-loop human-machine collaborative control system. S105: When a sudden obstacle is detected and the third-level braking mechanism is triggered, the IMU angular velocity deviation Δω is continuously monitored during the braking process. If |Δω| ≥ 0.5 rad / s², the risk of instability is determined, and the nearest safe state point is immediately called to fall back to a stable posture, and a warning message is broadcast through the voice module.

2. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 1, characterized in that: In S101, a multi-sensor module is installed on the walking-assistance robot, wherein the multi-sensor module includes: LiDAR, used to collect environmental depth information in real time and construct a 2D polar coordinate obstacle map; Binocular vision camera, used to collect RGB images of the environment in real time, generate disparity maps through the SGBM algorithm and convert them into 3D point clouds; Ultrasonic sensor array, used to obtain real-time distance information of close-range obstacles and detect step-like obstacles with a height of less than 20 cm; IMU module, used to obtain robot posture data information in real time.

3. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 2, characterized in that: Timestamp synchronization and coordinate system conversion matrix are used to unify multi-source data into the base coordinate system of the mobility assist robot, generating a fused dynamic obstacle sequence map, including: Establish a Euclidean distance clustering model for lidar point clouds and segment independent obstacle clusters using the DBSCAN algorithm; The binocular vision point cloud is matched with the center of the lidar cluster through KD-Tree accelerated nearest neighbor search. Successfully matched obstacles are assigned ID tags. When the ultrasonic data detects an obstacle at the same location in three consecutive frames, the map is updated. The sensor confidence weights are introduced into the fusion process, and the final obstacle position estimate is a weighted least squares solution.

4. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 3, characterized in that: In S102, based on the dynamic obstacle sequence map, a Kalman filter and LSTM neural network fusion model is used to predict the motion trajectory of dynamic obstacles and estimate the obstacle velocity vector and acceleration in real time, including: Based on the dynamic obstacle sequence map generated by the fusion of LiDAR and binocular vision, more than 30 consecutive frames of obstacle position time series data are extracted. Unique kinematic models are matched to different obstacle types. For pedestrians, a social force enhancement model is used, while for vehicles, a constant acceleration constraint model is used. A two-stage dynamic estimation is performed through Kalman filtering. The initial motion state at the current moment is predicted based on the historical trajectory. The latest sensor observations are then integrated for error correction, and high-precision velocity vectors and acceleration values ​​are output in real time. The velocity and acceleration vectors output by the Kalman filter are fused with the relative distance of the robot into a five-dimensional feature vector, which is then input into a two-layer LSTM neural network and the logical relationship data of the prediction results.

5. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 4, characterized in that: Input the two-layer LSTM neural network and the logical relationship data of the prediction results, including: The long-term temporal dependencies of obstacle motion are learned through the memory gate mechanism, which is particularly optimized for modeling nonlinear motions such as random turns of pedestrians and sudden stops of pets. Output the trajectory prediction position of 5 consecutive time points within the next 1 second, add a probability confidence to each point, and convert the prediction result into a spatiotemporal probability heat map; On the spatiotemporal probability heat map, with the current position of the obstacle as the center, a radial probability distribution cloud map is generated along the predicted direction. High-risk areas are marked with a red gradient and superimposed on the navigation interface.

6. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 5, characterized in that: In S103, local path optimization is started, a spatiotemporal cost function of the dynamic obstacle prediction trajectory is introduced, and the path with the minimum cost is solved by the gradient descent method, including: When a dynamic obstacle intrudes into the safety zone with a radius of 0.8 meters centered on the robot, the initial global path is immediately interrupted. The spatiotemporal intrusion risk value is calculated based on the predicted obstacle trajectory. Obstacles with a moving speed greater than 1m / s are weighted twice as much risk, and static obstacles are treated as having a basic risk value. This initiates a real-time obstacle avoidance response cycle. A spatiotemporal cost function that includes trajectory prediction probability is constructed, and a path curvature constraint is set. The path point coordinates are iteratively optimized through gradient descent to generate C²-continuous cubic B-spline curves. Each curve contains seven control points, and tangent-continuous transition segments are inserted at the inflection points to ensure that the steering angle change rate of the wheeled chassis is ≤15° / s.

7. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 6, characterized in that: In S104, the force sensor array detects the direction and magnitude of the user's thrust. When the user applies lateral force, the control model adjusts the movement direction and simultaneously transmits obstacle avoidance direction prompts to the user through vibration tactile feedback, including: The six-dimensional force sensor array distributed on the left and right armrests detects the user's thrust in real time. When the lateral thrust lasts >15N for more than 0.5 seconds, the impedance control model is activated. The steering angle compensation Δω is calculated based on the thrust direction angle θ. The compensation gain increases linearly from 0.1 to 0.5 as the thrust increases, while the maximum steering angle deviation is limited to ≤30°. Tactile feedback generates differentiated vibration codes based on the obstacle avoidance direction. Left obstacle avoidance triggers three short vibrations, and right obstacle avoidance triggers two long vibrations. The vibration intensity is inversely correlated with the distance to the obstacle. An orange arrow is simultaneously displayed on the navigation interface to indicate the obstacle avoidance direction, forming a visual-tactile dual-channel guidance.

8. The multi-sensor based obstacle avoidance control method for a walking-assistance robot according to claim 7, characterized in that: In S105, when a sudden obstacle is detected, the third-level braking mechanism is triggered. During the braking process, the IMU angular velocity deviation Δω is continuously monitored. If |Δω| ≥ 0.5 rad / s², the instability risk is determined, including: The criteria for sudden obstacles are: relative distance ≤ 0.3m or approach speed ≥ 1m / s, triggering three-level braking: Level 1: Motor torque PID control is reduced to 50%; Second level: electromagnetic friction plate energized braking; Level 3: reverse compensation current is injected into the motor; Acceleration gradient control during braking: 0-0.3s to -0.5m / s², 0.3-0.8s to -1.2m / s²; The IMU's three-axis angular velocity deviation is monitored in real time. When |Δω| ≥ 0.5 rad / s² for 100 ms, instability is determined. At this time, a safe state point within the last 5 seconds is extracted from the ring buffer. The safe state point includes posture, velocity, and map data. The robot is controlled to retreat to this point along the original path, with the retreat speed limited to 40% of the normal value.

9. A multi-sensor based obstacle avoidance control system for a walking-assist robot, characterized in that: Used to execute the walking-assist robot obstacle avoidance control method according to any one of claims 1 to 8, the walking-assist robot obstacle avoidance control system comprising: The multimodal perception module is used to collect environmental data through lidar, binocular vision camera, ultrasonic sensor array and IMU, and output a fused dynamic obstacle sequence map; The trajectory prediction module is used to estimate the obstacle velocity vector and acceleration in real time based on the Kalman filter and LSTM neural network fusion model, and generate a trajectory probability heat map within the next 1 second; The path planning module is used to generate an initial global path starting from the robot's current position and ending at the user's target point. When an obstacle intrudes into the safety threshold, local optimization is initiated and a C²-continuous cubic B-spline obstacle avoidance path is output. The human-machine collaborative control module generates chassis motion instructions based on the planned path, detects user thrust through force sensors and adjusts direction, and synchronizes the tactile feedback module to deliver obstacle avoidance prompts. The safety fault-tolerant module is used to trigger level 3 braking when encountering sudden obstacles, monitor IMU angular velocity deviations, and return to a safe state point, and broadcast warning information through the voice module.

10. The multi-sensor based obstacle avoidance control system for a walking-assistance robot according to claim 9, characterized in that: The trajectory prediction module and the path planning module are both deployed on a heterogeneous computing platform, which includes: FPGA acceleration unit for real-time Kalman filtering and spatiotemporal alignment of sensor data, configured with a pipelined parallel architecture to process lidar point clouds; The NPU inference unit is used to load the quantized LSTM trajectory prediction model and output prediction heat maps at a frame rate of 20Hz or higher. CPU decision unit: used to run efficient heuristic search algorithms and gradient descent path optimization for improved path planning, and implement planning cycles through ROS node scheduling.

Images