Automatic obstacle avoidance and navigation system of intelligent wheelchair

Abstract

The invention discloses an automatic obstacle avoidance and navigation system of an intelligent wheelchair, which relates to the technical field of intelligent wheelchairs and comprises a sensing unit, a processing unit, an execution unit and a man-machine interaction unit. The sensing unit transmits the acquired data to the processing unit and transmits the processed data to the execution unit; the execution unit transmits the received information to the man-machine interaction unit; the sensing unit comprises a laser radar assembly; the processing unit comprises a multi-sensor fusion module, a dynamic obstacle avoidance decision-making module and a global path planning module, and the multi-sensor fusion module carries out space-time synchronization, point cloud-image fusion and semantic segmentation processing and transmits processed data to the dynamic obstacle avoidance decision-making module to predict an obstacle movement track; and the predicted data is transmitted to a global path planning module for path planning. According to the invention, accurate perception and humanized obstacle avoidance of static / dynamic obstacles in a complex environment can be realized, and the path planning efficiency and the user experience are optimized.

Description

Technical Field

[0001] This invention belongs to the field of intelligent wheelchair technology, specifically relating to an automatic obstacle avoidance and navigation system for an intelligent wheelchair. Background Technology

[0002] Intelligent wheelchairs, also known as robotic wheelchairs, are assistive mobile devices that integrate visual navigation, voice interaction, and autonomous obstacle avoidance technologies. Their core functions include command recognition, dynamic environmental perception, multi-sensor fusion, and adaptive navigation control, aiming to enhance the autonomous mobility of people with mobility impairments through technological integration.

[0003] However, traditional smart wheelchairs are mainly controlled by operating buttons to move forward, backward, turn, and stop. When users with mobility impairments or who fail to notice obstacles in time, the wheelchair can easily collide with the obstacle, causing damage or accidents.

[0004] Current intelligent wheelchair obstacle avoidance and navigation technologies still have at least the following shortcomings: 1. Limited sensing range: Ultrasonic sensors have a short detection distance (usually <5 meters) and are easily affected by environmental interference (such as reflection from smooth surfaces); infrared sensors are easily affected by light.

[0005] 2. Low navigation accuracy: It lacks high-precision environmental modeling capabilities, making it difficult to plan the optimal path in complex dynamic scenarios (such as densely populated areas).

[0006] 3. Insufficient real-time performance: Traditional algorithms (such as simple threshold judgment) cannot efficiently handle the dynamic changes of multiple obstacles, resulting in delays in obstacle avoidance response.

[0007] 4. Limited functionality: Most systems only support basic obstacle avoidance and lack semantic understanding capabilities (such as recognizing special obstacles such as steps and door frames).

[0008] For example, Chinese invention patent CN119991738A discloses a real-time obstacle prediction and avoidance system for wheelchairs based on multi-target tracking. Its technical solution includes: a data acquisition and processing module for calculating the distance between obstacles and a reference point; an obstacle detection and tracking module for generating suspected obstacle regions based on multi-dimensional features; filtering out determined regions, then calculating the similarity of suspected dynamic obstacles within the determined regions, filtering out dynamic obstacles based on similarity, and filtering out determined obstacles based on confidence scores; a decision control module for determining optional paths based on a path evaluation function; and then using a trajectory evaluation and selection function to score the trajectory corresponding to the path planning and select the trajectory with the highest stability.

[0009] Therefore, it is necessary to design an automatic obstacle avoidance and navigation system for intelligent wheelchairs to solve some of the existing problems. Summary of the Invention

[0010] This invention addresses the problems existing in the prior art by providing an automatic obstacle avoidance and navigation system for intelligent wheelchairs.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An automatic obstacle avoidance and navigation system for an intelligent wheelchair includes a sensing unit, a processing unit, an execution unit, and a human-machine interaction unit. The sensing unit transmits acquired data to the processing unit and the processed data to the execution unit. The execution unit transmits received information to the human-machine interaction unit. The sensing unit includes a lidar component that measures the positions of the lowest and highest obstacles. The processing unit includes a multi-sensor fusion module, a dynamic obstacle avoidance decision module, and a global path planning module. The multi-sensor fusion module performs spatiotemporal synchronization, point cloud-image fusion, and semantic segmentation processing, and transmits the processed data to the dynamic obstacle avoidance decision module for obstacle trajectory prediction. The predicted data is then transmitted to the global path planning module for path planning.

[0012] Based on the above technical solution, the lidar component further includes a first lidar and a second lidar, and the first lidar and the second lidar are installed at different height positions of the intelligent wheelchair.

[0013] Based on the above technical solution, the sensing unit further includes a binocular camera, which is installed on the top of the backrest of the smart wheelchair.

[0014] Based on the above technical solution, the sensing unit further includes an inertial measurement unit, which is embedded inside the chair back.

[0015] Based on the above technical solution, the multi-sensor fusion module further performs point cloud-image fusion processing; the process is: P=a×P1+(1-a)×P2, where P is the fused point cloud data, P1 is the sum of the total point cloud data of the first lidar and the second lidar, P2 is the binocular visual point cloud data, a is the weight coefficient, and the value range of a is [0.4, 0.8].

[0016] Based on the above technical solution, the dynamic obstacle avoidance decision module further includes an interactive multi-model Kalman filter capable of predicting the trajectory of dynamic obstacles. The prediction process is as follows: X k =F k X k-1 +B k U k +W k , where X k F represents the system state vector, that is, the system state at time k; kX represents the transition matrix, used to describe the intrinsic dynamic characteristics of the system from time k-1 to k; k-1 B represents the state vector at the previous moment; k This represents the control input matrix, used to describe the external control input U. k Impact on system state; U k W represents the control input vector, which is the external control quantity at time k; k This represents the process noise vector.

[0017] Based on the above technical solution, the dynamic obstacle avoidance decision module further includes an improved DWA algorithm module, which is capable of processing cost function fusion safety margin, direction deviation, and velocity smoothness. The processing procedure is as follows: G(v, ω) = σ[α·(d1-d2) / d1+β·|θ1-θ2|+γ·(v1-v)], where G(v, ω) represents the path evaluation value, with a range of [0, 1]; v represents the linear velocity of the intelligent wheelchair, in m / s, where 0≤v≤v1; σ(·) represents the Sigmoid activation function, which maps the output to the (0, 1) interval; α represents the safety cost weight coefficient, preferably with a range of [0.5, 0.7]; β represents the direction cost weight coefficient, The preferred value range is [0.2, 0.4]; γ represents the speed cost weighting coefficient, with a preferred value range of [0.1, 0.2], and the priority of α, β, and γ decreases in that order; d1 represents the distance to the nearest obstacle, in meters; d2 represents the minimum safe distance, in meters; θ1 represents the target direction angle, in rads, with a value range of [-π, π]; θ2 represents the current direction angle, in rads, with a value range of [-π, π]; v1 represents the maximum speed allowed by the environment, in meters per second.

[0018] Based on the above technical solution, the global path planning module further includes an improved A* algorithm module, and the planning implementation process of the improved A* algorithm is as follows: f(n) = g(n) + h(n) + λR, where f(n) represents the total cost of the node, specifically the total cost of the path from the starting point to the ending point passing through node n; g(n) represents the actual cost, specifically the actual path cost from the starting point to the current node n; h(n) represents the heuristic cost, specifically the estimated cost from the current node n to the ending point; λ represents the semantic weight coefficient, and λ is greater than or equal to 0; R represents the semantic cost weight. Where, h(n) = [(x n -x 目标 ) 2 + (y n -y 目标 ) 2 ] 1 / 2In the formula, (x n y n (x) represents the current node's coordinates. 目标 y 目标 ) represents the coordinates of the target node on the path.

[0019] Based on the above technical solution, the execution unit further includes a motion control module and a safety mechanism module; wherein, the motion control module adjusts the rotation speed of the omnidirectional wheel through a PID controller, and the safety mechanism module automatically limits the speed according to the distance to the obstacle.

[0020] Based on the above technical solution, the human-computer interaction unit further includes a touch screen display module and a voice broadcast module; it performs real-time operation and displays corresponding data, while also providing voice reminders.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention provides an automatic obstacle avoidance and navigation system for intelligent wheelchairs, enabling accurate perception and human-centered obstacle avoidance of static / dynamic obstacles in complex environments, while optimizing path planning efficiency and user experience. It also achieves high-precision environmental modeling, real-time obstacle detection, and semantic recognition, combined with dynamic path planning algorithms to improve navigation safety and efficiency. Detailed Implementation

[0022] The present invention will be further described and illustrated below with reference to specific embodiments. The technical features of each embodiment of the present invention can be combined accordingly, provided that there is no mutual conflict.

[0023] To make the above-described objects, features, and advantages of the present invention more apparent and understandable, numerous specific details are set forth in the following description to provide a thorough understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the invention can be combined appropriately without conflict.

[0024] In the description of this invention, it should be understood that when an element is considered to be "connected" to another element, it can be a direct connection to the other element or an indirect connection, i.e., there is an intermediate element. Conversely, when an element is said to be "directly" connected to another element, there is no intermediate element.

[0025] Example This embodiment provides an automatic obstacle avoidance and navigation system for an intelligent wheelchair, which includes a sensing unit, a processing unit, an execution unit, and a human-computer interaction unit. The sensing unit transmits the acquired data to the processing unit for spatiotemporal synchronization, point cloud-image fusion, and semantic segmentation processing, and transmits the processed data to the execution unit. The execution unit controls the human-computer interaction unit to execute interactive commands. The sensing unit includes a lidar component, a binocular camera, and an inertial measurement unit (IMU). The lidar component, binocular camera, and inertial measurement unit are all independently installed on the smart wheelchair, and the data they acquire can be wirelessly transmitted to the processing unit. Specifically, the lidar component includes a first lidar and a second lidar, both of which can be selected as 16-line rotating lidars (such as the Livox Mid-70). This type of lidar can perform 360° omnidirectional rotation scanning in the horizontal direction (parallel to the ground) and has a 30° field of view in the vertical direction (perpendicular to the ground). Its detection range can reach 0.1m-70m, and its point cloud density is >100 points / ㎡. The first and second lidars are installed at different heights on the smart wheelchair to measure the positions of the lowest and highest obstacles. For example, the first lidar can be installed on the bottom of the smart wheelchair, which has omnidirectional wheels or similar structures—a standard setup that will not be elaborated upon. The second lidar is then installed above the first lidar, optionally on a structure where the user's arm rests. The monitoring by these two lidars can effectively determine whether there are obstacles in the path, and if so, can provide timely alerts to ensure user safety.

[0026] Preferably, the binocular camera has a baseline distance of 12cm, a resolution of 1920×1080@30fps, outputs RGB-D images, and supports OpenCV depth estimation; and the binocular camera is mounted on the top of the backrest of the smart wheelchair. The inertial measurement unit (IMU) has a 6-axis (accelerometer + gyroscope) structure and a sampling rate of 100Hz, which can provide attitude and motion data of the smart wheelchair; the inertial measurement unit is embedded inside the back of the chair. Furthermore, the processing unit includes a multi-sensor fusion module, a dynamic obstacle avoidance decision module, and a global path planning module. The multi-sensor fusion module can perform spatiotemporal synchronization, point cloud-image fusion, and semantic segmentation processing. Specifically, the spatiotemporal synchronization process is as follows: hardware triggering is used to synchronize the lidar component and the binocular camera data acquisition, with a timestamp error of <1ms; the hardware can be a main synchronization controller (FPGA), which can generate multiple synchronization signals. It is an existing component and will not be described in detail here. The point cloud-image fusion process is as follows: hardware (FPGA, etc.) is used to trigger the synchronous lidar component and the binocular camera to collect data, and the fused point cloud data is finally generated through a weighted fusion formula; The specific fusion process is as follows: P = a × P1 + (1-a) × P2, where P is the fused point cloud data, P1 is the sum of the total point cloud data from the first and second lidars, P2 is the binocular visual point cloud data, and a is the weighting coefficient, with a value ranging from [0.4, 0.8]. It should be noted that the point cloud data corresponding to P1 is the sum of the three-dimensional spatial point cloud data directly measured by the two lidars, while the point cloud data corresponding to P2 is the three-dimensional point cloud data generated by the binocular camera through depth estimation.

[0027] The semantic segmentation process is as follows: segmentation is achieved based on different obstacle types (people / vehicles / steps / doors / handrails, etc.) identified by the improved YOLOv7 model, with an accuracy rate of up to 95%.

[0028] The data processed by the multi-sensor fusion module is wirelessly transmitted to the dynamic obstacle avoidance decision module to predict the trajectory of the obstacle. The dynamic obstacle avoidance decision module includes an interactive multi-model (IMM) Kalman filter for predicting the trajectory of the dynamic obstacle; and an improved DWA algorithm module, which can perform cost function fusion, safety margin, orientation deviation and speed smoothness processing. The interactive multi-model (IMM) Kalman filter predicts the trajectory of dynamic targets (velocity ≥ 0.3 m / s), while targets with velocity < 0.3 m / s are considered static targets and do not require trajectory prediction. The specific dynamic prediction process is as follows: X k =F k X k-1 +B k U k +W k , where X k F represents the system state vector, that is, the system state at time k (such as the position and velocity of the obstacle); k X represents the transition matrix, used to describe the intrinsic dynamic characteristics of the system from time k-1 to k. It should be noted that intrinsic dynamic characteristics refer to the system's inherent dynamic behavior, determined by its physical structure, parameters, and internal mechanisms, and are unaffected by external inputs (or only by initial conditions); k-1 B represents the state vector at the previous moment; k This represents the control input matrix, used to describe the external control input U. k Impact on system state; U kW represents the control input vector, which is the external control quantity at time k (such as the acceleration / steering command of a smart wheelchair); k This represents the process noise vector, which signifies the inherent uncertainty of the system (such as environmental disturbances). Further, X can be understood as... k F represents the current state. k X k-1 B represents the natural evolution of the system. k U k Indicates external control influence, W k This indicates a random disturbance.

[0029] The specific processing procedure for this cost function, which integrates safety margin, direction deviation, and velocity smoothness, is as follows: G(v, ω) = σ[α·(d1-d2) / d1+β·|θ1-θ2|+γ·(v1-v)], where G(v, ω) represents the path evaluation value, ranging from [0, 1]. A larger value indicates a better path, and it is normalized by the Sigmoid activation function; v represents the linear velocity of the intelligent wheelchair, in m / s, where 0≤v≤v1; σ(·) represents the Sigmoid activation function, which maps the output to the (0, 1) interval; α represents the safety cost weight coefficient, preferably ranging from [0.5, 0.7]; β represents the direction cost weight coefficient, preferably ranging from [0.2, 0.4]; γ represents the speed cost weight coefficient, preferably ranging from [0.1, 0.2], and α, β, γ The priorities of the three decrease in the following order: d1 represents the distance to the nearest obstacle (the minimum distance measured in real time by the first and second lidars, and greater than 0), in meters; d2 represents the minimum safe distance, in meters, and a threshold can be set according to the actual situation, such as 0.8 meters; θ1 represents the target orientation angle, in rads, which is the ideal orientation angle given by global path planning, preferably [-π, π]; θ2 represents the current orientation angle, in rads, which is the orientation angle of the intelligent wheelchair measured in real time by the IMU, preferably [-π, π]; v1 represents the maximum speed allowed by the environment, in meters per second, which can be dynamically adjusted according to the actual situation, such as 1.5 m / s in open areas and 0.5 m / s in crowded areas.

[0030] In this embodiment, the data predicted by the dynamic obstacle avoidance decision module is transmitted to the global path planning module for path planning; the global path planning module includes an improved A* algorithm module, and the planning implementation process of the improved A* algorithm is as follows: f(n) = g(n) + h(n) + λR, where f(n) represents the total cost of the node, specifically the total cost of the path from the starting point to the ending point passing through node n (the smaller the value, the better the path); g(n) represents the actual cost, specifically the actual path cost from the starting point to the current node n (considering the movement constraints of the intelligent wheelchair, such as a turning radius limit of greater than 0.8m and a slope passability of less than 10 degrees); h(n) represents the heuristic cost, specifically the estimated cost from the current node n to the ending point (heuristic function); λ represents the semantic weight coefficient, which can adjust the influence of semantic information on path planning, and λ is greater than or equal to 0; R represents the semantic cost weight, specifically the semantic attribute penalty value of node n (such as the difficulty of passing obstacles such as steps and slopes). Furthermore, h(n) = [(x n -x 目标 ) 2 + (y n -y 目标 ) 2 ] 1 / 2 In the formula, (x n y n (x) represents the current node's coordinates. 目标 y 目标 () represents the coordinates of the target node along the path; Furthermore, regarding the design principles of R, for example, when the obstacle type is flat ground (i.e., the lowest first lidar does not detect any obstacle), R can be set to 0, meaning no resistance to passage; when the obstacle type is a step (i.e., the lowest first lidar detects an obstacle, but the higher second lidar does not), R can be set to 3, meaning passage is prohibited and the intelligent wheelchair cannot cross it; when the obstacle type is a doorway (i.e., the lowest first lidar detects an obstacle, and the higher second lidar also detects an obstacle), R can be set to 5, meaning there is a high risk of passage. It should be noted that this is not limited to the aforementioned obstacle types and can be set according to the actual situation.

[0031] It should be further explained that when encountering obstacles that pose a high risk of failure or are difficult to pass through, or when there is no feasible path, you can choose to detour or seek assistance from external forces to pass through. You can choose the appropriate method according to the actual situation.

[0032] In this embodiment, the execution unit includes a motion control module and a safety mechanism module. Specifically, the motion control module can adjust the rotation speed of the left and right omnidirectional wheels (accuracy ±0.1 rad / s) through a PID controller. The safety mechanism module can automatically limit the speed according to the distance to the obstacle; for example, when d1≤2m, v≤0.5m / s; when d1>2m, v≤3m / s. The safety mechanism module integrates an intelligent system of sensors and controllers. Its core objective is to avoid collisions by monitoring the environment in real time and dynamically adjusting the speed. The sensor can be an ultrasonic sensor, which emits ultrasonic waves and receives the echoes to calculate the flight time and then the distance. The controller can be a relay for basic speed limiting functions (such as "forced stop when distance <1m"). Furthermore, the execution unit transmits data to the human-computer interaction unit, which includes a touch screen display module and a voice broadcast module; it can perform real-time operations and display corresponding data, while providing voice prompts, such as broadcasting obstacle types and obstacle avoidance actions, such as "There is a pedestrian on the right, turning left to avoid them."

[0033] The main working principle of the system is as follows: Simultaneously acquire LiDAR point cloud, binocular RGB-D images, and IMU data; fuse multi-sensor data to generate a semantic grid map and label obstacle types; detect dynamic obstacles and predict their motion trajectory using IMM Kalman filtering; generate local obstacle avoidance paths in real time based on an improved DWA algorithm; use an improved A* algorithm to fuse semantic weights to plan a global path; convert path commands into motor control signals and execute motion control in a closed loop; display a real-time map on a touchscreen and provide voice navigation status updates.

[0034] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention do not depart from the essence and scope of the technical solution of the present invention.

Claims

1. An automatic obstacle avoidance and navigation system for a smart wheelchair, characterized in that, The perception unit, the processing unit, the execution unit and the human-computer interaction unit are included; The perception unit transmits the acquired data to the processing unit and transmits the processed data to the execution unit; The execution unit transmits the received information to the human-computer interaction unit; The perception unit includes a laser radar assembly, and the position of the lowest obstacle and the position of the highest obstacle are measured by the laser radar assembly; The processing unit includes a multi-sensor fusion module, a dynamic obstacle avoidance decision module and a global path planning module, wherein the multi-sensor fusion module performs time-space synchronization, point cloud-image fusion and semantic segmentation processing, and transmits the processed data to the dynamic obstacle avoidance decision module for prediction of the motion trajectory of the obstacle; the predicted data is transmitted to the global path planning module for path planning.

2. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The laser radar assembly includes a first laser radar and a second laser radar, and the first laser radar and the second laser radar are installed at different height positions of the intelligent wheelchair.

3. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The perception unit further includes a binocular camera, and the binocular camera is installed at the top of the backrest of the intelligent wheelchair.

4. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 3, wherein, The perception unit further includes an inertial measurement unit, and the inertial measurement unit is embedded inside the backrest.

5. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The multi-sensor fusion module performs point cloud-image fusion processing; the process is as follows: P = a * P1 + (1-a) * P2, wherein P is the fused point cloud data; P1 is the sum of the total point cloud data of the first laser radar and the second laser radar; P2 is the binocular vision point cloud data; a is a weight coefficient, and the value range of a is [0.4, 0.8].

6. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The dynamic obstacle avoidance decision module includes an interactive multiple model Kalman filter capable of predicting the trajectory of a dynamic obstacle, and the prediction process is as follows: X k =F k X k-1 +B k U k +W k where X k denotes the system state vector, i.e., the system state at time k; F k denotes the transition matrix, which describes the intrinsic dynamic characteristics of the system from time k-1 to k; X k-1 denotes the previous state vector; B k denotes the control input matrix, which describes the influence of external control input U k on the system state; U k denotes the control input vector, i.e., the external control quantity at time k; W k denotes the process noise vector.

7. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The dynamic obstacle avoidance decision module includes an improved DWA algorithm module, and the improved DWA algorithm module can perform cost function fusion safety margin, direction deviation and speed smoothness processing, and the processing process is as follows: G(v, ω) = σ [α·(d1-d2) / d1+β·|θ1-θ2|+γ·(v1-v)], wherein G(v, ω) represents the path evaluation value, and the value range is [0, 1]; v represents the linear speed of the intelligent wheelchair, and the unit is m / s, wherein 0≤v≤v1; σ(·) represents a Sigmoid activation function, which can map the output to the interval (0, 1); α represents a safety cost weight coefficient, and the preferred value range is [0.5, 0.7]; β represents a direction cost weight coefficient, and the preferred value range is [0.2, 0.4]; γ represents a speed cost weight coefficient, and the preferred value range is [0.1, 0.2], and the priority of α, β and γ decreases in turn; d1 represents the distance of the nearest obstacle, and the unit is m; d2 represents the minimum safety distance, and the unit is m; θ1 represents the target direction angle, and the unit is radian, and the value range is [-π, π]; θ2 represents the current direction angle, and the unit is radian, and the value range is [-π, π]; v1 represents the maximum speed allowed by the environment, and the unit is m / s.

8. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The global path planning module includes an improved A-star algorithm module, and the planning implementation process of the improved A-star algorithm is as follows: f(n) = g(n) + h(n) + λR, wherein, f(n) represents a total cost of a node, specifically, a total cost of a path only passing through the node n from a starting point to an ending point; g(n) represents an actual cost, specifically, an actual path cost from the starting point to the current node n; h(n) represents a heuristic cost, specifically, an estimated cost from the current node n to the ending point; λ represents a semantic weight coefficient, and λ is greater than or equal to 0; and R represents a semantic cost weight. wherein h(n) = [(x n -x 目标 ) 2 + (y n -y 目标 ) 2 ] 1 / 2 , wherein (x n , y n ) is the coordinate position of the current node, and (x 目标 , y 目标 ) is the coordinate position of the target node of the path.

9. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The execution unit comprises a motion control module and a safety mechanism module; wherein the motion control module adjusts the rotation speed of the universal wheel through a PID controller, and the safety mechanism module automatically limits the speed according to the distance of the obstacle.

10. The automatic obstacle avoidance and navigation system of an intelligent wheelchair according to claim 1, wherein, The human-computer interaction unit comprises a touch screen display module and a voice broadcast module; real-time operation is performed and corresponding data is displayed, and voice reminding is simultaneously performed.