System for an intelligent wheelchair with multimodal user control and obstacle avoidance

Abstract

A system for intelligent wheelchairs with multimodal user control and obstacle avoidance, comprising: a wheelchair chassis with a seat assembly, multiple drive wheels, and at least one steering wheel; a central processing unit mounted on the wheelchair chassis, comprising at least one processor and memory for storing executable instructions; a motor control processor communicatively connected to the central processing unit; a joystick input unit positioned next to the seat assembly and configured to generate directional navigation signals corresponding to the deflection of a manually operated joystick; a speech capture unit with at least one microphone configured to capture spoken commands from a user and transmit corresponding digital audio information to the central processing unit; and an image capture unit positioned tothat it continuously captures facial images of the user and transmits sequential image frames to the central processing unit; an obstacle detection unit consisting of a multitude of distance sensors distributed around the wheelchair chassis and configured to continuously generate distance measurements to objects in the environment; a communication unit that establishes bidirectional communication between the central unit, the motor control processor, the joystick input unit, the speech recognition unit, the image capture unit, and the obstacle detection unit; a motor driver unit electrically connected to the motor control processor; a multitude of electric motors mechanically connected to the drive wheels and configured to receive control signals from the motor driver unit; a power distribution unit configured to supply the central processing unit,the motor control processor, the joystick input unit, the speech capture unit, the image capture unit, the obstacle detection unit, the communication unit, the motor driver unit, and the multitude of electric drive motors with regulated electrical energy; and a user mode selection unit configured to selectively activate either a joystick navigation mode, a voice navigation mode, or a gaze-controlled navigation mode, wherein the central processing unit is configured to receive user navigation information from the activated navigation mode, receive distance measurements from the obstacle detection unit, determine whether a requested direction of movement intersects an obstacle within a predetermined safety distance, selectively inhibit movement in the direction of the detected obstacle while allowing movement in one or more remaining safe directions.validated movement instructions are transmitted via the communication unit to the motor control processor, and the motor control processor generates motor control signals to independently control the direction and speed of rotation of the multiple electric drive motors for wheelchair navigation.