Wheelchair system and method for following a companion

Abstract

Systems and methods of a wheelchair system having a companion control mode are disclosed.SOLUTION: The wheelchair system comprises a companion and a wheelchair. The wheelchair comprises one or more wheels, at least one actuator coupled to the one or more wheels, a processing unit, and a non-transitory processor-readable storage medium in communication with the processing unit. The non-transitory processor-readable storage medium includes one or more program instructions. The program instructions, when executed, cause the processing device to determine a companion, generate a companion profile based on the determined companion, determine a movement of the companion, and activate at least one actuator that drives one or more wheels to follow the movement of the companion. The operation of the at least one actuator for driving the one or more wheels to follow the movement of the companion is an autonomous control.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of and priority to U.S. Patent Application No. 17 / 091,599, filed November 6, 2020, and entitled "Wheelchair System And Methods To Follow A Companion," the contents of which are incorporated herein by reference in their entirety.

[0002] The present disclosure relates generally to wheelchairs, and more particularly to a wheelchair system that enables autonomous motion control when a companion is traveling along the side of the wheelchair. [Background technology]

[0003] Wheelchairs, such as power wheelchairs, allow people who cannot walk independently to move from one place to another. One type of wheelchair includes a power wheelchair that has one or more motors. The motors allow a user to control the power wheelchair so that the power wheelchair moves under power control. Typically, a user uses a joystick or other user input device to control the power wheelchair's motors and move through an environment. Some power wheelchairs have a "follow" function that uses sensors to autonomously control the power wheelchair and follow another person in front of or behind the power wheelchair. However, for power wheelchair users and accompanying companions, the follow mode can make it difficult to carry on a conversation because the accompanying companion must be in front of or behind the power wheelchair.

[0004] Therefore, there is a need for an alternative power wheelchair that allows an accompanying companion to be positioned to the side of the wheelchair. Summary of the Invention

[0005] In one embodiment, a wheelchair system includes a companion and a wheelchair. The wheelchair includes one or more wheels, at least one actuator coupled to the one or more wheels, a processing device, and a processor-readable storage medium in communication with the processing device. The processor-readable storage medium includes one or more programming instructions that, when executed, cause the processing device to generate a companion profile based on a plurality of determined identifying characteristics of the companion, determine a movement of the companion, and activate the at least one actuator to drive the one or more wheels to follow the movement of the companion. The activation of the at least one actuator to drive the one or more wheels to follow the movement of the companion is autonomous control.

[0006] In another embodiment, a wheelchair system includes a companion and a wheelchair. The wheelchair includes a frame, one or more wheels coupled to the frame, at least one actuator coupled to the one or more wheels, and a control unit. The control unit controls the wheelchair between a standard mode and a companion-controlled mode, such that in the companion-controlled mode, the control unit activates the at least one actuator to drive the one or more wheels to follow the movement of the companion. The activation of the at least one actuator to drive the one or more wheels to follow the movement of the companion is autonomous control.

[0007] In yet another embodiment, a method of operating a wheelchair includes receiving, by a control unit, an input corresponding to a request for a companion control mode from a user; determining, by the control unit, a companion; determining, by the control unit, a movement of the companion; and actuating, by the control unit, at least one actuator to drive one or more wheels of the wheelchair to follow the movement of the companion.

[0008] The above-mentioned objects and advantages, as well as additional objects and advantages provided by the embodiments described herein, will be more fully understood from a consideration of the following detailed description in conjunction with the drawings. [Brief explanation of the drawings]

[0009] The embodiments illustrated in the drawings are exemplary and explanatory in nature and are not intended to limit the subject matter defined by the claims. The following detailed description of exemplary embodiments can be understood when read in conjunction with the following drawings, in which like structure is designated with like reference numerals. [Figure 1] FIG. 1 schematically illustrates an exemplary wheelchair system in a companion following mode according to one or more embodiments described and illustrated herein. [Figure 2] FIG. 2 schematically illustrates a wheelchair of the wheelchair system of FIG. 1 according to one or more embodiments described and illustrated herein. [Figure 3A] FIG. 3A schematically illustrates components of an exemplary control unit of an exemplary wheelchair system according to one or more embodiments described and illustrated herein. [Figure 3B] FIG. 3B schematically illustrates logic modules of an exemplary memory element of the exemplary wheelchair system of FIG. 3A, according to one or more embodiments described and illustrated herein. [Figure 3C] FIG. 3C schematically illustrates exemplary data stored in an exemplary data storage device of the exemplary wheelchair system of FIG. 3A, according to one or more embodiments described and illustrated herein. [Figure 4] FIG. 4 diagrammatically depicts a flowchart of an exemplary method for a companion control mode, according to one or more embodiments described and illustrated herein. [Figure 5] FIG. 5 diagrammatically depicts a flowchart of an exemplary method for generating a companion profile according to one or more embodiments described and illustrated herein. DETAILED DESCRIPTION OF THE INVENTION

[0010] The systems and methods described herein generally relate to a power wheelchair with a companion following mode. The companion following mode enables autonomous motor control to follow a companion moving along the side of the power wheelchair. In this manner, the power wheelchair autonomously mimics or follows the companion's movement along the side of the power wheelchair. Furthermore, the companion may be identified via a companion profile. The companion profile may include data related to the companion's pants color, facial recognition of the companion, and / or walking cadence of the companion. The companion profile may thereby identify the proposed companion to be followed. Additionally, any background noise surrounding the companion may be filtered out, allowing the system to focus solely on the determined companion's movements. The power wheelchair may also include multiple sensors to detect undesirable objects in front of the power wheelchair, and automatically exit the companion following mode if contact with the undesirable object is unavoidable.

[0011] Current power wheelchairs do not include a companion following mode that can be activated to follow a companion along the side of the power wheelchair, which can make it difficult for a user to converse with a companion. As a non-limiting example, a user may want to interact with a companion at a supermarket. With current power wheelchairs, the companion would have to walk behind and / or in front of the current power wheelchair. Others may not be able to identify the companion based on facial recognition, gait, and / or clothing color, and may mistake another person for the companion.

[0012] In an embodiment, this problem is solved by creating a companion profile to identify the determined companion and following the determined companion as they move along the side of the powered wheelchair, allowing the user to freely converse with the companion while the powered wheelchair autonomously navigates in accordance with the companion's movements.

[0013] Various embodiments of the power wheelchair companion following mode are described in detail herein.

[0014] As used herein, the term "communicatively coupled" may mean that the coupled components can exchange data signals with each other, such as electrical signals over conductive or non-conductive media, electromagnetic signals over air, or optical signals over optical waveguides, e.g., via a network such as Wi-Fi, Bluetooth, etc.

[0015] As used herein, the term "system longitudinal direction" refers to the front-to-back direction of the system (i.e., the + / -X direction of the coordinate axes shown in FIG. 1). The term "system transverse direction" refers to the horizontal direction (i.e., the direction along the Y axis of the coordinate axes shown in FIG. 1), which is the direction across the longitudinal direction. The term "system vertical direction" refers to the up-down direction of the system (i.e., the + / -Z direction of the coordinate axes shown in FIG. 1). As used herein, "top" or "topmost" is generally defined as being oriented in the positive Z direction of the coordinate axes shown in the drawings. "bottom" or "bottommost" is generally defined as being oriented in the negative Z direction of the coordinate axes shown in the drawings.

[0016] 1-2, a schematic diagram of a wheelchair system generally designated 10 is provided. System 10 generally includes a wheelchair 12 and a companion 13. In some embodiments, the companion is a person who travels along the side of wheelchair 12. The wheelchair includes a control unit 14. Wheelchair 12 is a power wheelchair with powered components that allow a user 16 to electronically control the movement of the wheelchair. As such, an understanding of the various components of wheelchair 12 is necessary and will not be described in further detail herein. In some embodiments, wheelchair 12 may include a power base portion 20, a frame 22, and a seat 24 supported by frame 22. Frame 22 is supported by power base portion 20. As such, the frame 22 is positioned generally below the seat 24 in the system vertical direction (i.e., positioned in the -Z direction of the coordinate axes of FIG. 1 relative to the seat 24), and the power supply base portion 20 is positioned generally below the frame 22 in the system vertical direction (i.e., positioned in the -Z direction of the coordinate axes of FIG. 1 relative to the frame 22).

[0017] 1 , in some embodiments, the power base portion 20 may lift, tilt, or otherwise move the frame 22, and thus the seat portion 24. The frame 22 and seat portion 24 are generally configured to support the user 16 when the user 16 is seated in the wheelchair 12. In some embodiments, the seat portion 24 may include a pair of armrests 26 to which a controller 28 may be coupled. As described herein, the controller 28 may provide the user 16 with the ability to control the movement of the wheelchair 12. In some embodiments, the controller 28 may be a joystick-type controller in which the user 16 moves a joystick according to a desired direction and / or speed of movement. As such, the controller 28 may be communicatively coupled to the power base portion 20, including its various components, and may send signals to the power base portion 20 to cause the wheelchair 12 to react according to the input received by the controller 28. It should be understood that the joystick configuration is merely exemplary, and that in some embodiments, the controller 28 may utilize other designs, such as buttons, switches, voice controls, breathing controls, etc., to receive input from the user 16 via a user interface, etc.

[0018] In some embodiments, the seat 24 may include one or more handles 30 built into or coupled to the seat. The one or more handles 30 may provide an area for a user (e.g., a caregiver) to grip the wheelchair 12. For example, at least one of the one or more handles 30 may be located at the rear of the seat 24 so that a user may grasp the one or more handles 30 when moving behind the wheelchair 12.

[0019] The power base portion 20 may include, but is not limited to, one or more wheels 32, actuators 34, a battery 36, and a control unit 14. The control unit 14 may be an electronic control unit, or generally, a controller that controls the wheelchair 12 and / or one or more components thereof. As such, the control unit 14 may be communicatively coupled to various components of the wheelchair 12 such that one or more control signals may be transmitted from the control unit 14 to the various components, such as the actuators 34, as described in further detail herein. The actuators 34, which may be configured as one or more motors, are coupled to the wheels 32 to drive the movement of the wheels 32. The battery 36 may generally provide power to the various components of the wheelchair 12, such as the actuators 34. Furthermore, in some embodiments, the power base portion 20 may include a location sensor 40, such as a Global Positioning System (GPS) device, configured to transmit the location of the wheelchair 12 and / or receive the position of other objects relative to the wheelchair 12. The other components of the power base portion 20 should be understood generally and will not be described in further detail herein.

[0020] The one or more wheels 32 may be configured as any type of wheel. By way of non-limiting example, the one or more wheels 32 may be omnidirectional wheels, which may allow a user to easily move the wheelchair 12 in any direction in the case of manual fine motor control, as described below.

[0021] The control unit 14 may generally be a stand-alone control device that includes one or more components for controlling the movement of the wheelchair 12. While the control unit is shown in Figures 1-3C as part of the power base portion 20 of the wheelchair 12, it should be understood that this is a non-limiting example. That is, the control unit 14 may be a device that is separate from the power base portion 20, such as a device coupled to or integrated with a pair of armrests 26, the seat 24, etc. In some embodiments, the control unit 14 may be completely separate from the wheelchair 12, such as a computing device carried by the user, a user's mobile device, etc.

[0022] 1 through 3A, various exemplary components of the control unit 14 are schematically illustrated. The control unit 14 may be communicatively connected to a network 73. The network 73 may include a wide area network (WAN) such as the Internet, a local area network (LAN), a mobile communications network, a public service telephone network (PSTN), a personal area network (PAN), a metropolitan area network (MAN), a virtual private network (VPN), and / or another network that may electronically connect to the control unit 14. The network 73 may also communicatively couple the control unit 14 to personal electronic devices such as mobile phones, tablets, etc., as discussed in more detail herein.

[0023] In various embodiments, the control unit 14 includes a network interface 66, a processor unit 60, a data storage device 70, and a memory element 68. The processor unit 60, such as a computer processing unit (CPU), may be a central processing unit of the control unit 14 that performs calculations and logical operations to execute programs. The processor unit 60 alone or in combination with other components is an exemplary processing unit, computing device, processor, or combination thereof. The processor unit 60 may include any processing element configured to receive and execute instructions (e.g., from the memory element 68). A local interface 64, such as a bus, may interconnect the various components.

[0024] It should be appreciated that local interface 64 may be formed from any medium capable of transmitting a signal, such as, for example, conductive wires, conductive traces, optical waveguides, etc. In some embodiments, local interface 64 may facilitate the transmission of wireless signals, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), etc. Furthermore, local interface 64 may be formed from a combination of media capable of transmitting a signal. In one embodiment, local interface 64 comprises a combination of conductive traces, conductive wires, connectors, and buses that cooperate to enable the transmission of electrical data signals to components such as processors, memories, sensors, input devices, output devices, and communication devices.

[0025] In some embodiments, memory element 68 may be configured as a volatile and / or non-volatile computer-readable medium, and may therefore include random access memory (including SRAM, DRAM, and / or other types of random access memory), read-only memory (ROM), flash memory, registers, compact discs (CDs), digital versatile discs (DVDs), and / or other types of storage components. Additionally, memory element 68 may be non-transitory processor-readable memory. Memory element 68 may include one or more programming instructions that, when executed by processor unit 60, cause processor unit 60 to complete various processes, such as one or more of the processes described herein with respect to FIGS. 4-5.

[0026] 3B, the programming instructions stored in memory device 68 may be embodied as one or more software logic modules, where each logic module provides programming instructions for completing one or more functions, such as those described in more detail below with respect to FIGS. 4-5. For example, wheelchair operation logic module 74 may include one or more different logic portions, each of which may be embodied as a computer program, firmware, and / or software / hardware executable by processor unit 60 to cause wheelchair 12 to move and / or turn itself in response to commands provided by user 16.

[0027] Additionally, the wheelchair operation logic module 74 may include one or more different logic portions, each embodied as a computer program, firmware, and / or software / hardware that may be executable by the processor device 60 to cause the wheelchair 12 to move and / or turn itself in response to autonomous commands, such as when the wheelchair 12 is in a companion-controlled mode. Thus, in a companion-controlled mode, the wheelchair may autonomously follow or mimic the movements of the companion 13, such that the wheelchair 12 moves without the request or input of the user 16. That is, the actuator 34 may autonomously control the movement of one or more wheels 32 of the wheelchair to drive the wheelchair 12. It should be appreciated that in companion-controlled mode, the wheelchair operation logic module 74 may predict movements by the companion 13 based on history or other indicators, such as objects around the wheelchair 12, to cause the wheelchair 12 to move more slowly or "less jerky" in response to companion wobbles and sudden turns. Alternatively, the system may increase speed in turns and other maneuvers to ensure a more enjoyable experience for the user 16.

[0028] The companion logic module 76 may include one or more logic portions, each of which may be embodied as a computer program, firmware, and / or software / hardware that may be executable by the processor device 60 to determine the companion, determine the companion's movements, and communicate such movements to other systems of the wheelchair, such as the wheelchair operation logic module 74. In this manner, the companion logic module 76 may assist in identifying the companion and in directing the wheelchair to mimic or follow the companion's movements when the wheelchair is in a companion-controlled mode, as discussed in more detail herein.

[0029] In other embodiments, the companion logic module 76 may include one or more logic portions. Each of the logic portions may be embodied as a computer program, firmware, and / or software / hardware executable by the processor device 60 to determine the companion, how long to follow the companion, etc., based on a plurality of user preferences. For example, the plurality of user preferences may include manual entry of the type or color of clothing the companion is wearing, uploading of photos or other facial recognition features, gait information, etc. In some embodiments, user preferences may be uploaded via a mobile phone, tablet, or other personal electronic device using the Internet, Bluetooth, etc. In other embodiments, user preferences may be uploaded directly to the control unit 14 of the wheelchair 12. Any object recognition algorithm may be used to determine whether an object within range of the wheelchair 12 meets the criteria to be a companion 13. If so, the system initiates a companion control mode, as discussed in more detail herein.

[0030] Sensor logic module 78 may include one or more logic portions, each of which may be embodied as a computer program, firmware, and / or software / hardware executable by processor device 60 to receive process signals and / or data from one or more sensors, such as the sensors shown in Figures 2 and 3A.

[0031] The object detection logic 80 may include one or more logic portions. Each of the logic portions may be embodied as a computer program, firmware, and / or software / hardware that may be executable by the processor unit 60 to receive sensor data (e.g., image data from a camera) and detect objects around the wheelchair 12. As described in further detail below, the object detection logic 80 may use the sensor data to detect undesirable objects in the path of the wheelchair 12 when the wheelchair 12 is in a companion-controlled mode that could cause an undesirable condition if the wheelchair 12 remained in the companion-controlled mode. In this manner, the object detection logic 80 may transition the wheelchair from the companion-controlled mode to a standard mode. In the standard mode, the user may take full control of the wheelchair using, for example, a controller to enable a manual motion control mode.

[0032] As used herein, "companion-controlled mode" means that the wheelchair is in autonomous mode (i.e., moves without the user 16 manually controlling or directing the wheelchair) such that the wheelchair follows or mimics the companion 13. "Standard mode" means that the wheelchair is controlled or operated by the user 16, such as with the controller 28. "Undesirable object" means an object that has undesirable consequences for the user seated in the wheelchair 12. Non-limiting examples of undesirable objects include potholes, vehicles, signs, curbs, other people, animals, etc.

[0033] 3A , an input module 62 is provided to enable the user 16 to input controls to the control unit 14 and, therefore, the wheelchair 12. The input module 62 may be communicatively coupled to the controller 28 and / or another input device (e.g., switch 27), as discussed in more detail below. The input module 62 may, for example, communicate input signals to the processor unit 60 so that the user 16 may use the controller 28 to move the wheelchair via the actuators 34. Thus, as discussed in more detail herein, it should be understood that user controls may be transmitted to the control unit 14 via the input module 62. It should also be understood that the user 16 may select user controls via buttons, such as push buttons, switches, such as toggle switches, on the wheelchair 12, from the pair of armrests 26, or from programmed selections initiated on an external device, such as a portable computing device, smartphone, or the like, via a network interface 66, as described below.

[0034] The network interface 66 of the control unit 14 may include any wired or wireless networking hardware, such as a modem, a LAN port, a Wireless Fidelity (Wi-Fi) card, a WiMax card, mobile communication hardware, and / or other hardware for communicating with other networks and / or devices. Thus, communications between the control unit 14, the wheelchair 12, and / or other external devices may be provided via the network interface 66.

[0035] Data storage device 70, which may generally be a storage medium, may include one or more data repositories for storing received and / or generated data and may be any physical storage medium, including, but not limited to, a hard disk drive (HDD), memory, removable storage, etc. While data storage device 70 is shown as a local device, it should be understood that data storage device 70 may also be a remote storage device, such as, for example, a server computing device. Exemplary data that may be contained within data storage device 70 is described below with reference to FIG. 3C . This data includes, but is not limited to, companion data 82, rear sensor data 84, front sensor data 86, surface data 88, position data 90, and image data 92.

[0036] 1-3A, the wheelchair 12 may include multiple sensors that provide sensor data to implement the functionality described herein. The multiple sensors may include, but are not limited to, a surface sensor 38, a front sensor 37, a position sensor 40, a rear sensor 42, and an image capture device 43 or sensors. It should be understood that more or fewer sensors may be provided. Furthermore, it should be understood that the surface sensor 38, the front sensor 37, the position sensor 40, the rear sensor 42, and the image capture device 43, if provided, may be communicatively coupled to the local interface 64 and coupled to the processor device 60 via the local interface 64.

[0037] The surface sensor 38 is operable to generate surface data 88 indicative of the type of surface 48 on which the wheelchair 12 is operating. Different surfaces have different coefficients of friction for one or more wheels 32, which in turn generate different wheel torques on the one or more wheels 32. Non-limiting examples of surfaces include tile floors, laminate floors, wood floors, decks, concrete surfaces, dirt surfaces, gravel surfaces, etc. The surface type may be used to determine how to follow the companion in companion control mode. As a non-limiting example, the data storage device 70 may store examples of coefficients of friction for different surface types. The surface sensor 38 may, for example, be an image sensor that creates image data of the surface 48. As a non-limiting example, the surface sensor 38 may be positioned under the seat 24 and have a field of view of the surface 48. This image data may be used by the control unit 14 to determine the type of surface. Any object recognition algorithm may be used to determine the type of surface.

[0038] Some embodiments include a front sensor 37. If provided, the front sensor 37 may be positioned at the front of the wheelchair 12 (e.g., on a frame, armrest, etc., facing forward) and may generate data indicative of objects in front of the wheelchair 12. In a non-limiting example, the front sensor 37 is an image sensor that generates image data that can be processed using an object detection algorithm to detect objects, such as undesirable objects that cause undesirable conditions, as described in more detail below with respect to flowchart 400 of FIG. 4. It should be understood that other embodiments may not include a front sensor 37. In other embodiments, the front sensor 37 is a sensor such as a laser-based sensor, a proximity sensor, a level detection sensor, a pressure sensor, any combination thereof, and / or any other type of sensor as would be understood by one of ordinary skill in the art.

[0039] If provided, the rear sensor 42 may be positioned at the rear of the wheelchair 12 (e.g., on the seat back) and may generate data indicative of objects behind the wheelchair 12. In a non-limiting example, the rear sensor 42 is an image sensor that generates image data that can be processed using an object detection algorithm to detect objects, such as undesirable objects that cause undesirable conditions, as described in more detail below with respect to flowchart 400 of FIG. 4. It should be understood that other embodiments may not include the rear sensor 42. In other embodiments, the rear sensor 42 is a sensor such as a laser-based sensor, a proximity sensor, a level detection sensor, a pressure sensor, any combination thereof, and / or any other type of sensor as would be understood by one of ordinary skill in the art.

[0040] The wheelchair may also include a position sensor 40, which may be configured as, for example, a GPS sensor. The position sensor 40 provides data regarding the location of the wheelchair 12 within the environment. The position of the wheelchair 12 may be useful in determining objects around the wheelchair 12. The position sensor 40 may also include other sensors, such as a wireless Bluetooth® sensor operable to communicate with a wireless Bluetooth® beacon to triangulate the position of the wheelchair 12 and determine the location of the wheelchair 12 relative to objects.

[0041] 1-3A, image capture device 43, if equipped, may be any imaging device, sensor, or detector suitable for acquiring images. As used herein, the term "image" refers to a video image (i.e., a series of continuous images), a still image (including a still image separated from a video image), and / or image data. Any suitable commercially available image capture device 43 may be used without departing from the scope of the present disclosure. In some embodiments, image capture device 43 may be coupled to one or more other components that provide additional functionality for imaging, such as, for example, one or more sensors.

[0042] Image capture device 43 may include or be coupled to a lens (not shown). The lens is not limited by this disclosure and may generally be any optical component configured to focus light entering image capture device 43 so that an image can be properly captured. In some embodiments, the lens may be a fixed lens that is not adjustable. In other embodiments, the lens may be manually or automatically adjustable by processor unit 60 to zoom in on an object (i.e., companion 13), zoom out on an object (i.e., companion 13), and / or adjust the focus of light entering image capture device 43.

[0043] It should be understood that either or both sides of the wheelchair 12 may be equipped with an image capture device 43. Additionally, in some embodiments, either or both sides of the wheelchair 12 may be equipped with a number of other sensors, such as sensors that determine the depth of an object, such as the companion 13, from a fixed position, e.g., the distance from one or more wheels 32 or the frame 22 of the wheelchair 12. The other sensors may be proximity sensors, whiskers, pressure sensors such as toggle switches, etc.

[0044] 3C, data storage device 70 may store data for performing functions described herein, such as determining and following a companion in companion control mode and / or exiting companion control mode when an undesirable object is within the path of travel of wheelchair 12. An embodiment may include some, all, or more of the data shown in FIG.

[0045] Companion data 82 may be data determined from image capture device 43, entered user preferences, etc. Companion data 82 may include profile data generated about companions, such as the type or recognized color of companion 13's clothing, facial recognition or facial expressions, walking cadence and / or conversational speech. Historical companion data 82 may be saved in a data storage device. This data may be used by control unit 14, for example, to learn certain profile characteristics of companion 13 for future decisions.

[0046] Additionally, if rear sensors 42 are provided, rear sensor data 84 generated by the rear sensors 42 may be stored in the data storage device 70. As described above and in further detail below, the rear sensor data 84 may be used to detect undesirable objects and / or conditions and to take the wheelchair 12 out of companion-controlled mode so that the wheelchair 12 is operated in standard mode.

[0047] Additionally, if a front sensor 37 is provided, front sensor data 86 generated by the front sensor 37 may be stored in the data storage device 70. As described above and in further detail below, the front sensor data 86 may be used to detect undesirable objects and / or conditions and to take the wheelchair 12 out of companion-controlled mode so that the wheelchair 12 is operated in standard mode.

[0048] Additionally, if an image capture device 43 is provided, image data 92 generated by the image capture device 43 may be stored in the data storage device 70. As discussed above and in further detail below, the image data 92 may include multiple images of the companion 13 in the companion control mode, which are used by the processor device 60 to determine the movements of the companion 13 and direct the actuators 34 of the wheelchair 12 to mimic those movements.

[0049] Other data is in the form of position data 90 generated by position sensor 40 and surface sensor 38, if provided, and surface data 88 (e.g., surface image data) generated by surface sensor 38. Such data may be used to determine and follow companion 13 in companion control mode and / or to terminate companion control mode if an undesirable object is in the path of travel of wheelchair 12, as described in more detail below with respect to Figures 4-5.

[0050] 4, an exemplary method for companion control mode is illustrated by flowchart 400. If user 16 wishes to enable companion control mode, user 16 may use an input device (e.g., switch 27, controller 28, or any other input device) at block 405 to enter companion control mode such that wheelchair 12 autonomously follows companion 13, thereby allowing user 16 and companion 13 to converse side-by-side. It should be understood that user 16 may enter companion control mode at any time and does not need to wait for the companion to be next to wheelchair 12 first.

[0051] In some embodiments, the wheelchair 12 may automatically enter and exit the companion control mode without user input. In such embodiments, a user preference may instruct the companion control mode to begin using companion data 82 and image data 92 generated by the image capture device 43, and the companion control mode is activated once the data is detected and the user stops powered movement of the wheelchair 12. For example, any object recognition algorithm may be used to detect and classify individuals once they are identified as companions, and such companions may be re-recognized. For example, a user may converse with a particular individual or companion at approximately the same time each day, and the system may recognize that companion based on clothing, facial recognition, gait, etc. In some embodiments, a position sensor 40 may be used instead of or in addition to the image capture device 43 to detect when a companion approaches the user.

[0052] Once user input is received, image capture device 43 is activated at block 410. It should be understood that in embodiments, multiple sensors (i.e., proximity sensor, depth sensor, etc.) are activated without image capture device 43. Additionally, in embodiments, multiple other sensors (i.e., proximity sensor, depth sensor, etc.) are activated when image capture device 43 is activated. At block 415, system 10 determines the companion and may request companion confirmation at block 420 by presenting data to user 16 for confirmation. The data may be presented to the user via a mobile phone, tablet, or user interface on wheelchair 12.

[0053] In block 425, the system 10 generates a companion profile that includes certain data and characteristics about the companion 13, such as the companion's clothing type and / or color, facial recognition, and gait. Gait may be either a walking cadence and / or a speaking rhythm that the system 10 can recognize using multiple algorithms to establish a specific profile for each companion 13. Once the companion profile is generated, the system 10 begins a continuous image capture mode in block 430 and activates multiple sensors (e.g., surface sensor 38, front sensor 37, rear sensor 42, and position sensor 40) in block 435. The system determines whether the companion 13 has moved in block 440. If the companion has not moved, the system 10 waits and loops between the continuous image capture mode in block 430 and the activation of multiple sensors (e.g., surface sensor 38, front sensor 37, rear sensor 42, and position sensor 40) in block 435 until it is determined in block 440 that the companion has moved. Once it is determined in block 440 that the companion has moved, the system 10 determines the companion's movement in block 445 .

[0054] In the example of FIG. 2 , the user 16 and wheelchair 12 identify the companion 13 via the image capture device 43, as indicated by the companion determination 63. As discussed in more detail herein, the image capture device 43 captures an image of the companion, and the control unit 14 recognizes the companion by identifying certain identifying features of the companion 13. As shown, the companion 13 is positioned next to the wheelchair 12 and user 16 so that the companion 13 and user 16 may converse face-to-face. Also shown is an example of operation in which the companion 13 takes their eyes off the wheelchair 12. The system 10 directs or controls the wheelchair 12 by actuating the actuators 34 to drive one or more wheels 32 of the wheelchair 12 to follow the companion in an exemplary motion and to turn or move along the same arc as the companion 13. It should be understood that the image data 92, companion data 82, and / or position data 90 are used to recognize that the companion 13 has moved relative to the wheelchair 12, and the system 10 autonomously controls the wheelchair 12 to follow the companion 13. It should be understood that the operational examples are merely exemplary, and that the companion 13 may move in any direction, including toward entry into the wheelchair 12, behind the wheelchair 12, and / or in front of the wheelchair 12. In any of these operational examples, the system 10 may determine the movement of the companion 13 and instruct the actuators 34 to move the wheels 32 of the wheelchair 12 to follow or mimic the movement of the companion 13. In some embodiments, the wheelchair 12 may notify the user 16 that the wheelchair 12 has entered a companion-controlled mode.

[0055] Once companion movement is detected, the system determines whether there is an undesirable object or condition in the path of travel at block 450. If there is no undesirable object in the path of travel, the system controls the wheelchair 12 to mimic the movement of the companion 13 at block 455. On the other hand, if it is determined that there is an undesirable object or condition in the path of the wheelchair 12, the system 10 exits the companion-controlled mode at block 460 and sets the wheelchair 12 to a standard mode in which the user 16 controls all movement. That is, the system 10 may inhibit autonomous control of the actuators 34 to prevent the wheelchair 12 from autonomously driving one or more wheels 32 to follow the movement of the companion 13 via the actuators 34. For example, one or more sensors may be used to detect undesirable objects or conditions. Non-limiting examples of sensors used to detect undesirable objects or conditions include the rear sensor 42, front sensor 37, surface sensor 38, and image sensor 43 shown in FIG. 1 . Image data from one or more sensors may be used to detect undesirable objects. As used herein, the phrase "undesirable object" refers to any object that a user may accidentally collide with while in companion control mode. Non-limiting examples of undesirable objects include a pet that happens to be walking behind the wheelchair 12, an object that may tip over, stairs leading down, or objects that may cause damage to the wheelchair 12, such as potholes, curbs, parked cars, moving vehicles, bicycles, pedestrians, etc. Embodiments are not limited by the type of undesirable object.

[0056] 5, an exemplary method for generating a companion profile is illustrated by flowchart 500. If user 16 wishes to enable companion control mode, user 16 may use an input device (e.g., switch 27, controller 28, or any other input device) to enter companion control mode at block 405. As a result, wheelchair 12 autonomously follows companion 13 so that user 16 and companion 13 may converse side-by-side. It should be understood that user 16 can enter companion control mode at any time and does not need to first wait for the companion to be next to wheelchair 12.

[0057] Once user input is received, image capture device 43 is activated at block 410. It should be understood that in embodiments, multiple sensors (i.e., proximity sensor, depth sensor, etc.) are activated without image capture device 43. Furthermore, in embodiments, multiple other sensors (i.e., proximity sensor, depth sensor, etc.) are activated when image capture device 43 is activated. At block 505, system 10 filters the captured image to remove or filter background noise. It should be understood that system 10 filters the background noise to focus or match companion 13 in the image. Once the background noise has been filtered, the companion is determined from the image data at block 510, and identifying characteristics of companion 13 are captured and stored at block 515. It should be understood that this data may be stored as companion data 82 and / or image data 92. For example, specific identifying characteristics of the companion 13 that may be captured and stored to ensure that the system 10 is tracking the appropriate companion 13 include the color of the companion's clothing in block 520, the companion's facial features in block 525, and the companion's gait in block 530.

[0058] Once the companion's clothing color has been captured at block 520, the companion's 13 facial features have been captured at block 525, and the companion's 13 gait has been captured at block 530, the system 10 stores identifying characteristics of the companion 13, such as companion data 82 and / or image data 92 at block 535. At block 425, the system 10 generates a companion profile that includes specific data and characteristics about the companion 13, such as the companion's clothing type and / or color, facial recognition, and gait.

[0059] It should now be appreciated that the systems and methods described herein provide a wheelchair that can enter a companion-controlled mode such that a wheelchair user can converse with a companion moving along the side of the wheelchair and the wheelchair autonomously follows or mimics the companion's movements without the user having to control the wheelchair. Embodiments further determine when an undesirable object or condition is present that interferes with autonomous control of the wheelchair.

[0060] While particular embodiments have been illustrated and described herein, it should be understood that various other changes and modifications can be made without departing from the spirit and scope of the claimed subject matter. Moreover, although various aspects of the claimed subject matter have been described herein, such aspects need not be utilized in combination. Therefore, the appended claims are intended to cover all changes and modifications that are within the scope of the claimed subject matter.

Claims

1. Wheelchair system With a companion, A wheelchair, one or more wheels; at least one actuator coupled to the one or more wheels; a processing device; a processor-readable storage medium in communication with the processing unit and storing one or more programming instructions that, when executed, cause the processing unit to: generating a companion profile based on the determined plurality of distinguishing characteristics of the companion; determining movement of the companion; a processor-readable storage medium for causing the at least one actuator to operate to drive the one or more wheels to follow the movement of the companion; Equipped with the actuation of the at least one actuator to drive the one or more wheels to follow the movement of the companion is autonomous; Wheelchair system.

2. the companion is positioned to travel along the side of the wheelchair; The wheelchair system of claim 1 .

3. The companion profile may further comprise: activating an image capture device to capture an image of the companion; filtering background from a plurality of background noises; and determining the accompanying person. The wheelchair system of claim 1 .

4. the plurality of distinguishing characteristics of the determined companion for generating the companion profile are based on recognized colors of clothing of the companion captured by the image capture device; The wheelchair system of claim 3.

5. the plurality of identifying features of the determined companion for generating the companion profile are based on facial expressions of the companion captured by the image capture device; The wheelchair system of claim 3.

6. the plurality of distinguishing characteristics of the determined companion for generating the companion profile are based on a gait of the companion captured by the image capture device. The wheelchair system of claim 3.

7. further comprising a plurality of sensors coupled to the wheelchair and in communication with a non-transitory processor-readable storage medium and the processing unit. The wheelchair system of claim 1 .

8. The one or more programming instructions may further cause the processing unit to: detecting undesirable objects on a path of travel of the wheelchair; inhibiting the autonomous control of the at least one actuator. The wheelchair system of claim 7.

9. 1. A wheelchair system comprising: With a companion, A wheelchair, The frame and one or more wheels coupled to the frame; at least one actuator coupled to the one or more wheels; a wheelchair comprising a control unit; Equipped with the control unit controls the wheelchair between a standard mode and a companion control mode, such that in the companion control mode, the control unit activates the at least one actuator to drive the one or more wheels to follow the movement of the companion; and wherein the operation of the at least one actuator to drive the one or more wheels to follow the movement of the companion is autonomous. Wheelchair system.

10. further comprising at least one image capture device communicatively coupled to the control unit; 10. The wheelchair system of claim 9.

11. The control unit activating the at least one image capture device; determining the accompanying person; generating a companion profile based on the determined companion; further configured to determine movement of the companion. The wheelchair system of claim 10.

12. The control unit Filter background from multiple background noises, further configured to determine the companion; The wheelchair system of claim 11 , wherein the companion profile is generated from a plurality of identifying characteristics of the determined companion.

13. At least one of the plurality of distinguishing characteristics of the determined companion for generating the companion profile is based on a perceived color of clothing of the companion captured by the at least one image capture device.

13. The wheelchair system of claim 12.

14. at least one of the plurality of distinguishing characteristics of the determined companion for generating the companion profile is based on a facial expression of the companion captured by the at least one image capture device; 13. The wheelchair system of claim 12.

15. at least one of the plurality of distinguishing characteristics of the determined companion for generating the companion profile is based on a gait of the companion captured by the at least one image capture device; 13. The wheelchair system of claim 12.

16. further comprising a plurality of sensors coupled to the wheelchair and communicatively coupled to the control unit.

10. The wheelchair system of claim 9.

17. The control unit detecting undesirable objects in a path of travel of the wheelchair; further configured to inhibit the autonomous control of the at least one actuator.

17. The wheelchair system of claim 16.

18. the companion is positioned to travel along the side of the wheelchair; 10. The wheelchair system of claim 9.

19. 1. A method of operating a wheelchair, comprising: receiving, by the control unit, an input from a user corresponding to a request for a companion control mode; determining a companion by the control unit; determining, by the control unit, movement of the companion; activating, by the control unit, at least one actuator to drive one or more wheels of the wheelchair to follow the movement of the companion; A method comprising:

20. the actuation of the at least one actuator to drive the one or more wheels to follow the movement of the companion is autonomous; 20. The method of claim 19.

Images