Electric wheelchair control method based on artificial intelligence and device and system used for electric wheelchair control method

Abstract

The electric wheelchair control method based on artificial intelligence according to one embodiment of the present invention may comprise the steps of: pairing with a user device by executing an activation step as power is applied; acquiring user information from the server through the paired user equipment; determining an electric wheelchair work control mode based on the user information; recognizing a predefined start command by a sensor corresponding to the determined operation control mode in a state where the display screen is always closed; activating a closing picture according to the starting instruction, and outputting a user interface picture for controlling the electric wheelchair to a display to wait for a user to input a signal; performing machine learning on a user input signal input through a sensor to perform user recognition and authentication; and based on the success of the user authentication, identifying a control instruction corresponding to the user input signal to control the work of the electric wheelchair, the user information including at least one of disability type information and health state information.

Description

Technical Field

[0001] The present invention relates to electric wheelchair control technology, and in particular, to an electric wheelchair control method based on artificial intelligence, which can adaptively control the electric wheelchair according to the user's disability or health status based on artificial intelligence, and a device and system therefor. Background Art

[0002] Recently, cars can provide safety and convenience to drivers through various advanced driver assistance systems (ADAS) such as lane departure warning (LDW), adaptive cruise control (ACC), forward collision warning (FCW), blind spot detection (BSD) systems, etc.

[0003] The most important technology in this intelligent system is external object recognition technology. This technology can provide safety and convenience in driving and parking conditions by using various sensors such as cameras, radars, ultrasound, and lidar to detect lanes, as well as vehicles and pedestrians in the front, rear, and blind spots.

[0004] Cameras are necessary devices used in autonomous driving environments above Level 2, such as collision prevention, lane keeping, and parking assistance. With mandatory installation in many countries, their proportion is gradually increasing.

[0005] In addition, these sensors are linked with the Vehicle to Everything (V2X) communication system, which can effectively avoid accidents even in situations that are difficult to detect with environmental sensors, such as intersections.

[0006] In order to overcome the weaknesses of individual sensors, sensor fusion technology that combines sensor information is being actively developed, and the development of next-generation sensor technologies such as stereo cameras and three-dimensional image sensors is being actively carried out.

[0007] Recently, sensor technology applied to vehicles has also been adopted and utilized in electric wheelchairs.

[0008] Recently launched electric wheelchairs use the following technology, which uses cameras and ultrasonic sensors to detect obstacles in front of the user that are difficult for the user to perceive, such as speed bumps, stairs, puddles, cliffs, manholes, etc., and provide warning alerts to the user or automatically pause.

[0009] However, there is a problem that a hearing-impaired person cannot hear the warning alarm, and a visually-impaired person cannot confirm the warning alarm displayed on the display device.

[0010] Another problem is that it is difficult for quadriplegic patients, amyotrophic lateral sclerosis (ALS) patients, etc. to normally control the steering device such as the joystick provided on the electric wheelchair in response to the warning alarm, and they cannot travel to avoid obstacles.

[0011] In particular, electric wheelchairs used in medical institutions or nursing institutions are shared by multiple people, and therefore there is a problem that they cannot be used according to the user's disability status, health status, etc.

[0012] Therefore, there is a need for an artificial intelligence-based electric wheelchair that can work adaptively by considering the user's disability status and health status, etc. Summary of the invention

[0013] The object of the present invention is to provide an electric wheelchair control method based on artificial intelligence and a device and system therefor.

[0014] Another object of the present invention is to provide an artificial intelligence-based electric wheelchair that can automatically switch working modes based on the user's disability status and health status.

[0015] Another object of the present invention is to provide the following artificial intelligence-based electric wheelchair control method and device and system therefor, which obtains template data of each authentication unit by pre-learning sensing information collected from multiple authentication sensors, determines the authentication weighted value of each authentication unit by comparing the characteristic data of each authentication unit acquired in real time with the corresponding template data, and sets the optimal authentication level of each user based on the determined authentication weighted value, so that customized user authentication can be performed.

[0016] Another object of the present invention is to provide the following artificial intelligence-based electric wheelchair control method and device and system therefor, which obtains information related to the user's disability type and health status from a server through a user device paired with the electric wheelchair, and adaptively sets the working control mode of the electric wheelchair based on the acquired user information, thereby providing user convenience and driving safety.

[0017] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and ordinary technicians in the technical field to which the present invention belongs can clearly understand other technical problems not mentioned through the following description.

[0018] According to one embodiment of the present invention, an artificial intelligence-based electric wheelchair control method may include: performing a startup step as power is applied to pair with a user device; obtaining user information from a server through the paired user device; determining an operating control mode of the electric wheelchair based on the user information; in a state where an always-on display (AOD, Always On Display) screen is turned off, identifying a predefined startup instruction through a sensor corresponding to the determined operating control mode; activating the closed screen according to the startup instruction, outputting a user interface screen for controlling the electric wheelchair to a display, and then waiting for a user input signal; performing machine learning on the user input signal input through the sensor to perform user identification and authentication; and based on the success of the user authentication, identifying a control instruction corresponding to the user input signal to control the operation of the electric wheelchair, wherein the user information includes at least one of disability type information and health status information.

[0019] In an embodiment, the above method may also include setting an authentication application user mode, and the above user authentication includes: user-dependent authentication, performing authentication only on a pre-set specific user; and user-independent authentication, performing authentication on all users registered in the authentication database.

[0020] In an embodiment, the above-mentioned working control mode may include at least one of a manual control mode, a voice recognition control mode, a face recognition control mode, a gesture recognition control mode and an autonomous driving control mode.

[0021] In an embodiment, the above method may also include: based on the above working control mode being a voice recognition control mode, determining a view type corresponding to a recognized voice command; determining at least one camera to receive an image corresponding to the above determined view type image; constructing a view screen based on the image obtained from the above determined at least one camera; and displaying the above view screen on a screen of the above user device or the above display, the above view type including at least one of a front view, a left view, a right view, a rear view, a left / right view, a front surround view, a rear surround view and a bird's-eye view.

[0022] In an embodiment, the method may further include: detecting approaching obstacles using at least one of a Smart Parking Assistance System (SPAS) sensor, an ultrasonic sensor and a radar during the driving of the electric wheelchair, and displaying information of the detected approaching obstacles on one side of the view screen based on the detected approaching obstacles.

[0023] In an embodiment, the sensor may include a microphone, a camera and a biometric sensor, the biometric sensor includes at least one of a fingerprint sensor, an iris sensor and a body pressure sensor, and a user registration step is initiated according to a predetermined user menu selection on the user interface screen, the user registration step includes: extracting first characteristic data by recognizing the user's voice through the microphone; extracting second characteristic data by recognizing the user's iris from the image captured by the camera; extracting third characteristic data by recognizing the user's fingerprint through the fingerprint sensor; extracting fourth characteristic data by recognizing the user's face from the image captured by the camera; determining authentication weighted values ​​of each characteristic data based on pre-learned first template data, second template data, third template data and fourth template data, the pre-learned first template data, second template data, third template data and fourth template data corresponding to the first characteristic data, second characteristic data, third characteristic data and fourth characteristic data respectively; and determining the authentication level of the corresponding user based on the determined authentication weighted value of each characteristic data, storing and managing the extracted characteristic data and information related to the determined authentication level of each user in an internal memory, and performing the user authentication according to the determined authentication level of each user.

[0024] According to another embodiment of the present invention, an electric wheelchair control system based on artificial intelligence for controlling an electric wheelchair includes a peripheral board and an electric wheelchair control device, wherein the peripheral board may include: a display board for displaying a user interface screen on a display; a sensor board for generating sensing information based on a signal input from a sensor; a communication board for performing communication with the electric wheelchair and an external device; a drive board for controlling a motor provided in the electric wheelchair; and a steering board for controlling the steering of the electric wheelchair, wherein the electric wheelchair control device controls the overall operation of the electric wheelchair by interlocking with the peripheral board, wherein the electric wheelchair control device performs a startup step as power is applied to pair with a user device, obtains user information from a server through the paired user device, determines an operation control mode of the electric wheelchair based on the user information, and displays an always on display (AOD, Always On When the electric wheelchair display screen is closed, a predefined start instruction is identified through the sensor corresponding to the determined work control mode, the closed screen is activated according to the start instruction, a user interface screen for controlling the electric wheelchair is output to the display and then a user input signal is waited for, machine learning is performed on the user input signal input through the sensor to perform user identification and authentication, and based on the success of the authentication, a control instruction corresponding to the user input signal is identified to control the operation of the electric wheelchair, and the user information includes at least one of disability type information and health status information.

[0025] In an embodiment, the electric wheelchair control device can perform the user authentication in a pre-set authentication application user mode, and the user authentication includes: user-dependent authentication, which performs authentication only on a pre-set specific user; and user-independent authentication, which performs authentication on all users registered in the authentication database.

[0026] In an embodiment, the above-mentioned working control mode may include at least one of a manual control mode, a voice recognition control mode, a face recognition control mode, a gesture recognition control mode and an autonomous driving control mode.

[0027] In an embodiment, the sensor may include a microphone, a camera and a biosensor, the biosensor includes at least one of a fingerprint sensor, an iris sensor and a body pressure sensor, the electric wheelchair control device performs a user registration step according to a user menu selection on the user interface screen, the user registration step includes: extracting first characteristic data by recognizing the user's voice through the microphone; extracting second characteristic data by recognizing the user's iris from an image captured by the camera; extracting third characteristic data by recognizing the user's fingerprint through the fingerprint sensor; extracting fourth characteristic data by recognizing the user's face from the image captured by the camera; determining an authentication weight value of each characteristic data based on pre-learned first template data, second template data, third template data and fourth template data, the pre-learned first template data, second template data, third template data and fourth template data corresponding to the first characteristic data, second characteristic data, third characteristic data and fourth characteristic data respectively; determining an authentication level of a corresponding user based on the determined authentication weight value of each characteristic data, storing and managing the extracted characteristic data and information related to the determined authentication level of each user in an internal memory, and performing the user authentication according to the determined authentication level of each user.

[0028] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and ordinary technicians in the technical field to which the present invention belongs can clearly understand other technical problems not mentioned through the following description.

[0029] The advantage of the present invention is that it provides an electric wheelchair control method based on artificial intelligence and a device and system therefor.

[0030] Furthermore, the advantage of the present invention is that it provides an artificial intelligence-based electric wheelchair that can automatically switch the working control mode of the electric wheelchair based on the user's disability status and health status.

[0031] Furthermore, the advantage of the present invention is that it provides the following artificial intelligence-based electric wheelchair control method and a device and system therefor, which obtains template data of each authentication unit by pre-learning sensing information collected from multiple authentication sensors, determines the authentication weighted value of each authentication unit by comparing the characteristic data of each authentication unit acquired in real time with the corresponding template data, and sets the optimal authentication level of each user based on the determined authentication weighted value, thereby performing customized user authentication.

[0032] Furthermore, the advantage of the present invention is that it provides the following artificial intelligence-based electric wheelchair control method and a device and system therefor, which obtains information related to the user's disability type and health status from a server in real time through a user device paired with the electric wheelchair, and adaptively sets the electric wheelchair working control mode based on the acquired user information, thereby providing user convenience and driving safety.

[0033] Furthermore, the present invention has the advantage of providing an artificial intelligence-based electric wheelchair that can be effectively shared and used in medical institutions, nursing institutions, and the like.

[0034] Furthermore, various effects directly or indirectly grasped by this text can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 4 is a state diagram of an electric wheelchair control device based on artificial intelligence according to an embodiment.

[0036] Figure 2 FIG. 1 is a flowchart for explaining the initial startup procedure of the electric wheelchair control device of the embodiment.

[0037] Figure 3 1 is a flowchart for explaining the steps of setting the initial operation control mode of the electric wheelchair according to the embodiment.

[0038] Figure 4 FIG. 1 is a flowchart for explaining the electric wheelchair control method in the electric wheelchair control device according to the embodiment.

[0039] Figure 5 1 is a flowchart for explaining a user registration procedure in the electric wheelchair control device according to the embodiment.

[0040] Figure 6 This is a flowchart for explaining a user authentication step in the electric wheelchair control device according to the embodiment.

[0041] Figure 7 The flowchart is used to illustrate the user authentication steps based on the authentication application user mode according to the embodiment.

[0042] Figure 8A diagram for illustrating detailed operation of an electric wheelchair providing a camera view screen based on speech recognition artificial intelligence (AI) according to an embodiment.

[0043] Fig. 9 FIG. 4 is an illustration of a view screen provided by an electric wheelchair control device according to an embodiment.

[0044] Fig.10 FIG. 4 is a block diagram for illustrating the structure of an electric wheelchair system according to an embodiment of the present invention.

[0045] Fig.11 An example of an electric wheelchair according to the embodiment is shown. DETAILED DESCRIPTION

[0046] Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. When assigning reference numerals to structural elements in various drawings, the same reference numerals are assigned to the same structural elements as much as possible even if they are shown in different drawings. In addition, in the process of describing the present invention, if it is determined that the specific description of the relevant known structure or function hinders the understanding of the embodiments of the present invention, the detailed description thereof will be omitted.

[0047] When describing the structural elements of the embodiments of the present invention, terms such as first, second, A, B, (a), (b) and the like may be used. These terms are only used to distinguish the structural element from other structural elements, and the nature, order or sequence of the structural element are not limited to these terms. Furthermore, unless otherwise defined, the meanings of all terms used herein, including technical or scientific terms, are the same as those commonly understood by those of ordinary skill in the art to which the present invention belongs. The meanings of terms defined in commonly used dictionaries are the same as those in the context of the relevant technology, and shall not be interpreted as ideal or overly formalized meanings unless clearly defined in this application.

[0048] Figure 1 4 is a state diagram of an electric wheelchair control device based on artificial intelligence according to an embodiment.

[0049] Reference Figure 1 The electric wheelchair control status may generally include a standby status 10 (Standby Status), a wakeup status 20 (Wakeup Status), an analysis status 30 (Analysis Status), a result feedback and control status 40 (ResultFeedback and Control Status) and a lock status 50 (Lock Status).

[0050] When the initial startup steps are completed, the electric wheelchair control device may transition to a standby state 10 .

[0051] When the initial startup step is performed or completed, the electric wheelchair control device of the embodiment can automatically pair with a predefined (or close) user device, and obtain user information from the server through the paired user device. The user information may include at least one of information related to the type of disability and health status information. The server of the embodiment can be a server managed by a pre-registered medical institution and / or nursing institution or a dedicated server that operates separately for providing artificial intelligence-based electric wheelchair services.

[0052] The electric wheelchair control device may determine the initial working control mode of the electric wheelchair based on the user information obtained from the server. The initial working control mode may include a manual control mode, a voice recognition control mode, a face recognition control mode, a gesture recognition control mode, an autonomous driving control mode, etc., but is not limited thereto. In one example, when the user freely uses his hands, the initial working control mode may be set to a manual control mode, when the user has a hearing impairment, the initial working control mode may be set to a gesture recognition control mode, when the user has a hearing impairment, the initial working control mode may be set to an autonomous driving control mode, and when the user is a quadriplegic patient or an amyotrophic lateral sclerosis (ALS) patient, the initial working control mode may be set to a face recognition control mode or a voice recognition control mode.

[0053] When the electric wheelchair is stopped or in motion, and the electric wheelchair control device is in the Always On Display (AOD) working mode, the standby state 10 may be a sensor-specific input user state corresponding to the initial working control mode, that is, a state of waiting for a wakeup command. In the standby state 10, the display screen of the electric wheelchair control device, for example, a touch light emitting diode (LED) screen, may be in a closed state, that is, an OFF state.

[0054] In one example, when a preset / registered gesture, such as hand work, preset / registered facial movement, or preset / registered voice keyword, such as a wakeup voice command, is input by a user according to an initially set working control mode, the electric wheelchair control device in the standby state 10 can recognize the user's input through the equipped sensor, and when the recognized input signal is consistent with the preset / registered information, the closed state display screen can be switched to an activated state, i.e., an ON state. When the display screen is activated, the electric wheelchair control device can construct and display a user interface screen.

[0055] When the display screen is activated, the electric wheelchair control device can change from the standby state 10 to the awakening state 20, and can notify the electric wheelchair that it is in a state that can be controlled by the user according to the working control mode set through the speaker equipped on one side of the electric wheelchair or the paired user device or the equipped display screen.

[0056] The electric wheelchair control device may wait for a user input signal through a specific sensor corresponding to the operation control mode set in the awake state 20 , and transition to the analysis state 30 if the user input signal is detected.

[0057] The electric wheelchair control device can analyze the user input signal through the artificial intelligence engine carried by the device in the analysis state 30 to identify the user control command. The artificial intelligence engine can include at least one of a speech recognition engine, a face recognition engine, a gesture recognition engine, and a touch recognition engine.

[0058] When the analysis of the user input signal is completed normally, the electric wheelchair control device can be transformed into the result feedback state 40, and the predetermined control instruction corresponding to the analysis result is transmitted to the corresponding system (or device or board). In one example, when the analysis result of the user input signal is a steering control instruction, the electric wheelchair control device can transmit the corresponding steering control instruction to the driving board of the control motor. In another example, when the analysis result of the user input signal is a camera view screen control instruction, the electric wheelchair control device transmits the corresponding camera control instruction to the camera system to obtain the image captured by the camera, and the obtained image can constitute a view screen and be displayed on the corresponding display device, such as the user device screen or the display screen equipped with the electric wheelchair.

[0059] In an embodiment, the electric wheelchair may be equipped with multiple cameras, and the electric wheelchair control device may request and receive images captured by multiple cameras from the camera system according to a user input signal. In this case, the electric wheelchair control device may adaptively form a view screen according to the types of multiple cameras receiving the images and output them through a corresponding display screen. In one example, the camera may include a front camera, a left / right view camera, and a rear camera.

[0060] In one example, when the electric wheelchair control device receives an image captured by a front camera, the electric wheelchair control device may be controlled to display the image captured by the front camera on a user device screen or an equipped display screen.

[0061] In another example, when the electric wheelchair control device receives images captured by the left / right view camera, the constructed view screen can be divided into two areas, namely the first area (left side) and the second area (right side), and the electric wheelchair control device can be controlled to display the image captured by the left view camera in the first area and to display the image captured by the right view camera in the second area.

[0062] In another example, when the electric wheelchair control device receives images captured by the left / right view cameras and the front camera, the electric wheelchair control device can construct and output a front surround view picture based on the received multiple images.

[0063] In yet another example, when the electric wheelchair control device receives images captured by the left / right view cameras and the rear camera, the electric wheelchair control device can construct and output a rear surround view screen based on the received multiple images.

[0064] In yet another example, when the electric wheelchair control device receives images captured by a front camera, left / right view cameras, and a rear camera, the electric wheelchair control device can construct and output a bird's-eye view screen based on the received multiple images.

[0065] In the analysis state 30 , when the recognition of the user input signal fails, the electric wheelchair control device transitions to the result feedback state 40 and outputs a predetermined notification message notifying the user that the recognition of the user input signal fails through a speaker (or a user device).

[0066] According to the pre-set safety and / or locking mode, the electric wheelchair control device can perform machine learning (or deep learning) on ​​the user input signal and perform user identification and authentication.

[0067] If the user identification and authentication are successful, the electric wheelchair control device can control the electric wheelchair according to the user input signal. If the user identification or authentication fails, the electric wheelchair control device can output a predetermined warning notification message notifying the user of the identification or authentication failure, and transition from the result feedback state 40 to the awakening state 20.

[0068] When the number of times of user identification and authentication failure exceeds a predetermined number, the electric wheelchair control device may output a predetermined warning notification message notifying that the user is not a registered user and then transition to the locked state 50. In one example, in the locked state 50, the electric wheelchair control device may provide authentication failure information including current location information to the server (or a pre-set administrator terminal) through the user device. In this case, the current location of the electric wheelchair may be determined using a global positioning system (GPS) receiver equipped in the user device, but this is only an embodiment, and the global positioning system (GPS) receiver may also be installed on one side of the electric wheelchair.

[0069] In the locked state 50, the electric wheelchair control device may perform at least one biometric recognition of fingerprint recognition, iris recognition, face recognition, and weight recognition to perform user identification and authentication according to the pre-set security and / or lock mode level. If the biometric recognition is successful, the electric wheelchair control device may transition to the awake state 20 and wait for a user input signal.

[0070] In an embodiment, when the result feedback is completed, the electric wheelchair control device may return to the awake state 20 and wait for the next user instruction.

[0071] The electric wheelchair control device may drive a wake-up timer for a predefined time each time it enters the wake-up state 20. The electric wheelchair control device may wait for a user input signal for controlling the electric wheelchair only in a state where the wake-up timer is being driven. If, in the case where the wake-up timer expires, the electric wheelchair control device may transition to the standby state 10 and turn off the display screen.

[0072] According to the menu setting, in the awake state 20 , if the user identification and authentication fails for a predetermined number of times or more, the electric wheelchair control device of the embodiment may transition to the standby state 10 .

[0073] When a pre-registered quit command (Quit Command) is detected in any of the wake-up state 20 , the analysis state 30 , and the result feedback state 40 , the electric wheelchair control device of the embodiment may also transition to the standby state 10 .

[0074] Figure 2 FIG. 1 is a flowchart for explaining the initial startup procedure of the electric wheelchair control device of the embodiment.

[0075] When power is applied, the electric wheelchair control device may initialize an equipped main controller, ie, a main processor, by booting / loading (step S210 ).

[0076] The main controller may initialize a communication board for communicating with external devices and systems (step S220 ).

[0077] In one example, when the communication board is initialized, the electric wheelchair control device can search for a user device and pair with the searched user device. The electric wheelchair control device can be linked with an external device, such as an intersection (or traffic) signal system, a road side unit (RSU), a base station, etc., through the user device. In one example, the electric wheelchair control device can obtain current traffic light signal information from the intersection signal system through the user device. In another example, the electric wheelchair control device can also obtain user information through the base station and the server.

[0078] The main controller may initialize the display panel and activate an always-on display (AOD) operating mode (step S230 ).

[0079] The main controller may initialize the sensor board (step S240). The main controller may receive image signals captured by (multiple) cameras and user voice signals input through a microphone installed on the electric wheelchair through the sensor board. In addition, the main controller may also receive iris recognition signals, fingerprint recognition signals, gesture recognition signals, touch recognition signals, illumination sensing signals, etc. through the sensor board. In addition, the sensor board may also be connected to ultrasonic sensors, radars, smart parking system (SPAS, SmartParking Assistance) sensors, etc. and receive sensing signals from corresponding multiple sensors.

[0080] The main controller can perform calibration on the cameras and various sensors connected to the sensor board and maintain the resolution and sensing accuracy of multiple cameras and sensors at a predefined level.

[0081] The main controller may initialize the steering board and the driving board and complete the startup steps (step S250 and step S260). The driving board may be configured to control the motor(s) for traveling of the electric wheelchair.

[0082] The main controller of the embodiment can automatically connect with the pre-registered user device in a wireless or wired manner when initializing the communication board. In one example, the electric wheelchair control device and the user device can exchange information through short-range wireless communication, such as Bluetooth communication.

[0083] Figure 3 It is a flowchart for explaining the steps of setting the initial operation control mode of the embodiment.

[0084] Reference Figure 3 , when the initial startup steps are completed, the electric wheelchair control device can be paired with the user device (step S310).

[0085] The electric wheelchair control device may connect to the server through the paired user device to obtain user information (step S320).

[0086] The electric wheelchair control device may determine and set an initial operation control mode based on the acquired user information (step S330 ).

[0087] Through the above-mentioned embodiments, the present application adaptively determines and sets the initial working control mode according to the user's disability type and health status, etc., so that the user's inconvenience can be minimized and the electric wheelchair can be controlled more safely.

[0088] Figure 4 FIG. 1 is a flowchart for explaining the electric wheelchair control method in the electric wheelchair control device of the embodiment.

[0089] In detail, Figure 4 A diagram for explaining a method in which the electric wheelchair control device and the camera system are linked to provide a view screen when the initial operation control mode is set to the voice recognition control mode.

[0090] Reference Figure 4 In the state where the light-emitting diode (LED) screen in the always-on display (AOD) working mode is turned off, that is, when the screen is turned off, the electric wheelchair control device can detect the input user voice signal through the microphone (step S410).

[0091] The electric wheelchair control device can analyze the detected voice signal through the equipped voice recognition engine. When the analysis result is that the detected voice signal is a predefined start instruction, the display screen can be switched from the off state to the on state and the user interface screen can be displayed on the equipped display (step S420).

[0092] The electric wheelchair control device may wait for a user voice command (step S430).

[0093] The electric wheelchair control device detects the user's voice command. When the detected voice command is a camera control command, the image captured by at least one camera can be acquired by transmitting the corresponding control command to the camera system (step S440).

[0094] Among them, the camera control instructions may include front view display instructions, left view display instructions, right view display instructions, rear view display instructions, left / right view display instructions, front surround view display instructions, rear surround view display instructions, bird's-eye view display instructions, etc., but are not limited to these.

[0095] The electric wheelchair control device can construct a view image based on the acquired image(s) and display it on a display (or user device) (step S450 ).

[0096] Figure 5 FIG. 1 is a flowchart for explaining a user registration procedure in the electric wheelchair control device of the embodiment.

[0097] Reference Figure 5 , the electric wheelchair control device can initiate a user registration step (step S510) according to the user menu selection on the interface screen.

[0098] The electric wheelchair control device may recognize the user's voice through a microphone to extract first characteristic data (step S520).

[0099] The electric wheelchair control device may extract second characteristic data by recognizing the user's iris through the iris sensor (step S530 ).

[0100] The electric wheelchair control device may extract third characteristic data by identifying the user's fingerprint through the fingerprint sensor (step S540).

[0101] The electric wheelchair control device can determine the authentication weighted value of each characteristic data based on the pre-learned first template data, second template data and third template data, and the above-mentioned pre-learned first template data, second template data and third template data correspond to the first characteristic data, second characteristic data and third characteristic data respectively (step S550).

[0102] The corresponding user authentication level may be determined and registered based on the determined authentication weight value of each characteristic data (step S560). The extracted characteristic data and the determined authentication level information of each user may be stored and maintained in a user database and managed.

[0103] In one example, some users may have stronger voice characteristics than other users. This means that it is easier to identify and authenticate users through voice. Also, some users may have weak voice characteristics but stronger iris characteristics. In this case, the accuracy of user identification and authentication can be improved by setting a high iris authentication weighting value compared to voice. As described above, the present application can adaptively determine the authentication weighting value based on the authentication data characteristics of each user, which has the advantage of not only improving the reliability of user identification and authentication by determining the optimal authentication level, but also minimizing user inconvenience by minimizing unnecessary user authentications.

[0104] In the above Figure 5 In the embodiment, voice recognition, iris recognition and fingerprint recognition have been described as user authentication units, but this is only one embodiment, and other biometric recognition units such as face recognition and weight recognition may also be added.

[0105] Figure 6 FIG. 1 is a flowchart for explaining a user authentication procedure in the electric wheelchair control device of the embodiment.

[0106] Reference Figure 6 , when the user authentication step is initiated, the electric wheelchair control device may activate the biometric recognition engine (step S610).

[0107] Among them, the biometric recognition engine may include at least one of a voice recognition engine, an iris recognition engine, a fingerprint recognition engine, a face recognition engine, and a gesture recognition engine.

[0108] The electric wheelchair control device may obtain at least one characteristic data through at least one biometric recognition engine (step S620).

[0109] The electric wheelchair control device may select a candidate user from the user database based on the acquired characteristic data (step S630 ). There may be multiple candidate users according to the characteristic data.

[0110] The electric wheelchair control device may identify a pre-registered authentication level corresponding to the selected candidate user (step S640).

[0111] The electric wheelchair control device may determine whether additional authentication is required according to the identified authentication level (step S650).

[0112] In the case where additional authentication is required, the electric wheelchair control device may perform additional authentication steps according to the corresponding authentication level to perform final authentication on the candidate user(s) (step S660 ).

[0113] In the above step S650, when it is determined that additional authentication is not required, the electric wheelchair control device may determine the selected candidate user as the final authenticated user.

[0114] As described above, the present application adaptively determines the biometric recognition unit used to authenticate the user according to the pre-registered authentication level, thereby having the advantage that the user can be identified and authenticated more accurately.

[0115] Also, the present application performs a minimum number of authentication steps according to a predetermined authentication level of each user, thereby having an advantage in that user inconvenience can be minimized.

[0116] Figure 7 The flowchart is used to illustrate the user authentication steps based on the authentication application user mode according to the embodiment.

[0117] Reference Figure 7 , the electric wheelchair control device can be set to the authentication application user mode through the user's predetermined menu selection (step S710).

[0118] Among them, the authentication application user mode can be divided into user-dependent mode and user-independent mode.

[0119] The user dependent mode is a mode in which the user authentication procedure is executed only for users registered (selected) in advance by the user.

[0120] The user-independent mode is a mode in which the user authentication procedure is performed for all users registered in the authentication database.

[0121] In the case of the user-dependent mode, the user authentication step is performed only for a specific user. Therefore, its advantage is that user authentication can be performed more quickly. However, in the case of the user-dependent mode, there is a problem of restricting the use of the electric wheelchair due to authentication failure for unregistered (or selected) users.

[0122] In contrast, the user-independent mode performs user authentication on all users registered in the authentication database. Therefore, its advantage is that the probability of authentication failure is very low, but its disadvantage is that the time required for user authentication increases compared to the user-dependent mode.

[0123] When the user input signal is detected, the electric wheelchair device may initiate a user authentication step (step S720 ).

[0124] When the user authentication step is initiated, the electric wheelchair control device may determine whether the currently set authentication application user mode is a user-dependent type or a user-independent type (step S730 ).

[0125] If the result of the determination is the user-dependent mode, the electric wheelchair control device can perform user authentication only for a specific user who has been registered (or selected) in advance (step S740 ).

[0126] If the result of determination in step 730 is that the mode is user-independent, the electric wheelchair control device may perform user authentication on all users registered in the authentication database (step S750 ).

[0127] Figure 8 A diagram for illustrating detailed operation of an electric wheelchair providing a camera view screen based on speech recognition artificial intelligence (AI) according to an embodiment.

[0128] Reference Figure 8 , the electric wheelchair 800 may include a microphone 810 , a speech recognition engine 820 , a main controller 830 , a camera system 840 , and a display 850 .

[0129] Although the camera system 840 of the embodiment may include a left view camera 841, a right view camera 842, a front view camera 843, and a rear view camera 844, this is only an embodiment and may include more or less cameras than these.

[0130] In yet another embodiment, the camera system 840 may further include a Surround View Monitor (SVM) camera. The Surround View Monitor (SVM) camera may be installed on the front / rear / left / right sides of the vehicle to provide not only a wide view (front camera view) but also a front top view (or front surround view) (front / left / right camera composite view), a left side view (left camera view), a right side view (right camera view), a rear view (rear camera view), a rear top view (or rear surround view) (rear / left / right camera composite view), etc.

[0131] In another embodiment, the camera system 840 may also include a Smart Parking Assistance System (SPAS) sensor. In this case, the main controller 830 may detect short-distance obstacles based on sensing information received from the Smart Parking Assistance System (SPAS) sensor. When a short-distance obstacle is detected, the main controller 830 may display the short-distance obstacle detection information through the display 850. In one example, the main controller 830 may identify the detected obstacle position based on the sensing information received from the Smart Parking Assistance System (SPAS) sensor. The main controller 830 may match the camera view screen corresponding to the identified obstacle position with the identified obstacle. In one example, when an obstacle is detected on the left side of the vehicle, the main controller 830 may form a view screen of the obstacle detection result included in the image captured by the left camera and display it on the display 850. Therefore, its advantage is that the user can intuitively confirm where the short-distance obstacle is.

[0132] Hereinafter, the steps of forming a view screen of the electric wheelchair 800 based on voice recognition artificial intelligence (AI) will be described in detail.

[0133] A user voice signal input through the microphone 810 may be input to the voice recognition engine.

[0134] The voice recognition engine 820 may extract a keyword, that is, a user voice command, by performing natural language processing on the received user voice signal after removing noise included in the voice signal.

[0135] The main controller 830 may identify the view type based on the user voice command extracted by the voice recognition engine 820, and request an image corresponding to the view type identified by the camera system 840. The camera system 840 may transmit at least one camera-captured image corresponding to the identified view type to the main controller 830.

[0136] The main controller 830 may construct a view frame to be displayed on the display 850 based on the image received from the camera system 840, and display the constructed view frame through the display 850. Fig. 9 As shown, the view screen that can be configured by the main controller 830 can be variously configured according to the user's voice command.

[0137] Fig. 9 FIG. 4 is an illustration of a view screen provided by an electric wheelchair control device according to an embodiment.

[0138] Reference Fig. 9 The view screens that can be displayed on the display 850 can roughly include a left camera view screen 910, a right camera view screen 920, a side camera view screen 930, a front camera view screen 940, a front (or rear) surround view screen 950 and a bird's-eye view screen 960.

[0139] The electric wheelchair 800 can adaptively form a view image according to the user's voice command and display it on the equipped display 850.

[0140] In the case of detecting short-distance obstacles by linking with a smart parking assist system (SPAS) sensor, the electric wheelchair 800 of the embodiment can form a view screen on one side of the corresponding view screen to further display obstacle detection information. The obstacle detection information may include obstacle location information and obstacle type information, but is not limited thereto.

[0141] Fig.10 FIG. 4 is a block diagram for illustrating the structure of an electric wheelchair system according to an embodiment of the present invention.

[0142] like Fig.10 As shown, the electric wheelchair system 1000 may generally include: an electric wheelchair control device 1010 ; a peripheral control board connected to the electric wheelchair control device 1010 ; a user device 1051 ; and a server 1052 .

[0143] The peripheral control board may include a communication board 1050 , a display board 1060 , a sensor board 1070 , a steering board 1080 , and a driving board 1090 .

[0144] The communication board 1050 can set a communication channel with the user equipment 1051 by installing an antenna and a transceiver for communicating with an external device, and exchange information through the set communication channel. In one example, the communication board 1050 can provide at least one communication function of Bluetooth communication, fourth generation mobile communication technology (4G) long term evolution (LTE, Long Term Evolution) communication, fifth generation mobile communication technology (5G) new air interface (NR, New Radio) communication and wireless Internet communication.

[0145] Also, the communication board 920 can transmit the view screen formed by the electric wheelchair control device 1010 to the user device 1051 by connecting with the user device 1051. The user device 1051 can display the received view screen on one side of the equipped screen.

[0146] The display panel 1060 can be connected to a light emitting diode 1061 (LED) equipped with a touch sensor, and can display the user interface screen and various status information of the electric wheelchair 1000 on the light emitting diode 1061 (LED) according to the control signal of the electric wheelchair control device 1010. The display panel 1060 can control the light emitting diode 1061 (LED) in an always-on display (AOD) working mode.

[0147] The sensor board 1070 can be connected to at least one of the illumination sensor 1071, the proximity sensor 1072, the touch sensor 1073, the fingerprint sensor 1074, the iris sensor 1075, the ultrasonic sensor 1076, the radar 1077, the microphone 1078, and the camera system 1079, and can provide sensing information obtained from the corresponding sensor to the electric wheelchair control device 1010. The electric wheelchair control device 1010 can perform illumination recognition, gesture recognition, touch recognition, user authentication, user voice recognition, etc. based on the sensing information or voice signal received from at least one of the illumination sensor 1071, the proximity sensor 1072, the touch sensor 1073, the fingerprint sensor 1074, the iris sensor 1075, and the microphone 1078.

[0148] Furthermore, the sensor board 1070 can receive an image signal captured by at least one camera by connecting to the camera system 1079, and provide the received image signal to the electric wheelchair control device 1010. The electric wheelchair control device 1010 can construct a view screen based on the image signal received from the sensor board 1070 and transmit the constructed view screen to the display board 1060 and / or the communication board 1050.

[0149] Furthermore, the sensor board 1070 may also be connected to a smart parking assistance system (SPAS) sensor (not shown). The sensor board 1070 may provide sensing information received from the smart parking assistance system (SPAS) sensor to the electric wheelchair control device 1010. In one example, the electric wheelchair control device 1010 may detect a short-distance obstacle based on the information sensed from the smart parking assistance system (SPAS) sensor.

[0150] When initialized, the sensor board 1070 can perform calibration work on the connected multiple sensors and camera system 1079.

[0151] The illumination sensor 1071 can measure the illumination around the electric wheelchair control device 1010 and / or around the light emitting diode 1061 (LED). The electric wheelchair control device 1010 can use the illumination sensing information received from the sensor board 1070 to dynamically adjust the brightness of the light emitting diode 1061 (LED) screen.

[0152] The steering plate 1080 can detect a user's steering command by connecting to the steering rod 1081, and control a steering control device (not shown) equipped in the electric wheelchair 1000 according to the detected steering command.

[0153] The driving board 1090 may control the driving speed of the motor 1091 by being connected to the motor 1091 .

[0154] The electric wheelchair control device 1010 may include a main controller 1030 , a memory 1020 , various recognition engines, and a user authentication unit 1040 .

[0155] Among them, the recognition engine may include at least one of the following: a voice recognition engine 1031, which recognizes user control instructions by performing natural language processing on user voice signals; a face recognition engine 1032, which recognizes user facial movements from images captured by a camera to recognize user control instructions; a gesture recognition engine 1033, which recognizes user gestures and recognizes user control instructions corresponding to the recognized gestures; and a touch recognition engine, which recognizes user touch operations on the touch screen and recognizes user control instructions corresponding to the recognized touch operations.

[0156] The user authentication unit 1040 may include at least one of a voice recognition unit 1041 , an iris recognition unit 1042 , a fingerprint recognition unit 1043 , a face recognition unit 1044 , and an authentication level determination unit 1045 .

[0157] The voice recognition unit 1041 may extract user voice characteristics from a voice signal input by the user, and perform user recognition and authentication by comparing the extracted user voice characteristics with voice characteristic data pre-registered in the memory 1020 .

[0158] The voice recognition unit 1041 may extract voice characteristic data of the corresponding user based on the voice signal input by the user, and determine the authentication weight value of the corresponding voice characteristic data by comparing the extracted voice characteristic data with the learned voice template data.

[0159] The speech recognition unit 1041 can improve the speech recognition sensitivity by removing noise included in the speech signal.

[0160] The iris recognition unit 1042 can extract the user's iris characteristics from the iris image received by the iris sensor 1075 , and perform user identification and authentication by comparing the extracted user's iris characteristics with iris characteristic data pre-registered in the memory 1020 .

[0161] The iris recognition unit 1042 may extract iris characteristic data of the corresponding user based on the iris sensing information, and determine an authentication weight value of the corresponding iris characteristic data by comparing the extracted iris characteristic data with the learned iris template data.

[0162] The fingerprint recognition unit 1043 can extract the user's fingerprint characteristics from the fingerprint image received by the fingerprint sensor 1074 , and perform user identification and authentication by comparing the extracted user's fingerprint characteristics with the fingerprint characteristic data pre-registered in the memory 1020 .

[0163] The fingerprint recognition unit 1043 may extract corresponding user fingerprint characteristic data based on the fingerprint sensing information, and determine an authentication weight value of the corresponding fingerprint characteristic data by comparing the extracted fingerprint characteristic data with the learned fingerprint template data.

[0164] The face recognition unit 1044 may extract user facial features from the user facial image captured by the camera, and perform user recognition and authentication by comparing the extracted user facial features with facial feature data pre-registered in the memory 1020 .

[0165] The face recognition unit 1044 may extract facial feature data of the corresponding user based on the facial sensing information, and determine an authentication weighted value of the corresponding facial feature data by comparing the extracted facial feature data with the learned facial template data.

[0166] The authentication level determination unit 1044 may dynamically determine the authentication level for the corresponding user based on the authentication weight value determined by the above-mentioned identification unit(s).

[0167] The authentication level determination unit 1044 may sort the characteristic data in descending order of the authentication weight value of each user and sum up the authentication weight values ​​until the cumulative sum of the authentication weight values ​​of the sorted characteristic data reaches a predetermined threshold.

[0168] When the cumulatively added authentication weight value is greater than or equal to the threshold, the authentication level determination unit 1044 may determine the authentication level based on the added characteristic data type and quantity.

[0169] In one example, when the authentication weighted value corresponding to the voice characteristic data alone reaches the threshold, the authentication level may be determined as the first level. When the sum of the authentication weighted value corresponding to the voice characteristic data and the authentication weighted value corresponding to the iris characteristic data reaches or exceeds the threshold, the authentication level may be determined as the second level. When the sum of the authentication weighted value corresponding to the voice characteristic data, the authentication weighted value corresponding to the iris characteristic data, and the authentication weighted value corresponding to the fingerprint characteristic data reaches or exceeds the threshold, the authentication level may be determined as the third level. When the sum of the authentication weighted value corresponding to the voice characteristic data, the authentication weighted value corresponding to the iris characteristic data, the authentication weighted value corresponding to the fingerprint characteristic data, and the authentication weighted value corresponding to the face characteristic data reaches or exceeds the threshold, the authentication level may be determined as the fourth level.

[0170] The memory 1020 may maintain various firmware and software required to operate the electric wheelchair control device 1010. Also, for user identification and authentication, the memory 1020 may database and maintain pre-registered characteristic data of each user. Also, the memory 1020 may also maintain various parameter information including speaker patterns for authentication applications. In the memory 1020, characteristic data and authentication levels of each user may be stored and maintained. Also, in the memory 1020, pre-learned template data for each authentication type may be maintained.

[0171] The main controller 1030 may recognize at least one of user gestures, voice commands, facial movements, and touch operations according to a set operation control mode to activate a light emitting diode 1061 (LED) screen of an always on display (AOD) operation mode.

[0172] Furthermore, when the light emitting diode 1061 (LED) screen of the always on display (AOD) working mode is activated, the main controller 1030 can construct a user interface screen for controlling the electric wheelchair 1000 by transmitting the constructed user interface screen to the display panel 1060 and displaying it on the light emitting diode 1061 (LED) screen.

[0173] Also, pre-learned template data for each authentication type may be maintained in the memory 1020 .

[0174] In the above Fig.10 Although not explicitly shown in the figure, the electric wheelchair 1000 may also include a rechargeable battery, a wheelchair frame, footrests, wheels, speakers, lights, shock absorbers, etc.

[0175] The sensor plate 1070 of the embodiment can be linked to a body pressure sensor (not shown) that measures the body pressure distribution of the passenger on the seat.

[0176] The main controller 1030 can control the air supply to a seat cushion (not shown) disposed on one side of the seat based on the body pressure sensing information.

[0177] Fig.11 An example of an electric wheelchair according to the embodiment is shown.

[0178] Fig.11 Reference numeral 1110 is a left view of the electric wheelchair, reference numeral 1120 is a right view of the electric wheelchair, reference numeral 1130 is a front view of the electric wheelchair, and reference numeral 1140 is a rear view of the electric wheelchair.

[0179] Referring to reference numeral 1110, a left camera 1111 for photographing the left side of the electric wheelchair may be installed on the outer side (outer side) of the left handle 1116. In addition, one end of a bracket 1114 may be connected to one side of the backrest 1115, and a face camera 1113 for face recognition, a touch display 1112, and a microphone 1117 for voice recognition may be installed on the other end of the bracket 1114. In an embodiment, the bracket 1114 may be implemented in a manner that it can rotate left / right.

[0180] Referring to reference numeral 1120 , a right camera 1121 and a steering rod 1122 for photographing the right side of the electric wheelchair may be installed on the outer (outer) side of the right handle 1123 .

[0181] Referring to the figure mark 1130, a front camera 1131 for capturing images in front of the electric wheelchair can be installed on one side of the front surface of the right handle 1123, and an ultrasonic sensor 1132 and / or a radar (not shown) for detecting approaching obstacles can be provided on one side of the lower end of the front surface of the electric wheelchair.

[0182] The electric wheelchair control device 1010 and various boards may be mounted on one side of the electric wheelchair body.

[0183] The steps of the methods or algorithms described in connection with the embodiments disclosed in this specification may be implemented by hardware, software modules, or a combination of the two, executed by a processor. The software modules may reside in a storage medium (i.e., memory and / or storage device) such as a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable disk, a compact disk (CD-ROM), or the like.

[0184] An exemplary storage medium may be coupled to a processor, which may read information from the storage medium and may write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. The processor and the storage medium may reside in an application specific integrated circuit (ASIC). The application specific integrated circuit (ASIC) may also reside in a user terminal. Alternatively, the processor and the storage medium may also reside in a user terminal as separate components.

[0185] The above description is only an exemplary description of the technical concept of the present invention, and a person skilled in the art can make various modifications and variations without departing from the essential features.

[0186] Therefore, the embodiments disclosed in the present invention are only for illustration, and are not intended to limit the technical ideas of the present invention, and the scope of the technical ideas of the present invention is not limited to the above embodiments. The protection scope of the present invention should be interpreted by the protection scope of the attached invention claims, and all technical ideas within the equivalent scope should be interpreted as included in the protection scope of the present invention.

Claims

1. A method for controlling an electric wheelchair, based on artificial intelligence, characterized in that: include: performing a startup step to pair with a user device as power is applied; Obtain user information from the server through the paired user device; Determine the working control mode of the electric wheelchair based on the above user information; In a state where the display screen is always turned off, a predefined start instruction is identified through a sensor corresponding to the determined working control mode; Activate the closed screen according to the start instruction, output the user interface screen for controlling the electric wheelchair to the display and wait for a user input signal; Performing machine learning on the user input signal input through the above-mentioned sensor to perform user identification and authentication; and Based on the success of the above authentication, the control instruction corresponding to the above user input signal is identified to control the operation of the above electric wheelchair, The above-mentioned user information includes at least one of disability type information and health status information.

2. The electric wheelchair control method according to claim 1, characterized in that: Also includes: setting authentication application user mode, The above user authentication includes: User-dependent authentication, which performs authentication only on specific users that are set in advance; and User-independent authentication performs authentication on all users registered in the authentication database.

3. The electric wheelchair control method according to claim 1, characterized in that: The above-mentioned working control mode includes at least one of a manual control mode, a voice recognition control mode, a face recognition control mode, a gesture recognition control mode and an autonomous driving control mode.

4. The electric wheelchair control method according to claim 3, characterized in that: Also includes: Based on the working control mode being the voice recognition control mode, determining a view type corresponding to the recognized voice command; Determining at least one camera to receive an image corresponding to the determined view type; Constructing a view based on images acquired from the at least one camera; and Displaying the above-mentioned view screen on the screen of the above-mentioned user equipment or the above-mentioned display, The above view types include at least one of a front view, a left view, a right view, a rear view, a left / right view, a front surround view, a rear surround view and a bird's-eye view.

5. The electric wheelchair control method according to claim 4, characterized in that: The method further comprises: using at least one of an intelligent parking assistance system sensor, an ultrasonic sensor and a radar to detect an approaching obstacle during the driving of the electric wheelchair. Based on the detection of the approaching obstacle, information of the detected approaching obstacle is displayed on one side of the view screen.

6. The electric wheelchair control method according to claim 1, characterized in that: The above sensors include microphones, cameras and biometric sensors. The biometric sensor includes at least one of a fingerprint sensor, an iris sensor, and a body pressure sensor. Initiate a user registration step according to a predetermined user menu selection on the above user interface screen, The above user registration steps include: Recognizing the user's voice through the microphone to extract first characteristic data; Recognize the user's iris from the image captured by the camera to extract second characteristic data; Extracting third characteristic data by identifying the user's fingerprint through the fingerprint sensor; Recognize the user's face from the image captured by the camera to extract fourth characteristic data; determining an authentication weighted value of each characteristic data based on pre-learned first template data, second template data, third template data, and fourth template data, wherein the pre-learned first template data, second template data, third template data, and fourth template data correspond to the first characteristic data, second characteristic data, third characteristic data, and fourth characteristic data, respectively; and Determine the authentication level of the corresponding user based on the determined authentication weight value of each of the above characteristic data, The extracted characteristic data of each user and information related to the determined authentication level are stored and managed in an internal memory, and the user authentication is performed according to the determined authentication level of each user.

7. An electric wheelchair control system, an artificial intelligence-based electric wheelchair control system for controlling an electric wheelchair, characterized in that: Including peripheral panels and electric wheelchair controls, The above peripheral board includes: A display panel, which displays the user interface screen on a display; a sensor board that generates sensing information based on a signal input from a sensor; a communication board for performing communication with the electric wheelchair and external devices; A driving board for controlling a motor provided in the electric wheelchair; and A steering plate controls the steering of the electric wheelchair. The electric wheelchair control device controls the overall operation of the electric wheelchair by interlocking with the peripheral board. The electric wheelchair control device performs a startup step as power is applied to pair with the user device, obtains user information from a server through the paired user device, determines a working control mode of the electric wheelchair based on the user information, and identifies a predefined startup instruction through the sensor corresponding to the determined working control mode when the display screen is always turned off, activates the closed screen according to the startup instruction, outputs a user interface screen for controlling the electric wheelchair to the display, and waits for a user input signal, performs machine learning on the user input signal input through the sensor to perform user identification and authentication, and based on the success of the authentication, identifies a control instruction corresponding to the user input signal to control the operation of the electric wheelchair. The above-mentioned user information includes at least one of disability type information and health status information.

8. The electric wheelchair control system according to claim 7, characterized in that: The electric wheelchair control device performs the user authentication in a preset authentication application user mode. The above user authentication includes: User-dependent authentication, which performs authentication only on specific users that are set in advance; and User-independent authentication performs authentication on all users registered in the authentication database.

9. The electric wheelchair control system according to claim 7, characterized in that: The above-mentioned working control mode includes at least one of a manual control mode, a voice recognition control mode, a face recognition control mode, a gesture recognition control mode and an autonomous driving control mode.

10. The electric wheelchair control system according to claim 7, characterized in that: The above sensors include microphones, cameras and biometric sensors. The biometric sensor includes at least one of a fingerprint sensor, an iris sensor, and a body pressure sensor. The electric wheelchair control device performs a user registration step according to the user menu selection on the user interface screen. The above user registration steps include: Recognizing the user's voice through the microphone to extract first characteristic data; Recognize the user's iris from the image captured by the camera to extract second characteristic data; Extracting third characteristic data by identifying the user's fingerprint through the fingerprint sensor; Recognize the user's face from the image captured by the camera to extract fourth characteristic data; determining an authentication weighted value of each characteristic data based on pre-learned first template data, second template data, third template data, and fourth template data, wherein the pre-learned first template data, second template data, third template data, and fourth template data correspond to the first characteristic data, second characteristic data, third characteristic data, and fourth characteristic data, respectively; and Determine the authentication level of the corresponding user based on the determined authentication weight value of each of the above characteristic data, The extracted characteristic data of each user and information related to the determined authentication level are stored and managed in an internal memory, and the user authentication is performed according to the determined authentication level of each user.

Images