An automatic decision-making intelligent wheelchair device for ALS patients and a control method thereof

Abstract

The application relates to an automatic decision intelligent wheelchair device for ALS patients based on an artificial intelligence technology and a control method, which at least comprises a perception module and a path planning unit. An environmental perception module carried on a vehicle body perceives an environment to collect surrounding environment signals; different information is subjected to pattern recognition through a perception algorithm model, a deep learning model is used for target detection and region segmentation, objects around the wheelchair are identified and classified, and corresponding size, shape and distance discrimination is carried out; according to information calculation and object detection carried out by the perception module, an environmental map is constructed, a minimum distance or shortest time measurement action path configuration is carried out based on the generated map; the application is based on an artificial intelligence environmental perception and path planning algorithm technology, realizes the realization of the autonomous movement intelligent decision function of the wheelchair, greatly helps the movement inconvenient patients who are inconvenient in muscle control, and enables the patients in the group to carry out daily action auxiliary support.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of mobile robot object detection and path planning in artificial intelligence, and particularly relates to an automatic decision intelligent wheelchair device for ALS patients based on artificial intelligence technology and a control method. BACKGROUND

[0002] Most of the current related existing automatic wheelchairs are based on mechanical automation design schemes driven by motors. Most of the current mobile wheelchair structures are designed as manual power settings, and the force transmission is mainly driven by the force of external manpower (nurses and the user's own force).

[0003] There is no highly intelligent wheelchair device and technical scheme on the market at present. The current existing automatic wheelchair technology is based on mechanical automation design schemes driven by motors. The current wheelchair devices on the market cannot be better used by ALS users with muscle control force. Few intelligent wheelchairs are specially provided for ALS patients and can intelligently perceive the environment and intelligently plan the path.

[0004] Therefore, it is necessary for those skilled in the art to design an automatic moving wheelchair for ALS patients through artificial intelligence algorithm technology. SUMMARY

[0005] Therefore, the present application aims to overcome the shortcomings of the prior art and provide an automatic decision intelligent wheelchair device for ALS patients based on artificial intelligence technology. The automatic decision intelligent wheelchair device can combine the environmental intelligent perception and intelligent path planning algorithm of the artificial intelligence algorithm, so as to realize the autonomous movement and control of the wheelchair according to the surrounding environment, and facilitate the use of ALS users with muscle control force.

[0006] To achieve the above purpose, the first aspect of the present application provides an automatic decision intelligent wheelchair device for ALS patients, characterized by comprising wheels and a vehicle body arranged on the wheels; wherein the vehicle body is provided with at least:

[0007] A motor drive control system for driving and controlling the movement of the wheelchair device;

[0008] A perception module, wherein a plurality of source sensors are mounted on the vehicle body to collect surrounding environment signals; wherein the plurality of source sensors at least include a laser radar, a distance ultrasonic probe and a visual camera;

[0009] Further comprising a data processing unit, a signal transceiver module, and a data platform and a computing platform located in the cloud;

[0010] The laser radar, the distance ultrasonic probe and the visual camera are electrically connected with the data processing unit, and the data processing unit is used for data processing of surrounding environment signals collected by the multi-source sensors;

[0011] The data processing unit is electrically connected with the signal transceiver module, and the signal transceiver module is connected with the data platform and the computing platform in a wired or wireless manner;

[0012] Further, a path planning unit is arranged, and the path planning unit is communicatively connected with the data platform and the computing platform located in the cloud.

[0013] Further, an interaction and early warning system is arranged, and the interaction and early warning system is communicatively connected with the data platform and the computing platform located in the cloud; a voice recognition control system is also arranged, and the voice recognition control system is communicatively connected with the data platform and the computing platform located in the cloud; further comprising a braking system and a remote control system, the braking system is communicatively connected with the data platform and the computing platform located in the cloud; and the remote control system is communicatively connected with the data platform and the computing platform located in the cloud.

[0014] The second aspect of the present application provides an automatic decision intelligent wheelchair control method for ALS patients, characterized in that the control method adopts the wheelchair device described above, and the specific control method at least comprises the following steps:

[0015] An environment perception module mounted on the vehicle body is used to perceive the environment and collect surrounding environment signals;

[0016] The collected surrounding environment signals are transmitted to the data platform and the computing platform located in the cloud;

[0017] The data platform and the computing platform perform data fusion processing on the received surrounding environment signals;

[0018] Based on the environment perception information, different information is recognized by a perception algorithm model, a deep learning model is used for target detection and region segmentation, objects around the wheelchair are recognized and classified, and corresponding size, shape and distance are determined;

[0019] According to the information calculation and object detection of the perception module, an environment map is constructed, based on the generated map, a minimum distance or shortest time measurement action path configuration is performed;

[0020] Based on the generated mobile path map planning scheme, mobile interaction is carried out, and real-time environmental data feedback calculation is carried out to calculate risk obstacles, and obstacle avoidance algorithm configuration and early warning system implementation are carried out according to obstacle classification;

[0021] Based on the path planning and interaction and early warning algorithm information feedback, strategy iteration is carried out, combined with the wheelchair motor drive control system, intelligent control is carried out.

[0022] Further, in the step of perceiving the environment to collect the surrounding environment signals by the environmental perception module mounted on the vehicle body, the environmental perception module collects the surrounding environment signals by the multi-source sensor mounted on the vehicle body; wherein the multi-source sensor at least includes a laser radar, a distance ultrasonic probe and a visual camera;

[0023] The collected surrounding environment signal perception data is encoded and encrypted, and then transmitted to the data platform and computing platform located in the cloud through wired or wireless mode.

[0024] Further, the data analysis of the collected surrounding environment signals mainly includes:

[0025] a) filtering the original laser radar data by using a filtering algorithm to filter the noise signal trace, obtaining the environment point cloud imaging data based on the laser radar;

[0026] b) processing the ultrasonic sensor data by using an image reconstruction algorithm to calculate the environmental distance feature information;

[0027] c) based on the camera sensor data, the environment is perceived and reconstructed, and the surrounding environment data is restored to facilitate subsequent algorithm recognition and calculation.

[0028] Further, the sensor collected and processed environmental data characteristic values are sent into the algorithm model for model recognition and classification to detect the object distribution characteristics and distribution density in the environment;

[0029] According to the detected target object, the environmental map representation is constructed to provide data support for subsequent decision calculation.

[0030] Further, the environmental information calculation target recognition and target classification results are sent to the computing control unit and sent to the data platform and computing platform through high-speed transmission of information compression;

[0031] The computing platform designs different strategies of mobile path according to the reconstructed environmental representation information, adopts the shortest distance planning and shortest time planning algorithm, and defines the boundary of the surrounding objects by combining the graph method, so as to realize mobile obstacle avoidance.

[0032] Further, using the interaction and early warning system, based on the generated mobile path map planning scheme, mobile interaction is carried out, and real-time environmental data is returned to the calculation, the risk obstacle is calculated, the obstacle classification is carried out, the obstacle avoidance algorithm configuration and the early warning system implementation are carried out; specifically including

[0033] 1) After determining the two modes of the mobile path, including the shortest distance or the shortest time, the calculation platform generates a mobile route map, and the environmental data in the mobile process is continuously sent and updated in real time;

[0034] 2) For special obstacles found in the environment, the transmission is evaluated, the risk warning is carried out for the environment with a large risk factor, the background management personnel are notified to confirm whether to take over or stop walking, and the evaluation can select other paths.

[0035] Further, the control method can receive external feedback information, record and mark special changes in the surrounding environment, and automatically update the action measurement;

[0036] 2) According to the information change of the environment, the moving speed and direction are automatically adjusted, instructions are issued to the wheelchair moving control system, and braking is predicted to ensure the safety of the movement.

[0037] Further, the intelligent wheelchair control decision system automatically optimizes the function, and the wheelchair environment perception module real-time collects the data characteristics of the environmental information and returns them to the background database, and is included in the algorithm model for automatic correction, to establish a more accurate norm standard and improve the accuracy of the control algorithm model;

[0038] During the movement, in addition to supporting real-time path planning calculation, the data collected by various sensors is stored in the cloud data platform in parallel, to support model improvement and mobile strategy optimization;

[0039] Different types of data are preprocessed on a large scale on the data platform, special events in the environment and wheelchair walking habits are mined, combined with human environmental walking characteristics, and the algorithm is optimized.

[0040] The above technical scheme is adopted in the present application, and at least the following beneficial effects are achieved:

[0041] The present application is based on artificial intelligence environment perception and path planning algorithm technology, adopts reinforcement learning mechanism, realizes the realization of the autonomous mobile intelligent decision function of the wheelchair, greatly helps the mobile disabled patients with inconvenient muscle control. Make this kind of patient group can carry out daily action auxiliary support. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0043] Figure 1 is one of the structural schematic diagrams of the intelligent wheelchair device of the present application;

[0044] Figure 2 is another structural schematic diagram of the intelligent wheelchair device of the present application;

[0045] Figure 3 is a schematic diagram of the environment perception module of the intelligent wheelchair of the present application;

[0046] Figure 4 is a multi-source information fusion and path planning module of the intelligent wheelchair of the present application;

[0047] Figure 5 is an action control module of the intelligent wheelchair of the present application;

[0048] Figure 6 is a comprehensive control function diagram of the intelligent wheelchair of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0050] The technical solutions of the present application will be further described in detail below through the drawings and embodiments.

[0051] As shown in Figure 1 and Figure 2 , the present embodiment provides an automatic decision intelligent wheelchair device for ALS patients, characterized in that: comprising wheels and a vehicle body arranged on the wheels; wherein the vehicle body is provided with at least:

[0052] a motor drive control system, the motor drive control system is used for driving control wheelchair device movement;

[0053] a perception module, the perception module adopts a multi-source sensor carried on the vehicle body to collect surrounding environment signals; wherein the multi-source sensor at least includes a laser radar, a distance ultrasonic probe and a visual camera;

[0054] Further comprising a data processing unit, a signal transceiver module, and a data platform and a computing platform located in the cloud;

[0055] The laser radar, the distance ultrasonic probe, and the visual camera are electrically connected with the data processing unit, and the data processing unit is used for data processing of surrounding environment signals collected by the multi-source sensors;

[0056] The data processing unit is electrically connected with the signal transceiver module, and the signal transceiver module is connected with the data platform and the computing platform in a wired or wireless manner;

[0057] Further, a path planning unit is arranged, and the path planning unit is communicatively connected with the data platform and the computing platform located in the cloud.

[0058] Further, an interaction and early warning system is arranged, and the interaction and early warning system is communicatively connected with the data platform and the computing platform located in the cloud; a voice recognition control system is also arranged, and the voice recognition control system is communicatively connected with the data platform and the computing platform located in the cloud; further comprising a braking system and a remote control system, the braking system is communicatively connected with the data platform and the computing platform located in the cloud; and the remote control system is communicatively connected with the data platform and the computing platform located in the cloud.

[0059] In the embodiment, the surrounding environment signals are collected and perceived modeling through the specially-made wheelchair device carrying multi-source sensors (laser radar, ultrasonic probe, camera, etc.) and a driver, mainly collecting environment point cloud signals, front view image and video signals, and surrounding environment ultrasonic feedback signals.

[0060] Device features: the wheelchair device is a programmable motor driving mode, a laser radar is carried on the bottom of the wheelchair, ultrasonic probes are carried on the left and right sides of the middle of the wheelchair body, and a camera sensor is carried on the top of the back of the wheelchair; through this arrangement, the surrounding environment and distance can be accurately described and measured.

[0061] After the device is started, all sensors are activated, and data collection is performed in parallel, and the data collection period ends after the wheelchair device is turned off.

[0062] As shown in Figures 3 to 6 The second aspect of the present application provides an automatic decision intelligent wheelchair control method for ALS patients, characterized in that: the control method adopts the above-mentioned wheelchair device, and the specific control method at least includes the following steps:

[0063] An environment perception module carried on the vehicle body is used to perceive the environment and collect surrounding environment signals;

[0064] The collected ambient environment signals are transmitted to a data platform and a computing platform in the cloud;

[0065] The data platform and the computing platform perform data fusion processing on the received ambient environment signals;

[0066] Based on the environmental perception information, different information is recognized by a perception algorithm model, and a deep learning model is used for target detection and region segmentation to identify and classify the objects around the wheelchair and determine the corresponding size, shape and distance;

[0067] According to the information calculation and object detection of the perception module, an environmental map is constructed, and based on the generated map, a minimum distance or shortest time measurement action path configuration is performed;

[0068] Based on the generated mobile path map planning scheme, mobile interaction is performed, and real-time environmental data is returned for calculation to calculate risk obstacles, and an obstacle avoidance algorithm configuration and a warning system are implemented according to the obstacle classification;

[0069] Based on the path planning and interaction and warning algorithm information feedback, strategy iteration is performed, combined with the wheelchair motor drive control system, for intelligent control.

[0070] As a preferred embodiment, in the step of perceiving the environment to collect the ambient environment signals in the embodiment, the environmental perception module uses a multi-source sensor mounted on the vehicle body to collect the ambient environment signals; wherein the multi-source sensor at least includes a laser radar, a distance ultrasonic probe and a visual camera;

[0071] The collected ambient environment signals are encoded and encrypted, and then transmitted to the data platform and the computing platform in the cloud through wired or wireless means.

[0072] As a preferred embodiment, in the embodiment, the signals collected by the multi-source sensor of the perception system mounted on the special wheelchair are amplified, converted and encoded, and transmitted to the local embedded computing system and the cloud data storage and computing platform through wireless connection; the transmission mode of various sensor data can be any wireless connection mode, not limited to Bluetooth, 5G data flow and WiFi, preferably, the application uses 5G Bluetooth to transmit data to the backend computing platform;

[0073] The data analysis of the collected ambient environment signals mainly includes:

[0074] a) The original laser radar data is filtered by a filtering algorithm to filter out noise signal traces, and the environmental point cloud imaging data based on the laser radar is obtained;

[0075] b) using an image reconstruction algorithm to process the ultrasonic sensor data to calculate environmental distance feature information;

[0076] c) based on camera sensor data, environmental perception and environmental reconstruction are performed to restore the surrounding environment data, which facilitates subsequent algorithm recognition and calculation.

[0077] As a preferred embodiment, in this embodiment, the sensor collects and transmits the processed environmental data feature values to the algorithm model for model recognition and classification, and detects the object distribution characteristics and distribution density in the environment;

[0078] It should be noted that 1) in this step, the classification algorithm model used is several commonly used classic models: SVM, decision tree, KNN, random forest, naive Bayes classification, least squares method, logistic regression, etc.

[0079] 2) the environmental object recognition model used is mainly RNN, faster-RNN, Yolo, etc. target detection deep learning model;

[0080] 3) according to the detected target object, the environmental map representation is constructed to provide data support for subsequent decision calculation;

[0081] According to the detected target object, the environmental map representation is constructed to provide data support for subsequent decision calculation.

[0082] As a preferred embodiment, in this embodiment, the path planning algorithm carried on the edge of the special wheelchair and the cloud is used to construct the environmental map based on the information calculation and object detection of the perception module, and based on the generated map, the minimum distance or shortest time measurement action path configuration is performed:

[0083] The environmental information calculation target recognition and target classification result of the previous step is sent to the calculation control unit (data platform and / or calculation platform), and is sent to the data platform and calculation platform through high-speed transmission of information compression;

[0084] The calculation platform performs different strategy mobile path design based on the reconstructed environmental representation information, uses distance shortest planning and time shortest planning algorithm, and defines the boundary of the surrounding object through the combination of graphics method, so as to realize mobile obstacle avoidance.

[0085] As a preferred embodiment, in this embodiment, the interaction and early warning system is used to perform mobile interaction based on the generated mobile path map planning scheme, and real-time environmental data is returned for calculation to calculate the risk obstacle, and the obstacle avoidance algorithm configuration and early warning system implementation are performed according to the obstacle classification; specifically including

[0086] 1) When the two modes of the moving path are determined, including the shortest distance or the shortest time, the computing platform generates a moving route map, and the environmental data during the moving process is continuously sent and updated in real time;

[0087] 2) For special obstacles found in the environment, risk warning is performed on the environment with a large risk factor, and the background management personnel are notified to confirm whether to take over or stop walking, and to evaluate other paths.

[0088] As a preferred embodiment, in this embodiment, the control algorithm carried on the edge of the wheelchair and the cloud is based on path planning and interactive and warning algorithm information feedback, and performs strategy iteration, combined with wheelchair braking and driving control system, to perform intelligent control;

[0089] The control method can receive external feedback information, record and mark special changes in the surrounding environment, and automatically update the action measurement;

[0090] According to the information change of the environment, the moving speed and direction are automatically adjusted, and instructions are issued to the wheelchair moving control system, and prediction braking is performed to ensure the safety of the movement.

[0091] As a preferred embodiment, the intelligent wheelchair control decision system in this embodiment automatically optimizes the function of returning the data characteristics of the environmental information collected by the wheelchair environmental perception module to the background database in real time, and incorporates it into the algorithm model for automatic correction, to establish a more accurate norm standard and improve the accuracy of the control algorithm model;

[0092] During the moving process, in addition to supporting real-time path planning calculation, the data collected by various sensors are stored in the cloud data platform in parallel, supporting model improvement and moving strategy optimization;

[0093] The different types of data are preprocessed on a large scale on the data platform, special events in the environment and wheelchair walking habits are mined, and the algorithm is optimized in combination with human environmental walking characteristics.

[0094] The control method in this embodiment also has other auxiliary intelligent function configurations

[0095] 1) The device is also equipped with a voice recognition control system, which supports the use of other people (with normal voice expression function), and controls the wheelchair through voice function;

[0096] 2) The device is configured with a button control, which is suitable for people with normal hand movement function;

[0097] 3) The device supports manual takeover, and supports manual control switching by external caregivers while the automatic function is performed;

[0098] 4) The device has intelligent emergency braking and remote control functions, and can perform emergency braking on abnormal conditions;

[0099] The application is based on artificial intelligence environment sensing and path planning algorithm technology, adopts a reinforcement learning mechanism, and realizes autonomous mobile intelligent decision function of the wheelchair, greatly assisting the patients with inconvenient movement due to muscle control inconvenience. The patients in this group can perform daily action auxiliary support.

[0100] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. The terms "first", "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance. The term "multiple" refers to two or more, unless otherwise explicitly limited.

Claims

1. An automatic decision-making intelligent wheelchair control method for ALS patients, characterized in that: The control method is implemented by a wheelchair device, and the specific control method comprises at least the following steps: An environmental perception module mounted on the vehicle body is used to perceive the environment and collect surrounding environment signals; The collected surrounding environment signals are transmitted to a data platform and a computing platform located in the cloud; The data platform and the computing platform perform data fusion processing on the received surrounding environment signals; Based on the environmental perception information, different information is identified by a perception algorithm model, target detection and region segmentation are performed using a deep learning model, objects around the wheelchair are identified and classified, and corresponding size, shape and distance are determined; Based on the information calculation and object detection of the perception module, an environmental map is constructed, and based on the generated map, a minimum distance or shortest time measurement action path is configured; Based on the generated mobile path map planning scheme, mobile interaction is performed, and real-time environmental data is fed back for calculation to calculate risk obstacles, and an obstacle avoidance algorithm configuration and a warning system are implemented according to the obstacle classification; Based on the path planning and the information feedback of the interaction and warning algorithm, strategy iteration is performed, and the intelligent control is combined with the wheelchair motor drive control system; The wheelchair device comprises wheels and a vehicle body provided on the wheels; wherein the vehicle body is provided with at least: A motor drive control system for driving and controlling the wheelchair device to move; A perception module, wherein a multi-source sensor mounted on the vehicle body is used to collect surrounding environment signals; wherein the multi-source sensor at least comprises a laser radar, a distance ultrasonic probe and a visual camera; Further comprising a data processing unit, a signal transceiver module and a data platform and a computing platform located in the cloud; The laser radar, the distance ultrasonic probe and the visual camera are electrically connected with the data processing unit, and the data processing unit is used to process the surrounding environment signals collected by the multi-source sensor; The data processing unit is electrically connected with the signal transceiver module, and the signal transceiver module is connected with the data platform and the computing platform in a wired or wireless manner; Further provided is a path planning unit, which is communicatively connected with the data platform and the computing platform located in the cloud; The sensor collects and transmits the processed environmental data characteristic values into the algorithm model for model recognition and classification to detect the object distribution characteristics and distribution density in the environment; According to the detected target objects, an environmental map is constructed to provide data support for subsequent decision calculation; 1) The control method can receive external feedback information, record and mark special changes in the surrounding environment, and automatically update the action measurement; 2) According to the information change of the environment, the moving speed and direction are automatically adjusted, and instructions are sent to the wheelchair moving control system, and prediction braking is performed to ensure the safety of movement; The intelligent wheelchair control decision system automatically optimizes the function of transmitting the data characteristics of the real-time environmental information collected by the wheelchair environmental perception module back to the background database, and the algorithm model is automatically corrected, a more accurate norm standard is established, and the accuracy of the control algorithm model is improved; In the process of moving, various types of sensor data are acquired, which not only supports real-time path planning calculation, but also stores the collected data to the cloud data platform to support model improvement and mobile strategy optimization; The different types of data are preprocessed on a large scale on the data platform, the special events in the environment and the walking habits of the wheelchair are mined, and the algorithm is optimized in combination with the walking characteristics of human beings in the environment; The data analysis of the collected surrounding environment signals mainly includes: a) Filtering the original laser radar data using a filtering algorithm to filter out noise signals, obtaining laser radar-based environment point cloud imaging data; b) Processing the ultrasonic sensor data using an image reconstruction algorithm to calculate the environmental distance feature information; c) Environment perception and environment reconstruction based on camera sensor data to restore the surrounding environment data for subsequent algorithm recognition and calculation; The environment information calculation target recognition and target classification results are sent to the computing control unit and transmitted to the data platform and computing platform through high-speed transmission of information compression; The computing platform designs a mobile path according to the reconstructed environment representation information, uses the shortest distance planning and shortest time planning algorithm, and defines the boundaries of the surrounding objects through a combination of graphics methods to achieve mobile obstacle avoidance.

2. The automated decision intelligent wheelchair control method for ALS patients as claimed in claim 1, wherein: An interaction and warning system is also provided, which is in communication connection with the data platform and computing platform located in the cloud; a voice recognition control system is also provided, which is in communication connection with the data platform and computing platform located in the cloud; a braking system and a remote control system are also included, which are in communication connection with the data platform and computing platform located in the cloud.

3. The automated decision intelligent wheelchair control method for ALS patients according to claim 2, characterized in that: In step, the environment perception module mounted on the vehicle body perceives the environment and collects the surrounding environment signals; the environment perception module uses a multi-source sensor mounted on the vehicle body to collect the surrounding environment signals; wherein the multi-source sensor at least includes a laser radar, a distance ultrasonic probe and a visual camera; The collected surrounding environment signal perception data are encoded and encrypted, and then transmitted to the data platform and computing platform located in the cloud through wired or wireless means.

4. The automated decision intelligent wheelchair control method for ALS patients according to any one of claims 1 to 3, characterized in that: Using the interaction and warning system, based on the generated mobile path map planning scheme, the mobile interaction is carried out, and the real-time environment data is returned for calculation, the risk obstacles are calculated, the obstacle avoidance algorithm configuration and the warning system are implemented according to the obstacle classification; specifically including 1) After determining the two modes of the mobile path, including the shortest distance or the shortest time, the computing platform generates a mobile route map, and the environment data in the moving process is continuously transmitted and updated in real time; 2) For special obstacles found in the environment, risk warning is carried out, and the background management personnel are notified to confirm whether to take over the walking or stop the action, and to evaluate other paths.

Images