An intelligent wheelchair safety response method and system

Abstract

The present invention provides an intelligent wheelchair safety response method and system, belonging to the field of intelligent wheelchairs. The intelligent wheelchair safety response method includes: collecting the current image of the user on the wheelchair; based on the current image, determining the current user state according to the state detection model; if the current user state is operating the wheelchair, collecting the current wheelchair data; determining the current wheelchair state according to the current wheelchair data; if the current wheelchair state is a fault, cutting off the actions of the wheelchair and alarming; if the current user state is stationary, collecting the health index data of the user; determining the health state of the user according to the health index data; if the health state is coma, cutting off the actions of the wheelchair and alarming; if the health state is sleep, saving the health index data. By performing real-time detection on the data of the wheelchair itself and the health index data of the user, when the wheelchair fails or the user is in a coma state, the actions can be cut off in time and an alarm can be given, improving the safety of the wheelchair.

Description

Technical Field

[0001] The present invention relates to the field of intelligent wheelchairs, and particularly to an intelligent wheelchair safety response method and system. Background Art

[0002] In recent years, some elderly people have been affected by chronic diseases and need wheelchairs or walking aids to assist in movement. The safety of such devices is of crucial importance. However, most current walking aids or wheelchairs do not have an active fault detection function and a safety response mechanism, and users cannot detect the faults of the walking aids or wheelchairs in time, resulting in huge safety hazards. At the same time, the current health monitoring of the elderly still mainly adopts traditional management and service models that mainly rely on manual labor. Such methods are inefficient, costly, and require a relatively high level of technology to complete, and the elderly cannot understand their own health status in real time. Summary of the Invention

[0003] The purpose of the present invention is to provide an intelligent wheelchair safety response method and system, which can improve the safety of the wheelchair and can monitor the health status of the user in real time.

[0004] To achieve the above purpose, the present invention provides the following solutions:

[0005] An intelligent wheelchair safety response method includes:

[0006] Collecting the current image of the user on the wheelchair;

[0007] Based on the current image and a state detection model, determining the current user state; the state detection model is obtained by pre-training a first neural network with a first training sample set; the first neural network includes a target detection network and a first classification network connected in sequence; the first training sample set includes multiple historical images and the user states in each historical image; the user states include operating the wheelchair and being stationary;

[0008] If the current user state is operating the wheelchair, collecting the current wheelchair data;

[0009] Based on the current wheelchair data, determining the current wheelchair state; the current wheelchair state is normal or faulty; if the current wheelchair state is faulty, cutting off the action of the wheelchair and alarming according to the type of the fault;

[0010] If the current user state is stationary, collecting the health index data of the user;

[0011] Based on the health index data, determining the health state of the user; the health state is coma or sleep; if the health state is coma, cutting off the action of the wheelchair and alarming; if the health state is sleep, saving the health index data.

[0012] Optionally, the current wheelchair data is collected by sensors provided on the wheelchair.

[0013] Optionally, determining the current wheelchair state according to the current wheelchair data specifically includes:

[0014] Filtering and denoising the current wheelchair data to obtain smooth wheelchair data;

[0015] Performing discrete sampling on the smooth wheelchair data to obtain discretized wheelchair data;

[0016] Determining the current wheelchair state based on the discretized wheelchair data and a fault diagnosis model; the fault diagnosis model is obtained by pre-training a second classification network with a second training sample set; the second training sample set includes multiple groups of historical discretized wheelchair data and corresponding wheelchair states.

[0017] Optionally, the categories of the faults include general faults and serious faults; general faults include low battery, damaged sensor, remote sensing not properly reset, and control circuit unresponsive; serious faults include wheelchair tipping and brake failure;

[0018] If the current wheelchair state is a fault, then cut off the actions of the wheelchair and alarm according to the category of the fault, specifically including:

[0019] If the category of the fault is a general fault, then generate a corresponding prompt message and alarm;

[0020] If the category of the fault is a serious fault, then cut off the actions of the wheelchair, generate a corresponding prompt message and send it to a remote terminal for alarm.

[0021] Optionally, the intelligent wheelchair safety response method further includes:

[0022] Generate recommended measures according to the category of the fault, and send the category of the fault and the corresponding recommended measures to a remote terminal.

[0023] Optionally, the health index data includes heart rate, blood pressure, respiratory rate, and electroencephalogram signal;

[0024] Determining the health state of the user according to the health index data specifically includes;

[0025] Determining a heart rate feature representation according to the heart rate by a first autoencoder;

[0026] Determining a blood pressure feature representation according to the blood pressure by a second autoencoder;

[0027] Determining a respiratory feature representation according to the respiratory rate by a third autoencoder;

[0028] Determine the electroencephalogram feature representation according to the electroencephalogram signal by a fourth autoencoder;

[0029] Determine the overall feature representation by a fifth autoencoder according to the heart rate feature representation, the blood pressure feature representation, the respiration feature representation and the electroencephalogram feature representation;

[0030] Determine the health status of the user based on the overall feature representation and a pre-trained support vector machine classifier.

[0031] Optionally, the intelligent wheelchair safety response method further includes:

[0032] Obtain multiple groups of standard health data; the standard health data includes a standard heart rate feature representation, a standard blood pressure feature representation, a standard respiration feature representation and a standard electroencephalogram feature representation;

[0033] Determine a standard heart rate average value, a standard heart rate standard deviation, a standard blood pressure average value, a standard blood pressure standard deviation, a standard respiration rate average value, a standard respiration rate standard deviation, a standard electroencephalogram average value and a standard electroencephalogram standard deviation according to the multiple groups of standard health data;

[0034] Determine a heart rate index value according to the heart rate feature representation, the standard heart rate average value and the standard heart rate standard deviation;

[0035] Determine a blood pressure index value according to the blood pressure feature representation, the standard blood pressure average value and the standard blood pressure standard deviation;

[0036] Determine a respiration index value according to the respiration feature representation, the standard respiration rate average value and the standard respiration rate standard deviation;

[0037] Determine an electroencephalogram index value according to the electroencephalogram feature representation, the standard electroencephalogram average value and the standard electroencephalogram standard deviation;

[0038] Send the heart rate index value, the blood pressure index value, the respiration index value and the electroencephalogram index value to a remote terminal to monitor the health status of the user in real time.

[0039] To achieve the above object, the present invention also provides the following solution:

[0040] An intelligent wheelchair safety response system, comprising:

[0041] An image acquisition unit for acquiring a current image of a user on the wheelchair;

[0042] A user status determination unit, connected to the image acquisition unit, for determining the current user status based on the current image and a status detection model; the status detection model is obtained by pre-training a first neural network with a first training sample set; the first neural network includes a target detection network and a first classification network connected in sequence; the first training sample set includes multiple historical images and the user status in each historical image; the user status includes operating a wheelchair and being stationary.

[0043] A wheelchair data acquisition unit, disposed on the wheelchair and connected to the user status determination unit, for acquiring current wheelchair data when the current user status is operating the wheelchair.

[0044] A wheelchair status determination unit, connected to the wheelchair data acquisition unit, for determining the current wheelchair status based on the current wheelchair data; the current wheelchair status is normal or faulty.

[0045] A health data acquisition unit, connected to the user status determination unit, for acquiring the user's health index data when the current user status is stationary.

[0046] A health status determination unit, connected to the health data acquisition unit, for determining the user's health status based on the health index data; the health status is coma or sleep.

[0047] An alarm unit, respectively connected to the wheelchair status determination unit and the health status determination unit, for cutting off the actions of the wheelchair and alarming according to the type of the fault when the current wheelchair status is faulty, and cutting off the actions of the wheelchair and alarming when the health status is coma.

[0048] A storage unit, connected to the health status determination unit, for saving the health index data when the health status is sleep.

[0049] Optionally, the health data acquisition unit is a smart wearable device.

[0050] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention: First, collect the current image of the user on the wheelchair; then, based on the state detection model, determine the current user state according to the current image; if the current user state is to operate the wheelchair, collect the current wheelchair data; determine the current wheelchair state according to the current wheelchair data; if the current wheelchair state is a fault, cut off the actions of the wheelchair and alarm according to the type of the fault; by detecting the data of the wheelchair itself, it is determined whether the wheelchair has a fault, and when a fault occurs, the actions are cut off in time and an alarm is given, which improves the safety of the wheelchair; if the current user state is static, collect the health index data of the user; determine the health state of the user according to the health index data; if the health state is coma, cut off the actions of the wheelchair and alarm; if the health state is sleep, save the health index data. When the user is static, it is further determined whether it is coma or sleep according to the health index data. If the user is in a coma state, the actions can be cut off in time and an alarm can be given, and the health state of the user can be monitored in real time, which further improves the safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0052] Figure 1 It is a flowchart of the intelligent wheelchair safety response method of the present invention;

[0053] Figure 2 It is an overall flowchart of the intelligent wheelchair safety response method;

[0054] Figure 3 It is a schematic structural diagram of the target detection network;

[0055] Figure 4 It is a schematic structural diagram of the feature extraction module;

[0056] Figure 5 It is a schematic structural diagram of the first residual block in the feature extraction module;

[0057] Figure 6 It is a schematic structural diagram of the second residual block in the feature extraction module;

[0058] Figure 7 It is a schematic structural diagram of the first classification network;

[0059] Figure 8 It is a schematic structural diagram of the first 3 basic units in the first classification network;

[0060] Figure 9 Schematic diagram of the structures of the last two basic units in the first classification network;

[0061] Figure 10 Schematic diagram of the wheelchair state determination process;

[0062] Figure 11 Schematic diagram of the structure of the second classification network;

[0063] Figure 12 Schematic diagram of the health state determination process;

[0064] Figure 13 Schematic diagram of the structure of a traditional autoencoder;

[0065] Figure 14 Schematic diagram of the structure of the first autoencoder of the present invention;

[0066] Figure 15 Schematic diagram of the structure of a decoder;

[0067] Figure 16 Schematic diagram of health condition monitoring;

[0068] Figure 17 Schematic diagram of an overall intelligent wheelchair, a cloud platform, and intelligent wearable devices;

[0069] Figure 18 Schematic diagram of the overall operation process of intelligent wheelchair safety response;

[0070] Figure 19 Schematic diagram of the modules of the intelligent wheelchair safety response system of the present invention.

[0071] Symbol description:

[0072] Image acquisition unit - 1, user state determination unit - 2, wheelchair data acquisition unit - 3, wheelchair state determination unit - 4, health data acquisition unit - 5, health state determination unit - 6, alarm unit - 7, storage unit - 8. Specific embodiments

[0073] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0074] In view of the current imperfect safety response mechanism of intelligent wheelchairs and the fact that the elderly cannot intuitively understand their own health status, the present invention provides an intelligent wheelchair safety response method and system. Based on the current state of the user, by detecting the data of the wheelchair itself and the health index data of the user, when the wheelchair fails or the user is in a coma, the action is cut off in a timely manner and an alarm is given to ensure the safe operation of the wheelchair user, and when an emergency occurs, the children and medical staff can be informed immediately.

[0075] To make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] As Figure 1 and Figure 2 shown, the intelligent wheelchair safety response method of the present invention includes:

[0077] S1: Collect the current image of the user on the wheelchair.

[0078] S2: Based on the current image and a state detection model, determine the current user state. The state detection model is obtained by pre-training a first neural network with a first training sample set. The first training sample set includes multiple historical images and the user states in each historical image. The user states include operating the wheelchair and being stationary. The first neural network includes a target detection network and a first classification network connected in sequence.

[0079] In this embodiment, the current image of the user is captured in real time by a camera. And each frame of the image is input into the target detection network for human detection, so as to determine the area where the user is located and delimit the ROI (region of interest) to reduce the influence of surrounding useless information on subsequent judgments. The ROI is input into the first classification network to discriminate the state of the user.

[0080] The state monitoring method of the present invention based on the combination of image and neural network has lower cost and is more convenient to operate compared with the consciousness detection method based on brain waves.

[0081] S3: If the current user state is operating the wheelchair, collect the current wheelchair data. Specifically, the current wheelchair data is collected by sensors arranged on the wheelchair.

[0082] S4: Based on the current wheelchair data, determine the current wheelchair state. The current wheelchair state is normal or faulty. No response is made when it is normal.

[0083] S5: If the current wheelchair state is faulty, cut off the action of the wheelchair and give an alarm according to the type of the fault.

[0084] Specifically, the types of the faults include general faults and serious faults. General faults are those that do not affect the operation of the wheelchair and the safety of the user, including low battery, damaged sensors, incorrect return of the remote sensor, and no response from the control circuit. Serious faults include wheelchair tipping and brake failure.

[0085] If the type of the fault is a general fault, corresponding prompt information is generated and an alarm is issued, and the corresponding fault is uploaded to the cloud platform for recording, which is convenient for technicians to perform subsequent maintenance. If the type of the fault is a serious fault, the actions of the wheelchair are cut off, corresponding prompt information is generated and sent to the remote terminal or the cloud platform for alarm.

[0086] S6: If the current user state is stationary, the health index data of the user is collected. Specifically, stationary includes sitting still and unresponsive states. If the user is in a sitting still state, no response is made, otherwise the health index data of the user is collected.

[0087] In this embodiment, the health index data includes heart rate, blood pressure, respiratory rate, and electroencephalogram signals. There are two sources of the health index data: 1. A series of parameters such as blood flow velocity, heart rate, blood pressure, blood oxygen, and respiratory rate of the user are collected through wearable devices (sensors worn daily), stored locally and transmitted to the cloud for recording, for the children of the elderly and medical staff to view. 2. Various data obtained from the user's regular physical examinations are automatically uploaded by the hospital after the physical examination and are called by the medical staff. The data is graphically processed by the local micro host and reflected on the touch screen of the interaction device, so that medical staff can observe the trend of data changes, and the user himself and his children, who are non-professionals, can also intuitively understand the health status of the wheelchair user.

[0088] S7: Determine the health state of the user according to the health index data. The health state is coma or sleep.

[0089] S8: If the health state is coma, cut off the actions of the wheelchair and alarm.

[0090] S9: If the health state is sleep, save the health index data. This can avoid the danger caused by the out-of-control wheelchair due to the user's blurred consciousness or accidental operation during rest.

[0091] In this embodiment, if the user's health status is sleep, the sleep duration is recorded, the sleep quality is evaluated, and the data is uploaded to the cloud platform. If the user is in a coma, the operation of the wheelchair is cut off, and an alarm is sent to the user's children or medical staff through the cloud platform. During the preliminary research, it was found that there are significant differences in the four indicators of heart rate, blood pressure, respiratory rate, and electroencephalogram (EEG) signals when the elderly are in a coma or sleep state. Therefore, in the present invention, by monitoring the above four indicators of the user, the state of the user can be further judged and timely responses can be made.

[0092] Further, in step S2, an open-source dataset is used to train the object detection network. By collecting a video database of the states of wheelchair users, each frame of the video is separated, and each frame is labeled as "operating the wheelchair", "sitting still", or "unresponsive". The labeled training samples are used to train the first classification network.

[0093] In practical applications, an image of the user is collected every 5 seconds and input into the trained object detection network to delimit the region of interest (ROI) where the user is located. Then, the ROI is input into the first classification network to determine the state of the user.

[0094] In this embodiment, the object detection network uses the CenterNet framework, as Figure 3 shown. First, a feature extraction module is used to extract the features of the input image to obtain a feature map. Then, heatmap prediction, center point prediction, and width and height prediction are respectively performed on the feature map to determine the target bounding box. Finally, the feature map is cropped according to the target bounding box to obtain the ROI where the user is located. Specifically, the target bounding box is subjected to coordinate transformation to directly crop the feature map. This can avoid repeated calculations of features, make the structure of the object detection network more compact, and improve the overall efficiency.

[0095] Specifically, in the present invention, the structure of the residual block (ResBlock) in the residual neural network (ResNet) is used in the feature extraction module, and the number of batch normalization (BN) layers is increased, thereby accelerating the convergence of the network and improving the training speed of the network. The structure of the feature extraction module is as Figure 4 shown, including a convolutional layer Conv0, a max-pooling layer Maxpool, and four residual blocks Block1, Block2, Block3, and Block4 connected in sequence. Batch normalization BN and ReLu activation operations are performed between the convolutional layer Conv0 and the max-pooling layer Maxpool.

[0096] Among them, the convolution kernel size of Conv0 is 7*7, the padding is 3, and the stride is 2. After the input image passes through Conv0, BN and ReLu operations are performed. Then it is input into the max pooling layer (Maxpool), where the pooling kernel size of this layer is 3*3, the padding is 1, and the stride is 2. The data after pooling is input into the subsequent 4 residual blocks to obtain the output. The size of the image input into the backbone network is 512*512, the number of channels is 3, and the final output feature size is 16*16, with the number of channels being 512.

[0097] The structure of the first residual block Block1 is as Figure 5 shown. Among them, the convolution kernel sizes of the convolutional layers Conv1-Conv4 are all 3*3, the stride is 1, the padding is 1, and BN and ReLu are connected after each convolutional layer.

[0098] The structures of the second to fourth residual blocks are the same, as Figure 6 shown. Among them, the convolution kernel size of the convolutional layer Conv5 is 3*3, the padding is 1, and the stride is 2; the convolution kernel sizes of the convolutional layers Conv6-Conv8 are 3*3, the padding is 1, and the stride is 1. The convolution kernel size of the convolutional layer Conv9 is 1*1, the padding is 0, and the stride is 2. BN and ReLu are connected after each convolutional layer.

[0099] The structure of the first classification network is as Figure 7 shown, including 5 basic units, 2 fully connected layers, and 1 Softmax classification layer connected in sequence. The structures of the first 3 basic units are as Figure 8 shown, including 1 convolutional layer and 1 max pooling layer connected in sequence, and BN and ReLu are connected between the convolutional layer and the max pooling layer. The structures of the last 2 basic units are as Figure 9 shown, including 2 convolutional layers and 1 max pooling layer connected in sequence, and BN and ReLu are connected after each convolutional layer.

[0100] The convolution kernel size of each convolutional layer in the 5 basic units is 3*3, the padding is 1, and the stride is 1. The kernel size of each max pooling layer is 2*2, the padding is 0, and the stride is 2. After the input feature map passes through the 5 basic units, the size of the obtained feature vector is 7*7*512. The feature vector passes through two fully connected layers and one Softmax classification layer, and the output result is a 1*number of classes feature vector. The position of the element with the largest value in the feature vector is the corresponding recognition result.

[0101] When training the network, first use the open-source human detection dataset to train the object detection network part separately and store the weight parameters. Then use the collected database to train the entire network, update the weights of the object detection network part, and obtain the weights of the classification network part.

[0102] The results show that, compared with the traditional method of detecting first and then classifying, the method proposed by the present invention has the same accuracy, and the time is only about 2 / 3 of that of the traditional algorithm, greatly improving the efficiency of state detection.

[0103] Since the intelligent wheelchair is a relatively complex system, during operation, the data captured by the sensors is constantly changing. Therefore, when an abnormality occurs in the wheelchair, although the fluctuations in the sensor output data can be detected, the type of abnormality cannot be accurately determined. There may be multiple factors affecting a certain output of a sensor. Using the method of mathematical modeling for fault diagnosis is error-prone and cannot cover all aspects. Therefore, the present invention uses a data-driven fault diagnosis method, which is convenient to operate, has high accuracy, and can detect a more comprehensive range of fault types.

[0104] Further, step S4 includes:

[0105] (1) Filter and denoise the current wheelchair data to obtain smoothed wheelchair data. Specifically, the data collected by each sensor on the wheelchair, such as the current sensors at the joystick, the battery current sensor, the IMU and other signals, are first filtered and denoised to remove the noise generated due to environmental factors or electromagnetic interference of the components inside the sensors, so that the waveform output by the sensors can be smoother.

[0106] (2) Discretely sample the smoothed wheelchair data to obtain discretized wheelchair data.

[0107] (3) Based on the discretized wheelchair data and the fault diagnosis model, determine the current wheelchair state. The overall process is as Figure 10 shown.

[0108] Among them, the fault diagnosis model is obtained by pre-training a second classification network with a second training sample set. The second training sample set includes multiple groups of historical discretized wheelchair data and the corresponding wheelchair states.

[0109] Specifically, the fault diagnosis model performs a convolution operation on the discretized wheelchair data, finally obtains the feature vector corresponding to the result, and classifies the feature vector to determine the current wheelchair state. When a fault occurs in the wheelchair, the fault diagnosis model can output the type of the fault. When general faults such as insufficient battery power, damaged sensor, incorrect joystick return, and no response from the control circuit occur in the wheelchair, the present invention only records them and reminds the user that a fault has occurred. However, when problems such as wheelchair tipping, out of control, and brake failure occur, the wheelchair operation is immediately cut off and an alarm is sent through the cloud platform to inform the children or medical staff.

[0110] When annotating historical discrete wheelchair data, the solutions to various faults are also assisted in annotation. Therefore, the present invention can simultaneously inform the user of the type of fault and the recommended measures, and send them to a remote terminal or a cloud platform, so that the user can maintain the wheelchair even without professional personnel.

[0111] The structure of the second classification network is as Figure 11 shown, including 4 CBMR modules, 2 CBR modules, 2 CBMR modules and a Softmax classification layer connected in sequence. Among them, the CBMR module includes a convolutional layer, a batch normalization layer, a max pooling layer and a Relu activation layer connected in sequence. The CBR module includes a convolutional layer, a batch normalization layer and a Relu activation layer connected in sequence.

[0112] Further, as Figure 12 shown, step S7 specifically includes:

[0113] (1) Determine the heart rate feature representation according to the heart rate through the first autoencoder.

[0114] (2) Determine the blood pressure feature representation according to the blood pressure through the second autoencoder.

[0115] (3) Determine the respiration feature representation according to the respiration rate through the third autoencoder.

[0116] (4) Determine the electroencephalogram feature representation according to the electroencephalogram signal through the fourth autoencoder.

[0117] (5) Determine the overall feature representation according to the heart rate feature representation, the blood pressure feature representation, the respiration feature representation and the electroencephalogram feature representation through the fifth autoencoder. The heart rate feature representation, the blood pressure feature representation, the respiration feature representation, the electroencephalogram feature representation and the overall feature representation are all n-dimensional feature vectors. In this embodiment, n = 5.

[0118] In this embodiment, an autoencoder is constructed based on an unsupervised learning method. Since the scale of the health index data is small, a fully connected method is used to link between layers. When training the model, the input data is the same as the output data, so that the autoencoder can autonomously complete the compression and decompression process. The structure of a traditional autoencoder is as Figure 13 shown, including an encoder and a decoder. The part from the input to the encoding is intercepted, which is the first autoencoder, the second autoencoder, the third autoencoder, the fourth autoencoder and the fifth autoencoder used in the present invention. The structures of the first autoencoder, the second autoencoder, the third autoencoder, the fourth autoencoder and the fifth autoencoder are the same. The structures of the first autoencoder, the second autoencoder, the third autoencoder, the fourth autoencoder and the fifth autoencoder are as Figure 14As shown, it includes two CBR modules, two CBRM modules, two CBR modules, and two CBRM modules connected in sequence. The structure of the decoder is as shown in Figure 15 As shown, it includes four CBR modules, two TConv (transposed convolution) modules, one CBR module, and one TConv module connected in sequence. The CBR modules and CBRM modules herein have the same structure as the CBR modules and CBRM modules of the second classification network, and thus will not be elaborated herein.

[0119] (6) Based on the overall feature representation, determine the health status of the user based on a pre-trained support vector machine classifier. That is, perform binary classification using the support vector machine to accurately obtain the health status of the user.

[0120] Furthermore, the intelligent wheelchair safety response method of the present invention further includes:

[0121] S701: Obtain multiple groups of standard health data. The standard health data includes standard heart rate feature representation, standard blood pressure feature representation, standard respiratory feature representation, and standard electroencephalogram feature representation. Specifically, collect the health data of the healthy elderly group, and through the above-mentioned first autoencoder, second autoencoder, third autoencoder, fourth autoencoder, and fifth autoencoder, generate a series of feature representations of various physical functions of the healthy elderly, as well as the overall feature representation, and use such data to construct a multivariate Gaussian distribution.

[0122] S702: Determine the standard heart rate average value, standard heart rate standard deviation, standard blood pressure average value, standard blood pressure standard deviation, standard respiratory rate average value, standard respiratory rate standard deviation, standard electroencephalogram average value, and standard electroencephalogram standard deviation according to multiple groups of standard health data.

[0123] Specifically, taking the heart rate as an example, the standard heart rate average value includes the average value of each dimension of the standard heart rate feature representation, and the standard heart rate standard deviation includes the standard deviation of each dimension of the standard heart rate feature representation. According to the standard heart rate feature representation in multiple groups of standard health data, calculate the average value and standard deviation of each dimension of the standard heart rate feature representation.

[0124] That is, use the formula to calculate the average value of the i-th dimension, and use the formula to calculate the standard deviation of the i-th dimension. Wherein, m represents the number of standard heart rate feature representations, T represents the transpose operation, and x ij represents the data of the i-th dimension of the j-th standard heart rate feature representation.

[0125] S703: Determine the heart rate index value according to the heart rate feature representation, the standard heart rate average value, and the standard heart rate standard deviation.

[0126] Specifically, first calculate the index values of each dimension of the heart rate characteristics, and then determine the heart rate index value according to the index values of all dimensions.

[0127] Use the formula to calculate the index value of the i-th dimension. Where p i (x i ) is the index value of the i-th dimension of the heart rate characteristics, xi is the data of the i-th dimension of the heart rate characteristics, u i is the average value of the i-th dimension of the standard heart rate characteristics, and σ i is the standard deviation of the i-th dimension of the standard heart rate characteristics.

[0128] Use the formula to calculate the heart rate index value. Where p(x) is the heart rate index value and n is the number of dimensions.

[0129] S704: Determine the blood pressure index value according to the blood pressure characteristics representation, the standard blood pressure average value and the standard blood pressure standard deviation.

[0130] S705: Determine the respiratory index value according to the respiratory characteristics representation, the standard respiratory rate average value and the standard respiratory rate standard deviation.

[0131] S706: Determine the electroencephalogram index value according to the electroencephalogram characteristics representation, the standard electroencephalogram average value and the standard electroencephalogram standard deviation.

[0132] The calculation methods of blood pressure, respiratory rate and electroencephalogram are the same as those of heart rate, and will not be elaborated here.

[0133] S707: Send the heart rate index value, the blood pressure index value, the respiratory index value and the electroencephalogram index value to the remote terminal to monitor the health status of the user in real time.

[0134] As a specific implementation manner, the health index data may further include indicators such as transaminase, uric acid, fasting blood glucose, etc. Determine the overall health score according to the index values of each item. The overall process of health status monitoring is as Figure 16 shown.

[0135] The higher the index value, the better the physical function of this item. On the contrary, it indicates that the physical function of this item is poor. By comparing the characteristic representations of the user with the standard data and calculating the corresponding index values, the present invention can fuse the cumbersome items, and then reflect the physical functions of the user, and display them in the form of numerical scores, which can enable the elderly and their children to intuitively understand the health status of the elderly.

[0136] Such as Figure 17The figure shows a schematic diagram of an overall intelligent wheelchair, a cloud platform, and intelligent wearable devices. Data processing is achieved through the cloud platform. Wheelchair data is collected by sensors on the intelligent wheelchair and uploaded to the cloud platform for relevant data processing. Health index data is collected by intelligent wearable devices and uploaded to the cloud platform for relevant data processing. As Figure 18 The figure shows a schematic diagram of the overall operation process of the intelligent wheelchair safety response. First, a database is constructed based on user health data and wheelchair sensor data. The local processor is used to train to obtain a state detection model and a fault diagnosis model for user awareness detection, health analysis, fault discrimination, and safety response. Relevant data is uploaded to the cloud database and informed the children or medical staff of the elderly in a timely manner. At the same time, relevant data can also be visually displayed through the interaction interface.

[0137] The present invention can monitor the safety status of the wheelchair and the health status of the user, actively respond to safety, and thus improve the safety of the wheelchair.

[0138] As Figure 19 As shown in the figure, the intelligent wheelchair safety response system of the present invention includes: an image acquisition unit 1, a user state determination unit 2, a wheelchair data acquisition unit 3, a wheelchair state determination unit 4, a health data acquisition unit 5, a health state determination unit 6, an alarm unit 7, and a storage unit 8.

[0139] Among them, the image acquisition unit 1 is used to acquire the current image of the user on the wheelchair.

[0140] The user state determination unit 2 is connected to the image acquisition unit 1. The user state determination unit 2 is used to determine the current user state based on the state detection model according to the current image. The state detection model is obtained by pre-training a first neural network with a first training sample set. The first neural network includes a target detection network and a first classification network connected in sequence. The first training sample set includes multiple historical images and the user states in each historical image. The user states include operating the wheelchair and being stationary.

[0141] The wheelchair data acquisition unit 3 is arranged on the wheelchair and is connected to the user state determination unit 2. The wheelchair data acquisition unit 3 is used to acquire the current wheelchair data when the current user state is operating the wheelchair.

[0142] The wheelchair state determination unit 4 is connected to the wheelchair data acquisition unit 3. The wheelchair state determination unit 4 is used to determine the current wheelchair state according to the current wheelchair data; the current wheelchair state is normal or faulty.

[0143] The health data acquisition unit 5 is connected to the user status determination unit 2. The health data acquisition unit 5 is used to collect the user's health index data when the current user status is static. Specifically, the health data acquisition unit 5 is a smart wearable device.

[0144] The health status determination unit 6 is connected to the health data acquisition unit 5. The health status determination unit 6 is used to determine the user's health status according to the health index data. The health status is coma or sleep.

[0145] The alarm unit 7 is respectively connected to the wheelchair status determination unit 4 and the health status determination unit 6. The alarm unit 7 is used to cut off the actions of the wheelchair and alarm according to the type of the fault when the current wheelchair status is faulty, and to cut off the actions of the wheelchair and alarm when the health status is coma.

[0146] The storage unit 8 is connected to the health status determination unit 6. The storage unit 8 is used to save the health index data when the health status is sleep.

[0147] The present invention provides a safety response system that combines visual and wearable health detection devices, wheelchair self-sensors, and a cloud platform, which can enable children to know at any time whether the elderly encounter sudden situations, intuitively understand the health status of the elderly, and can notify the user immediately when the wheelchair breaks down.

[0148] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0149] Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An intelligent wheelchair safety response method, characterized in that, the intelligent wheelchair safety response method includes: Collecting the current image of the user on the wheelchair; Based on the current image and a state detection model, determining the current user state; the state detection model is obtained by pre-training a first neural network with a first training sample set; the first neural network includes a target detection network and a first classification network connected in sequence; the first training sample set includes multiple historical images and the user states in each historical image; the user states include operating the wheelchair and being stationary; If the current user state is operating the wheelchair, collecting the current wheelchair data; Determining the current wheelchair state according to the current wheelchair data, specifically including: filtering and denoising the current wheelchair data to obtain smoothed wheelchair data; discretely sampling the smoothed wheelchair data to obtain discretized wheelchair data; based on the discretized wheelchair data and a fault diagnosis model, determining the current wheelchair state; the fault diagnosis model is obtained by pre-training a second classification network with a second training sample set; the second training sample set includes multiple groups of historical discretized wheelchair data and the corresponding wheelchair states; the current wheelchair state is normal or faulty; if the current wheelchair state is faulty, cutting off the actions of the wheelchair and giving an alarm according to the type of the fault; If the current user state is stationary, collecting the health index data of the user; Determining the health state of the user according to the health index data; the health state is coma or sleep; if the health state is coma, cutting off the actions of the wheelchair and giving an alarm; if the health state is sleep, saving the health index data.

2. The intelligent wheelchair safety response method according to claim 1, characterized in that, Collecting the current wheelchair data through sensors arranged on the wheelchair.

3. The intelligent wheelchair safety response method according to claim 1, characterized in that, The types of the faults include general faults and serious faults; general faults include low battery, damaged sensor, remote sensing not correctly positioned, control circuit not responding; serious faults include wheelchair tipping over, brake failure; If the current wheelchair state is faulty, cutting off the actions of the wheelchair and giving an alarm according to the type of the fault, specifically including: If the type of the fault is a general fault, generating a corresponding prompt message and giving an alarm; If the type of the fault is a serious fault, cutting off the actions of the wheelchair, generating a corresponding prompt message and sending it to a remote terminal for alarm.

4. The intelligent wheelchair safety response method according to claim 1, characterized in that, The intelligent wheelchair safety response method further includes: Generating recommended measures to be taken according to the type of the fault, and sending the type of the fault and the corresponding recommended measures to a remote terminal.

5. The intelligent wheelchair safety response method according to claim 1, characterized in that, The health index data includes heart rate, blood pressure, respiratory rate and electroencephalogram signal; Determining the health state of the user according to the health index data, specifically including; Determining the heart rate feature representation according to the heart rate through a first autoencoder; Determine a blood pressure feature representation based on the blood pressure through a second autoencoder; Determine a respiratory feature representation based on the respiratory rate through a third autoencoder; Determine an electroencephalogram (EEG) feature representation based on the EEG signal through a fourth autoencoder; Determine an overall feature representation through a fifth autoencoder based on the heart rate feature representation, the blood pressure feature representation, the respiratory feature representation, and the EEG feature representation; Based on the overall feature representation, determine the health status of the user based on a pre-trained support vector machine classifier.

6. The intelligent wheelchair safety response method according to claim 5, wherein, the intelligent wheelchair safety response method further includes: Obtain multiple sets of standard health data; the standard health data includes a standard heart rate feature representation, a standard blood pressure feature representation, a standard respiratory feature representation, and a standard EEG feature representation; Based on the multiple sets of standard health data, determine the standard heart rate average value, the standard heart rate standard deviation, the standard blood pressure average value, the standard blood pressure standard deviation, the standard respiratory rate average value, the standard respiratory rate standard deviation, the standard EEG average value, and the standard EEG standard deviation; Based on the heart rate feature representation, the standard heart rate average value, and the standard heart rate standard deviation, determine the heart rate index value; Based on the blood pressure feature representation, the standard blood pressure average value, and the standard blood pressure standard deviation, determine the blood pressure index value; Based on the respiratory feature representation, the standard respiratory rate average value, and the standard respiratory rate standard deviation, determine the respiratory index value; Based on the EEG feature representation, the standard EEG average value, and the standard EEG standard deviation, determine the EEG index value; Send the heart rate index value, the blood pressure index value, the respiratory index value, and the EEG index value to a remote terminal to monitor the health status of the user in real time.

7. An intelligent wheelchair safety response system, wherein, the intelligent wheelchair safety response system includes: An image acquisition unit for acquiring a current image of the user on the wheelchair; A user state determination unit connected to the image acquisition unit for determining the current user state based on the current image and a state detection model; the state detection model is obtained by pre-training a first neural network with a first training sample set; the first neural network includes a target detection network and a first classification network connected in sequence; the first training sample set includes multiple historical images and the user states in each historical image; the user states include operating the wheelchair and being stationary; A wheelchair data acquisition unit disposed on the wheelchair and connected to the user state determination unit for acquiring current wheelchair data when the current user state is operating the wheelchair. A wheelchair state determination unit, connected to the wheelchair data acquisition unit, is configured to determine the current wheelchair state according to the current wheelchair data. Specifically, it includes: filtering and denoising the current wheelchair data to obtain smoothed wheelchair data; discretely sampling the smoothed wheelchair data to obtain discretized wheelchair data; determining the current wheelchair state based on the discretized wheelchair data and a fault diagnosis model; the fault diagnosis model is obtained by pre-training a second classification network with a second training sample set; the second training sample set includes multiple groups of historical discretized wheelchair data and corresponding wheelchair states; the current wheelchair state is normal or faulty; A health data acquisition unit, connected to the user state determination unit, is configured to collect the user's health index data when the current user state is stationary; A health state determination unit, connected to the health data acquisition unit, is configured to determine the user's health state according to the health index data; the health state is coma or sleep; An alarm unit, respectively connected to the wheelchair state determination unit and the health state determination unit, is configured to cut off the actions of the wheelchair and alarm according to the type of fault when the current wheelchair state is faulty, and cut off the actions of the wheelchair and alarm when the health state is coma; A storage unit, connected to the health state determination unit, is configured to save the health index data when the health state is sleep.

8. The intelligent wheelchair safety response system according to claim 7, wherein, the health data acquisition unit is an intelligent wearable device.

Images