A hand rehabilitation assistance system with flexible manipulator grasping state feedback

Abstract

The present invention discloses a hand rehabilitation assistance system with feedback on the gripping state of a flexible manipulator, which belongs to the field of medical rehabilitation assistance. The present invention cooperates a gripping state prediction module, a gripping state perception feedback module and a flexible manipulator to improve the patient's tactile perception ability of the current gripping state when using a flexible manipulator device for hand gripping task assistance and rehabilitation training, and enables the patient to obtain real-time feedback on the gripping state. The network model of Grasp Classification Net designed by the present invention innovatively combines the advantages of temporal and spatial feature extraction, providing a more accurate and stable solution for the gripping state classification of the flexible manipulator. The use of an improved loss function based on Focal Loss further improves the prediction accuracy of the gripping state classification. Designing a function of the vibration frequency size according to the predicted probability of various gripping classification states is beneficial for patients to carry out rehabilitation training with higher precision requirements.

Description

Technical Field

[0001] The present invention belongs to the field of medical rehabilitation assistance, and more specifically, relates to a hand rehabilitation assistance system with flexible manipulator grasping state feedback. Background Art

[0002] Hemiplegic patients often suffer from upper limb motor dysfunction, especially the loss of fine motor skills in their hands, which seriously affects their ability to take care of themselves in daily life. Therefore, rehabilitation training is indispensable in their motor recovery process. At present, although auxiliary training equipment can help improve upper limb function after stroke, most devices only focus on passive assisted movement, ignoring the real-time recognition and feedback of grasping intentions, and are difficult to adapt to the personalized needs of patients. In addition, most devices lack tactile perception capabilities and cannot accurately identify the dynamic contact state during the grasping process. It is difficult to simulate real grasping scenarios, resulting in a single training feedback and restricting the effect of neurological function remodeling in hemiplegic patients.

[0003] However, most existing devices are unable to effectively distinguish and process the three key stages of the grasping process: the initial state when just contacting the object, the dynamic state where the object may slip, and finally the stable state where a firm grasp is achieved. This incomplete perception of the grasping process prevents the device from providing targeted feedback, resulting in patients lacking an accurate understanding of their own movement status during training and difficulty in making effective self-adjustments. In addition, most existing devices fail to provide patients with real-time tactile feedback, making it impossible for patients to feel tactile information similar to real grasping during training, thus affecting the authenticity and immersion of the training.

[0004] Therefore, developing a hand rehabilitation assistance system that provides real-time tactile feedback and accurately classifies grip states is crucial for enhancing the intelligence and interactivity of hand rehabilitation training. Providing real-time tactile feedback and accurately classifying grip states can help patients better perceive their movement status, improving the effectiveness and efficiency of training. Furthermore, this feedback mechanism can stimulate patients' active participation, increasing their enthusiasm and confidence in training. Furthermore, accurate grip state classification and real-time tactile feedback can help promote neural remodeling, accelerate the rehabilitation process, and enhance patients' ability to care for themselves.

[0005] Designing algorithms that ensure accurate and real-time predictions of the grasping state of flexible manipulators and designing relevant hand rehabilitation assistance systems by combining classification algorithms, sensing technologies, and feedback modules are crucial. Research has proposed the LSTM (Long Short Term Memory) network model, which incorporates memory cells and a gating mechanism to process and predict long-term dependencies in time series data, effectively extracting temporal features from the data. The ResNet network, by introducing residual blocks, addresses the vanishing and exploding gradient problems in deep networks and exhibits robustness to noise and outliers. Combining the two networks can combine their strengths, processing both temporal and spatial features of data and improving grasping state classification prediction. However, existing research, due to the loss of some positional information in the LSTM network, has typically placed the ResNet network before the LSTM. This results in high-dimensional and large-scale input data to the ResNet network, slowing training and prediction operations and hindering its application in systems that prioritize temporal features. Therefore, designing a joint network model to reduce training and prediction time and computing power costs, address the existing network's position information corruption problem, and better predict grasping state classification is a major challenge. Furthermore, to allow users to better perceive the flexible manipulator's grasp of an object and improve training effectiveness and efficiency, designing a feedback device that can provide real-time feedback to the user based on the model's predictions while ensuring human sensitivity to the received information without causing any harm to the human body is also a pressing issue. Summary of the Invention

[0006] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a hand rehabilitation assistance system with feedback on the grasping state of a flexible robotic arm. Its purpose is to improve the patient's tactile perception ability of the current grasping state when using a flexible robotic arm device for hand grasping task assistance and rehabilitation training, assist hemiplegic patients in completing some daily grasping tasks, and improve the patient's sense of participation and the authenticity of the experience, realize real-time tactile feedback during the training process, and thus improve the neurological function remodeling effect of hemiplegic patients.

[0007] To achieve the above objectives, the present invention provides a hand rehabilitation assistance system with flexible manipulator grasping state feedback, comprising a hand bending detection module, a controller, a flexible manipulator, a tactile information acquisition module, a grasping state prediction module, and a grasping state perception feedback module;

[0008] The hand bending detection module includes a bending sensor, which is used to detect the bending of the patient's finger part and generate an electrical signal corresponding to the bending;

[0009] The controller is used to generate an instruction to control the air pump according to the electrical signal, and the instruction is used to control the air pump to inflate / deflate the flexible manipulator;

[0010] The flexible manipulator is used to grasp an object by inflating and deflating air;

[0011] The tactile information acquisition module includes a tactile sensor, which is used to obtain real-time pressure distribution data generated when the flexible manipulator grasps an object;

[0012] a gripping state prediction module, configured to predict the gripping state based on the real-time pressure distribution data;

[0013] The grip state perception feedback module is used to receive the predicted grip state and provide feedback to the patient in the form of vibration, where different vibration frequencies match different grip states.

[0014] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0015] 1. The present invention combines a grasping state prediction module, a grasping state perception feedback module and a flexible manipulator to improve the patient's tactile perception ability of the current grasping state when using a flexible manipulator device for hand grasping task assistance and rehabilitation training, and enables the patient to obtain real-time feedback on the grasping state.

[0016] 2. The Grasp Classification Net network model designed in this invention effectively solves the problems of adding a ResNet network before an LSTM network, which results in high-dimensional and large-scale input data to the ResNet network, leading to slow training and prediction operations and high computational costs, as well as the problem that the LSTM network can cause partial destruction of position information. Furthermore, by constructing a transfer function, the accuracy, stability, and robustness of classification are improved. This innovative network structure combines the advantages of temporal and spatial feature extraction, providing a more accurate and stable solution for grasping state classification in flexible manipulators.

[0017] 3. The improved loss function based on Focal Loss used in the present invention can dynamically adapt to the difficulty distribution of different samples, solve the problem of possibly suppressing "medium difficulty" samples, and further improve the prediction accuracy of grasping state classification.

[0018] 4. The gripping state perception feedback module designed in the present invention designs a function of the vibration frequency size according to the predicted probability of N kinds of gripping classification states, ensuring that the user (such as a hemiplegic patient) can clearly feel the specific situation of the grip in the current time step and the change in the grip tightness between the flexible manipulator and the object in the previous and next time steps, which is beneficial for patients to carry out rehabilitation training with higher precision requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic structural diagram of a hand rehabilitation assistance system with feedback of the object grasping state of a flexible manipulator provided in an embodiment of the present invention.

[0020] Figure 2 Schematic diagram of the hand bending detection module in the present invention.

[0021] Figure 3 Flowchart for prediction using the Grasp Classification Net model designed for the present invention.

[0022] Figure 4 A schematic diagram of processing using LSTM, ResNet networks and feature fusion layers.

[0023] Figure 5 Schematic diagram of post-processing of the transfer matrix introduced in the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0025] The present invention provides a hand rehabilitation assistance system with gripping state feedback of a flexible manipulator, comprising a hand bending detection module, a controller, a flexible manipulator, a tactile information acquisition module, a gripping state prediction module, and a gripping state perception feedback module;

[0026] The hand bending detection module includes a bending sensor, which is used to detect the bending of the patient's finger part and generate an electrical signal corresponding to the bending;

[0027] The controller is used to generate an instruction to control the air pump according to the electrical signal, and the instruction is used to control the air pump to inflate / deflate the flexible manipulator;

[0028] The flexible manipulator is used to grasp an object by inflating and deflating air;

[0029] The tactile information acquisition module includes a tactile sensor, which is used to obtain real-time pressure distribution data generated when the flexible manipulator grasps an object;

[0030] a gripping state prediction module, configured to predict the gripping state based on the real-time pressure distribution data;

[0031] The grip state perception feedback module is used to receive the predicted grip state and provide feedback to the patient in the form of vibration, where different vibration frequencies match different grip states.

[0032] The embodiment of the present invention provides a hand rehabilitation assistance system with feedback of the gripping state of a flexible manipulator, such as Figure 1 Shown, including:

[0033] Hand bending detection module, such as Figure 2 As shown, a wearable glove and a bend sensor embedded in the glove's fingers detect the patient's finger bending motion. Upon detecting hand bending, an electrical signal is generated. A controller, based on this electrical signal, generates an instruction to control an air pump, which in turn inflates and deflates the flexible robotic hand. Preferably, the controller and hand bend detection module can be integrated into the same module.

[0034] A flexible robotic arm used to grasp objects by inflating and deflating air.

[0035] The tactile information acquisition module is a tactile sensor attached to both sides of the flexible manipulator to obtain the real-time pressure distribution data generated by the flexible manipulator during the grasping of the object. The single tactile sensor used is The data value of each small square is the relative size of the pressure in the square position, and each flexible manipulator has two tactile sensors on the left and right sides, that is, one frame of information consists of 224 data. Expressed as ,in express time step data, and That is, the feature dimension is 224, Represents the relative value of the pressure in each small square in the array tactile sensor.

[0036] The grasp state prediction module, such as Figure 3 As shown in the figure, a Grasp Classification Net model is designed for data processing training and prediction. In previous studies, due to the disadvantage that LSTM network processing will cause the loss of position information, most of them add LSTM network after ResNet network. However, since this rehabilitation assistance system is to predict the grasping state in real time, it has a high requirement for a shorter prediction time. Adding LSTM network before ResNet network can reduce the feature dimension of the data input to ResNet network and improve the calculation speed of training and prediction. Therefore, in this invention, LSTM network is added before ResNet network and a feature fusion layer is added to solve the problem of position information loss. The structure of LSTM, ResNet and feature fusion layer is shown in the figure. Figure 4 The designed model is as follows:

[0037] First, the collected data is filtered using a low-pass filter (cutoff frequency of 77Hz) and a median filter (window length 5, to remove impulse noise) in the time dimension and a one-dimensional Gaussian filter (sigma = 2, to smooth the spatial noise between the 112 sensor data) in the spatial dimension. The data is then preprocessed by loading the original data list and deleting the all-zero rows. 30 rows of data are loaded into the dataset as a sample and then fed into the LSTM network model. The shape is , indicating that the batch size (batch_size) is 32, the sequence length (seq_len) is 30, and the feature dimension (input_size) of each time step is 224.

[0038] The LSTM model (Long Short Term Memory) is a recurrent neural network in which each unit depends on the previous unit and the current time series information. Suppose its input sequence is , where each is the time step The LSTM network has an input vector of The calculation can be expressed as:

[0039] Input Gate: ;

[0040] Forget Gate: ;

[0041] Output gate: ;

[0042] Cell status: ;

[0043] ;

[0044] Hidden state: .

[0045] In the process of the present invention, the input is , processed by LSTM network, LSTM network parameters are input_size=224, hidden_size=448, the hidden state of the last time step of the output .

[0046] In this system, after LSTM processes the time series features, it will destroy the data location information. However, this system application needs to retain the data location information. Therefore, an innovative feature fusion network is added to fuse the global time series features extracted by LSTM with the samples of the most recent time step to retain the location features before inputting them into the ResNet network. The process is as follows:

[0047] 1. Feature stitching: , then in the present invention there is ;

[0048] in, is the global temporal feature extracted by LSTM (i.e., the hidden state of the last time step), is the feature of the last time step of the original sequence data.

[0049] 2. Perform linear transformation: , then in the present invention there is ;

[0050] in, is the weight matrix, is the bias term.

[0051] 3. Activation function (GELU) processing: , then in the present invention there is ;

[0052] Among them, the GELU function can be expressed as , erf is the error function.

[0053] 4. Add Dropout: , then in the present invention there is .

[0054] Among them, Dropout is a regularization calculation used to reduce the risk of model overfitting.

[0055] 5. Layer Normalization: , then in the present invention there is .

[0056] Furthermore, this data is converted into a feature map. The process is as follows:

[0057] 1. Convert to a two-dimensional feature map: ;

[0058] 2. Add padding: ;

[0059] 3. Normalization: .

[0060] The feature map after normalization and adding channel dimensions is input into the ResNet101 network (residual network). The network is built with residual blocks. The core is residual connection, which allows the input to be directly passed to subsequent layers, alleviating the difficulty of deep network training. The specific calculation process is as follows:

[0061] 1. Initial convolutional layer and pooling layer: The input data shape is [batch_size, 1, 16, 18], and the convolution layer is After convolution, the output shape is [batch_size, 64, 8, 9]. After batch normalization, ReLU activation and max pooling, the output shape is [batch_size, 64, 4, 5].

[0062] 2. Four residual layers are processed using residual blocks (BasicBlock), and the final output shape is [batch_size, 512, 1, 1].

[0063] 3. Perform global average pooling and fully connected layer processing, and the output shape is: [batch_size, num_classes].

[0064] 4. Softmax activation: Use the softmax function to convert each element Convert to probability , specifically: .

[0065] The final output is the probability that each input sequence data is classified into different categories, with a shape of [batch_size, num_classes], that is, each element It represents the probability that the i-th input sequence data is classified as the j-th category.

[0066] Furthermore, due to the uneven distribution of collected samples across categories and the observed differences in easy and difficult classifications observed during early training, we chose to use a function based on Focal Loss to mitigate this imbalance. Focal Loss is a dynamically scaled cross-entropy loss function. Using a dynamic scaling factor, Focal Loss dynamically reduces the weight of easily distinguishable samples during training, thereby quickly focusing on difficult-to-distinguish samples.

[0067] First, for the multi-classification problem, assuming that there are C categories to be classified, the cross-entropy loss (Cross-EntropyLoss) function is expressed as:

[0068] ;

[0069] in, Represents the one-hot encoding of the true label, Code model prediction category probability.

[0070] Focal Loss introduces a modulation factor based on cross entropy and weights To dynamically adjust the sample weight. The specific function is as follows

[0071] ;

[0072] Easy to know, when The closer it is to 1, the more accurate the sample classification is, that is, it is easy to classify, the smaller the Focal Loss is, and the loss is suppressed; similarly, when The closer it is to 0, the less accurate the sample classification is, that is, it is difficult to classify, and most of its Focal Loss is retained, thereby achieving the purpose of focusing on difficult-to-distinguish samples.

[0073] The original Focal Loss function is fixed Although the value simplifies the implementation, in actual application, there are still problems such as being unable to dynamically adapt to the difficulty distribution of different samples and possibly suppressing "medium difficulty" samples. Based on such problems, the original Focal Loss function is improved and realized by associating it with the accuracy of the previous round of training. Dynamic adjustments during different stages of training.

[0074] Considering that as the accuracy rate reaches a high threshold, excessive attention to difficult-to-classify samples may cause the accuracy rate to drop. To ensure the smoothness of dynamic adjustment, the model is constructed based on the hyperbolic tangent function, and the following is obtained:

[0075] ;

[0076] in, represents the accuracy of the model on the validation set in the previous round of training, and is the preset threshold range, parameter 、 They are used to control the rate of change and determine the inflection point of the change. Through experiments, a set of hyperparameters with excellent performance is obtained as follows: , , , Based on this set of parameters, the accuracy of this model on the validation set increased from 93.13% to 96.28%.

[0077] Furthermore, after the output of the LSTM-ResNet joint network, it was observed that there were problems with unstable prediction and reduced success rate when grasping state transitions during the grasping process. In addition, in many grasping processes, the transition of grasping state had a relatively fixed probability distribution over time. To solve this problem, a well-defined transfer probability matrix was innovatively introduced for processing, such as Figure 5 ,for ,in , is the probability of transitioning to grasp state j at time step t+1 given the grasp state i at time step t (in this invention, i=0 is defined as the initial contact state, i=1 as the sliding state, and i=2 as the firmly grasped state, with N=3. Different sliding states can be further divided according to the sliding speed, thereby allowing N to take different values). This matrix is ​​calculated from the experimentally collected dataset of the complete grasping dynamic process. The processing and calculation process is as follows:

[0078] Each sample in the dataset is labeled to indicate the grasping state represented by the sample. Since the samples are collected in time series, the grasping state of the time step represented by the current sample in the dynamic grasping process is marked. The data is obtained by statistics. , is the number of cases where the grasping state j occurs at time step t+1 under the condition that the grasping state i occurs at time step t, so we can get

[0079] ;

[0080] Then we can construct the transfer probability matrix .

[0081] The observation probability matrix is ,in, is the probability of grasping state j predicted at time step t, M is the length of the tactile time series; the initial probability distribution is defined as , that is, the three grasping states appear with equal probability, and then Viterbi decoding (including recursion and backtracking processes) is used to output the most likely grasping state sequence for each input sequence. Finally, the prediction is completed and passed to the grasping state perception feedback module. This method improves the prediction success rate and enhances the stability and robustness of the prediction.

[0082] The process is as follows:

[0083] initialization , the confidence probability of each initial grasping state is: ;

[0084] in, is the probability of grasping state i at t=1, ;

[0085] for , combined with the confidence probability of the grasping state of the previous time step and the state transition probability matrix , and the confidence probability of the grasping state at the current time step to update the confidence probability of the grasping state at the current time step:

[0086] ;

[0087] in, is the confidence probability of grasp state i at the previous time step;

[0088] This results in the confidence probability of predicting various grasping states at each time step;

[0089] The confidence probabilities of various grasping states predicted at each time step are compared, and the grasping state with the highest probability is taken as the predicted grasping state at the current time step, thereby obtaining the most likely grasping state for each tactile time series.

[0090] The grip state perception feedback module receives the results predicted by the data calculation and processing and grip state classification modules, and transmits this prediction result information to a vibration module with adjustable vibration frequency. Different grip states are matched according to different vibration frequencies. That is, the vibration frequency of the vibration module gradually increases from just touching to sliding to grasping.

[0091] Since the grasping process is a continuous dynamic process, and is divided into three states such as just touching, sliding and grasping, if different vibration frequencies are matched only according to different states, users (such as hemiplegic patients) cannot clearly feel the specific situation of the grasping in the current time step and the changes in the tightness of the grasping between the flexible manipulator and the object in the previous and next time steps, which is not conducive to patients to carry out rehabilitation training with high precision requirements. Therefore, based on the predicted probability of each grasping state output by the network and the sensitivity of the human body to frequency and the safety of vibration frequency, a function is designed to realize continuous vibration frequency feedback from loose to tight grasping. Specifically, suppose the predicted probability of each grasping state output by the network is: the probability of just touching is , the sliding probability is , the probability of holding on is In order to avoid damage to the human body and the human body's relatively strong perception of vibration, 80~150 Hz is selected, so the function of the vibration frequency is set as:

[0092] ;

[0093] Vibration at this frequency will provide real-time grip status feedback to the user. In order to better remind the user of the currently predicted grip status, a buzzer module is used. When the grip status changes, different numbers of sound reminders are made in different states to achieve real-time feedback. This device can stimulate the reconstruction of motor neural circuits through real-time tactile feedback and promote the recovery of neural function. This vibration and sound feedback not only reminds the patient of the current grip status and the changes in the tightness of the grip on the object during the process, but the vibration information also acts as a sensory-motor stimulus to activate related neuromuscular pathways. Long-term training enables the patient's brain to gradually adapt to and integrate this feedback signal, enhance neural plasticity, thereby promoting the reconstruction and optimization of motor neural circuits, improving the patient's motor control ability and coordination, and enhancing the rehabilitation effect.

[0094] In the ongoing experiment, the collected data set was split into an 80% training set and a 20% test set. Experiments were conducted on three objects in total, and the prediction success rate was 97.08%. This proves that the system of the present invention can effectively complete the classification of the grasping state of objects by the flexible manipulator, and can provide users with real-time tactile information feedback on the grasping state, which is beneficial for assisting hemiplegic patients to rebuild motor neural circuits through tactile feedback stimulation and accelerate the recovery process.

[0095] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hand rehabilitation assistance system based on tactile signal-based flexible manipulator grasping state feedback, characterized in that: It includes a hand bending detection module, a controller, a flexible manipulator, a tactile information acquisition module, a grasping state prediction module, and a grasping state perception feedback module; The hand bending detection module includes a bending sensor, which is used to detect the bending of the patient's finger part and generate an electrical signal corresponding to the bending; The controller is used to generate an instruction to control the air pump according to the electrical signal, and the instruction is used to control the air pump to inflate / deflate the flexible manipulator; The flexible manipulator is used to grasp an object by inflating and deflating air; The tactile information acquisition module includes a tactile sensor, which is used to obtain real-time pressure distribution data generated when the flexible manipulator grasps an object; a gripping state prediction module, configured to predict the gripping state based on the real-time pressure distribution data; The grasping state prediction module includes an LSTM network, a feature fusion layer, and a ResNet network. The prediction process includes: The pre-processed real-time pressure distribution data is organized into a tactile time series as the initial network input, which is then fed into the LSTM network to extract global time series features. fusing the global temporal features with the tactile time series of the most recent time step in a feature fusion layer to obtain a reconstructed feature, and converting the reconstructed feature into a feature map; Input the feature map into the ResNet network and output the confidence probability of various grasping states; The grip state perception feedback module is used to receive the predicted grip state and provide feedback to the patient in the form of vibration, where different vibration frequencies match different grip states.

2. The system according to claim 1, wherein The global temporal features are fused with the tactile time series of the most recent time step in a feature fusion layer to reconstruct a feature map to obtain a reconstructed feature, and the reconstructed feature is converted into a feature map, specifically including: The global temporal features are concatenated with the tactile time series of the most recent time step and then linearly transformed. The features after linear transformation are processed using a GELU function and then subjected to regularization calculation and layer normalization to obtain reconstructed features. The reconstructed features are converted into a two-dimensional feature map, and then subjected to padding and normalization processing in sequence.

3. The system according to claim 1, wherein: The loss function of the ResNet network is ; in, Represents the accuracy of the validation set in the previous round of training, is the preset threshold range, parameter 、 They are used to control the rate of change and determine the inflection point of the change respectively.

4. The system according to claim 1, wherein: The method further includes a post-processing module, wherein the post-processing module is configured to use the gripping state with the highest confidence probability as the predicted gripping state, including: initialization , the confidence probability of each initial grasping state is: in, The predicted grasping state at t=1 i The probability of ; for , combined with the confidence probability of the grasping state of the previous time step and the state transition probability matrix A , and the confidence probability of the grasping state at the current time step to update the confidence probability of the grasping state at the current time step: in, is the grasping state at the previous time step i The confidence probability of is in the grasping state at time step t i Under the condition of j The probability of i=1,2,···,N,j=1,2,···,N, N is the number of grasping states, To transfer the probability matrix, it is calculated from the prior data set of the complete grasping dynamic process collected in the experiment. The calculation process includes: marking each sample in the prior data set, marking the grasping state of the time step represented by the current sample in the dynamic grasping process, and obtaining the , is in the grasping state at time step t i Under the condition of being in grasping state at time step t+1 j The number of cases that occur, , and then construct the transfer probability matrix ; is the time step t Predicted grasping state j The probability of is the observation probability matrix, M is the length of the tactile time series; This results in the confidence probability of predicting various grasping states at each time step; The confidence probabilities of various grasping states predicted at each time step are compared, and the grasping state with the highest probability is taken as the predicted grasping state at the current time step, thereby obtaining the most likely grasping state for each tactile time series.

5. The system according to claim 1, wherein: The preprocessing includes low-pass filtering and median filtering in the time dimension, one-dimensional Gaussian filtering in the space dimension, and deletion of all-zero rows.

6. The system according to claim 1, wherein: The gripping state perception feedback module includes a vibration module, the vibration frequency of which matches different gripping states, wherein the vibration frequency is determined according to the predicted probabilities of different gripping states: in,[ f 1, f N ] is the preset vibration frequency range, f 2··· f N-1 is the middle value in the vibration frequency range, N is the number of grasping states, p 1. p 2. ··· p N are the probabilities of the corresponding grasping states.

7. The system according to claim 1, wherein: The gripping state perception feedback module further includes a buzzer module, which is used to make different sound reminders at different predicted gripping states when the gripping state changes, thereby feeding back the real-time gripping state to the user.

Images