A multi-sensor fusion prosthetic hand grip force feedback control method

Abstract

The present invention proposes a multi-sensor fusion prosthetic hand gripping force feedback control method, including a multi-sensor fusion and feedback method, a gripping force prediction method and a PD-based CMAC manipulator gripping control method; the multi-sensor fusion and feedback method includes collecting surface electromyographic signals of muscles, collecting the temperature, contact force, sliding state of the object gripped by the prosthetic hand, collecting the joint acceleration and angular velocity corresponding to the gripping action, and collecting the actual gripping force of the prosthetic hand's gripping action; the gripping force prediction method predicts the gripping force of the current gripping action; the multi-sensor fusion and feedback method fuses the data to generate the prosthetic end perception information of the current gripping action; the present invention allows the user to adjust the actual gripping force and gripping action in time to achieve accurate and stable gripping of objects, while detecting the real-time temperature and contact force of the gripped object during the gripping process, and feeding back the tactile information to the user through the vibration and display module.

Description

Technical Field

[0001] The present invention relates to the technical field of prosthetic limbs, and in particular to a multi-sensor fusion prosthetic hand grasping force feedback control method. Background Art

[0002] At present, prosthetic hand technology mainly realizes specific pattern recognition of the hand, and only has opening and closing control, lacks force control, and cannot stably grasp objects of different stiffness and weight. There is a correlation between surface electromyography (sEMG) and muscle contraction force. Characterizing the change of prosthetic hand grasping force through sEMG of related muscles can effectively reflect the user's grasping intention.

[0003] Using only electromyographic signals to predict the gripping force of a prosthetic hand, due to the lack of environmental perception, it is impossible to perceive the stability of the grasped object during the grasping process, making it difficult to adaptively adjust the gripping force to ensure stable grasping of the object. The reason why the human hand can use the corresponding gripping force for different objects lies in the receptors on the human hand, which respond to external mechanical stimuli through touch, slip, pressure and pain, and respond appropriately to the corresponding stimuli.

[0004] The present invention discloses a multi-sensor fusion prosthetic hand grasping force feedback control method based on surface electromyography signals. By combining an improved TSK fuzzy recursive cerebellar model neural network and a nonlinear cerebellar model joint controller (CMAC), and using multiple sensors such as pressure, tactile sensation, acceleration, angular velocity, etc. as feedback signals to adjust the actual grasping force and grasping action, an object can be grasped accurately and stably. At the same time, the real-time temperature and contact force of the grasped object are detected during the grasping process, and the tactile information is fed back to the user through a vibration and display module. Summary of the invention

[0005] The present invention proposes a multi-sensor fusion prosthetic hand grasping force feedback control method, which combines an improved TSK fuzzy recursive cerebellar model neural network and a nonlinear cerebellar model joint controller (CMAC), and uses multiple sensors such as pressure, tactile sensation, acceleration, angular velocity, etc. as feedback signals, so that the user can adjust the actual grasping force and grasping action in time to achieve accurate and stable grasping of objects. At the same time, the real-time temperature and contact force of the grasped object are detected during the grasping process, and the tactile information is fed back to the user through a vibration and display module.

[0006] The present invention adopts the following technical solutions.

[0007] A multi-sensor fusion prosthetic hand gripping force feedback control method, the prosthetic hand includes a PD-based CMAC manipulator gripping force controller, and its control method includes a multi-sensor fusion and feedback method, a gripping force prediction method and a PD-based CMAC manipulator gripping control method;

[0008] In the multi-sensor fusion and feedback method, a sensor array composed of multiple sensors is used to collect feedback signals and perception signals from the forearm and the prosthetic hand, and the collected signals are fused, including collecting surface electromyographic signals of the forearm muscles related to the hand grasping movement, collecting the temperature, contact force, sliding state of the object grasped by the prosthetic hand, collecting the joint acceleration and angular velocity corresponding to the grasping action, and collecting the actual gripping force of the prosthetic hand's grasping action;

[0009] The grip force prediction method predicts the grip force of the current gripping action by using the effective feature information in the preprocessed electromyographic signal through an improved TSK fuzzy recursive cerebellar model;

[0010] The PD-based CMAC manipulator grasping control method is a nonlinear quantized CMAC-PD control method in an improved cerebellar model joint controller;

[0011] The multi-sensor fusion and feedback method is to fuse the signal acquisition data to generate the prosthetic end perception information of the current grasping action.

[0012] In the multi-sensor fusion and feedback method, a multi-sensor fusion signal processing module is used to fuse signal acquisition data and generate prosthetic end perception information; the sensors in the sensor array include a fused surface electromyography sensor arranged at the forearm of the prosthetic hand for collecting electromyographic signals, an acceleration sensor arranged at the prosthetic hand and an angular velocity sensor arranged at the prosthetic hand, and also include distributed pressure sensors and distributed touch and slip sensors arranged at the fingers of the prosthetic hand, and an infrared temperature sensor arranged at the center of the palm of the prosthetic hand.

[0013] In the perception information feedback method, the generated prosthetic end perception information is fed back to the prosthetic hand user through a vibration unit or a display screen; the vibration unit is configured at the user's arm and can emit a vibration signal; the display screen is a display screen provided at the user's wristband or a display screen provided at the prosthetic hand.

[0014] In the grip force prediction method, the surface electromyographic signals of the forearm muscles related to the hand grasping movement are filtered and preprocessed, and the time domain feature RMS is selected as the feature vector for grip force prediction. The signal features are processed by windowing and averaged within the window to obtain the required number of data points. Finally, the feature values ​​are put into the recursive cerebellar model neural network combined with TSK fuzzy reasoning for training and prediction to obtain the required grip force.

[0015] The recursive cerebellar model calculates the correlation coefficient and the root mean square error between the predicted value and the reference force, and uses them as evaluation indicators of the model to analyze the linearity and relative error of the prediction results.

[0016] The forearm muscles related to hand grasping movement include radial flexor carpi, ulnar flexor carpi, extensor digitorum, superficial flexor digitorum, extensor pollicis longus, etc.

[0017] When the recursive cerebellar model neural network predicts the grasping force of the prosthetic hand, the recursive cerebellar model neural network is combined with TSK fuzzy reasoning, and the wavelet function is used as the membership function to improve the generalization ability and learning performance of the network. The model structure of the recursive cerebellar model neural network includes an input layer, an associative memory layer, a receiving field layer, a weight memory layer, an output layer and a recursive unit;

[0018] The recursive cerebellar model neural network includes the following fuzzy inference rules

[0019] R λ : If x1 is μ 1jk ,x2 is μ 2jk ,......, and x n is μ njk ,

[0020] Then jk =c 1jk x1+c 2jk x2+……c njk x n ,j=1,2,...,n j ,k=1,2,...,n k

[0021] Among them, xi is the i-th input value, μ ijk is the fuzzy set of the i-th input, j-th layer, and k-th block, w jk is the weight of the subsequent partial TSK-type output;

[0022] The associative memory layer accumulates multiple elements into a block, and different blocks are obtained after moving each element. Each block executes a receiving domain basis function. The Gaussian wavelet function is used as the membership function, that is, the wavelet function is constructed using the first-order derivative of the Gaussian function. The specific calculation formula is:

[0023]

[0024] Among them, x rijk is the input signal, m ijk and v ijk are the translation and expansion parameters of the wavelet function, corresponding to the membership basis function of the i-th input, j-th layer, and k-th block.

[0025] The recursive unit function is as follows:

[0026] x rijk (t) = x i (t)+wrijk μ ijk (t-1) Formula 2;

[0027] Among them, t is the time series of the data, w rijk is the weight of the recursive unit, indicating the influence of the output of the previous moment on this moment, μ ijk (t-1) represents the output result of the previous moment, μ ijk is the output result of the associative memory layer. Therefore, the output at this moment contains the data at this moment and the influence of the previous moment on it;

[0028] The receiving domain layer uses AND operation to realize fuzzy product reasoning, and its formula is:

[0029]

[0030] The weight memory layer has the weight of the TSK fuzzy output, and uses the input polynomial function to obtain a more accurate approximation capability. The formula is

[0031]

[0032] where c ijk is a constant, representing the connection weight between the j-th layer, the k-th block and the i-th input;

[0033] The output y of the recursive cerebellar model neural network is the algebraic sum of the receiving field and the activation weight, which is the predicted grasping force, and the formula is

[0034]

[0035] The update parameter of the recursive cerebellar model is m ijk 、v ijk 、w rijk and c ijk , which is used to describe the error function of the update algorithm is defined as

[0036]

[0037] The momentum gradient descent method is used to update the parameters, and the weights are updated by calculating the exponential weighted average of the gradient, where w is m ijk 、v ijk 、w rijk and c ijk , the formula is

[0038] V dw =β*V dw +(1-β)*dw Formula 7;

[0039] w=w+α*V dw Formula 8;

[0040] Where β is the momentum coefficient, β*V dw is the momentum term, (1-β)*dw is the gradient descent amount, V dw It is the exponential weighted average sum, multiplied by the learning decay rate α to get the parameter update amount.

[0041] In the multi-sensor fusion and feedback method, a sensor array composed of multiple sensors is used to collect perception feedback signals from the forearm and the prosthetic hand, including collecting surface electromyographic signals of the forearm muscles related to the hand grasping movement, collecting the temperature, contact force and sliding state of the object grasped by the prosthetic hand, and collecting the actual grip force of the prosthetic hand's grasping action;

[0042] In the multi-sensor fusion and feedback method, a multi-sensor fusion signal processing module is used to fuse signal acquisition data and generate prosthetic end perception information; the sensors in the sensor array include a fused surface electromyography sensor arranged at the forearm of the prosthetic hand for collecting electromyographic signals, an acceleration sensor arranged at the prosthetic hand and an angular velocity sensor arranged at the prosthetic hand, and also include distributed pressure sensors and distributed touch and slip sensors arranged at the fingers of the prosthetic hand, and an infrared temperature sensor arranged at the center of the palm of the prosthetic hand.

[0043] In the perception information feedback method, the generated prosthetic end perception information is fed back to the prosthetic hand user through a vibration unit or a display screen; the vibration unit is configured at the user's arm and can emit a vibration signal; the display screen is a display screen provided at the user's wristband or a display screen provided at the prosthetic hand.

[0044] The PD-based CMAC manipulator gripping force control method is used for the nonlinear quantized CMAC-PD controller in the improved cerebellum model joint controller;

[0045] When the CMAC manipulator gripping force controller of the cerebellar model joint controller controls the prosthetic hand, the user's gripping intention and the user's force output intention are judged by the collected electromyographic signals. The control system of the prosthetic hand sends each state point sampled during the force application process of the prosthetic hand as an input signal to CMAC, and uses the predicted value obtained by the cerebellar model in response to the user's electromyographic signal as a driving signal to control the next time point of the prosthetic hand's force adjustment. That is, CMAC is used for feedforward control to improve the response speed and control accuracy of the system, PD is used for negative feedback control, and the CMAC weights are adjusted using multi-fusion signals and PD control algorithms to speed up the system approximation speed and ensure the stability of the system. When sampling, a non-uniform quantization method is used to fit the sinusoidal density of the sampling distribution.

[0046] The control system of the prosthetic hand collects signals from distributed pressure sensors, distributed touch-slip sensors, acceleration sensors, angular velocity sensors, and infrared temperature sensors, obtains multiple outputs such as the model grasping force output by the fuzzy fusion model, and feeds back the output to the PD link and CMAC controller of the improved cerebellar model joint controller for driving and control, and adjusts the current action of the prosthetic hand, specifically: multiple sensor signals are fused as feedback signals, distributed touch-slip sensors are used to detect the contact between the prosthetic hand and the object, and then the actual grasping force collected by the distributed pressure sensor and the slip state detected by the touch-slip sensor are adjusted through the improved cerebellar model joint controller; acceleration and angular velocity signals are collected at the same time and combined with the touch-slip signals to adjust the joint angle of the prosthetic hand in real time; multiple sensor signals are fused as feedback signals, and temperature information and contact force information are fed back to the prosthetic hand user through a vibration unit or a display screen to optimize the stability of grasping objects.

[0047] The vibration unit is a pressure, touch and slip vibration sensing module set at the prosthetic hand. When the prosthetic hand touches an object, as the force applied increases, the configured vibration module will gradually increase the vibration sensation; if relative sliding occurs when grabbing the object, the vibration module will change the vibration mode to remind the user that the object may slip.

[0048] When the prosthetic hand approaches the object to be grasped, the infrared temperature sensor detects the temperature of the object, and the detected temperature is displayed on the display in real time.

[0049] The purpose of the present invention is to provide a multi-sensor fusion prosthetic hand grasping force control method based on surface electromyography signals. During the grasping process, surface electromyography signals are used to predict the grasping force through an improved TSK fuzzy cerebellar model neural network, and pressure and touch-slip sensors are used to collect the actual grasping force and the sliding state of the grasped object, which are fed back to the nonlinear quantized CMAC-PD controller, and acceleration and angular velocity sensors are added to adjust the grasping action and grasping force of the prosthetic hand in real time. The double closed-loop feedback control ensures the stability and reliability of the grasping, and objects of different hardness can be grasped without damaging the objects. At the same time, by detecting the temperature of the object for real-time display, the user is provided with pressure and vibration perception, which provides safety protection for the prosthetic hand to grasp events and contact the surrounding environment, and can provide more humane help to the user and improve the user experience.

[0050] The present invention utilizes surface electromyography signals and combines pressure signals, slip signals, acceleration signals and angular velocity signals to implement the grasping operation of the prosthetic hand. The double closed-loop feedback control ensures the stability and reliability of the grasping, and provides safety protection for the prosthetic hand to grasp and contact the surrounding environment through temperature sensing feedback.

[0051] The present invention combines an improved TSK fuzzy recursive cerebellar model neural network with a nonlinear cerebellar model joint controller (CMAC), performs filtering and other preprocessing on the surface electromyographic signals of the relevant muscles of the forearm, extracts effective feature information, uses an improved TSK fuzzy recursive cerebellar model to predict the size of the gripping force, and uses multiple sensors such as pressure, tactile sensation, acceleration, angular velocity, etc. as feedback signals to notify the user, so that the user can adjust the actual gripping force and gripping action in time to achieve accurate and stable gripping of objects. At the same time, the real-time temperature and contact force of the gripped object are detected during the gripping process, and the sensed information is fed back to the user through the vibration and display module, which can achieve multi-sensory closed-loop feedback control of the prosthetic hand.

[0052] In order to better control the robot to grasp objects stably, the present invention proposes a PD-based CMAC robot gripping force controller, and uses a nonlinear quantization method to perform adaptive division at the input layer to improve the response speed and control accuracy of the system. An infrared temperature sensor is placed on the myoelectric prosthesis to detect the temperature of the object when the prosthetic hand is close to the contact object. The detected temperature is displayed in real time on the bracelet (or a temperature display screen is configured on the back of the prosthetic hand), and the user can instantly understand the temperature of the contact object; and set a reminder for excessively high or low temperatures, pre-set different ranges of temperature, and when the detected temperature is within a certain range, the vibration unit configured on the arm emits a corresponding range of vibration intensity to alert the user.

[0053] The scheme of the present invention proposes an improved cerebellum model neural network, which combines the TSK fuzzy system with the CMNN. In the TSK fuzzy inference model, the constants in the general fuzzy inference process are replaced by polynomial functions of the input variables based on TSK, which can enrich the mapping, obtain more accurate approximate performance, and be more powerful in processing multivariable uncertain nonlinear systems. At the same time, the wavelet function is used as the membership function. Since the wavelet function has the time-frequency localization characteristics, it can better process stationary signals and non-stationary signals.

[0054] The present invention combines multi-sensor fusion technology with a double closed-loop front-feedback control system. While ensuring that the prosthetic hand can accurately apply the expected gripping force to the object, the gripping force is finely adjusted to ensure that the prosthetic hand can stably grasp objects of different hardness without damaging the object. In addition, in the process of contacting the object, the temperature of the grasped object is fed back to the user in real time, which can provide more humane help to the user and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0056] Attached Figure 1 It is a schematic diagram of the principle of the present invention;

[0057] Attached Figure 2 It is a schematic diagram of the process of grip force prediction of the present invention (prediction of prosthetic hand grip force based on surface electromyography signal);

[0058] Attached Figure 3 It is a schematic diagram of the model structure of the cerebellum model neural network of the present invention;

[0059] Attached Figure 4 It is a schematic diagram of the principle structure of the cerebellum model joint controller of the present invention. DETAILED DESCRIPTION

[0060] As shown in the figure, a multi-sensor fusion prosthetic hand gripping force feedback control method, the prosthetic hand includes a PD-based CMAC manipulator gripping force controller, and its control method includes a multi-sensor fusion and feedback method, a gripping force prediction method and a PD-based CMAC manipulator gripping control method;

[0061] In the multi-sensor fusion and feedback method, a sensor array composed of multiple sensors is used to collect feedback signals and perception signals from the forearm and the prosthetic hand, and the collected signals are fused, including collecting surface electromyographic signals of the forearm muscles related to the hand grasping movement, collecting the temperature, contact force, sliding state of the object grasped by the prosthetic hand, collecting the joint acceleration and angular velocity corresponding to the grasping action, and collecting the actual gripping force of the prosthetic hand's grasping action;

[0062] The grip force prediction method predicts the grip force of the current gripping action by using the effective feature information in the preprocessed electromyographic signal through an improved TSK fuzzy recursive cerebellar model;

[0063] The PD-based CMAC manipulator grasping control method is an improved nonlinear quantized CMAC-PD control method in the cerebellum model joint controller.

[0064] The multi-sensor fusion and feedback method is to fuse the signal acquisition data to generate the prosthetic end perception information of the current grasping action.

[0065] In the multi-sensor fusion and feedback method, a multi-sensor fusion signal processing module is used to fuse signal acquisition data and generate prosthetic end perception information; the sensors in the sensor array include a fused surface electromyography sensor arranged at the forearm of the prosthetic hand for collecting electromyographic signals, an acceleration sensor arranged at the prosthetic hand and an angular velocity sensor arranged at the prosthetic hand, and also include distributed pressure sensors and distributed touch and slip sensors arranged at the fingers of the prosthetic hand, and an infrared temperature sensor arranged at the center of the palm of the prosthetic hand.

[0066] In the perception information feedback method, the generated prosthetic end perception information is fed back to the prosthetic hand user through a vibration unit or a display screen; the vibration unit is configured at the user's arm and can emit a vibration signal; the display screen is a display screen provided at the user's wristband or a display screen provided at the prosthetic hand.

[0067] In the grip force prediction method, the surface electromyographic signals of the forearm muscles related to the hand grasping movement are filtered and preprocessed, and the time domain feature RMS is selected as the feature vector for grip force prediction. The signal features are processed by windowing and averaged within the window to obtain the required number of data points. Finally, the feature values ​​are put into the recursive cerebellar model neural network combined with TSK fuzzy reasoning for training and prediction to obtain the required grip force.

[0068] The recursive cerebellar model calculates the correlation coefficient and the root mean square error between the predicted value and the reference force, and uses them as evaluation indicators of the model to analyze the linearity and relative error of the prediction results.

[0069] The forearm muscles related to hand grasping movement include radial flexor carpi, ulnar flexor carpi, extensor digitorum, superficial flexor digitorum, extensor pollicis longus, etc.

[0070] When the recursive cerebellar model neural network predicts the grasping force of the prosthetic hand, the recursive cerebellar model neural network is combined with TSK fuzzy reasoning, and the wavelet function is used as the membership function to improve the generalization ability and learning performance of the network. The model structure of the recursive cerebellar model neural network includes an input layer, an associative memory layer, a receiving field layer, a weight memory layer, an output layer and a recursive unit;

[0071] The recursive cerebellar model neural network includes the following fuzzy inference rules

[0072] R λ : If x1 is μ 1jk ,x2 is μ 2jk ,......, and x n is μ njk ,

[0073] Then jk =c 1jk x1+c 2jk x2+......c njk x n ,j=1,2,...,n j ,k=1,2,...,n k

[0074] Among them, xi is the i-th input value, μ ijk is the fuzzy set of the i-th input, j-th layer, and k-th block, w jk is the weight of the subsequent partial TSK-type output;

[0075] The associative memory layer accumulates multiple elements into a block, and different blocks are obtained after moving each element. Each block executes a receiving domain basis function. The Gaussian wavelet function is used as the membership function, that is, the wavelet function is constructed using the first-order derivative of the Gaussian function. The specific calculation formula is:

[0076]

[0077] Among them, x rijk is the input signal, m ijk and v ijk are the translation and expansion parameters of the wavelet function, corresponding to the membership basis function of the i-th input, j-th layer, and k-th block.

[0078] The recursive unit function is as follows:

[0079] x rijk (t) = x i (t)+w rijk μ ijk (t-1) Formula 2;

[0080] Among them, t is the time series of the data, w rijk is the weight of the recursive unit, indicating the influence of the output of the previous moment on this moment, μ ijk (t-1) represents the output result of the previous moment, μ ijk is the output result of the associative memory layer. Therefore, the output at this moment contains the data at this moment and the influence of the previous moment on it;

[0081] The receiving domain layer uses AND operation to realize fuzzy product reasoning, and its formula is:

[0082]

[0083] The weight memory layer has the weight of the TSK fuzzy output, and uses the input polynomial function to obtain a more accurate approximation capability. The formula is

[0084]

[0085] where c ijk is a constant, representing the connection weight between the j-th layer, the k-th block and the i-th input;

[0086] The output y of the recursive cerebellar model neural network is the algebraic sum of the receiving field and the activation weight, which is the predicted grasping force, and the formula is

[0087]

[0088] The update parameter of the recursive cerebellar model is m ijk 、vijk 、w rijk and c ijk , which is used to describe the error function of the update algorithm is defined as

[0089]

[0090] The momentum gradient descent method is used to update the parameters, and the weights are updated by calculating the exponential weighted average of the gradient, where w is m ijk 、v ijk 、w rijk and c ijk , the formula is

[0091] V dw =β*V dw +(1-β)*dw Formula 7;

[0092] w=w+α*V dw Formula 8;

[0093] Where β is the momentum coefficient, β*V dw is the momentum term, (1-β)*dw is the gradient descent amount, V dw It is the exponential weighted average sum, multiplied by the learning decay rate α to get the parameter update amount.

[0094] In the multi-sensor fusion and feedback method, a multi-sensor fusion signal processing module is used to fuse signal acquisition data and generate prosthetic end perception information; the sensors in the sensor array include a fused surface electromyography sensor arranged at the forearm of the prosthetic hand for collecting electromyographic signals, an acceleration sensor arranged at the prosthetic hand and an angular velocity sensor arranged at the prosthetic hand, and also include distributed pressure sensors and distributed touch and slip sensors arranged at the fingers of the prosthetic hand, and an infrared temperature sensor arranged at the center of the palm of the prosthetic hand.

[0095] In the multi-sensor fusion and feedback method, the generated prosthetic end perception information is fed back to the prosthetic hand user through a vibration unit or a display screen; the vibration unit is configured at the user's arm and can emit a vibration signal; the display screen is a display screen provided at the user's wristband or a display screen provided at the prosthetic hand.

[0096] The PD-based CMAC manipulator gripping force control method is used for the nonlinear quantized CMAC-PD controller in the improved cerebellum model joint controller;

[0097] The control system of the prosthetic hand includes a distributed pressure sensor, a distributed touch-slip sensor, an acceleration sensor, an angular velocity sensor, and an infrared temperature sensor. After a fuzzy fusion model, multiple outputs such as the model grasping force outputted by the sensor are obtained, and the outputs are fed back to the PD link and CMAC controller of the improved cerebellum model joint controller for driving and control, so as to adjust the current action of the prosthetic hand. Specifically, the multi-sensor signals are fused as feedback signals, and the distributed touch-slip sensor is used to detect the contact between the prosthetic hand and the object. Then, the actual grasping force collected by the distributed pressure sensor and the slip state detected by the touch-slip sensor are used to adjust the grasping force through the improved cerebellum model joint controller; the acceleration and angular velocity signals are collected at the same time and combined with the touch-slip signal to adjust the joint angle of the prosthetic hand in real time; the multi-sensor signals are fused as feedback signals, and the temperature information and contact force information are fed back to the prosthetic hand user through a vibration unit or a display screen to optimize the stability of grasping the object.

[0098] When the CMAC manipulator gripping force controller of the cerebellar model joint controller controls the prosthetic hand, the user's gripping intention and the user's force output intention are judged by the collected electromyographic signals. The control system of the prosthetic hand sends each state point sampled during the force application process of the prosthetic hand as an input signal to CMAC, and uses the predicted value obtained by the cerebellar model in response to the user's electromyographic signal as a driving signal to control the next time point of the prosthetic hand's force adjustment, that is, CMAC is used for feedforward control to improve the response speed and control accuracy of the system, PD is used for negative feedback control, and the CMAC weights are adjusted using multi-fusion signals and PD control algorithms to speed up the system approximation speed and ensure the stability of the system;

[0099] When sampling is performed, a non-uniform quantization method is used to fit the sinusoidal density of the sampling distribution.

[0100] The vibration unit is a pressure, touch and slip vibration sensing module set at the prosthetic hand. When the prosthetic hand touches an object, as the force applied increases, the configured vibration module will gradually increase the vibration sensation; if relative sliding occurs when grabbing the object, the vibration module will change the vibration mode to remind the user that the object may slip.

[0101] When the prosthetic hand approaches the object to be grasped, the infrared temperature sensor detects the temperature of the object, and the detected temperature is displayed on the display in real time.

Claims

1. A multi-sensor fusion prosthetic hand grip force feedback control method, characterized in that: The prosthetic hand includes a PD-based CMAC manipulator gripping force controller, and its control method includes a multi-sensor fusion and feedback method, a gripping force prediction method, and a PD-based CMAC manipulator gripping control method; In the multi-sensor fusion and feedback method, a sensor array composed of multiple sensors is used to collect feedback signals and perception signals from the forearm and the prosthetic hand, and the collected signals are fused, including collecting surface electromyographic signals of the forearm muscles related to the hand grasping movement, collecting the temperature, contact force, sliding state of the object grasped by the prosthetic hand, collecting the joint acceleration and angular velocity corresponding to the grasping action, and collecting the actual gripping force of the prosthetic hand's grasping action; The grip force prediction method predicts the grip force of the current gripping action by using the effective feature information in the preprocessed electromyographic signal through an improved TSK fuzzy recursive cerebellar model; The PD-based CMAC manipulator grasping control method is a nonlinear quantized CMAC-PD control method in an improved cerebellar model joint controller; When the improved cerebellar model neural network predicts the grasping force of the prosthetic hand, the surface electromyographic signals of the forearm muscles related to the grasping movement of the hand are filtered and preprocessed, the recursive cerebellar model neural network is combined with TSK fuzzy reasoning, and the wavelet function is used as the membership function to improve the generalization ability and learning performance of the network. The model structure of the recursive cerebellar model neural network includes an input layer, an associative memory layer, a receiving field layer, a weight memory layer, an output layer and a recursive unit; The recursive cerebellar model neural network includes the following fuzzy inference rules R λ : If x1 is μ 1jk ,x2 is μ 2jk ,......, and x n is μ njk , Then w jk = c 1jk x1 + c 2jk x2 + …… c njk x n where j = 1, 2, ..., n j and k = 1, 2, ..., n k Among them, x n is the nth input value, μ ijk is the fuzzy set of the i-th input, j-th layer, and k-th block, w jk is the weight of the subsequent partial TSK-type output; The associative memory layer accumulates multiple elements into a block, and obtains different blocks after moving each element. Each block executes a receiving domain basis function. The Gaussian wavelet function is used as the membership function, that is, the wavelet function is constructed using the first-order derivative of the Gaussian function. The specific calculation formula is: Among them, x rijk is the input signal, m ijk and v ijk are the translation and expansion parameters of the wavelet function, corresponding to the membership basis function of the i-th input, j-th layer, and k-th block; The recursive unit function is as follows: x rijk (t) = x i (t)+w rijk μ ijk (t-1) Formula 2; Among them, t is the time series of the data, w rijk is the weight of the recursive unit, indicating the influence of the output of the previous moment on this moment, μ ijk (t-1) represents the output result of the previous moment, μ ijk is the output result of the associative memory layer; therefore, the output at this moment contains the data at this moment and the influence of the previous moment on it; The receiving domain layer uses AND operation to realize fuzzy product reasoning, and its formula is: The weight memory layer has the weight of the TSK fuzzy output, and uses the input polynomial function to obtain a more accurate approximation capability. The formula is where c ijk is a constant, representing the connection weight between the j-th layer, the k-th block and the i-th input; The output y of the recursive cerebellar model neural network is the algebraic sum of the receiving field and the activation weight, which is the predicted grasping force, and the formula is The update parameter of the recursive cerebellar model is m ijk 、v ijk 、w rijk and c ijk , which is used to describe the error function of the update algorithm is defined as The momentum gradient descent method is used to update the parameters, and the weights are updated by calculating the exponential weighted average of the gradient, where w is m ijk 、v ijk 、w rijk and c ijk , the formula is V dw =β*V dw +(1-β)*dw Formula 7; w=w+α*V dw Formula 8; Where β is the momentum coefficient, β*V dw is the momentum term, (1-β)*dw is the gradient descent amount, V dw It is the exponential weighted average sum, multiplied by the learning decay rate α to get the parameter update amount; In the multi-sensor fusion and feedback method, a multi-sensor fusion signal processing module is used to fuse the signal acquisition data and generate the prosthetic end perception information; the sensors in the sensor array include a fusion surface electromyography sensor arranged at the forearm of the prosthetic hand for collecting electromyographic signals, an acceleration sensor arranged at the prosthetic hand, and an angular velocity sensor arranged at the prosthetic hand, and also include a distributed pressure sensor and a distributed touch and slip sensor arranged at the fingers of the prosthetic hand, and an infrared temperature sensor arranged at the center of the palm of the prosthetic hand; the PD-based CMAC manipulator gripping force control method is used for the nonlinear quantized CMAC-PD controller in the improved cerebellar model joint controller; The control system of the prosthetic hand collects signals from distributed pressure sensors, distributed touch-slip sensors, acceleration sensors, angular velocity sensors, and infrared temperature sensors, obtains the model grasping action output by the fuzzy fusion model, and feeds back the output to the PD link and CMAC controller of the improved cerebellum model joint controller for driving and control, and adjusts the current action of the prosthetic hand, specifically: fuses the multi-sensor signals as feedback signals, uses distributed touch-slip sensors to detect the contact between the prosthetic hand and the object, and then adjusts the grasping force through the improved cerebellum model joint controller by combining the actual grasping force collected by the distributed pressure sensor with the slip state detected by the touch-slip sensor; collects acceleration and angular velocity signals at the same time and adjusts the joint angle of the prosthetic hand in real time in combination with the touch-slip signals; fuses the multi-sensor signals as feedback signals, and feeds back the temperature information and contact force information to the prosthetic hand user through a vibration unit or a display screen, so as to optimize the stability of grasping the object; When the CMAC manipulator grip force controller of the cerebellar model joint controller controls the prosthetic hand, the user's grip intention and the user's force intention are judged by the collected electromyographic signals. The control system of the prosthetic hand sends each state point sampled during the force application process of the prosthetic hand as an input signal to CMAC, and uses the predicted value obtained by the cerebellar model in response to the user's electromyographic signal as a driving signal to control the next time point of the prosthetic hand to adjust the force. That is, CMAC is used for feedforward control to improve the response speed and control accuracy of the system. Use PD as negative feedback control, and use multi-fusion signal and PD control algorithm to adjust the CMAC weights. To speed up the system approach and ensure the stability of the system; When sampling is performed, a non-uniform quantization method is used to fit the sinusoidal density of the sampling distribution.

Images