Multimodal perception feedback system and method based on myoelectric prosthetic hand

Abstract

The invention discloses a multi-mode perception feedback system and method based on a myoelectric prosthetic hand, the system comprises the myoelectric prosthetic hand, a feedback control module, an electrical stimulation feedback module and a vibration feedback module, a user controls the myoelectric prosthetic hand to make different actions and interact with an environment object, and a force tactile sensor in the myoelectric prosthetic hand is connected with the feedback control module. The motor encoder and the infrared distance measuring sensor respectively obtain force touch sense, proprioceptive sense and proximity sense sensing information, and transmit the information to the feedback control module; after receiving the sensing information, the feedback control module converts the sensing information into a feedback normal form and outputs a stimulation feedback output signal; the vibration feedback module and the electrical stimulation feedback module receive the stimulation feedback output signal and carry out vibration feedback and electrical stimulation feedback. According to the method, collaborative feedback of multiple types of stimulation can be realized, and the feedback efficiency and effect of sensing information are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-modal perception and feedback of human-machine co-integrated bio-electromechanical devices, and mainly relates to a multi-modal perception feedback system and method based on a myoelectric prosthetic hand. Background Art

[0002] The research on artificial prosthetics can be applied to many fields such as high-end medical equipment, bio-electromechanical integration intelligent robots, hazardous environment exploration, disaster rescue equipment, and assisting disabled people in rehabilitation engineering training. Its scientific and technological achievements can radiate, so it has important strategic significance. So far, the hand structure of humanoid robots has been very mature, and single-point or even multi-point array force / tactile sensors can be integrated to endow them with force / tactile perception functions. However, there is currently no good solution to intuitively feedback the force / tactile information, proprioceptive information, and proximity information perceived by the prosthetic hand to the prosthetic user, which also limits the application of the prosthetic hand. Summary of the Invention

[0003] In view of the problem in the prior art that myoelectric prosthetics lack perception information feedback and cannot achieve closed-loop control, the present invention proposes a multi-modal perception feedback system and method based on a myoelectric prosthetic hand, including a myoelectric prosthetic hand, a feedback control module, an electrical stimulation feedback module, and a vibration feedback module. The user controls the myoelectric prosthetic hand to make different movements and interact with environmental objects. The proximity sensor in the myoelectric prosthetic hand detects the distance of the object in front of the palm, the force / tactile sensor senses the pressure information received by the fingers and the palm, and the motor encoder obtains the perception information of the hand and wrist movements and transmits the information to the feedback control module. After receiving the perception information, the feedback control module converts the perception information into a feedback paradigm and outputs a stimulation feedback output signal. The vibration feedback module and the electrical stimulation feedback module receive the stimulation feedback output signal and perform vibration feedback and electrical stimulation feedback. The method of the present invention can achieve the collaborative feedback of multiple types of stimuli and improve the feedback efficiency and effect of perception information.

[0004] To achieve the above object, the technical solution adopted by the present invention is: a multi-modal perception feedback system based on a myoelectric prosthetic hand, at least including a myoelectric prosthetic hand, a feedback control module, an electrical stimulation feedback module, and a vibration feedback module.

[0005] The myoelectric prosthetic hand: the fingers and palm are provided with force tactile sensors, which can obtain force tactile information of the fingers and palm; the finger motor is equipped with an encoder, which can measure the stroke information of the finger motor movement, that is, the finger proprioceptive information; the two motors driving the wrist to flip and rotate are each equipped with an encoder, which can respectively obtain the rotation and flip angle information of the wrist, that is, the wrist proprioceptive information; the prosthetic hand is equipped with an infrared ranging sensor at the base of the prosthetic hand, which can obtain the distance information between the prosthetic hand and the object in front, that is, the proximity perception information, which is used to obtain the perception information of the movement of the myoelectric prosthetic hand and transmit the information to the feedback control module; the perception information includes force tactile perception information, proprioceptive perception information and proximity perception information;

[0006] The feedback control module is connected to the myoelectric prosthetic hand, the electrical stimulation feedback module and the vibration feedback module, respectively, and is used to convert the sensory information of the myoelectric prosthetic hand movement into a stimulation feedback output signal, and transmit the stimulation feedback output signal to the electrical stimulation feedback module and the vibration feedback module;

[0007] The electrical stimulation feedback module: inputs 4-channel micro-current pulse signals, outputs 4-channel electrical stimulation, and is located on the outside of the upper arm;

[0008] The vibration feedback module: inputs 9-channel 3.3V PWM signal, outputs 9-channel vibration stimulation, and is located on the inner side of the upper arm.

[0009] As an improvement of the present invention, the vibration feedback module includes a vibration motor array, a vibration feedback module flexible printed circuit board and a vibration feedback module FPC connecting line. The vibration feedback module FPC connecting line is used to connect the interface of the vibration feedback module flexible printed circuit board and the feedback control module. The vibration motor array is 9 eccentric vibration motors arranged in a 3×3 manner and installed at the circular pad of the flexible printed circuit board.

[0010] As another improvement of the present invention, the electrical stimulation feedback module includes electrical stimulation electrodes, an electrical stimulation array flexible printed circuit board and an electrical stimulation feedback module FPC connecting line. The electrical stimulation feedback module FPC connecting line is used to connect the interface between the electrical stimulation array flexible printed circuit board and the feedback control module. The electrical stimulation electrodes are arranged in a cross shape through 12 metal copper sheets and installed on the circular pads of the electrical stimulation array flexible printed circuit board.

[0011] In order to achieve the above object, the present invention also adopts a technical solution: a multimodal perception feedback method based on myoelectric prosthetic hand, comprising the following steps:

[0012] S1: The user controls the myoelectric prosthetic hand to perform different actions and interact with environmental objects. The proximity sensor in the myoelectric prosthetic hand detects the distance of the object in front of the palm. The force and tactile sensor senses the pressure information received by the fingers and the palm. The motor encoder obtains the perception information of the hand and wrist movements and transmits the information to the feedback control module;

[0013] S2: The feedback control module receives the perception information obtained in step S1;

[0014] S3: The feedback control module converts the perception information into a feedback paradigm and outputs a stimulus feedback output signal;

[0015] S4: The vibration feedback module and the electrical stimulation feedback module receive the stimulus feedback output signal output in step S3 and perform vibration feedback and electrical stimulation feedback.

[0016] As an improvement of the present invention, the process of converting the perception information into a stimulus feedback output signal in step S3 is specifically as follows:

[0017] S31: After the feedback control module receives the perception information, it judges whether the proximity information of the myoelectric prosthetic hand reaches the set feedback threshold. If the threshold is reached, it selects the proximity feedback paradigm and calculates the electrical stimulation intensity and frequency to be output according to the proximity distance information size; if the proximity threshold is not reached, it judges whether there is force and tactile information. If there is force and tactile information, it selects the force and tactile feedback paradigm and selects the electrical stimulation intensity and frequency to be output according to the force and tactile size;

[0018] S32: Judge whether the wrist of the myoelectric prosthetic hand is moving. If it is moving, select the wrist proprioceptive feedback paradigm and calculate the intensity and frequency of the vibration feedback according to the rotation angle; if there is finger proprioceptive bending, select the finger proprioceptive feedback paradigm and calculate the intensity and frequency of the vibration feedback according to the finger bending degree.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a multi-modal perception feedback system and method based on a myoelectric prosthetic hand, which can feedback the proprioceptive information, force and tactile information, and proximity information of the myoelectric prosthetic hand, and realize the perception-control closed loop of the myoelectric prosthetic hand; the present invention adopts a modular integration method, and the control module, vibration feedback module, and electrical stimulation feedback module can be independently replaced, saving costs; compared with the existing technology, it can realize the collaborative feedback of multiple types of stimuli, reduce the user's perception fatigue, and improve the perception feedback effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic structural diagram of the multi-modal perception feedback system based on the myoelectric prosthetic hand of the present invention;

[0021] Figure 2It is a schematic structural diagram of the vibration feedback module in the system of the present invention;

[0022] Figure 3 It is a schematic structural diagram of the electrical stimulation feedback module in the system of the present invention;

[0023] Figure 4 It is a schematic diagram of the skin - facing - away surface in the usage state of the system of the present invention;

[0024] Figure 5 It is a schematic diagram of the skin - contacting surface in the usage state of the system of the present invention;

[0025] Figure 6 It is a step - flow chart of the multi - modal perception feedback method based on the myoelectric prosthetic hand of the present invention;

[0026] Figure 7 It is a schematic diagram of the numbering of the vibration unit and the electrical stimulation electrodes in Embodiment 2 of the present invention;

[0027] Figure 8 It is a schematic diagram of the proximity - sense feedback paradigm in Embodiment 2 of the present invention;

[0028] Figure 9 It is a schematic diagram of the force - touch feedback paradigm in Embodiment 2 of the present invention;

[0029] Figure 10 It is a schematic diagram of the wrist proprioception feedback paradigm in Embodiment 2 of the present invention;

[0030] Figure 11 It is a schematic diagram of the wrist movement in Embodiment 2 of the present invention;

[0031] Figure 12 It is a schematic diagram of the finger movement in Embodiment 2 of the present invention;

[0032] Figure 13 It is a schematic diagram of the finger proprioception feedback paradigm in Embodiment 2 of the present invention;

[0033] In the figure: 1. Feedback control module; 2. Vibration feedback module; 3. Electrical stimulation feedback module; 4. Myoelectric prosthetic hand; 5. Vibration motor array; 6. Flexible printed circuit board of the vibration module; 7. FPC connecting wire of the vibration module; 8. Electrical stimulation electrode; 9. Flexible printed circuit board of the electrical stimulation array; 10. FPC connecting wire of the electrical stimulation module Specific embodiments

[0034] The following further clarifies the present invention in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0035] Embodiment 1

[0036] The multi - modal perception feedback system based on the myoelectric prosthetic hand, such asFigure 1 As shown in the figure, it includes a feedback control module 1, a vibration feedback module 2, an electrical stimulation feedback module 3, and an electromyographic prosthetic hand 4. The electromyographic prosthetic hand 4 has finger force touch, palm force touch, finger travel proprioception, wrist rotation proprioception, and proximity sense perception functions, and transmits the above perception information to the feedback control module 1.

[0037] The vibration feedback module 2 includes a vibration motor array 5, a vibration module flexible printed circuit board 6, and a vibration module FPC connecting wire 7, specifically as Figure 2 shown. The vibration motor array 5 uses 9 eccentric vibration motors, arranged in a 3×3 layout, and is installed at the corresponding positions on the vibration module flexible printed circuit board 6. The vibration module FPC connecting wire 7 connects the vibration module flexible printed circuit board 6 to the interface of the feedback control module 1.

[0038] The electrical stimulation feedback module 3 includes electrical stimulation electrodes 8, an electrical stimulation array flexible printed circuit board 9, and an electrical stimulation module FPC connecting wire 10, specifically as Figure 3 shown. The electrical stimulation electrodes 8 use 12 metal copper sheets, of which 8 copper sheets have a diameter of 8 mm and the other 4 copper sheets have a diameter of 10 mm, forming a cross-shaped layout and are installed at the corresponding positions on the electrical stimulation array flexible printed circuit board 9. The electrical stimulation module FPC connecting wire 10 connects the electrical stimulation array flexible printed circuit board 9 to the interface of the feedback control module 1.

[0039] When in use, the system of this embodiment is pasted on the armband, specifically as Figure 4 and Figure 5 shown. Figure 4 and Figure 5 are respectively schematic diagrams of the system in the working state. From Figure 4 it can be seen the control box equipped with the feedback control module. Two FPC connecting wires extend from both sides of the control box, pass through the armband to the other side, that is, the vibration module FPC connecting wire 7 and the electrical stimulation module FPC connecting wire 10. Figure 5 it can be seen that the two FPC connecting wires are respectively connected to the vibration feedback module 2 and the electrical stimulation feedback module 3.

[0040] When the system of this embodiment works, the feedback control module 1 receives and processes various perception information of the electromyographic prosthetic hand 4, converts it into a paradigm of corresponding stimulation feedback, transmits the vibration feedback signal to the vibration feedback module 2 through the vibration module FPC connecting wire 7, and transmits the electrical stimulation feedback signal to the electrical stimulation feedback module 3 through the electrical stimulation module FPC connecting wire 10. The vibration feedback module 2 and the electrical stimulation feedback module 3 receive the stimulation feedback output signal, perform vibration feedback and electrical stimulation feedback, complete the perception feedback, and realize the perception-control closed loop of the electromyographic prosthetic hand.

[0041] Embodiment 2

[0042] A multi-modal perception feedback method based on an electromyographic prosthetic hand, the perception feedback process of which is as shown in FIG. 6, includes the following steps:

[0043] Step S1: Let the user wear a multi-modal perception feedback armband with the system described in Embodiment 1, and connect the control unit circuit board and the battery; the numbers of each vibration unit and electrical stimulation electrode on the armband are as Figure 7 shown. Let the user control the electromyographic prosthetic hand to make different movements and interact with environmental objects, and the electromyographic prosthetic hand transmits various perception information data to the feedback control module.

[0044] Step S2: The feedback control module receives the perception information data and converts it into a stimulation feedback output signal.

[0045] Step S3: The feedback control module converts the perception information into a feedback paradigm and outputs a stimulation feedback output signal.

[0046] S31: After receiving the perception information, the feedback control module determines whether the proximity information of the electromyographic prosthetic hand reaches the set feedback threshold. If the threshold is reached, the proximity feedback paradigm is selected, and the electrical stimulation intensity and frequency to be output are calculated according to the proximity distance information; if the proximity threshold is not reached, it is determined whether there is force tactile information. If there is force tactile information, the force tactile feedback paradigm is selected, and the electrical stimulation intensity and frequency to be output are selected according to the force tactile magnitude.

[0047] Proximity feedback paradigm: Eight small electrodes near the center point in the electrical stimulation feedback module form proximal electrodes, and four large electrodes at the four outer ends of the cross-shaped FPC form distal electrodes. Proximity feedback is achieved by alternately activating the proximal electrodes and the distal electrodes to realize the internal and external alternating stimulation sensation. The maximum distance that the proximity sensor can sense is 20 cm, the maximum interval time for alternating stimulation of the proximal and distal electrodes is 100 ms, and the maximum current intensity of the electrical stimulation is 20 mA. When the proximity sensor senses that an object is approaching, the alternating time decreases and the current intensity increases. The specific numerical calculation method: Calculate the ratio of the current object distance information to the proximity range (20 cm). The magnitude of the electrical stimulation output current is equal to the maximum current intensity (20 mA) multiplied by the difference between 1 and this ratio, and the alternating interval time is equal to the longest interval time (100 ms) multiplied by this ratio.

[0048] Force tactile feedback paradigm: The electrical stimulation electrodes in the electrical stimulation feedback module can be divided into 8 electrodes in a vertical column and 8 electrodes in a horizontal row. The vertical electrodes and the horizontal electrodes respectively represent the normal force tactile and tangential force tactile in the force tactile information. When there is force tactile information, the electrodes in the corresponding direction will perform dot-like current pulse stimulation, that is, the 8 electrodes in the vertical or horizontal direction will simultaneously start and stop intermittent current output. The output current stimulation intensity and the start-stop frequency are both positively correlated with the corresponding force tactile magnitude. Among them, the maximum range value of the force tactile is 20N, the maximum start-stop interval is 500ms, and the maximum current intensity of the electrical stimulation is 20mA. The specific numerical calculation method: Calculate the ratio of the magnitude of the longitudinal / normal force tactile to the maximum range of the force tactile (20N). The start-stop interval of the electrical stimulation output by the longitudinal / normal electrodes is equal to the maximum start-stop time interval (500ms) multiplied by the difference between 1 and this ratio. The output electrical stimulation current intensity is equal to the maximum current value (20mA) multiplied by this ratio.

[0049] S32: Determine whether the wrist of the myoelectric prosthesis is moving. If it is moving, select the wrist proprioceptive feedback paradigm and calculate the intensity and frequency of the vibration feedback according to the rotation angle. If there is finger proprioceptive bending, select the finger proprioceptive feedback paradigm and calculate the intensity and frequency of the vibration feedback according to the finger bending degree.

[0050] Wrist proprioceptive feedback paradigm: When the wrist rotates, first determine the direction and magnitude of the wrist rotation angle. Set the position where the palm is perpendicular to the ground and the thumb is above the four fingers as the neutral position of the wrist rotation angle, that is, the zero position of the rotation angle. When the wrist rotates from the neutral position to the direction where the palm faces the ground, it is the internal rotation direction, and when it rotates to the direction where the back of the hand faces the ground, it is the external rotation direction. The maximum rotation angle of the wrist is 90°. Divide the 9 vibration motors in the vibration feedback module into three columns of vibration unit columns arranged vertically. Each vibration unit column will vibrate sequentially in the direction from left to right (corresponding to external rotation) or from right to left (corresponding to internal rotation). The larger the rotation angle, the greater the vibration intensity of the vibration motor, that is, the larger the duty cycle of the PWM signal driving the vibration motor. The specific numerical calculation method: Calculate the ratio of the current wrist rotation angle to the maximum angle value of 90°. At the initial moment, the duty cycle of the PWM signal of the first column of vibration units is 50% of this ratio. After 100 ms, the first column of vibration units stops vibrating, and the second column starts vibrating. Its PWM signal duty cycle is 75% of this ratio. After 150 ms, the second column stops vibrating, and the third column starts vibrating. Its PWM signal is equal to this ratio. After 200 ms, the third column stops vibrating, and the above process repeats continuously. When the wrist flips, set the position where the palm plane is parallel to the forearm as the neutral position, the position where the palm faces the body as the wrist varus direction, and vice versa as the wrist valgus direction. The maximum flip angle of the wrist is 90°. Divide the 9 vibration motors in the vibration feedback module into three rows of vibration unit rows arranged horizontally. Each vibration unit row will vibrate sequentially in the direction from bottom to top (corresponding to valgus) or from top to bottom (corresponding to varus). The larger the flip angle, the greater the vibration intensity of the vibration motor, that is, the larger the duty cycle of the PWM signal driving the vibration motor. The specific numerical calculation method is similar to that of wrist rotation.

[0051] Finger proprioceptive feedback paradigm: Set the finger position when the palm is fully opened as the initial position. At this time, the bending angles of all fingers are 0°. When the thumb, index finger, and the remaining three fingers of the myoelectric prosthetic hand bend, the angle between the proximal phalanx of the finger and the initial position is the bending angle of the finger. Vertically divide the 9 vibration motors in the vibration feedback module into three columns of vibration unit columns. The three columns of vibration motors correspond to the thumb, index finger, and the remaining three fingers from left to right respectively. When the bending angle of the corresponding finger is greater than 0°, the vibration motor in this column vibrates continuously. The larger the finger bending angle, the stronger the vibration intensity. The specific numerical calculation method: The duty cycle value of the PWM signal is equal to the ratio of the finger bending angle to 180°.

[0052] The various paradigms in this embodiment are as Figures 8 - 13 shown, Figure 8It is a schematic diagram of the proximity sense feedback paradigm. The proximal electrodes are composed of 8 small electrodes close to the center in the electrical stimulation electrodes, and the distal electrodes are composed of 4 large electrodes close to the outside. The proximity sense feedback realizes the internal and external alternating stimulation by alternately activating the proximal electrodes and the distal electrodes. The alternating frequency, the electrical stimulation intensity, and the proximity sense distance information are positively correlated. The proximity sense feedback coding table is shown in Table 1 below:

[0053] Table 1

[0054]

[0055] where I x represents the ratio of the current intensity output by the electrical stimulation electrode with serial number x to the maximum output current of 20 mA. D represents the approaching distance. The maximum range of the proximity sensor is 20 cm. Among them, the electrodes numbered 1, 4, 5, and 8 and the electrodes numbered 2, 3, 6, and 7 alternately output electrical stimulation currents. The stimulation intensity is inversely proportional to the approaching distance D, and the alternating stimulation time t is proportional to the approaching distance D. The longest alternating stimulation time t is 100 ms.

[0056] If the proximity sense information does not reach the threshold, it is judged whether there is force touch information. If so, the force touch feedback paradigm of the electrical stimulation feedback module is selected, as specifically Figure 9 shown Figure 9 It shows a schematic diagram of the force touch feedback paradigm. The longitudinal electrodes and the transverse electrodes in the electrical stimulation electrodes in the figure respectively represent the normal force touch and the tangential force touch in the force touch information. When there is force touch information, the electrodes in the corresponding directions will perform dotting current pulse stimulation. The stimulation intensity is positively correlated with the dotting frequency and the corresponding force touch magnitude. The force touch feedback coding table is shown in Table 2

[0057] Table 2

[0058]

[0059]

[0060] where F t represents the tangential force touch value, F n represents the normal force touch value, and n is a positive integer. Among them, the electrical stimulation electrodes numbered 1, 2, 3, and 4 simultaneously perform dotting electrical stimulation for tangential force touch feedback, and the electrodes numbered 5, 6, 7, and 8 perform dotting stimulation feedback for normal force touch. The output current intensity is proportional to the corresponding force touch magnitude. The maximum range value of the force touch is 20 N. The start-stop interval t of the dotting electrical stimulation is inversely proportional to the force touch value, and the maximum start-stop interval is 500 ms.

[0061] Meanwhile, it is determined whether the prosthetic hand has made a wrist movement. If there is a wrist movement, the vibration feedback module performs the feedback of wrist proprioception, and the feedback paradigm is as follows Figure 10 as shown in (a). It can be seen from the figure that the three columns of vibration motors stimulate the wrist proprioception feedback paradigm in sequence from left to right or from right to left.

[0062] Suppose the wrist movement is a wrist rotation movement as shown in Figure 11 (a). The vibration array is divided into three columns of vertical vibration unit columns. Each column of vibration units vibrates in sequence, and the vibration feedback in the form of running water from left to right or from right to left is formed. The running water direction corresponds to the direction of wrist rotation, and the vibration intensity and the running water speed are positively correlated with the magnitude of the wrist rotation angle. The specific feedback coding table is shown in Table 3:

[0063] Table 3

[0064]

[0065]

[0066] Among them, P x represents the PWM duty cycle of the vibration motor with the serial number x, represents the magnitude of the wrist rotation angle. It is set that the wrist rotation angle when the palm is perpendicular to the ground is decreases when rotating counterclockwise and increases when rotating clockwise. Increases.

[0067] Similarly, if the wrist movement is a wrist flexion movement as shown in Figure 11 (b), the vibration array is divided into three rows of horizontal vibration unit rows, and the feedback paradigm is as shown in Figure 10 (b). The specific feedback coding table is shown in Table 4:

[0068] Table 4

[0069]

[0070]

[0071] Among them, θ represents the magnitude of the wrist flexion angle. It is set that the flexion angle when the wrist is in the neutral position is θ = 0, θ decreases when the wrist is in varus, and θ increases when the wrist is in valgus.

[0072] If the finger makes a movement as shown in Figure 12The movements shown, 12(a) represents the movement of the thumb, 12(b) represents the movement of the index finger, and 12(c) represents the movement of the remaining three fingers. Among them, α, β, and γ respectively represent the bending angles of the thumb, index finger, and the remaining three fingers (the middle finger, ring finger, and little finger are generally linked fingers in the myoelectric prosthetic hand), and they are corresponding to three columns of vibration unit columns in the vibration array. The vibration feedback paradigm is as follows Figure 13 as shown Figure 13 It can be seen from Figure 13 that the vibration feedback module is divided into three columns of vibration motor columns, corresponding to the thumb, index finger, and the remaining three fingers from left to right. When the corresponding finger bends, the corresponding vibration column will generate vibration stimuli with an intensity magnitude positively correlated with the bending angle. The specific feedback coding table is shown in Table 5:

[0073] Table 5

[0074]

[0075] In Table 5, the vibration motors numbered 1, 4, and 7 vibrate simultaneously to feedback the bending angle α of the thumb. When the thumb bends, the vibration intensity of the corresponding vibration motor increases; the same applies to the index finger and the remaining three fingers.

[0076] Step S4: Finally, the vibration feedback module and the electrical stimulation feedback module receive the stimulation feedback output signal, so as to perform vibration feedback and electrical stimulation feedback.

[0077] In summary, the multi-modal perception feedback system and method based on a myoelectric prosthetic hand disclosed by the present invention include a vibration feedback module, an electrical stimulation feedback module, a feedback control module, and a myoelectric prosthetic hand. The vibration feedback module and the electrical stimulation feedback module are respectively installed on both sides of the magic tape armband. The feedback control module receives and processes various perception information of the myoelectric prosthetic hand, and converts it into a corresponding stimulation feedback paradigm, transmits the vibration feedback signal to the vibration feedback module, and transmits the electrical stimulation feedback signal to the electrical stimulation feedback module, so as to perform stimulation feedback. The structure is simple, the effect is obvious, it is suitable for actual needs, and the application range is wide.

[0078] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements all fall within the protection scope of the claims of the present invention.

Claims

1. A multi-modal perception feedback system based on an electromyographic prosthetic hand, characterized in that, At least including an electromyographic prosthetic hand, a feedback control module, an electrical stimulation feedback module, and a vibration feedback module, The electromyographic prosthetic hand: Force and tactile sensors are provided at the fingers and the palm to obtain force and tactile information of the finger and palm parts; Finger motors with encoders are also provided on the fingers to obtain finger proprioceptive information; Two motors for driving flipping and rotating actions are installed at the wrist, and each of the two motors is provided with an encoder to obtain wrist proprioceptive information, and the wrist proprioceptive information at least includes the rotation and flipping angle information of the wrist; An infrared ranging sensor is installed at the thumb gap of the prosthetic hand to obtain proximity perception information and transmit the information to the feedback control module; The feedback control module: Is respectively connected to the electromyographic prosthetic hand, the electrical stimulation feedback module, and the vibration feedback module, and is used to convert the perception information of the movement of the electromyographic prosthetic hand into a stimulation feedback output signal, and transmit the stimulation feedback output signal to the electrical stimulation feedback module and the vibration feedback module; The perception information at least includes force and tactile perception information, proprioceptive perception information, and proximity perception information; The electrical stimulation feedback module: Inputs a 4-channel microcurrent pulse signal and outputs 4-channel electrical stimulation, and is located on the outer side of the upper arm; The vibration feedback module: Inputs a 9-channel 3.3V PWM signal and outputs 9-channel vibration stimulation, and is located on the inner side of the upper arm.

2. The multimodal perception feedback system based on an electromyographic prosthetic hand according to claim 1, characterized in that: The vibration feedback module includes a vibration motor array, a vibration feedback module flexible printed circuit board, and a vibration feedback module FPC connecting wire. The vibration feedback module FPC connecting wire is used to connect the interface of the vibration feedback module flexible printed circuit board and the feedback control module. The vibration motor array is 9 eccentric vibration motors arranged in a 3×3 grid pattern. The 8 outer motors are located at a position 3 cm away from the edge of the circuit board in the vertical direction of the square flexible printed circuit board and are arranged parallel to the circuit board edge at an interval of 3 cm. The middle 1 motor is located at the center of the circuit board.

3. The multi-modal perception feedback system based on an electromyographic prosthetic hand according to claim 1, wherein: The electrical stimulation feedback module includes electrical stimulation electrodes, an electrical stimulation array flexible printed circuit board, and an electrical stimulation feedback module FPC connecting wire. The electrical stimulation feedback module FPC connecting wire is used to connect the interface of the electrical stimulation array flexible printed circuit board and the feedback control module. The electrical stimulation electrodes are arranged in a cross shape through 12 copper sheets and are installed at the circular pads of the electrical stimulation array flexible printed circuit board.

4. A multimodal perception feedback method for an electromyogram prosthetic hand using the system according to claim 1, characterized in that, Including the following steps: S1: The user controls the electromyographic prosthetic hand to make different actions and interact with environmental objects. The proximity sensor in the electromyographic prosthetic hand detects the distance of the object in front of the palm. The force and tactile sensors sense the pressure information received by the fingers and the palm. The motor encoders obtain the perception information of the hand and wrist movements and transmit the information to the feedback control module; S2: The feedback control module receives the perception information obtained in step S1; S3: The feedback control module converts the perception information into a feedback paradigm and outputs a stimulation feedback output signal; S4: The vibration feedback module and the electrical stimulation feedback module receive the stimulation feedback output signal output in step S3 and perform vibration feedback and electrical stimulation feedback.

5. The multimodal perception feedback method based on an electromyographic prosthetic hand according to claim 4, characterized in that: The process of converting the perception information into a stimulation feedback output signal in step S3 is specifically as follows: S31: After the feedback control module receives the perception information, it determines whether the proximity sense information of the myoelectric prosthetic hand reaches the set feedback threshold. If the threshold is reached, it selects the proximity sense feedback paradigm and calculates the electrical stimulation intensity and frequency to be output according to the magnitude of the proximity sense distance information. If the proximity sense threshold is not reached, it determines whether there is force touch information. If there is force touch information, it selects the force touch feedback paradigm and selects the electrical stimulation intensity and frequency to be output according to the magnitude of the force touch. S32: It determines whether the myoelectric prosthetic hand wrist is moving. If it is moving, it selects the wrist proprioception feedback paradigm and calculates the intensity and frequency of the vibration feedback according to the rotation angle. If there is finger proprioceptive bending, it selects the finger proprioception feedback paradigm and calculates the intensity and frequency of the vibration feedback according to the degree of finger bending.

6. The multimodal perception feedback method based on a myoelectric prosthetic hand according to claim 5, wherein: When the proximity sense feedback paradigm is selected, 8 small electrodes near the center point in the electrical stimulation feedback module form the proximal electrodes, and 4 large electrodes at the 4 ends outside the cross-shaped FPC form the distal electrodes. The proximity sense feedback realizes the internal and external alternating stimulation feeling by alternately activating the proximal electrodes and the distal electrodes. When the proximity sensor senses the approach of an object, the alternating time decreases and the current intensity increases. When the force touch feedback paradigm is selected, the electrical stimulation electrodes in the electrical stimulation feedback module are divided into a column of 8 electrodes in the longitudinal direction and a row of 8 electrodes in the horizontal direction. The longitudinal electrodes and the horizontal electrodes respectively represent the normal force touch and the tangential force touch in the force touch information. The 8 electrodes in the longitudinal or horizontal direction simultaneously perform the start and stop of intermittent current output, and the output current stimulation intensity and the start and stop frequency are both positively correlated with the corresponding force touch magnitude. When the wrist proprioception feedback paradigm is selected, if it is a wrist rotation, the 9 vibration motor arrays in the vibration feedback module are divided into vibration unit columns arranged in three vertical rows. Each vibration unit column will vibrate sequentially in the direction from left to right or from right to left. The larger the rotation angle, the greater the vibration intensity of the vibration motor. If it is a wrist flip, the 9 vibration motor arrays in the vibration feedback module are divided into vibration unit rows arranged in three horizontal rows. Each vibration unit row will vibrate sequentially in the direction from bottom to top or from top to bottom. The larger the flip angle, the greater the vibration intensity of the vibration motor. When the finger proprioception feedback paradigm is selected, the 9 vibration motors in the vibration feedback module are longitudinally divided into three columns of vibration unit columns. The three columns of vibration motors correspond to the thumb, index finger, and the remaining three fingers from left to right. When the bending angle of the corresponding finger is greater than 0°, the vibration motor in this column vibrates continuously. The greater the finger bending angle, the stronger the vibration intensity.

7. The multimodal perception feedback method based on an electromyographic prosthetic hand according to claim 6, characterized in that: When the proximity sense feedback paradigm is selected, the specific numerical calculation method is as follows: Calculate the ratio of the current object distance information to the proximity sense range. The magnitude of the electrical stimulation output current is equal to the maximum current intensity multiplied by the difference between 1 and this ratio. The alternating interval time is equal to the longest interval time multiplied by this ratio. When the force-tactile feedback paradigm is selected, the specific numerical calculation methods are as follows: calculate the ratio of the longitudinal / normal force-tactile magnitude to the maximum range of the force-tactile. The start-stop interval of the electrical stimulation output by the longitudinal / normal electrode is equal to the maximum start-stop time interval multiplied by the difference between 1 and this ratio. The intensity of the output electrical stimulation current is equal to the maximum current value multiplied by this ratio. When the wrist proprioceptive feedback paradigm is selected, the specific numerical calculation methods are as follows: calculate the ratio of the wrist rotation angle to the maximum angle value of 90°. The duty cycle of the PWM signal of the first column of vibration units at the initial moment is equal to 50% of this ratio. After an interval of 100 ms, the first column of vibration units stops vibrating, and the second column starts vibrating, with its PWM signal duty cycle being 75% of this ratio. After an interval of 150 ms, the second column stops vibrating, and the third column starts vibrating, with its PWM signal being equal to this ratio. After an interval of 200 ms, the third column stops vibrating, and the above steps are continuously cycled. When the finger proprioceptive feedback paradigm is selected, the duty cycle ratio of the PWM signal is equal to the ratio of the finger bending angle to 180°.

Images