A duck footed pneumatic glove and a hand mirror image control method thereof

Abstract

The application discloses a duck web type pneumatic glove and a hand mirror image control method thereof, and steps of the method comprise the following steps: 1, initialization of the duck web type pneumatic glove; 2, selection and preparation of an active hand and a passive hand; 3, collection of a finger motion track and a joint bending angle of the active hand by a binocular camera; 4, visualization of the finger motion track of the active hand; 5, three-dimensional modeling and mirror image processing of the finger motion track of the active hand; 6, acquisition of a finger motion track and a fingertip pressure of the passive hand by the duck web type pneumatic glove; 7, comparison of the finger motion tracks of the active hand and the passive hand to obtain actual errors of the two; and 8, control of joint angles of the duck web type pneumatic glove according to the actual errors of the two to achieve mirror image motion tracks of the active hand. The application can more accurately and reliably realize hand mirror image by collecting motion tracks of the active hand and the passive hand by the binocular camera and the duck web type pneumatic glove respectively, and controlling fingers of the duck web type pneumatic glove to spread, open and close.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hand control, in particular to a duck webbing type pneumatic glove and a hand mirror image control method thereof. BACKGROUND

[0002] Traditional hand function robots are mostly mechanical structures, and the wearing comfort is poor. In order to control conveniently, auxiliary force is generally applied to the distal end of the finger, only limited control can be provided to the joints near the distal end of the finger, the movement dimension of the finger is single, the finger movement is lack of parallel movement, and abnormal hand movement forms are prone to occur. The existing pneumatic glove has mechanical design defects. The traditional multi-section air bag adopts a hard support such as a plastic connecting strip, which is easy to cause local compression to the metacarpophalangeal joint and the interphalangeal joint when inflated. At the same time, it also depends on external vacuum pumps and electromagnetic valves for control, and the inflation rate and pressure regulation accuracy are insufficient. The traditional mirror therapy only relies on plane mirror reflection, lacks three-dimensional motion capture and real-time data quantization, can only identify specific gestures for finger movement training, and cannot distinguish the motion difference of the finger joints, resulting in a large deviation of passive hand motion from active hand mirror trajectory. The user passively receives visual stimulation, and the biological signal feedback is not integrated to enhance the motion intention driving. SUMMARY

[0003] The present application is to solve the above-mentioned problems of the prior art, and proposes a duck webbing type pneumatic glove and a hand mirror image control method thereof, so as to control the parallel, opening and closing of the fingers by using a safer glove air bag technology, and effectively identify the finger motion trajectory, thereby improving the hand motion control precision and accuracy.

[0004] In order to achieve the above-mentioned application purposes, the following technical solutions are adopted:

[0005] The duck webbing type pneumatic glove is characterized in that it comprises a glove body, a pressure sensor, an electromyographic sensor, an air bag, a single-chip microcomputer and an electromagnetic valve.

[0006] The pressure sensor and the electromyographic sensor are respectively arranged at the fingertips of the five fingers of the glove body.

[0007] The air bag is arranged between the five fingers and the center of the palm of the glove body, and each air bag is respectively communicated or disconnected with the air source of the vacuum pump through the electromagnetic valve.

[0008] The single-chip microcomputer controls the opening and closing of the electromagnetic valve according to the finger pressure, bending angle and electromyographic signal collected by the pressure sensor and the electromyographic sensor, so as to control the inflation or deflation of the air bag.

[0009] The hand mirror image control method based on the duck webbing type pneumatic glove is characterized in that it is performed according to the following steps:

[0010] Step 1, initialize the joint angles of the duck webbing mirror image inflatable glove, including: the thumb metacarpophalangeal joint of the duck webbing mirror image inflatable glove is straight at one hundred and eighty degrees, the four finger metacarpophalangeal joints are straight at one hundred and eighty degrees, the four finger proximal interphalangeal joints are straight at one hundred and eighty degrees, and the four finger distal interphalangeal joints are straight at one hundred and eighty degrees;

[0011] Step 2, select the active hand and passive hand of the hand mirror image control, and make the passive hand wear the initialized duck webbing pneumatic glove, and the active hand is placed at the recognition position of the binocular camera;

[0012] Step 3, when the active hand performs hand movement, the binocular camera collects image data of the active hand after preprocessing, extracts key point data of the active hand and analyzes it to capture finger movement trajectory and joint bending angle of the active hand in real time;

[0013] Step 4, display the finger movement trajectory and joint bending angle of the active hand through real-time curve and three-dimensional visualization graphics to display the movement state of the active hand in real time;

[0014] Step 5, mirror image processing and three-dimensional modeling are performed on the finger movement trajectory and joint bending angle of the active hand to obtain the mirror image action trajectory of the active hand;

[0015] Step 6, when the passive hand starts to move, the duck webbing pneumatic glove obtains finger pressure, bending angle and electromyographic signal data of the passive hand and performs three-dimensional modeling processing to obtain finger action trajectory and fingertip pressure of the passive hand;

[0016] Step 7, compare the mirror image action trajectory of the active hand and the finger action trajectory of the passive hand to obtain the flexion and extension degree and the finger joint movement error of the two, wherein the finger joint movement error includes angle error and spacing error;

[0017] Step 8, according to the flexion and extension degree and the finger joint movement error of the two, and control the vacuum pump to inflate or deflate the air bag of the duck webbing pneumatic glove, so that the joint angles of the duck webbing pneumatic glove reach the mirror image action trajectory of the active hand, wherein the air bag at the palm controls the flexion or extension of the hand, and the four air bags between the five fingers control the separation or convergence of the five fingers.

[0018] Further, the step 7 includes:

[0019] Step 7.1, mark the finger key points in the active hand mirror image movement trajectory and the passive hand movement trajectory respectively, including: wrist joint, four finger metacarpophalangeal joint, thumb metacarpophalangeal joint, thumb proximal interphalangeal joint, thumb tip, four finger proximal interphalangeal joint, four finger distal interphalangeal joint, four finger tip;

[0020] Step 7.2: Obtain the lines MB between the thumb metacarpophalangeal joint, the four finger metacarpophalangeal joints and the wrist joint in the active hand mirror motion trajectory. ACT,1 , MB ACT,2 , MB ACT,3 , MB ACT,4 , MB ACT,5 PP, the line between the metacarpophalangeal joint of the thumb and the proximal interphalangeal joint of the thumb ACT,1 , and the lines PP between the proximal interphalangeal joints of the four fingers and the corresponding metacarpophalangeal joints of the four fingers ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 , and calculate MB ACT,1 , MB ACT,2 , MB ACT,3 , MB ACT,4 , MB ACT,5 PP ACT,1 PP ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 The angle θ between the fingers ACT,1 ,θ ACT,2 ,θ ACT,3 ,θ ACT,4 ,θ ACT,5 ;

[0021] Obtain the line PP between the thumb metacarpophalangeal joint and the thumb proximal interphalangeal joint in the active hand mirror motion trajectory ACT,1 , and the line DP connecting the proximal interphalangeal joint of the thumb and the tip of the thumb ACT,1 , and calculate PP ACT,1 With DP ACT,1 The angle θ between the fingers ACT，6 ;

[0022] Obtain the lines MP between the proximal interphalangeal joints of the four fingers and the corresponding distal interphalangeal joints of the four fingers in the active hand mirror motion trajectory ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and the lines PP between the proximal interphalangeal joints of the four fingers and the corresponding metacarpophalangeal joints of the four fingers ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 , and calculate MPACT , 2. MPACT , 3. MPACT , 4. MP ACT,5 PP ACT,2 PP ACT,3PP ACT,4 PP ACT,5 The angle θ between the fingers ACT,7 ,θ ACT,8 ,θ ACT,9 ,θ ACT,10 ;

[0023] Obtain the line MP between the proximal interphalangeal joints of the four fingers and the corresponding distal interphalangeal joints of the four fingers in the active hand mirror motion trajectory ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and the lines DP between the distal interphalangeal joints of the four fingers and the fingertips of the corresponding four fingers ACT,2 , DP ACT,3 , DP ACT,4 , DP ACT,5 , and calculate MP ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 DPs that correspond to each other ACT,2 , DP ACT,3 , DP ACT,4 , DP ACT,5 The angle θ between the fingers ACT,11 ,θ ACT,12 ,θ ACT,12 ,θ ACT,14 ;

[0024] Thus, the angle error set θ of the active hand is constructed ACT ={θ ACT,i | i =1,2,3,…,14}, where θACT,i represents the motion angle of the i-th finger of the active hand;

[0025] Step 7.3: Follow the process of step 7.2 to obtain the angular error set θ of the passive hand PAS ={θ PAS,i | i =1,2,3,…,14}, where, θ PAS,i represents the motion angle of the i-th finger of the passive hand;

[0026] Step 7.4: Calculate the motion angle error Δθ of the i-th finger motion angle i =θ ACT,i -θ PAS,i ;

[0027] Step 7.5: Mark the key points of the fingers in the active hand mirror motion trajectory and the passive hand motion trajectory, including the proximal interphalangeal joint of the thumb, the proximal interphalangeal joint of the index finger, the proximal interphalangeal joint of the middle finger, the proximal interphalangeal joint of the ring finger, the proximal interphalangeal joint of the little finger, and the metacarpophalangeal joint of the index finger;

[0028] Step 7.6, obtain the finger gap distance D between the proximal interphalangeal joint of the index finger and the proximal interphalangeal joint of the middle finger in the mirror movement trajectory of the active hand ACT,1 ;

[0029] obtain the finger gap distance D between the proximal interphalangeal joint of the middle finger and the proximal interphalangeal joint of the ring finger in the mirror movement trajectory of the active hand ACT,2 ;

[0030] obtain the finger gap distance D between the proximal interphalangeal joint of the ring finger and the proximal interphalangeal joint of the little finger in the mirror movement trajectory of the active hand ACT,3 ;

[0031] obtain the finger gap distance D between the proximal interphalangeal joint of the thumb and the metacarpophalangeal joint of the index finger in the mirror movement trajectory of the active hand ACT,4 ;

[0032] Thus, the distance error set D of the active hand is constructed ACT ={D ACT,j | j=1,2,3,4}, wherein D ACT,j represents the jth finger gap distance number of the active hand, j=1,2,3,4;

[0033] Step 7.7, obtain the distance error set D of the passive hand according to the process of step 7.6 PAS ={D PAS,j | j=1,2,3,4}, wherein D PAS,j represents the jth finger gap distance number of the passive hand, j=1,2,3,4;

[0034] Step 7.6, calculate the movement distance error ΔD of the jth finger gap j =D ACT,j -D PAS,j .

[0035] Further, in step 8, according to the flexion and extension degrees and the finger joint movement error of the two, the vacuum pump is controlled to inflate or deflate the air bag of the duck web pneumatic glove according to the following process;

[0036] If Δθ i >0, the palm air bag is deflated to make the passive hand flex one step;

[0037] If Δθ i <0, the palm air bag is inflated to make the passive hand further extend,

[0038] If Δθ i =0, the palm air bag is not controlled to maintain the current state of the passive hand;

[0039] If ΔD j> 0, control four air bags between five fingers to inflate;

[0040] If ΔD j < 0, control four air bags between five fingers to deflate;

[0041] If ΔD j = 0, do not control four air bags between five fingers.

[0042] The electronic device comprises a memory and a processor, and the memory is used for storing a program supporting the processor to execute the hand mirror control method, and the processor is configured to execute the program stored in the memory.

[0043] The computer readable storage medium stores a computer program, and the computer program is executed by the processor to execute the steps of the hand mirror control method.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] 1. Traditional hand-controlled robots are mostly mechanical structures, and the wearing comfort is poor. In order to control conveniently, auxiliary force is generally applied to the distal end of the finger, and only limited control can be provided to the joints near the distal end of the finger. Compared with the existing hydraulic rod technology, the present application adopts safer air bag technology, and the fingers are spread, opened and closed by air bag inflation, which expands the motion dimension and reduces the occurrence of passive hand injury.

[0046] 2. Traditional hand mirror only relies on plane mirror reflection, and the user passively accepts visual stimulation, without integrating biological feedback, lacking of quantitative real-time data. The hand mirror of the present application can distinguish the subtle motion differences of finger joints and obtain the motion trajectories of the active hand and the passive hand, form effective comparison and error record; compared with common hand mirror training, the motion dimension is expanded, and the motion efficiency of the fingers is improved by training finger spread, opening and closing. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart of the hand mirror control method based on the duck web type pneumatic glove of the present application;

[0048] Figure 2 The air bag structure diagram of the duck web type pneumatic glove of the present application;

[0049] Figure 3 The air bag control composition structure diagram of the duck web type pneumatic glove of the present application;

[0050] In the figure: 101 air bag; 102 two-way three-way electromagnetic valve; 103 connecting valve; 104 adapter; 105 SMC air pipe; 106 vacuum pump quick connector; 107 vacuum pump; 108 slide rail. DETAILED DESCRIPTION

[0051] In this embodiment, a hand mirror control method based on duck web type pneumatic gloves is to use binocular camera and duck web type pneumatic gloves to collect the motion trajectory of the active hand and the passive hand respectively, and the duck web type pneumatic gloves control the passive hand to realize more accurate and reliable hand mirror. Specifically, the duck web type pneumatic glove is to use air bag to complete the relaxation and finger extension of the fingers, including: connecting mechanism, air bag mechanism, control mechanism and feedback mechanism. The connecting mechanism includes: glove body, magic tape, air bag fixing frame, motor fixing frame, electromagnetic valve fixing frame, etc.; the air bag mechanism is connected with the two-way three-way electromagnetic valve 102 through the SMC air pipe 105 and the air pipe quick connector, the other end of the electromagnetic valve is connected with the vacuum pump 107 through the threaded connector, the vacuum pump is controlled by the control panel, the air bag mechanism includes: soft rubber material printed air bag 101, quick connector 104, control mechanism includes: control panel, single-chip microcomputer, electromagnetic valve 102, etc.; the feedback mechanism includes: pressure sensor, electromyographic sensor, calibration point, etc. Figure 2

[0052] The glove body is composed of knitted gloves, magic tape and air bag fixing frame, and is fixed by knitting. The air bag mechanism is composed of five air bags 101 printed by soft rubber material. The control mechanism is composed of STM32F407 single-chip microcomputer, 24V vacuum pump 107, high-frequency electromagnetic valve and other control devices. The transmission mechanism is composed of SMC air pipe 105, which is connected with the air outlet of the vacuum pump 107 and the air bag 101 through the connecting valve 103 and the pneumatic interface. The feedback mechanism is composed of pressure sensors arranged at the fingertips of the five fingers and electromyographic sensors arranged at the joints of the fingertips of the five fingers. When the glove works, as shown in Figure 3

[0053] In this embodiment, a hand mirror control method of duck web type pneumatic gloves is as shown in Figure 1 , which is performed according to the following steps:

[0054] ​​Step 1, initializing the joint angles of the duck-billed mirror inflatable glove, including: the thumb metacarpophalangeal joint of the duck-billed mirror inflatable glove is straight at one hundred and eighty degrees, the four finger metacarpophalangeal joints are straight at one hundred and eighty degrees, the four finger proximal interphalangeal joints are straight at one hundred and eighty degrees, and the four finger distal interphalangeal joints are straight at one hundred and eighty degrees;

[0055] Step 2, selecting the active hand and passive hand of the hand mirror control, and making the passive hand wear the initialized duck-billed pneumatic glove, and the active hand is placed at the recognition position of the binocular camera;

[0056] Step 3, when the active hand performs hand movement, the binocular camera collects image data of the active hand and performs preprocessing, extracts key point data of the active hand and performs analysis, to capture the finger movement trajectory and joint bending angle of the active hand in real time;

[0057] Step 4, displaying the finger movement trajectory and joint bending angle of the active hand through real-time curve and three-dimensional visualization graphics, to display the movement state of the active hand in real time;

[0058] Step 5, mirror processing and three-dimensional modeling of the finger movement trajectory and joint bending angle of the active hand, to obtain the mirror action trajectory of the active hand;

[0059] Step 5.1, registering the two images collected by the binocular camera to eliminate the geometric difference between the two cameras, so that the images can be aligned;

[0060] Step 5.2, finding corresponding feature points in the two images through SURF feature extraction algorithm, and performing feature point matching;

[0061] Step 5.3, calculating the coordinates of the feature points in the three-dimensional space according to the known camera position, internal parameter and relative position through triangulation method, and the triangulation formula is formula (1):

[0062] (1)

[0063] In formula (1), Z is the depth of the target point (i.e. the distance from the camera). f is the focal length of the camera. B is the baseline distance between the two cameras (i.e. the horizontal distance between the cameras). d is the parallax value, which is the horizontal difference of corresponding points in the two images. These three-dimensional coordinates represent the spatial positions of the key parts of the active hand;

[0064] Step 5.4, calculating the depth information of each pixel according to the parallax value d in formula (1) to generate the depth map of the active hand and further construct the three-dimensional hand model;

[0065] Step 6, when the passive hand starts to move, the duck webbing pneumatic glove acquires the passive hand's finger pressure, bending angle and electromyographic signal data and performs three-dimensional modeling processing to obtain the passive hand's finger motion trajectory and fingertip pressure;

[0066] Step 6.1, the bending angle θ, air bag pressure P and electromyographic signal EMG of the user's fingers are synchronously collected by the flexible pressure sensor array (sampling rate ≥ 100 Hz) and surface electromyographic sensor (sEMG, bandwidth 20-500 Hz) built in the passive hand glove; Step 6.2, edge computing is performed by using an STM32F407 single-chip microcomputer, the original data are subjected to Kalman filtering denoising, and a pressure-angle mapping relationship is established by polynomial fitting:

[0067] (2)

[0068] In formula (2), θ represents the bending angle of the finger joint (unit: °), which is calculated by fusing the flexible pressure sensor array and the electromyographic signal. P is the internal pressure of the air bag (unit: kPa), which is measured by the flexible pressure sensor in real time, and the range covers 0-500 kPa; is a polynomial coefficient, which is determined by least square fitting, and represents the relationship between the nonlinear deformation characteristics of the air bag and the joint motion;

[0069] Step 6.3, the data processed by formula (2) are packaged into Protocol Buffers format, and uploaded to the host computer at a rate of 1 Mbps through a USB-CDC virtual serial port;

[0070] Step 6.4, the host computer calls the dynamic time warping (DTW) algorithm to match the current motion mode with the pre-stored template library, and generates PID control parameters:

[0071] (3)

[0072] (4)

[0073] In formula (3), formula (4), represents the control output, which is used to adjust the action strength of the actuator (such as an air pump or an electromagnetic valve); respectively represent the proportional, integral and differential coefficients, which are generated by matching from the pre-stored template library through the dynamic time warping (DTW) algorithm; is used to control the error, which is defined as the deviation between the target value and the actual value, i.e. , wherein, is the set joint target angle, is the actual angle fed back by the sensor; is calculated from formula (5) and serves as the initial reference pressure.

[0074] Step 6.5: Drive the high-speed solenoid valve (response time ≤ 10ms) through PWM modulation to achieve closed-loop control of the airbag pressure.

[0075] Step 7: Compare the mirror motion trajectory of the active hand with the finger motion trajectory of the passive hand to obtain the flexion and extension degrees of the two and the finger joint motion error, wherein the finger joint motion error includes angle error and spacing error;

[0076] Step 7.1. Mark the key points of the fingers in the active hand mirror motion trajectory and the passive hand motion trajectory, including the wrist joint, the metacarpophalangeal joints of the four fingers, the metacarpophalangeal joints of the thumb, the proximal interphalangeal joints of the thumb, the tip of the thumb, the proximal interphalangeal joints of the four fingers, the distal interphalangeal joints of the four fingers, and the fingertips of the four fingers;

[0077] Step 7.2: Obtain the lines MB between the thumb metacarpophalangeal joint, the four finger metacarpophalangeal joints and the wrist joint in the active hand mirror motion trajectory. ACT,1 , MB ACT,2 , MB ACT,3 , MB ACT,4 , MB ACT,5 PP, the line between the metacarpophalangeal joint of the thumb and the proximal interphalangeal joint of the thumb ACT,1 , and the lines PP between the proximal interphalangeal joints of the four fingers and the corresponding metacarpophalangeal joints of the four fingers ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 , and calculate MB ACT,1 , MB ACT,2 , MB ACT,3 , MB ACT,4 , MB ACT,5 PP ACT,1 PP ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 The angle θ between the fingers ACT,1 ,θ ACT,2 ,θ ACT,3 ,θ ACT,4 ,θ ACT,5 ;

[0078] Obtain the line PP between the thumb metacarpophalangeal joint and the thumb proximal interphalangeal joint in the active hand mirror motion trajectory ACT,1 , and the line DP connecting the proximal interphalangeal joint of the thumb and the tip of the thumb ACT,1 , and calculate PP ACT,1 With DP ACT,1 The angle θ between the fingers ACT，6 ;

[0079] MP ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and PP ACT,2 , PP ACT,3 , PP ACT,4 , PP ACT,5 , and calculate the finger motion angle θ , 2, θ , 3, θ , 4, θ ACT,5 between MP ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and the one-to-one corresponding PP ACT,7 , PP ACT,8 , PP ACT,9 , PP ACT,10 ;

[0080] MP ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and DP ACT,2 , DP ACT,3 , DP ACT,4 , DP ACT,5 , and calculate the finger motion angle θ ACT,2 2, θ ACT,3 3, θ ACT,4 4, θ ACT,5 between MP ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and the one-to-one corresponding DP ACT,11 , DP ACT,12 , DP ACT,12 , DP ACT,14 ;

[0081] Thus, the angle error set θ ACT ={θ ACT,i | i =1,2,3,…,14} of the active hand is constructed, wherein θACT,i represents the i-th finger motion angle of the active hand;

[0082] Step 7.3, the angle error set θPAS ={θ PAS,i | i =1,2,3,…,14}, where, θ PAS,i represents the motion angle of the i-th finger of the passive hand;

[0083] Step 7.4: Calculate the motion angle error Δθ of the i-th finger motion angle i =θ ACT,i -θ PAS,i ;

[0084] Step 7.5: Mark the key points of the fingers in the active hand mirror motion trajectory and the passive hand motion trajectory, including the proximal interphalangeal joint of the thumb, the proximal interphalangeal joint of the index finger, the proximal interphalangeal joint of the middle finger, the proximal interphalangeal joint of the ring finger, the proximal interphalangeal joint of the little finger, and the metacarpophalangeal joint of the index finger;

[0085] Step 7.6: Obtain the interphalangeal distance D between the proximal interphalangeal joint of the index finger and the proximal interphalangeal joint of the middle finger in the active hand mirror motion trajectory. ACT,1 ;

[0086] Get the interphalangeal distance D between the proximal interphalangeal joint of the middle finger and the proximal interphalangeal joint of the ring finger in the active hand mirror motion trajectory ACT,2 ;

[0087] Get the interphalangeal distance D between the proximal interphalangeal joint of the ring finger and the proximal interphalangeal joint of the little finger in the active hand mirror motion trajectory ACT,3 ;

[0088] Get the interphalangeal distance D between the proximal interphalangeal joint of the thumb and the metacarpophalangeal joint of the index finger in the active hand mirror motion trajectory ACT,4 ;

[0089] Thus construct the spacing error set D of the active hand ACT ={D ACT,j | j=1,2,3,4}, where D ACT,j Indicates the j-th interdigit spacing number of the active hand, j=1,2,3,4;

[0090] Step 7.7: Follow the process of step 7.6 to obtain the distance error set D of the passive hand. PAS ={D PAS,j | j=1,2,3,4}, where D PAS,j Indicates the j-th inter-finger spacing number of the passive hand, j=1,2,3,4;

[0091] Step 7.6: Calculate the motion distance error ΔD of the j-th finger gap j =D ACT,j -D PAS,j ;

[0092] Step 8. According to the flexion and extension degrees of both and the finger joint motion error, control the vacuum pump to inflate or deflate the airbags of the duck-billed pneumatic glove, so that the joint angles of the duck-billed pneumatic glove reach the mirror action trajectory of the active hand, wherein the palm airbag controls the flexion or extension of the hand, and the four airbags between the fingers control the separation or convergence of the five fingers.

[0093] If Δθ i > 0, control the palm airbag to deflate, so that the passive hand flexes one step;

[0094] If Δθ i < 0, control the palm airbag to inflate, so that the passive hand further extends,

[0095] If Δθ i = 0, do not control the palm airbag to maintain the current state of the passive hand;

[0096] If ΔD j > 0, control the four airbags between the fingers to inflate;

[0097] If ΔD j < 0, control the four airbags between the fingers to deflate;

[0098] If ΔD j = 0, keep the four airbags between the fingers unchanged.

[0099] In specific implementation, first set the target joint angle parameter and the maximum airbag pressure threshold ; Then generate the pneumatic control sequence:

[0100] (5)

[0101] (6)

[0102] In formula (5), formula (6), represents the initial inflation pressure (unit: kPa); is the set maximum safety pressure threshold (unit: kPa), the default range is 50-400 kPa; is the joint physiological limit angle (unit: °), which is preset according to general ergonomics parameters (such as metacarpophalangeal joint 90°, interphalangeal joint 120°); is the set target joint angle (unit: °), which needs to satisfy 0≤ ≤ ; represents the inflation slope constraint, which is the inflation process time constant (unit: second), which controls the airbag pressure rising rate; is the maximum allowable joint angle rate of change, based on biomechanical prior experiment setting;

[0103] The control instructions are sent to the STM32F407 single-chip microcomputer through the RS485 bus to drive the 24V vacuum pump (flow ≥ 15L / min) and the proportional electromagnetic valve (linearity ± 1% FS) to perform the staged inflation.

[0104] When the real-time pressure The safety relief valve is triggered to prevent overstretching of the user's tendon.

[0105] In this embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the control method described above, and the processor is configured to execute the program stored in the memory.

[0106] In this embodiment, a computer readable storage medium has a computer program stored thereon, and the computer program is run by a processor to execute the steps of the control method described above.

Claims

1. A hand mirror control method based on duck-webbed pneumatic gloves, characterized in that: The duck-web pneumatic glove includes: glove body, pressure sensor, myoelectric sensor, airbag, single chip microcomputer, solenoid valve; Pressure sensors and myoelectric sensors are respectively provided on the fingertips of the five fingers of the glove body; Airbags are provided between the five fingers of the glove body and at the center of the palm, and each airbag is connected or disconnected with the air source of the vacuum pump through a solenoid valve; The single chip microcomputer controls the opening and closing of the solenoid valve according to the finger pressure, bending angle and electromyographic signals collected by the pressure sensor and the electromyographic sensor to control the inflation or deflation of the airbag. The hand mirror control method is carried out according to the following steps: Step 1, initializing the joint angles of the duck-webbed pneumatic glove, including: the metacarpophalangeal joint of the thumb of the duck-webbed pneumatic glove is extended at an angle of 180 degrees, the metacarpophalangeal joints of the four fingers are extended at an angle of 180 degrees, the proximal interphalangeal joints of the four fingers are extended at an angle of 180 degrees, and the distal interphalangeal joints of the four fingers are extended at an angle of 180 degrees; Step 2: Select the active and passive hands for hand mirror control, and make the passive hand wear the initialized duck-web pneumatic glove. Place the active hand at the recognition position where the binocular camera is located. Step 3: When the active hand performs hand movements, the binocular camera collects and pre-processes the image data of the active hand, extracts and analyzes the key point data of the active hand to capture the finger movement trajectory and joint bending angle of the active hand in real time; Step 4: Display the finger motion trajectory and joint bending angle of the active hand through real-time curve graphs and three-dimensional visualization graphics to display the motion state of the active hand in real time; Step 5: Mirror the finger motion trajectory and joint bending angle of the active hand and perform three-dimensional modeling to obtain the mirror motion trajectory of the active hand; Step 6: When the passive hand starts to move, the duck-web pneumatic glove obtains the finger pressure, bending angle, and electromyographic signal data of the passive hand and performs three-dimensional modeling processing to obtain the passive hand's finger movement trajectory and fingertip pressure; Step 7: Compare the mirror motion trajectory of the active hand with the finger motion trajectory of the passive hand to obtain the flexion and extension degrees of the two and the finger joint motion error, wherein the finger joint motion error includes angle error and spacing error; Step 8: Based on the flexion and extension degrees of the two hands and the kinematic errors of the finger joints, the vacuum pump is controlled to inflate or deflate the airbags of the duck-webbed pneumatic glove, so that the joint angles of the duck-webbed pneumatic glove reach the mirror image motion trajectory of the active hand. The airbag in the palm controls the flexion or extension of the hand, and the four airbags between the five fingers simultaneously control the separation or closing of the five fingers.

2. The hand mirror control method based on duck-webbed pneumatic gloves according to claim 1, characterized in that: The step 7 includes: Step 7.

1. Mark the key points of the fingers in the active hand mirror motion trajectory and the passive hand motion trajectory, including the wrist joint, the metacarpophalangeal joints of the four fingers, the metacarpophalangeal joints of the thumb, the proximal interphalangeal joints of the thumb, the tip of the thumb, the proximal interphalangeal joints of the four fingers, the distal interphalangeal joints of the four fingers, and the fingertips of the four fingers; Step 7.2: Obtain the lines MB between the thumb metacarpophalangeal joint, the four finger metacarpophalangeal joints and the wrist joint in the active hand mirror motion trajectory. ACT,1 , MB ACT,2 , MB ACT,3 , MB ACT,4 , MB ACT,5 PP, the line between the metacarpophalangeal joint of the thumb and the proximal interphalangeal joint of the thumb ACT,1 , and the lines PP between the proximal interphalangeal joints of the four fingers and the corresponding metacarpophalangeal joints of the four fingers ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 , and calculate MB ACT,1 , MB ACT,2 , MB ACT,3 , MB ACT,4 , MB ACT,5 PP ACT,1 PP ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 The angle θ between the fingers ACT,1 ,θ ACT,2 ,θ ACT,3 ,θ ACT,4 ,θ ACT,5 ; Obtain the line PP between the thumb metacarpophalangeal joint and the thumb proximal interphalangeal joint in the active hand mirror motion trajectory ACT,1 , and the line DP connecting the proximal interphalangeal joint of the thumb and the tip of the thumb ACT,1 , and calculate PP ACT,1 With DP ACT,1 The angle θ between the fingers ACT,6 ; Obtain the lines MP between the proximal interphalangeal joints of the four fingers and the corresponding distal interphalangeal joints of the four fingers in the active hand mirror motion trajectory ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and the lines PP between the proximal interphalangeal joints of the four fingers and the corresponding metacarpophalangeal joints of the four fingers ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 , and calculate MPACT , 2. MPACT , 3. MPACT , 4. MP ACT,5 PP ACT,2 PP ACT,3 PP ACT,4 PP ACT,5 The angle θ between the fingers ACT,7 ,θ ACT,8 ,θ ACT,9 ,θ ACT,10 ; Obtain the line MP between the proximal interphalangeal joints of the four fingers and the corresponding distal interphalangeal joints of the four fingers in the active hand mirror motion trajectory ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 and the lines DP between the distal interphalangeal joints of the four fingers and the fingertips of the corresponding four fingers ACT,2 , DP ACT,3 , DP ACT,4 , DP ACT,5 , and calculate MP ACT,2 , MP ACT,3 , MP ACT,4 , MP ACT,5 DPs corresponding to each other ACT,2 , DP ACT,3 , DP ACT,4 , DP ACT,5 The angle θ between the fingers ACT,11 ,θ ACT,12 ,θ ACT,12 ,θ ACT,14 ; Thus construct the angle error set θ of the active hand ACT ={θ ACT,i | i =1,2,3,…,14}, where θACT,i represents the motion angle of the i-th finger of the active hand; Step 7.3: Follow the process of step 7.2 to obtain the angular error set θ of the passive hand PAS ={θ PAS,i | i =1,2,3,…,14}, where, θ PAS,i represents the motion angle of the i-th finger of the passive hand; Step 7.4: Calculate the motion angle error Δθ of the i-th finger motion angle i =θ ACT,i -θ PAS,i ; Step 7.5: Mark the key points of the fingers in the active hand mirror motion trajectory and the passive hand motion trajectory, including the proximal interphalangeal joint of the thumb, the proximal interphalangeal joint of the index finger, the proximal interphalangeal joint of the middle finger, the proximal interphalangeal joint of the ring finger, the proximal interphalangeal joint of the little finger, and the metacarpophalangeal joint of the index finger; Step 7.6: Obtain the interphalangeal distance D between the proximal interphalangeal joint of the index finger and the proximal interphalangeal joint of the middle finger in the active hand mirror motion trajectory. ACT,1 ; Get the interphalangeal distance D between the proximal interphalangeal joint of the middle finger and the proximal interphalangeal joint of the ring finger in the active hand mirror motion trajectory ACT,2 ; Get the interphalangeal distance D between the proximal interphalangeal joint of the ring finger and the proximal interphalangeal joint of the little finger in the active hand mirror motion trajectory ACT,3 ; Get the interphalangeal distance D between the thumb proximal interphalangeal joint and the index finger metacarpophalangeal joint in the active hand mirror motion trajectory ACT,4 ; Thus construct the spacing error set D of the active hand ACT ={D ACT,j | j=1,2,3,4}, where D ACT,j Indicates the j-th interdigit spacing number of the active hand, j=1,2,3,4; Step 7.7: Follow the process of step 7.6 to obtain the distance error set D of the passive hand. PAS ={D PAS,j | j=1,2,3,4}, where D PAS,j Indicates the j-th inter-finger spacing number of the passive hand, j=1,2,3,4; Step 7.6: Calculate the motion distance error ΔD of the j-th finger gap j =D ACT,j -D PAS,j .

3. The hand mirror control method based on duck-webbed pneumatic gloves according to claim 2, characterized in that: In step 8, the vacuum pump is controlled to inflate or deflate the airbag of the duck-webbed pneumatic glove according to the following process based on the flexion and extension degrees of the two and the kinematic error of the finger joints; If Δθ i >0, the palm airbag is controlled to deflate, causing the passive hand to flex in one step; If Δθ i <0, the palm airbag is controlled to inflate, causing the passive hand to stretch further. If Δθ i =0, the palm airbag is not controlled to maintain the current state of the passive hand; If ΔD j >0, it controls the inflation of the four air bags between the five fingers; If ΔD j <0, the four air bags between the five fingers will be deflated; If ΔD j =0, the four air bags between the five fingers are not controlled.

4. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the hand mirror control method according to any one of claims 1 to 3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hand mirror control method according to any one of claims 1 to 3 are executed.

Images