Hand electronic skin based on ultrasonic transducer and three-dimensional gesture reconstruction method thereof

Abstract

The application discloses a hand electronic skin based on an ultrasonic transducer and a three-dimensional gesture reconstruction method thereof, and relates to the technical field of human-computer interaction.The hand electronic skin comprises a sticky layer arranged on the surface of hand skin, a flexible strain sensor layer, flexible electrodes, a composite electrode, a surface packaging layer and an ultrasonic transducer.The flexible strain sensor layer comprises a plurality of flexible strain sensors, and each finger joint is provided with a flexible strain sensor for independently measuring the bending angle thereof.The ultrasonic transducer is partially embedded in the surface packaging layer and is used for emitting ultrasonic waves.The electronic skin is mainly composed of flexible stretchable strain sensors and piezoelectric micromechanical ultrasonic transducers, has a simple and reliable structure, low manufacturing cost, is not disturbed by environmental light, has low time drift rate, and is convenient and comfortable to wear.The hand electronic skin is suitable for virtual reality, robot control, medical rehabilitation and other fields, and can significantly improve the interactive experience and operation convenience of users.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of human-computer interaction, and relates to the fields of flexible sensor technology and wearable devices, in particular to a hand electronic skin based on an ultrasonic transducer and a three-dimensional gesture reconstruction method thereof. BACKGROUND

[0002] In recent years, with the rapid development of virtual reality (VR), augmented reality (AR) and human-computer interaction technology, hand motion capture and three-dimensional reconstruction technology have attracted more and more attention. These technologies have a wide range of applications in medical rehabilitation, robot control, game entertainment and education. However, the existing hand reconstruction technology has some shortcomings. Traditional hand tracking methods mainly rely on optical cameras. Although this method can achieve high-precision gesture reconstruction, it has high cost, complex installation and is easily affected by environmental light, which limits its application in practice. In addition, although the data glove-based solution can provide better flexibility, it often leads to insufficient accuracy and real-time performance of the data due to the inconvenience of wearing and the drift phenomenon of inertial measurement units (IMU).

[0003] In order to overcome these problems, some emerging sensor technologies have been introduced into the field of hand reconstruction in recent years. For example, hand electronic skin based on stretchable sensors can achieve real-time monitoring of hand movements while maintaining flexibility and comfort. However, the existing hand electronic skin cannot provide detailed angle of each finger joint and accurate reconstruction of overall hand position and inclination, which limits its application in complex scenarios. In addition, the commonly used hand electronic skin based on PDMS usually has small adhesion and a Young's modulus greater than 1 MPa, which causes large internal stress in the electronic skin during finger bending, making it easy to fall off the hand. Although the hand electronic skin based on hydrogel has relatively large adhesion, the flexible strain sensor based on hydrogel substrate is complex to manufacture, and its physical and chemical properties are not as stable as PDMS, so its sensing performance is unstable, with large drift over time and low precision.

[0004] Therefore, there is an urgent need for a new hand reconstruction solution that can integrate multiple sensors to detect the motion state of the fingers in real time and accurately obtain the three-dimensional position and inclination of the hand, thereby improving the accuracy and convenience of human-computer interaction. The present application aims to solve the above problems and proposes a hand electronic skin based on flexible stretchable strain sensors and MEMS ultrasonic transducers, which can achieve high-precision three-dimensional reconstruction of the hand. The hand electronic skin has the advantages of low cost, simple manufacturing, stable and reliable performance, and is not affected by environmental magnetic field and light, etc. It has high application value and wide market potential in the fields of human-computer interaction, medical rehabilitation and robot control. SUMMARY

[0005] The present application aims to overcome the defects in the prior art and provide a hand electronic skin based on an ultrasonic transducer and a three-dimensional gesture reconstruction method thereof.The three-dimensional gesture reconstruction method has the advantages of low cost, high stability, simple production, strong environmental anti-interference ability and natural user experience.The present application provides a hand electronic skin manufacturing process based on PDMS, which adjusts the mass ratio of PDMS, PDMS curing agent and polyethylene glycol oligomer, and the curing temperature and time, to control the cross-linking degree of the cured PDMS, and further control the Young's modulus and surface adhesion of the electronic skin.The present application integrates a multi-channel PDMS-based flexible strain sensor and a MEMS ultrasonic transducer chip on the above-mentioned electronic skin, which can realize high-precision three-dimensional hand reconstruction while the user's hand is moving, and significantly improve the accuracy and convenience of human-computer interaction.

[0006] The specific technical solutions adopted by the present application are as follows:

[0007] In a first aspect, the present application provides a hand electronic skin based on an ultrasonic transducer, comprising an adhesive layer, a flexible strain sensor layer, a flexible electrode, a composite electrode, a surface packaging layer and an ultrasonic transducer arranged on the surface of the skin; the adhesive layer, the flexible strain sensor layer, the flexible electrode, the composite electrode and the surface packaging layer are sequentially stacked from bottom to top, and chemical cross-linking is formed between adjacent layers; the flexible strain sensor layer comprises a plurality of flexible strain sensors, each finger joint is provided with a flexible strain sensor for independently measuring the bending angle thereof; the ultrasonic transducer is partially embedded in the surface packaging layer and used for emitting ultrasonic waves.

[0008] Preferably, the ultrasonic transducer comprises a packaging shell and a piezoelectric micromechanical ultrasonic transducer; the piezoelectric micromechanical ultrasonic transducer is partially embedded in the surface packaging layer and is sealed as a whole by the packaging shell.

[0009] Further, the piezoelectric micromechanical ultrasonic transducer has a working frequency range of 40-300 kHz, comprises a cavity base, an insulating layer, a structure layer and a seed layer which are sequentially stacked from bottom to top, a plurality of transducer units arranged in a rectangular array are arranged on the upper surface of the seed layer, and all the transducer units are in electrical parallel connection; the transducer unit comprises a bottom electrode, a piezoelectric layer and a top electrode which are sequentially stacked from bottom to top.

[0010] Preferably, the base material of at least one of the adhesive layer, the composite electrode and the surface packaging layer is at least partially the same as the base material of the flexible strain sensor and is polydimethylsiloxane.

[0011] Preferably, the raw materials of the adhesive layer and the flexible strain sensor include polydimethylsiloxane, polydimethylsiloxane curing agent and polyethylene glycol oligomer; the molecular weight of the polyethylene glycol oligomer ranges from 200 to 600, the mass ratio of the polyethylene glycol oligomer to the polydimethylsiloxane is 1:1, and the mass ratio of the polydimethylsiloxane curing agent to the polydimethylsiloxane is 1:1.

[0012] Further, the preparation method of the adhesive layer is specifically as follows:

[0013] S1: uniformly mix the polydimethylsiloxane and the polyethylene glycol oligomer according to the proportion;

[0014] S2: add the polydimethylsiloxane curing agent to the mixture obtained in S1 and uniformly mix;

[0015] S3: pour the mixture obtained in S2 into a mold or use a spin coating method to control the thickness;

[0016] S4: cure the product obtained in S3 in an environment with a temperature of 50-70°C, and the curing time is not more than 2 hours.

[0017] Preferably, the raw materials of the flexible strain sensor and the composite electrode include at least one of carbon nanotubes, carbon powder and graphene, and the resistance of the flexible strain sensor increases with the increase of strain.

[0018] Preferably, the hand electronic skin satisfies the following characteristics:

[0019] The adhesive force of the adhesive layer is 0.5-1.2 N, and the Young's modulus is 100-400 kPa;

[0020] The total thickness of the hand electronic skin is 0.3-1 mm, and the elongation at break is 60%-200%.

[0021] In the second aspect, the application provides a use of the hand electronic skin based on the ultrasonic transducer as described in any one of the first aspect for fixing on the back of the hand for virtual reality interaction, robot control or medical rehabilitation.

[0022] In the third aspect, the application provides a three-dimensional gesture reconstruction method of the hand electronic skin based on the ultrasonic transducer as described in any one of the first aspect, which is specifically as follows:

[0023] By measuring the resistance values of the flexible strain sensors in real time, the resistance value, the resistance change rate at the current time and the resistance value, the angle value and the resistance change rate of the previous several frames are taken as the input of the neural network model, and the output is the angle value at the current time, and then the bending angles of the finger joints are obtained.

[0024] The ultrasonic transducer emits ultrasonic waves, more than three ultrasonic wave receivers outside the hand electronic skin are fixed to receive ultrasonic signals and convert them into electric signals; the ultrasonic signals are processed and extracted to obtain the time-of-flight difference and / or the voltage amplitude and / or the energy amplitude; the distance of the palm reaching a specific position is calculated according to the time-of-flight difference, so as to determine the spatial position of the palm; the inclination angle of the palm relative to the external ultrasonic receiver is calculated according to the voltage amplitude or the energy amplitude because the signal amplitude attenuates differently at different angles.

[0025] The finger joint bending angle calculated by each flexible strain sensor and the palm position and inclination angle data calculated by the ultrasonic transducer are integrated to generate a three-dimensional reconstruction model of the hand in the upper computer; the model can dynamically reflect the posture and position changes of the fingers and the palm in real time, so as to realize accurate three-dimensional gesture reconstruction.

[0026] Compared with the prior art, the present application has the following beneficial effects:

[0027] The structure design of the present application is simple, the manufacturing process cost is low, and it is convenient for large-scale production and popularization and application, and has a wide market prospect.

[0028] 1) Compared with other hand motion monitoring methods, the three-dimensional gesture reconstruction method based on the hand electronic skin proposed by the present application has the characteristics of three-dimensional gesture reconstruction without calibration. This not only simplifies the operation process, but also enhances the universality of the method, making long-time continuous monitoring possible. In addition, the method does not depend on complex circuit design and signal processing algorithm, thereby reducing the computational burden of the system, reducing the requirements of software and hardware, and improving the overall efficiency.

[0029] 2) Compared with other electronic skins, the polydimethylsiloxane (PDMS) based hand electronic skin manufactured by the screen printing process adopted by the present application has significant advantages in cost and production efficiency, while maintaining the stability of long-time wearing and can be repeatedly worn. By adding and adjusting the molecular weight, mass ratio of polyethylene glycol and mass ratio of PDMS curing agent, a high-viscosity and high-softness hand electronic skin is obtained. The electronic skin is comfortable and adaptive when worn, can effectively prevent falling off caused by hand movement, and is suitable for various human-computer interaction scenes.

[0030] 3) The manufacturing process of this electronic skin is simple, has good repeatability and biocompatibility, and the double-layer electrode design further improves the yield of the multi-channel electronic skin. These characteristics make the electronic skin of the present application not only suitable for entertainment in a home environment, but also suitable for medical rehabilitation in a clinical environment. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1is a physical diagram of the hand electronic skin of the present application;

[0032] Figure 2 is an exploded diagram of the structure of each layer of the hand electronic skin device of the present application;

[0033] Figure 3 is a structural schematic diagram of the piezoelectric micromechanical ultrasonic transducer of the present application;

[0034] Figure 4 is an exploded diagram of the structure of each layer of the piezoelectric micromechanical ultrasonic transducer of the present application;

[0035] Figure 5 is a principle diagram of the hand electronic skin calculating the three-dimensional position and inclination angle of the present application;

[0036] Figure 6 is a flow chart of the preparation process of the hand electronic skin of the present application;

[0037] Figure 7 is a response diagram of the relative resistance change of the flexible strain sensor of the present application with time under different bending frequencies of the finger;

[0038] Figure 8 is a response diagram of the relative resistance change of the flexible strain sensor of the present application with time under different bending angles of the finger;

[0039] In the drawings, the reference signs are: hand skin 1, hand electronic skin 2, adhesive layer 21, flexible strain sensor layer 22, flexible electrode 23, composite electrode 24, surface encapsulation layer 25, encapsulation shell 26, piezoelectric micromechanical ultrasonic transducer 27, top electrode 31, piezoelectric layer 32, bottom electrode 33, seed layer 34, structure layer 35, insulating layer 36, cavity base 37, first ultrasonic receiver 51, second ultrasonic receiver 52, third ultrasonic receiver 53; conductive material 601, organic solvent 602, flexible strain sensor substrate 603, polydimethylsiloxane curing agent 604, transfer table 605, adhesive layer raw material 606, first flexible electrode raw material 607, second flexible electrode raw material 608, surface encapsulation layer raw material 609, flexible circuit board 610, ultrasonic dispersion S61, magnetic stirring S62, heating stirring S63, screen printing flexible sensor layer S64, heating and curing flexible sensor layer S65, heating and curing adhesive layer S66, heating and curing flexible electrode S67, making composite electrode S68, making surface encapsulation layer S69. DETAILED DESCRIPTION

[0040] The present application will be further described and explained with reference to the accompanying drawings and specific embodiments. The technical features of each embodiment of the present application can be combined accordingly without conflict.

[0041] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0042] The terms "first" and "second" are used for descriptive purposes only, to distinguish objects, such as substances, from one another, and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. For example, without departing from the scope of the embodiments of this application, the first XX may also be referred to as the second XX, and similarly, the second XX may also be referred to as the first XX.

[0043] like Figure 1 and 2 As shown, this invention provides an electronic hand skin based on an ultrasonic transducer. The electronic hand skin mainly includes an adhesive layer 21, a flexible strain sensor layer 22, a flexible electrode 23, a composite electrode 24, a surface encapsulation layer 25, and an ultrasonic transducer, all disposed on the surface of the hand skin 1. The adhesive layer 21, flexible strain sensor layer 22, flexible electrode 23, composite electrode 24, and surface encapsulation layer 25 are stacked sequentially from bottom to top, forming a sandwich structure. Adjacent layers are chemically cross-linked to form a stable connection, improving overall reliability and durability, and preventing layer separation under normal conditions.

[0044] In a preferred embodiment of the present invention, the substrate of at least one of the adhesive layer 21, the composite electrode 24, and the surface encapsulation layer 25 is at least partially the same as the substrate of the flexible strain sensor and is polydimethylsiloxane.

[0045] As a preferred embodiment of the present invention, the adhesive layer 21 is made of polydimethylsiloxane (PDMS) material, which not only has excellent flexibility and appropriate adhesion, but also maintains stability under different environmental conditions, ensuring that irreversible deformation does not easily occur during long-term wear.

[0046] Further, the raw material of the adhesive layer 21 also includes a polydimethylsiloxane curing agent and a polyethylene glycol oligomer. Specifically, the present application wraps part of the PDMS cured platinum catalyst with the polyethylene glycol oligomer, thereby interfering with the degree of PDMS cross-linking reaction, to obtain a PDMS / polyethylene glycol mixed elastomer that takes into account both high surface adhesion and low Young's modulus. As a preferred, the molecular weight range of the polyethylene glycol oligomer is 200-600, the mass ratio of the polyethylene glycol oligomer to the polydimethylsiloxane is 1:19-99, and the mass ratio of the polydimethylsiloxane curing agent to the polydimethylsiloxane is 1:11-15. At this time, the viscosity of the elastomer can satisfy the styling contact with the skin during hand movement, and since the skin strain will exceed 60% when the fingers are bent, if the degree of PDMS cross-linking is further reduced, the elastomer can be prone to breakage.

[0047] The preparation method of the adhesive layer 21 is specifically as follows:

[0048] S1: uniformly mix the polydimethylsiloxane and the polyethylene glycol oligomer in the above proportions;

[0049] S2: add the polydimethylsiloxane curing agent to the mixture obtained in S1 in the above proportions and mix uniformly;

[0050] S3: pour the mixture obtained in S2 into a mold or use a spin coating method to control the thickness;

[0051] S4: cure the product obtained in S3 in an environment of 50-70°C, and the curing time is not more than 2 hours. Preferably, the curing temperature of the PDMS and the polyethylene glycol elastomer is 60°C, and the time is 2 hours, at which time the cured elastomer is sticky and not prone to breakage.

[0052] As a preferred embodiment of the present application, the flexible electrode raw material uses stretchable silver paste, and the composite electrode uses the same raw material as the flexible strain sensor, to increase the bonding force between the composite electrode and the flexible strain sensor layer, and to prevent the occurrence of broken contact.

[0053] In the present application, the flexible strain sensor layer 22 includes a plurality of flexible strain sensors, and a flexible strain sensor is arranged at each finger joint. The flexible strain sensor is used to independently measure the bending angle of the finger at the location. Since the present application is used to reconstruct gestures with high precision, each finger joint corresponds to a flexible strain sensor, and therefore the hand electronic skin has a large number of channels. The present application uses two kinds of electrode materials as a conductive network, to increase the signal stability of the electronic skin, especially the electrical stability under large deformation of the fingers.

[0054] As a preferred embodiment of the present application, the material of the flexible strain sensor can adopt at least one of carbon nanotubes, carbon powder and graphene, so as to ensure that the resistance changes sensitively when deformed (the resistance increases with the increase of strain), thereby realizing accurate monitoring of finger actions, and meanwhile, the nanoscale carbon material has stable physical and chemical properties, wear resistance and corrosion resistance, so as to improve the stability and accuracy of sensor measurement.

[0055] In the present application, the ultrasonic transducer is partially embedded in the surface packaging layer 25 for emitting ultrasonic waves.

[0056] As a preferred embodiment of the present application, the ultrasonic transducer mainly comprises a packaging shell 26 and a piezoelectric micromechanical ultrasonic transducer 27. The piezoelectric micromechanical ultrasonic transducer 27 is partially embedded in the upper surface of the surface packaging layer 25, and the whole piezoelectric micromechanical ultrasonic transducer 27 is sealed through the packaging shell 26. The working frequency range of the piezoelectric micromechanical ultrasonic transducer 27 is 40-300 kHz. If the working frequency decreases, the distance resolution is low; if the working frequency increases, the ultrasonic propagation distance is short. For example, Figure 3 and 4 As shown in the drawings, the piezoelectric micromechanical ultrasonic transducer 27 mainly comprises a cavity base 37, an insulating layer 36, a structure layer 35 and a seed layer 34 which are sequentially stacked from bottom to top, and a plurality of transducer units arranged in a rectangular array are arranged on the upper surface of the seed layer 34. All the transducer units in the same piezoelectric micromechanical ultrasonic transducer 27 are in an electrical parallel state, so as to improve the receiving and transmitting efficiency of ultrasonic signals. The transducer unit comprises a bottom electrode 33, a piezoelectric layer 32 and a top electrode 31 which are sequentially stacked from bottom to top.

[0057] The hand electronic skin 2 of the present application should meet the following characteristics:

[0058] (1) The adhesive peeling force of the adhesive layer 21 is 0.5-1.2 N;

[0059] (2) The Young's modulus of the adhesive layer 21 is 100-400 kPa;

[0060] (3) The total thickness of the hand electronic skin 2 is 0.3-1 mm, which meets the requirements of good flexibility and comfort. Preferably, the thickness of the hand electronic skin 2 should not exceed 0.5 mm, so as to avoid the electronic skin from falling off when the finger is bent due to excessive thickness.

[0061] (4) The elongation at break of the hand electronic skin 2 is 60%-200%.

[0062] In actual use, the hand electronic skin based on the ultrasonic transducer of the present application can be fixed on the back of the hand, so as to realize the application in the fields of virtual reality interaction, robot control or medical rehabilitation.

[0063] Based on the above hand electronic skin based on ultrasonic transducer, the application further provides a three-dimensional gesture reconstruction method, which is specifically as follows:

[0064] First step: finger joint bending angle capture. The resistance change of the plurality of flexible strain sensors is measured, and the finger joint angle information at the corresponding position can be calculated, thereby completing the high-degree-of-freedom reconstruction of the finger joints of the whole hand. Specifically as follows:

[0065] By measuring the resistance values of the flexible strain sensors in real time, the resistance value, resistance change rate at the current moment and the resistance values, angle values and resistance change rates of the previous several frames are taken as the inputs of the neural network model, and the output is the angle value at the current moment, and then the bending angles of the finger joints are obtained.

[0066] Since the flexible strain sensors are in the middle layer of the hand electronic skin, the outer packaging layer adopts a mixed elastomer of PDMS and polyethylene glycol. Therefore, the hand electronic skin also has good waterproof performance and electromagnetic interference shielding characteristics.

[0067] Second step: three-dimensional position and inclination angle calculation of the hand.

[0068] The ultrasonic transducer emits ultrasonic waves, and more than three ultrasonic receivers are fixed outside the hand electronic skin 2 to receive ultrasonic signals and convert them into electrical signals. The ultrasonic signals are processed and extracted to obtain the time-of-flight difference and / or voltage amplitude and / or energy amplitude. The distance from the palm to a specific position is calculated according to the time-of-flight difference, thereby determining the spatial position of the palm. Since the signal amplitude attenuates differently at different angles, the inclination angle of the palm relative to the external ultrasonic receiver is calculated according to the voltage amplitude or energy amplitude.

[0069] In this embodiment, as shown in Figure 5 three ultrasonic receivers, i.e., the first ultrasonic receiver 51, the second ultrasonic receiver 52 and the third ultrasonic receiver 53, can be fixed outside the hand electronic skin 2. By calculating the time-of-flight and voltage amplitude of the ultrasonic transmitter on the electronic skin to the three fixed ultrasonic receivers in space, the spatial position and inclination angle of the ultrasonic transmitter on the back of the hand are obtained, and the six-axis attitude information of the hand can be realized without being affected by the ambient light and without real-time calibration.

[0070] In addition, the hand electronic skin can be placed with one ultrasonic transducer. If multiple ultrasonic transducers are placed, the position and inclination angle information of several points on the back of the hand can be obtained to improve the accuracy and robustness of gesture reconstruction.

[0071] Step 3: 3D Hand Reconstruction. By combining the angles of each finger joint and the six-axis position information of several points on the back of the hand from the previous two steps, the hand gesture can be directly reconstructed. This method is simple, reliable, computationally inexpensive, robust, and real-time. Details are as follows:

[0072] The finger joint bending angles calculated by various flexible strain sensors and the palm position and tilt angle data calculated by the ultrasonic transducer are integrated to generate a 3D reconstructed model of the hand in a host computer. The model can dynamically reflect the posture and position changes of the fingers and palm in real time, thereby achieving accurate 3D gesture reconstruction.

[0073] The electronic hand skin and method of the present invention will be described in detail below with reference to embodiments.

[0074] Example

[0075] like Figure 2 The diagram shown is an exploded view of the structure of a hand electronic skin 2 provided in this embodiment. The hand electronic skin 2 mainly includes an adhesive layer 21, a flexible strain sensor layer 22, a flexible electrode 23, a composite electrode 24, a surface encapsulation layer 25, an ultrasonic transducer encapsulation shell 26, and a piezoelectric micromechanical ultrasonic transducer 27. The adhesive layer 21, flexible strain sensor layer 22, flexible electrode 23, composite electrode 24, and surface encapsulation layer 25 are cross-linked and fixed together, and layer separation will not occur under normal circumstances. This embodiment takes a device containing eight flexible strain sensors as an example, corresponding to eight joint positions of the thumb, index finger, and middle finger, as shown below. Figure 1 As shown. Figure 1 This is a physical image of the electronic skin for the hand in this embodiment. To use it, open your palm, align each flexible strain sensor with its corresponding joint, and gently press it onto the back of your hand. This electronic skin is easy to wear and comfortable to use.

[0076] The structure of each component in this embodiment will be described in detail below.

[0077] The flexible strain sensor layer 22 includes several tension-sensitive flexible strain sensor elements that change resistance as the finger bends, used to measure the angle of the finger joint. Its length is designed to be approximately equal to the joint length to obtain the largest possible rate of resistance change and improve the sensor's sensitivity.

[0078] The flexible electrodes 23 and composite electrodes 24 in the finger area are wavy / serpentine, with good elasticity, and can adapt to different hand movements and deformations. The adhesive layer 21 has good stretchability and adhesion, allowing the electronic skin to adhere tightly to the skin and maintain its shape with hand movements.

[0079] The packaging shell 26 of the ultrasonic transducer is a plastic shell with metal electrodes, and the upper and lower surfaces each have two electrodes and are connected in series. The top electrode 31 and the bottom electrode 33 on the piezoelectric micro-mechanical ultrasonic transducer 27 are connected to the two electrodes on the upper surface of the packaging shell 26 through gold wires, and then the two electrodes on the lower surface of the shell are connected to the flexible electrode 23 line, and the gesture information is transmitted to the upper computer through the flexible circuit (FPC) and the flexible strain sensor.

[0080] Figure 3 and Figure 4 are respectively a structural schematic diagram and an exploded view of the piezoelectric micro-mechanical ultrasonic transducer 27 of the present application, and the piezoelectric micro-mechanical ultrasonic transducer 27 includes a top electrode 31, a piezoelectric layer 32, a bottom electrode 33, a seed layer 34, a structural layer 35, an insulating layer 36, and a cavity base 37. The piezoelectric micro-mechanical ultrasonic transducer 27 contains a plurality of rectangularly arranged transducer units, and all the units in one transducer are in an electrical parallel state in the present application, so as to improve the transmitting and receiving sensitivity. In the present embodiment, the ultrasonic transducer includes 6*7 units.

[0081] Figure 5 is a principle diagram for calculating the three-dimensional position and inclination angle of the hand by the hand electronic skin 2 of the present application. In the present embodiment, a piezoelectric micro-mechanical ultrasonic transducer 27 is installed on the hand electronic skin 2 as a transmitter, and three ultrasonic transducers (i.e. a first ultrasonic receiver 51, a second ultrasonic receiver 52, and a third ultrasonic receiver 53) are fixed in the external environment as receivers. The three receivers are not on the same straight line, and by extracting the three time-of-flight values and voltage amplitudes of the ultrasonic waves from the transmitter to the receivers, as well as the coordinates and axis directions of the three receivers, the three-dimensional position and inclination angle of the ultrasonic transducer on the back of the hand can be calculated.

[0082] Figure 6 is a flow chart of the preparation process of the hand electronic skin 2 of the present embodiment, and the specific steps are as follows:

[0083] Ultrasonic dispersion S61: the conductive material 601 of the flexible strain sensor is mixed with an organic solvent 602 (such as toluene / dichloromethane, etc.), and a uniform suspension is obtained by ultrasonic dispersion.

[0084] Magnetic stirring S62: the flexible strain sensor base material 603 (a mixture of PDMS and an appropriate amount of polyethylene glycol in the present embodiment) is mixed with the organic solvent 602, and the mixture is uniformly mixed by magnetic stirring.

[0085] Heating and stirring S63: the two solutions obtained in S61 and S62 are mixed, and then heated to above the boiling point of the organic solvent 602 while being magnetically stirred. Until the organic solvent 602 is almost completely evaporated, a mixture of uniformly dispersed conductive material 601 and flexible strain sensor base material 603 is obtained.

[0086] Screen printing flexible sensor layer S64: Add an appropriate amount of PDMS curing agent 604 to the mixed solution obtained in S63 and continue stirring for 10-15 min to prepare a coating liquid. The coating liquid is brushed onto the transfer platform 605 by screen printing to make a flexible strain sensor. The transfer platform 605 can be any material that does not adhere to PDMS (such as PET, stainless steel, etc.).

[0087] Heating and curing the flexible sensor layer S65: Place the printed transfer platform 605 in a drying oven at 60°C for 2h to cure the elastomer and volatilize the remaining small amount of organic solvent 602.

[0088] Heating and curing the adhesive layer S66: After the flexible strain sensor is completely cured, coat / spin a layer of adhesive layer raw material 606 (a mixture of PDMS, polyethylene glycol, and PDMS curing agent, prepared in a similar manner to steps S62 and S64) on it and place it in a drying oven at 60°C for 2h.

[0089] Heating and curing the flexible electrode S67: Prepare the first flexible electrode raw material 607, which is stretchable silver paste in this embodiment, and print the flexible electrode 23 on the substrate by screen printing, and place it in a drying oven at 100°C for 10 min.

[0090] Making a composite electrode S68: Further screen print a layer of second flexible electrode raw material 608 (similar composition to the sensor layer raw material) on the flexible electrode 23, and place it in a drying oven at 60°C for 2h to obtain the composite electrode 24.

[0091] Making a surface packaging layer S69: Align the lower electrode of the packaging shell 26 of the ultrasonic transducer with the corresponding flexible electrode position, and connect the piezoelectric micromechanical ultrasonic transducer 27 to the conductive network of the hand electronic skin 2. Then connect the FPC port (i.e. flexible circuit board) 610 of the subsequent circuit to the conductive network port of the hand electronic skin 2. Finally, spin the surface packaging layer raw material 609 (a mixture of PDMS, polyethylene glycol, and PDMS curing agent), and place it in a drying oven at 60°C for 2h to obtain the complete hand electronic skin 2.

[0092] Figure 6 The flowchart shown introduces the complete process from raw material mixing to electronic skin assembly. Through screen printing, vacuum, heating and curing, etc., the low-cost, mass-produced efficient manufacturing process of the hand electronic skin 2 in this embodiment is realized.

[0093] Figure 7is the response diagram of the relative resistance change (R / R0) of the flexible strain sensor in different bending frequencies (1Hz, 2Hz, 3Hz) of the finger over time in the embodiment. It is shown from the graph that the flexible strain sensor can quickly respond to different frequency stimuli and has good dynamic response capability. This characteristic is very important in hand dynamic posture tracking applications and helps to achieve accurate detection of different speed movements.

[0094] Figure 8 is the response diagram of the relative resistance change of the flexible strain sensor over time in different bending angles of the finger in the embodiment. The graph shows that the flexible strain sensor can respond to different bending angles, and as the angle increases, the amplitude of the resistance change also increases accordingly. This characteristic shows that the sensor has high angle resolution capability and is suitable for detecting different degrees of bending and is suitable for real-time monitoring of hand joint angles.

[0095] The hand electronic skin of the application mainly consists of a flexible stretchable strain sensor and a piezoelectric micro-mechanical ultrasonic transducer, has a simple and reliable structure, low manufacturing cost, is not disturbed by environmental light, has low time drift rate, and is convenient and comfortable to wear. By integrating a multi-channel flexible strain sensor and an ultrasonic transducer, the electronic skin can simultaneously detect the bending angle of the finger joints and the spatial position and inclination angle of the hand, realizing three-dimensional real-time reconstruction of the user's hand. The flexible strain sensor independently measures the bending angles of the finger joints, and the ultrasonic transducer measures the positions and inclination angles of several points of the hand by calculating the flight time. The hand electronic skin of the application is suitable for virtual reality, robot control and medical rehabilitation fields, and can significantly improve the user's interaction experience and operation convenience.

[0096] The above-described embodiments are only a preferred scheme of the application, and are not intended to limit the application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the application. Therefore, any technical scheme obtained by equivalent replacement or equivalent transformation falls within the protection scope of the application.

Claims

1. A hand electronic skin based on an ultrasonic transducer, characterized in that, The device includes an adhesive layer (21), a flexible strain sensor layer (22), a flexible electrode (23), a composite electrode (24), a surface encapsulation layer (25), and an ultrasonic transducer, all disposed on the surface of the skin (1) of the hand. The adhesive layer (21), flexible strain sensor layer (22), flexible electrode (23), composite electrode (24), and surface encapsulation layer (25) are stacked sequentially from bottom to top, and chemical cross-linking is formed between adjacent layers. The flexible strain sensor layer (22) includes several flexible strain sensors, and each finger joint is provided with a flexible strain sensor for independently measuring its bending angle. The ultrasonic transducer is partially embedded in the surface encapsulation layer (25) and is used to emit ultrasonic waves. The substrate of at least one of the adhesive layer (21), composite electrode (24), and surface encapsulation layer (25) is at least partially the same as the substrate of the flexible strain sensor and is polydimethylsiloxane. The raw materials of the adhesive layer (21) and the flexible strain sensor layer (22) include polydimethylsiloxane, polydimethylsiloxane curing agent and polyethylene glycol oligomer; the molecular weight range of the polyethylene glycol oligomer is 200~600, the mass ratio of polyethylene glycol oligomer to polydimethylsiloxane is 1:(19~99), and the mass ratio of polydimethylsiloxane curing agent to polydimethylsiloxane is 1:(11~15).

2. The electronic hand skin based on an ultrasonic transducer according to claim 1, characterized in that, The ultrasonic transducer includes an encapsulation shell (26) and a piezoelectric micromechanical ultrasonic transducer (27); the piezoelectric micromechanical ultrasonic transducer (27) is partially embedded in the surface encapsulation layer (25) and is sealed as a whole through the encapsulation shell (26).

3. The electronic hand skin based on an ultrasonic transducer according to claim 2, characterized in that, The piezoelectric micromechanical ultrasonic transducer (27) operates in the frequency range of 40~300kHz and includes a cavity substrate (37), an insulating layer (36), a structural layer (35) and a seed layer (34) stacked sequentially from bottom to top. Several transducer units arranged in a rectangular array are provided on the upper surface of the seed layer (34), and all transducer units are in an electrically parallel state. The transducer unit includes a bottom electrode (33), a piezoelectric layer (32) and a top electrode (31) stacked sequentially from bottom to top.

4. The electronic hand skin based on an ultrasonic transducer according to claim 3, characterized in that, The specific method for preparing the adhesive layer (21) is as follows: S1: Mix polydimethylsiloxane and polyethylene glycol oligomer in a certain proportion until homogeneous; S2: Add polydimethylsiloxane curing agent to the mixture obtained in S1 and mix thoroughly; S3: Pour the mixture obtained in S2 into a mold, or use spin coating to control the thickness; S4: Cur the product obtained in S3 in an environment of 50℃~70℃, and the curing time shall not exceed 2 hours.

5. The electronic hand skin based on an ultrasonic transducer according to claim 1, characterized in that, The raw materials of the flexible strain sensor and the composite electrode (24) include at least one of carbon nanotubes, carbon powder, and graphene, and the resistance of the flexible strain sensor increases with the increase of strain.

6. The electronic hand skin based on an ultrasonic transducer according to claim 1, characterized in that, The electronic skin on the hand (2) has the following characteristics: The adhesive peeling force of the adhesive layer (21) is 0.5~1.2N, and the Young's modulus is 100~400kPa; The total thickness of the electronic skin (2) of the hand is 0.3~1mm, and the elongation at break is 60%~200%.

7. An application of attaching an ultrasonic transducer-based electronic skin for the hand as described in any one of claims 1 to 6 to the back of the hand for use in virtual reality interaction, robot control, or medical rehabilitation.

8. A method for three-dimensional gesture reconstruction of hand electronic skin based on an ultrasonic transducer as described in any one of claims 1 to 6, characterized in that, Specifically as follows: By measuring the resistance value of each flexible strain sensor in real time, the resistance value, resistance change rate, and resistance value, angle value, and resistance change rate of the previous several frames are used as input to the neural network model, and the output is the angle value at the current moment, thereby obtaining the bending angle of each finger joint. Ultrasonic waves are emitted through the ultrasonic transducer, and three or more ultrasonic receivers are fixed outside the electronic skin of the hand (2) to receive ultrasonic signals and convert them into electrical signals; the ultrasonic signals are processed and extracted to obtain the time difference of flight and / or the voltage amplitude and / or the energy amplitude; the distance from the palm to a specific position is calculated based on the time difference of flight, thereby determining the spatial position of the palm; since the signal amplitude attenuates differently at different angles, the tilt angle of the palm relative to the external ultrasonic receiver is calculated based on the voltage amplitude or the energy amplitude. The finger joint bending angles calculated by each flexible strain sensor and the palm position and tilt angle data calculated by the ultrasonic transducer are integrated to generate a three-dimensional reconstruction model of the hand in the host computer. The model can dynamically reflect the posture and position changes of the fingers and palm in real time, thereby realizing accurate three-dimensional gesture reconstruction.

Images