Hand electronic skin based on ultrasonic transducer and three-dimensional gesture reconstruction method thereof

Abstract

The invention discloses a hand electronic skin based on an ultrasonic transducer and a three-dimensional gesture reconstruction method thereof, and relates to the technical field of man-machine interaction. The hand electronic skin comprises a viscous layer, a flexible strain sensor layer, a flexible electrode, a composite electrode, a surface packaging layer and an ultrasonic transducer which are arranged on the surface of the hand skin; the flexible strain sensor layer comprises a plurality of flexible strain sensors, and each finger joint is provided with a flexible strain sensor used for independently measuring the bending angle of the finger joint. The ultrasonic transducer is partially embedded in the surface packaging layer and used for emitting ultrasonic waves. The electronic skin is mainly composed of a flexible stretchable strain sensor and a piezoelectric micromechanical ultrasonic transducer, and is simple and reliable in structure, low in manufacturing cost, free of interference of ambient light, low in time drift rate and convenient and comfortable to wear. The hand electronic skin is suitable for the fields of virtual reality, robot control, medical rehabilitation and the like, and the interaction experience and operation convenience of the user can be remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of human-computer interaction technology, relates to the field of flexible sensor technology and wearable devices, and specifically relates to an ultrasonic transducer-based hand electronic skin and a three-dimensional gesture reconstruction method thereof. Background Art

[0002] In recent years, with the rapid development of virtual reality (VR), augmented reality (AR) and human-computer interaction technology, hand motion capture and three-dimensional reconstruction technology have received increasing attention. These technologies are widely used in medical rehabilitation, robot manipulation, game entertainment and education. However, existing hand reconstruction technologies generally have some shortcomings. Traditional hand tracking methods mainly rely on optical cameras. Although this method can achieve high-precision gesture reconstruction, it is expensive, complex to install and easily affected by ambient light, which limits its practical application. In addition, although data glove-based solutions can provide better flexibility, they often lead to insufficient data accuracy and real-time performance due to the inconvenience of wearing and the drift of the inertial measurement unit (IMU).

[0003] In order to overcome these problems, some emerging sensor technologies have been introduced into the field of hand reconstruction in recent years. For example, hand electronic skin based on stretchable sensors can achieve real-time monitoring of hand movements while maintaining flexibility and comfort. However, the existing hand electronic skin cannot simultaneously provide the detailed angle of each finger joint and the accurate reconstruction of the overall hand position and inclination, which limits its application in complex scenarios. In addition, the common hand electronic skin based on PDMS is usually less viscous, and the Young's modulus is greater than 1MPa, which leads to large internal stress of the electronic skin during finger bending and is very easy to fall off the hand. Although the hand electronic skin based on hydrogel is relatively viscous, the flexible strain sensor based on the hydrogel substrate is complex to manufacture, and its physicochemical properties are not as stable as PDMS, so its sensing performance is unstable, drifts greatly over time, and has low accuracy.

[0004] Therefore, there is an urgent need for a new hand reconstruction solution that can integrate multiple sensors, detect the movement state of fingers in real time, and accurately obtain the three-dimensional position and inclination of the hand, thereby improving the accuracy and convenience of human-computer interaction. The present invention aims to solve the above problems and proposes a hand electronic skin based on a flexible stretchable strain sensor and a MEMS ultrasonic transducer, which can achieve high-precision three-dimensional reconstruction of the hand. The hand electronic skin has the advantages of low cost, simple manufacturing, stable and reliable performance, and is not affected by environmental magnetic fields and light. It has high application value and broad market potential in the fields of human-computer interaction, medical rehabilitation and robot control. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects in the prior art and provide an ultrasonic transducer-based electronic skin for hands and a three-dimensional gesture reconstruction method thereof. The three-dimensional gesture reconstruction method of the present invention has the advantages of low cost, high stability, simple production, strong environmental interference resistance and natural user experience. The present invention provides a PDMS-based electronic skin manufacturing process for hands. By adjusting the mass ratio of PDMS, PDMS curing agent and polyethylene glycol oligomer, as well as the curing temperature and curing time, the degree of cross-linking after PDMS curing is regulated, thereby controlling the Young's modulus and surface adhesion of the electronic skin. The present invention integrates a multi-channel PDMS-based flexible strain sensor and a MEMS ultrasonic transducer chip on the above-mentioned electronic skin, which can achieve high-precision three-dimensional hand reconstruction while the user's hand moves, significantly improving the accuracy and convenience of human-computer interaction.

[0006] The specific technical solutions adopted by the present invention are as follows:

[0007] In the first aspect, the present invention provides an ultrasonic transducer-based electronic skin for the hand, comprising a viscous layer, a flexible strain sensor layer, a flexible electrode, a composite electrode, a surface packaging layer and an ultrasonic transducer arranged on the surface of the hand skin; the viscous layer, the flexible strain sensor layer, the flexible electrode, the composite electrode and the surface packaging layer are stacked in sequence from bottom to top, and chemical cross-links are formed between adjacent layers; the flexible strain sensor layer comprises a plurality of flexible strain sensors, and each finger joint is provided with a flexible strain sensor for independently measuring its bending angle; the ultrasonic transducer is partially embedded in the surface packaging layer for emitting ultrasonic waves.

[0008] Preferably, the ultrasonic transducer comprises a packaging shell and a piezoelectric micromechanical ultrasonic transducer; the piezoelectric micromechanical ultrasonic transducer is partially embedded in the surface packaging layer, and the overall sealing is achieved through the packaging shell.

[0009] Furthermore, the operating frequency range of the piezoelectric micromechanical ultrasonic transducer is 40 to 300 kHz, and it includes a cavity substrate, an insulating layer, a structural layer and a seed layer stacked in sequence from bottom to top, and a plurality of transducer units arranged in a rectangular array are arranged on the upper surface of the seed layer, and all the transducer units are in an electrically parallel state; the transducer unit includes a bottom electrode, a piezoelectric layer and a top electrode stacked in sequence from bottom to top.

[0010] Preferably, the substrate of at least one of the adhesive layer, the composite electrode and the surface packaging layer is at least partially the same as the substrate of the flexible strain sensor and is polydimethylsiloxane.

[0011] Preferably, the raw materials of the adhesive layer and the flexible strain sensor include polydimethylsiloxane, polydimethylsiloxane curing agent and polyethylene glycol oligomer; the molecular weight range of the polyethylene glycol oligomer is 200-600, the mass ratio of polyethylene glycol oligomer to polydimethylsiloxane is 1:, and the mass ratio of polydimethylsiloxane curing agent to polydimethylsiloxane is 1:.

[0012] Furthermore, the preparation method of the adhesive layer is as follows:

[0013] S1: Mix polydimethylsiloxane and polyethylene glycol oligomer in proportion;

[0014] S2: adding polydimethylsiloxane curing agent to the mixture obtained in S1 and mixing evenly;

[0015] S3: pouring the mixture obtained in S2 into a mold, or using a spin coating method to control the thickness;

[0016] S4: curing the product obtained in S3 in an environment of 50° C. to 70° C., and the curing time shall not exceed 2 hours.

[0017] Preferably, the raw materials of the flexible strain sensor and the composite electrode include at least one of carbon nanotubes, carbon powder and graphene, and the resistance of the flexible strain sensor increases with increasing strain.

[0018] Preferably, the hand electronic skin meets the following characteristics:

[0019] The adhesive layer has an adhesive peeling force of 0.5 to 1.2 N and a Young's modulus of 100 to 400 kPa;

[0020] The total thickness of the hand electronic skin is 0.3-1 mm, and the elongation at break is 60%-200%.

[0021] In a second aspect, the present invention provides a method of fixing an ultrasonic transducer-based hand electronic skin as described in any one of the first aspects on the back of a hand for use in virtual reality interaction, robot control or medical rehabilitation.

[0022] In a third aspect, the present invention provides a method for reconstructing a three-dimensional gesture of an electronic skin of a hand based on an ultrasonic transducer as described in any one of the first aspects, which is as follows:

[0023] By measuring the resistance value of each flexible strain sensor in real time, the resistance value at the current moment, the resistance change rate, and the resistance value, angle value, and resistance change rate of several previous frames are used as inputs of the neural network model, and the output is the angle value at the current moment, thereby obtaining the bending angle of each finger joint;

[0024] Ultrasonic waves are emitted through the ultrasonic transducer, and three or more ultrasonic receivers are fixed on the outside of the electronic skin of the hand to receive ultrasonic signals and convert them into electrical signals; the ultrasonic signals are processed and extracted to obtain the flight time difference and / or the voltage amplitude and / or the energy amplitude; the distance from the palm to a specific position is calculated according to the flight time difference, so as to determine the spatial position of the palm; since the attenuation of the signal amplitude is different at different angles, the inclination angle of the palm relative to the external ultrasonic receiver is calculated according to the voltage amplitude or the energy amplitude;

[0025] The finger joint bending angles calculated by each flexible strain sensor and the palm position and inclination data calculated by the ultrasonic transducer are integrated to generate a three-dimensional reconstruction model of the hand in the host computer; the model can dynamically reflect the posture and position changes of the fingers and palm in real time, thereby achieving accurate three-dimensional gesture reconstruction.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The invention has a simple structural design, low manufacturing process cost, is easy to mass produce and popularize and apply, and has broad market prospects.

[0028] 1) Compared with other hand motion monitoring methods, the 3D gesture reconstruction method based on hand electronic skin proposed in the present invention has the characteristic of being able to perform 3D gesture reconstruction without calibration. This not only simplifies the operation process, but also enhances the versatility of the method, making long-term continuous monitoring possible. In addition, the method does not rely on complex circuit design and signal processing algorithms, thereby reducing the computational burden of the system, lowering the requirements for software and hardware, and improving overall efficiency.

[0029] 2) Compared with other electronic skins, the polydimethylsiloxane (PDMS)-based hand electronic skin manufactured by the screen printing process adopted by the present invention has significant advantages in cost and production efficiency, while maintaining the stability of long-term wear and can be worn repeatedly. By adding and adjusting the molecular weight and mass ratio of polyethylene glycol and the mass ratio of PDMS curing agent, a highly viscous and highly soft hand electronic skin is obtained. This makes the electronic skin comfortable and adaptable when worn, can effectively prevent shedding due to hand movement, and is suitable for a variety of human-computer interaction scenarios.

[0030] 3) The manufacturing process of this electronic skin is simple, with good repeatability and biocompatibility, and the double-layer electrode design further improves the yield of multi-channel electronic skin. These characteristics make the electronic skin of the present invention not only suitable for entertainment in home environments, but also for medical rehabilitation in clinical environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1This is a physical picture of the electronic hand skin of the present invention;

[0032] Figure 2 It is an exploded diagram of the structure of each layer of the hand electronic skin device of the present invention;

[0033] Figure 3 Schematic diagram of the structure of the piezoelectric micromechanical ultrasonic transducer of the present invention;

[0034] Figure 4 It is an exploded diagram of the structure of each layer of the piezoelectric micromechanical ultrasonic transducer of the present invention;

[0035] Figure 5 This is a schematic diagram of the principle of the electronic skin of the hand of the present invention for calculating the three-dimensional position and inclination of the hand;

[0036] Figure 6 It is a flow chart of the preparation process of the electronic skin for hand of the present invention;

[0037] Figure 7 is a response diagram of the relative resistance change of the flexible strain sensor of the present invention at different bending frequencies of the finger over time;

[0038] Figure 8 is a response diagram of the relative resistance change of the flexible strain sensor of the present invention at different bending angles of the finger over time;

[0039] The accompanying drawings in the figure are marked as: hand skin 1, hand electronic skin 2, adhesive layer 21, flexible strain sensor layer 22, flexible electrode 23, composite electrode 24, surface packaging layer 25, packaging shell 26, piezoelectric micromechanical ultrasonic transducer 27, top electrode 31, piezoelectric layer 32, bottom electrode 33, seed layer 34, structural layer 35, insulating layer 36, cavity substrate 37, first ultrasonic receiver 51, second ultrasonic receiver 52, third ultrasonic receiver 53; conductive material 601, organic solvent 602, flexible strain sensor Sensor substrate 603, polydimethylsiloxane curing agent 604, transfer table 605, viscous layer raw material 606, first flexible electrode raw material 607, second flexible electrode raw material 608, surface packaging layer raw material 609, flexible circuit board 610, ultrasonic dispersion S61, magnetic stirring S62, heating stirring S63, screen printing flexible sensor layer S64, heating and curing flexible sensor layer S65, heating and curing viscous layer S66, heating and curing flexible electrode S67, making composite electrode S68, making surface packaging layer S69. DETAILED DESCRIPTION

[0040] The present invention is further described and illustrated below in conjunction with the accompanying drawings and specific embodiments. The technical features of each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0041] In this application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0042] The terms "first" and "second" are used only for descriptive purposes to distinguish objects such as substances from each other, and should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. For example, without departing from the scope of the embodiments of the present application, the first XX may also be referred to as the second XX, and similarly, the second XX may also be referred to as the first XX.

[0043] like Figure 1 and 2 As shown, an ultrasonic transducer-based hand electronic skin provided by the present invention mainly comprises a viscous layer 21, a flexible strain sensor layer 22, a flexible electrode 23, a composite electrode 24, a surface encapsulation layer 25 and an ultrasonic transducer arranged on the surface of the hand skin 1. The viscous layer 21, the flexible strain sensor layer 22, the flexible electrode 23, the composite electrode 24 and the surface encapsulation layer 25 are stacked in sequence from bottom to top to form a sandwich structure, and the adjacent layers are firmly connected by chemical cross-linking, thereby improving the overall reliability and durability, and the layer separation phenomenon will not occur under normal circumstances.

[0044] As a preferred embodiment of the present invention, the substrate of at least one of the adhesive layer 21, the composite electrode 24, and the surface packaging layer 25 is at least partially the same as the substrate of the flexible strain sensor and is polydimethylsiloxane.

[0045] As a preferred embodiment of the present invention, the adhesive layer 21 is made of polydimethylsiloxane (PDMS) material, which not only has excellent flexibility and appropriate viscosity, but also can maintain stability under different environmental conditions, ensuring that irreversible deformation is not easily caused during long-term wear.

[0046] In addition, the raw materials of the viscous layer 21 also include polydimethylsiloxane curing agent and polyethylene glycol oligomer. Specifically, the present invention wraps part of the PDMS-cured platinum catalyst with polyethylene glycol oligomer, thereby interfering with the degree of PDMS cross-linking reaction, so as to obtain a PDMS / polyethylene glycol mixed elastomer with both high surface viscosity and low Young's modulus. Preferably, the molecular weight range of polyethylene glycol oligomer is 200-600, the mass ratio of polyethylene glycol oligomer to polydimethylsiloxane is 1:19-99, and the mass ratio of polydimethylsiloxane curing agent to polydimethylsiloxane is 1:11-15. At this time, the viscosity of the elastomer can meet the conformal contact with the skin during hand movement. Since the strain of part of the skin will exceed 60% when the finger is bent, if the degree of PDMS cross-linking is further reduced, the elastomer may be easily broken.

[0047] The preparation method of the adhesive layer 21 is as follows:

[0048] S1: uniformly mix polydimethylsiloxane and polyethylene glycol oligomer according to the above proportions;

[0049] S2: adding polydimethylsiloxane curing agent to the mixture obtained in S1 according to the above proportion and mixing evenly;

[0050] S3: pouring the mixture obtained in S2 into a mold, or using a spin coating method to control the thickness;

[0051] S4: Curing the product obtained in S3 in an environment of 50°C to 70°C for no more than 2 hours. Preferably, the curing temperature of the PDMS and polyethylene glycol elastomers is 60°C for 2 hours, at which point the cured elastomers are highly viscous and not easily broken.

[0052] As a preferred embodiment of the present invention, the flexible electrode material uses stretchable silver paste, and the composite electrode uses the same material as the flexible strain sensor to increase the bonding force between the composite electrode and the flexible strain sensor layer, making it less likely to cause contact breakage.

[0053] In the present invention, the flexible strain sensor layer 22 includes a plurality of flexible strain sensors, and each finger joint is provided with a flexible strain sensor, which is used to independently measure the bending angle of the finger at that location. Since the present invention is to reconstruct gestures with high precision, each finger joint corresponds to a flexible strain sensor, so the number of channels of the hand electronic skin is relatively large. Therefore, the present invention adopts two electrode materials as a conductive network together, aiming to increase the signal stability of the electronic skin, especially the electrical stability under large deformation of the finger.

[0054] As a preferred embodiment of the present invention, the material of the flexible strain sensor can be at least one of carbon nanotubes, carbon powder, and graphene to ensure that the resistance changes sensitively during deformation (the resistance increases with the increase of strain), thereby achieving accurate monitoring of finger movements. At the same time, the nano-scale carbon material has stable physical and chemical properties, wear resistance and corrosion resistance, which can improve the stability and accuracy of sensor measurements.

[0055] In the present invention, the ultrasonic transducer is partially embedded in the surface packaging layer 25 for emitting ultrasonic waves.

[0056] As a preferred embodiment of the present invention, the ultrasonic transducer mainly includes a packaging shell 26 and a piezoelectric micromechanical ultrasonic transducer 27. The piezoelectric micromechanical ultrasonic transducer 27 is partially embedded in the upper surface of the surface packaging layer 25, and the piezoelectric micromechanical ultrasonic transducer 27 is sealed as a whole by the packaging shell 26. The operating frequency range of the piezoelectric micromechanical ultrasonic transducer 27 is 40 to 300 kHz. If the operating frequency decreases, the distance resolution is low; if the operating frequency increases, the ultrasonic propagation distance is short. Figure 3 and 4 As shown, the piezoelectric micromechanical ultrasonic transducer 27 mainly includes a cavity substrate 37, an insulating layer 36, a structural layer 35 and a seed layer 34 stacked in sequence from bottom to top, and a plurality of transducer units arranged in a rectangular array are arranged on the upper surface of the seed layer 34. All transducer units in the same piezoelectric micromechanical ultrasonic transducer 27 are in an electrically parallel state to improve the efficiency of receiving and transmitting ultrasonic signals. The transducer unit includes a bottom electrode 33, a piezoelectric layer 32 and a top electrode 31 stacked in sequence from bottom to top.

[0057] The hand electronic skin 2 of the present invention should meet the following characteristics:

[0058] (1) The adhesive peeling force of the adhesive layer 21 is 0.5 to 1.2 N;

[0059] (2) The Young's modulus of the adhesive layer 21 is 100 to 400 kPa;

[0060] (3) The total thickness of the hand electronic skin 2 is 0.3-1 mm, which meets the requirements of good flexibility and comfort. Preferably, the thickness of the hand electronic skin 2 should not exceed 0.5 mm to avoid the electronic skin falling off when the finger is bent due to excessive thickness.

[0061] (4) The breaking elongation of the hand electronic skin 2 is 60% to 200%.

[0062] In actual use, the ultrasonic transducer-based electronic skin for hand of the present invention can be fixed on the back of the hand to realize applications in the fields of virtual reality interaction, robot control or medical rehabilitation.

[0063] Based on the above-mentioned hand electronic skin based on ultrasonic transducer, the present invention also provides a three-dimensional gesture reconstruction method, which is specifically as follows:

[0064] Step 1: Capture the bending angle of finger joints. By measuring the resistance changes of multiple flexible strain sensors, the angle information of the finger joints at their corresponding positions can be calculated, thereby completing the high-degree-of-freedom reconstruction of the finger joints of the entire hand. The details are as follows:

[0065] By measuring the resistance value of each flexible strain sensor in real time, the current resistance value, resistance change rate, and the resistance value, angle value, and resistance change rate of the previous frames are used as the input of the neural network model, and the output is the angle value at the current moment, thereby obtaining the bending angle of each finger joint.

[0066] Since the flexible strain sensor is in the middle layer of the hand electronic skin, and the outer encapsulation layer uses a mixed elastomer of PDMS and polyethylene glycol, the hand electronic skin also has good waterproof performance and electromagnetic interference shielding properties.

[0067] Step 2: Calculate the three-dimensional position and inclination of the hand.

[0068] Ultrasonic waves are emitted by an ultrasonic transducer, and three or more ultrasonic receivers are fixed on the outside of the hand electronic skin 2 to receive ultrasonic signals and convert them into electrical signals. The ultrasonic signals are processed and extracted to obtain the flight time difference and / or the voltage amplitude and / or the energy amplitude. The distance from the palm to a specific position is calculated based on the flight time difference, thereby determining the spatial position of the palm. Since the attenuation of the signal amplitude is different at different angles, the inclination angle of the palm relative to the external ultrasonic receiver is calculated based on the voltage amplitude or the energy amplitude.

[0069] In this embodiment, if Figure 5 As shown, three ultrasonic receivers can be fixed outside the hand electronic skin 2, namely, the first ultrasonic receiver 51, the second ultrasonic receiver 52, and the third ultrasonic receiver 53. By calculating the flight time and voltage amplitude from the ultrasonic transmitter on the electronic skin to the three ultrasonic receivers fixed in space, the spatial position and inclination of the ultrasonic transmitter on the back of the hand are obtained, and the six-axis hand posture information that is not affected by ambient light and does not require calibration in real time can be realized.

[0070] In addition, the electronic skin of the hand can be placed with less than one ultrasonic transducer. If multiple ultrasonic transducers are placed, the position and inclination information of several points on the back of the hand can be obtained to improve the accuracy and robustness of gesture reconstruction.

[0071] Step 3: 3D hand reconstruction. Combining the six-axis position information of the finger joint angles and several points on the back of the hand in the above two steps, the gesture can be directly restored. This method is simple and reliable, with low computational cost and high robustness and real-time performance. The details are as follows:

[0072] The finger joint bending angles calculated by the flexible strain sensors and the palm position and inclination data calculated by the ultrasonic transducer are integrated to generate a 3D reconstruction model of the hand in the host computer. The model can dynamically reflect the posture and position changes of the fingers and palm in real time, thereby achieving accurate 3D gesture reconstruction.

[0073] The electronic hand skin and method of the present invention will be specifically described below with reference to embodiments.

[0074] Example

[0075] like Figure 2 As shown in the figure, it is an exploded view of the structure of a hand electronic skin 2 provided in this embodiment. The hand electronic skin 2 mainly includes a viscous layer 21, a flexible strain sensor layer 22, a flexible electrode 23, a composite electrode 24, a surface packaging layer 25, an ultrasonic transducer packaging shell 26 and a piezoelectric micromechanical ultrasonic transducer 27. The viscous layer 21, the flexible strain sensor layer 22, the flexible electrode 23, the composite electrode 24, and the surface packaging layer 25 are fixed by material cross-linking, and layer separation will not occur under normal circumstances. This embodiment takes a device containing 8 flexible strain sensors as an example, corresponding to 8 joint positions of the thumb, index finger, and middle finger, respectively, as shown in FIG. Figure 1 shown. Figure 1 This is a real picture of the electronic skin on the hand of this embodiment. When using it, you need to open your palm, then align each flexible strain sensor with the corresponding joint, and gently press it on the back of your hand. The electronic skin is easy to wear and comfortable to use.

[0076] The structure of each component of this embodiment will be described in detail below.

[0077] The flexible strain sensor layer 22 includes a plurality of stretch-sensitive flexible strain sensor elements, which can change resistance as the finger bends, and is used to measure the angle of the finger joint. Its length is designed to be roughly equal to the length of the joint to obtain the largest possible resistance change rate and improve the sensitivity of the sensor.

[0078] The flexible electrodes 23 and composite electrodes 24 at the finger parts are wavy / snake-shaped, have good elasticity, and can adapt to different movements and deformations of the hand. The adhesive layer 21 has good stretchability and viscosity, so that the electronic skin can adhere to the skin and move in a shape-preserving manner with the hand movements.

[0079] The package shell 26 of the ultrasonic transducer is a plastic shell with metal electrodes, and has two electrodes on the upper and lower surfaces, and the upper and lower surfaces are connected. First, the top electrode 31 and the bottom electrode 33 on the piezoelectric micromechanical ultrasonic transducer 27 are connected to the two electrodes on the upper surface of the package shell 26 through gold wires, and then the two electrodes on the lower surface of the shell are connected to the flexible electrode 23 circuit, and the gesture information is transmitted to the host computer through the flexible circuit (FPC) and the flexible strain sensor.

[0080] Figure 3 and Figure 4 They are a schematic diagram and an exploded view of the structure of the piezoelectric micromechanical ultrasonic transducer 27 of the present invention, and the piezoelectric micromechanical ultrasonic transducer 27 includes a top electrode 31, a piezoelectric layer 32, a bottom electrode 33, a seed layer 34, a structural layer 35, an insulating layer 36, and a cavity substrate 37. The piezoelectric micromechanical ultrasonic transducer 27 contains a plurality of transducer units arranged in a rectangular manner. In the present invention, all units in a transducer are in an electrically parallel state to improve the transmission and reception sensitivity. In this embodiment, the ultrasonic transducer includes 6*7 units.

[0081] Figure 5 This is a schematic diagram of the hand electronic skin 2 of the present invention for calculating the three-dimensional position and inclination of the hand. In this embodiment, a piezoelectric micromechanical ultrasonic transducer 27 is installed on the hand electronic skin 2 as a transmitter, and three ultrasonic transducers (i.e., a first ultrasonic receiver 51, a second ultrasonic receiver 52, and a third ultrasonic receiver 53) are fixed in the external environment as receivers. These three receivers are not in the same straight line. By extracting the three flight time values ​​and voltage amplitudes of the ultrasonic wave from the transmitter to the receiver, as well as the coordinates and axis directions of the three receivers, the three-dimensional position and inclination of the ultrasonic transducer on the back of the hand can be calculated.

[0082] Figure 6 : is a flow chart of the preparation process of the hand electronic skin 2 of this embodiment, and the specific steps are as follows:

[0083] Ultrasonic dispersion S61: The conductive material 601 of the flexible strain sensor is mixed with an organic solvent 602 (such as toluene / dichloromethane, etc.), and a uniform suspension is obtained by ultrasonic dispersion.

[0084] Magnetic stirring S62: The flexible strain sensor substrate 603 (a mixture of PDMS and an appropriate amount of polyethylene glycol in this example) and the organic solvent 602 are mixed and uniformly mixed by magnetic stirring.

[0085] Heating and stirring S63: The two solutions obtained in S61 and S62 are mixed, and then heated to a temperature above the boiling point of the organic solvent 602 and magnetically stirred at the same time, until the organic solvent 602 is almost completely evaporated, and a mixed solution of the conductive material 601 and the flexible strain sensor substrate 603 uniformly dispersed is obtained.

[0086] Screen printing flexible sensor layer S64: Add an appropriate amount of PDMS curing agent 604 to the mixed solution obtained in S63 and continue stirring for 10 to 15 minutes to prepare a coating solution. The coating solution is brushed onto a transfer table 605 by screen printing to make a flexible strain sensor. The transfer table 605 can be any material that does not adhere to PDMS (such as PET, stainless steel, etc.).

[0087] Heat-curing the flexible sensor layer S65: placing the printed transfer table 605 in a drying oven at 60°C for 2 hours to cure the elastomer and volatilize the remaining small amount of organic solvent 602.

[0088] Heat-curing the adhesive layer S66: After the flexible strain sensor is completely cured, a layer of adhesive layer raw material 606 (a mixture of PDMS, polyethylene glycol, and PDMS curing agent, prepared in a similar manner to steps S62 and S64) is coated / spin-coated on it, and placed in a drying oven at 60°C for curing for 2 hours.

[0089] Heating and curing the flexible electrode S67: prepare a first flexible electrode raw material 607, which is a stretchable silver paste in this embodiment, and print a flexible electrode 23 on the substrate by screen printing, and place it in a drying oven at 100°C for curing for 10 minutes.

[0090] Preparation of composite electrode S68: A layer of second flexible electrode raw material 608 (with similar composition to the sensor layer raw material) is further screen-printed on the flexible electrode 23, and placed in a drying oven at 60° C. for curing for 2 hours to obtain a composite electrode 24.

[0091] Make the surface encapsulation layer S69: align the lower electrode of the ultrasonic transducer package shell 26 and place it on the corresponding flexible electrode position, and connect the piezoelectric micromechanical ultrasonic transducer 27 to the conductive network of the hand electronic skin 2. Then connect the FPC port (i.e., flexible circuit board) 610 of the subsequent circuit to the conductive network port of the hand electronic skin 2. Finally, spin-coat the surface encapsulation layer raw material 609 (PDMS, polyethylene glycol, PDMS curing agent mixture), and after curing in a drying oven at 60°C for 2h, a complete hand electronic skin 2 is obtained.

[0092] Figure 6 The flowchart shown introduces the complete process from raw material mixing to electronic skin assembly. Through screen printing, vacuum, heating and curing and other processes, a low-cost, mass-producible and efficient manufacturing process for the hand electronic skin 2 in this embodiment is achieved.

[0093] Figure 7The graph is a response diagram of the relative resistance change (R / R0) of the flexible strain sensor in this embodiment at different finger bending frequencies (1Hz, 2Hz, 3Hz) over time. The graph shows that the flexible strain sensor can quickly respond to different frequency stimuli and has good dynamic response capability. This feature is very important in the application of dynamic hand posture tracking and helps to achieve accurate detection of different speed movements.

[0094] Figure 8 is a response diagram of the relative resistance change of the flexible strain sensor at different bending angles of the finger in this embodiment over time. The figure shows that the flexible strain sensor can respond to different bending angles, and as the angle increases, the amplitude of the resistance change also increases accordingly. This characteristic shows that the sensor has a high angle resolution capability, is suitable for detecting different degrees of bending, and is suitable for real-time monitoring of hand joint angles.

[0095] The hand electronic skin of the present invention is mainly composed of a flexible stretchable strain sensor and a piezoelectric micromechanical ultrasonic transducer. It has a simple and reliable structure, low manufacturing cost, is not affected by ambient light, has a low time drift rate, and is convenient and comfortable to wear. By integrating a multi-channel flexible strain sensor and an ultrasonic transducer, the electronic skin can simultaneously detect the bending angle of the finger joints and the spatial position and inclination of the hand, thereby realizing three-dimensional real-time reconstruction of the user's hand. The flexible strain sensor independently measures the bending angle of each finger joint, and the ultrasonic transducer measures the position and inclination of several points on the hand by calculating the flight time. The hand electronic skin of the present invention is suitable for fields such as virtual reality, robot control, and medical rehabilitation, and can significantly improve the user's interactive experience and operational convenience.

[0096] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. An electronic hand skin based on an ultrasonic transducer, characterized in that: The invention comprises an adhesive layer (21), a flexible strain sensor layer (22), a flexible electrode (23), a composite electrode (24), a surface packaging layer (25) and an ultrasonic transducer arranged on the surface of the hand skin (1); the adhesive layer (21), the flexible strain sensor layer (22), the flexible electrode (23), the composite electrode (24) and the surface packaging layer (25) are stacked in sequence from bottom to top, and chemical cross-links are formed between adjacent layers; the flexible strain sensor layer (22) comprises a plurality of flexible strain sensors, and each finger joint is provided with a flexible strain sensor for independently measuring its bending angle; the ultrasonic transducer is partially embedded in the surface packaging layer (25) and is used to emit ultrasonic waves.

2. The hand electronic skin based on ultrasonic transducer according to claim 1, characterized in that: The ultrasonic transducer comprises a packaging shell (26) and a piezoelectric micromechanical ultrasonic transducer (27); the piezoelectric micromechanical ultrasonic transducer (27) is partially embedded in the surface packaging layer (25), and the entirety of the transducer is sealed by the packaging shell (26).

3. The hand electronic skin based on ultrasonic transducer according to claim 2, characterized in that: The piezoelectric micromechanical ultrasonic transducer (27) has an operating frequency range of 40 to 300 kHz and comprises a cavity substrate (37), an insulating layer (36), a structural layer (35) and a seed layer (34) which are stacked in sequence from bottom to top; a plurality of transducer units arranged in a rectangular array are arranged on the upper surface of the seed layer (34), and all the transducer units are in an electrically parallel state; the transducer unit comprises a bottom electrode (33), a piezoelectric layer (32) and a top electrode (31) which are stacked in sequence from bottom to top.

4. The hand electronic skin based on ultrasonic transducer according to claim 1, characterized in that: The base material of at least one of the adhesive layer (21), the composite electrode (24), and the surface packaging layer (25) is at least partially the same as the base material of the flexible strain sensor and is polydimethylsiloxane.

5. The hand electronic skin based on ultrasonic transducer according to claim 1, characterized in that: The raw materials of the adhesive layer (21) and the flexible strain sensor layer (22) include polydimethylsiloxane, a polydimethylsiloxane curing agent and a polyethylene glycol oligomer; the molecular weight of the polyethylene glycol oligomer is in the range of 200 to 600, the mass ratio of the polyethylene glycol oligomer to the polydimethylsiloxane is 1:(19 to 99), and the mass ratio of the polydimethylsiloxane curing agent to the polydimethylsiloxane is 1:(11 to 15).

6. The hand electronic skin based on ultrasonic transducer according to claim 5, characterized in that: The preparation method of the adhesive layer (21) is specifically as follows: S1: Mix polydimethylsiloxane and polyethylene glycol oligomer in proportion; S2: adding polydimethylsiloxane curing agent to the mixture obtained in S1 and mixing evenly; S3: pouring the mixture obtained in S2 into a mold, or using a spin coating method to control the thickness; S4: curing the product obtained in S3 in an environment of 50° C. to 70° C., and the curing time shall not exceed 2 hours.

7. The hand electronic skin based on ultrasonic transducer according to claim 1, characterized in that: The raw materials of the flexible strain sensor and the composite electrode (24) include at least one of carbon nanotubes, carbon powder and graphene, and the resistance of the flexible strain sensor increases as the strain increases.

8. The hand electronic skin based on ultrasonic transducer according to claim 1, characterized in that: The hand electronic skin (2) meets the following characteristics: The adhesive layer (21) has an adhesive peeling force of 0.5 to 1.2 N and a Young's modulus of 100 to 400 kPa; The total thickness of the hand electronic skin (2) is 0.3-1 mm, and the elongation at break is 60%-200%.

9. An application of fixing the hand electronic skin based on ultrasonic transducer as claimed in any one of claims 1 to 8 on the back of a hand in virtual reality interaction, robot control or medical rehabilitation.

10. A three-dimensional gesture reconstruction method of hand electronic skin based on ultrasonic transducer according to any one of claims 1 to 8, characterized in that: The details are as follows: By measuring the resistance value of each flexible strain sensor in real time, the resistance value at the current moment, the resistance change rate, and the resistance value, angle value, and resistance change rate of several previous frames are used as inputs of the neural network model, and the output is the angle value at the current moment, thereby obtaining the bending angle of each finger joint; Ultrasonic waves are emitted by the ultrasonic transducer, and three or more ultrasonic receivers are fixed outside the electronic skin (2) of the hand to receive ultrasonic signals and convert them into electrical signals; the ultrasonic signals are processed and extracted to obtain a flight time difference and / or a voltage amplitude and / or an energy amplitude; the distance from the palm to a specific position is calculated based on the flight time difference, thereby determining the spatial position of the palm; since the attenuation of the signal amplitude is different at different angles, the inclination angle of the palm relative to the external ultrasonic receiver is calculated based on the voltage amplitude or the energy amplitude; The finger joint bending angles calculated by each flexible strain sensor and the palm position and inclination data calculated by the ultrasonic transducer are integrated to generate a three-dimensional reconstruction model of the hand in the host computer; the model can dynamically reflect the posture and position changes of the fingers and palm in real time, thereby achieving accurate three-dimensional gesture reconstruction.

Images