Bidirectional electronic skin system with multimodal sensing and feedback stimulation and application

Abstract

The invention relates to the technical field of sensors, in particular to a multimodal sensing and feedback stimulation two-way electronic skin system and application, and the system comprises a two-way electronic skin and a control circuit connected with the two-way electronic skin; the control circuit is used for respectively establishing a corresponding relation between temperature sensing and cold and hot stimulation of the semiconductor chilling plate, a relation between force sensing and electrical stimulation and a corresponding relation between resistance sensing and vibration motor stimulation; the bidirectional electronic skin comprises a multi-modal stimulation electronic skin and a multi-modal synchronous sensing electronic skin; the bidirectional electronic skin is installed on the wearable prosthetic hand, and flexible fingers of the wearable prosthetic hand are integrated with the multi-mode synchronous sensing electronic skin so as to sense external stimulation. The multi-mode stimulation electronic skin is attached to the surface of a human body to stimulate the touch sense of the human body. According to the bidirectional electronic skin system provided by the invention, the temperature, resistance and force sensed by touch are fed back to the human body by converting corresponding temperature, vibration and electrical stimulation, so that the touch sensing of the human body is enhanced, and the man-machine interaction capability is improved.

Description

Technical Field

[0001] This application relates to the technical field of sensors, and particularly to a two-way electronic skin system and application for multi-modal sensing and feedback stimulation. Background Art

[0002] Generally, amputees lack the tactile perception feedback function due to the installation of prosthetic limbs, making it impossible for them to complete many normal behaviors in life. Although some researchers have developed some simple-function intelligent prostheses, which mainly assist amputees to complete uncomplicated grasping, moving and other actions by collecting simple myoelectric signals, they are powerless for a large number of refined actions in daily life. By adding the multi-modal perception feedback stimulation function, not only can the tactile function of the human body be enhanced through feedback stimulation, but also the human-computer interaction ability can be improved, and the difference between the prosthetic limb and the normal limb can be reduced. Currently, the perception and feedback stimulation of single-modal and two-modal (pressure and temperature) are relatively common, but it still faces challenges for multi-modal sensors to detect more than three kinds of stimuli. Summary of the Invention

[0003] The embodiments of this application provide a two-way electronic skin system and application for multi-modal sensing and feedback stimulation, which convert the temperature, resistance, and force of tactile perception into corresponding temperature, vibration, and electrical stimulation and feedback them to the human body to enhance the human tactile perception and improve the human-computer interaction ability.

[0004] To solve the above technical problems, in the first aspect, the embodiments of this application provide a two-way electronic skin system for multi-modal sensing and feedback stimulation. The system includes: a two-way electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish the corresponding relationship between temperature perception and the hot and cold stimulation of a thermoelectric cooler, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and the stimulation of a vibration motor; the two-way electronic skin includes a multi-modal stimulation electronic skin and a multi-modal synchronous perception electronic skin; when in use, the two-way electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multi-modal synchronous perception electronic skin to sense external stimuli; the multi-modal stimulation electronic skin is attached to the human body surface to stimulate human touch, and enhances human touch through perception feedback stimulation, improving the human-computer interaction ability.

[0005] In some exemplary embodiments, the multi-modal stimulation electronic skin is a three-modal stimulation electronic skin with vibration, temperature, and electrical stimulation; the three-modal stimulation electronic skin is composed of a vibration motor, a thermoelectric cooler, a stimulation electrode, and an electromyogram electrode array.

[0006] In some exemplary embodiments, the three-modal stimulation electronic skin is used to output vibration stimulation by the vibration motor, output hot and cold stimulation by the thermoelectric cooler, and output electrical stimulation by the electromyogram electrode array under the control of the control circuit.

[0007] In some exemplary embodiments, a method for preparing a tri-modal stimulation electronic skin includes: attaching double-sided tape to the bottom of a mold, attaching a vibration motor and a thermoelectric cooler to the surface of the double-sided tape, and injecting a packaging material into the mold and heating for curing; after the packaging material is cured, removing the double-sided tape from the silicone, forming an electrode circuit by 3D printing or printing, encapsulating the electrode lead circuit and leaving stimulation sites and electromyogram acquisition sites; coating conductive silicone at the stimulation sites and coating low-impedance hydrogel at the electromyogram acquisition sites to complete the preparation of the tri-modal stimulation electronic skin.

[0008] In some exemplary embodiments, the bottoms of the vibration motor, the thermoelectric cooler, the stimulation electrodes, and the electromyogram electrode array are all exposed on the surface of the packaging material; the packaging material is a flexible and stretchable polymer material; the packaging material includes one of polydimethylsiloxane, silicone, and ion elastomer.

[0009] In some exemplary embodiments, the multi-modal synchronous sensing electronic skin is a tri-modal synchronous sensing electronic skin for temperature, resistance, and force sensing; the tri-modal synchronous sensing electronic skin is made of copper paste, silver paste, and graphite.

[0010] In some exemplary embodiments, the multi-modal synchronous sensing electronic skin prepares a pressure sensing unit, a temperature sensing unit, and a resistance sensing unit layer by layer with copper paste, silver paste, and graphite on a flexible substrate to realize the packaging and production of tri-modal sensors per unit area.

[0011] In some exemplary embodiments, a method for preparing a multi-modal synchronous sensing electronic skin includes the following steps: using a 3D printer or template printing to print a silver paste circuit on a bottom flexible substrate to form a graphite sensing layer in the force sensing area, and then coating an encapsulation adhesive on the graphite sensing layer; arranging copper paste temperature sensing lines at the center position of the upper surface of the top flexible substrate, and making resistance measurement contact lines on both sides of the upper surface of the top flexible substrate; arranging symmetric silver paste circuits on the lower surface of the top flexible substrate, bonding and encapsulating the lower surface of the top flexible substrate with the surface of the bottom flexible substrate coated with the encapsulation adhesive, and leading out the sensing unit wires.

[0012] In some exemplary embodiments, the flexible substrate is a flexible polymer; the flexible substrate includes one of a polydimethylsiloxane film, polyethylene terephthalate, thermoplastic polyurethane, and a polyimide film.

[0013] In a second aspect, the present application also provides an application of the multi-modal sensing and feedback stimulation bidirectional electronic skin system as described in the above embodiments in the field of wearable prosthetic hands. The technical solutions provided by the embodiments of the present application have at least the following advantages: An embodiment of the present application provides a two-way electronic skin system with multimodal sensing and feedback stimulation and an application. The system includes: a two-way electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish the correspondence between temperature sensing and the hot and cold stimulation of a thermoelectric cooler, the relationship between force sensing and electrical stimulation, and the correspondence between resistance sensing and the stimulation of a vibration motor; the two-way electronic skin includes a multimodal stimulation electronic skin and a multimodal synchronous sensing electronic skin; when in use, the two-way electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the human body surface to stimulate human touch, and enhances human touch through sensing and feedback stimulation, improving the human-computer interaction ability.

[0014] An embodiment of the present application provides a two-way electronic skin system with multimodal sensing and feedback stimulation. By integrating multimodal sensing and multimodal stimulation into one, it realizes the two-way function of sensing and stimulation; moreover, the prosthetic dexterous hand is integrated with a three-modal synchronous sensing electronic skin to sense external stimuli, especially the two basic units of temperature and force, enabling the prosthetic dexterous hand to have a bionic tactile function and meet the basic daily needs; finally, the three-modal stimulation electronic skin can realize temperature stimulation, vibration stimulation, and electrical stimulation, and through scene simulation, various human-computer interaction scenarios can be realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily illustrated by the pictures in the corresponding drawings. These exemplary illustrations do not limit the embodiments, unless otherwise stated, and the figures in the drawings do not constitute a proportional limitation.

[0016] Figure 1 It is a schematic diagram of the two-way electronic skin worn by a prosthetic dexterous hand for the disabled provided by an embodiment of the present application.

[0017] Figure 2 It is a schematic diagram of the unit structure layout design of the three-modal stimulation electronic skin provided by an embodiment of the present application.

[0018] Figure 3 It is a schematic diagram of the process flow of the preparation method of the three-modal stimulation electronic skin provided by an embodiment of the present application.

[0019] Figure 4 It is a schematic diagram of the prosthetic dexterous finger integrated with the three-modal synchronous sensing electronic skin provided by an embodiment of the present application.

[0020] Figure 5 It is a schematic diagram of the process flow of the preparation method of the three-modal synchronous sensing electronic skin provided by an embodiment of the present application.

[0021] DESCRIPTION OF THE REFERENCE NUMERALS 1. Machine finger, 11. Finger tip, 2. Prosthetic dexterous hand, 3. Control circuit, 4. Conductive, 5. Tristimulus electronic skin, 51. Stimulating electrode, 52. Semiconductor refrigerating sheet, 53. Vibration motor, 54. Electromyogram electrode array, 55. Encapsulation material, 6. Fixing shell of prosthetic dexterous hand, 7. Tristimulus synchronous sensing electronic skin, 71. Two endpoints for resistance measurement, 72. Temperature sensing unit, 73. Pressure sensing unit, 74. Flexible substrate. Detailed implementation mode

[0022] As can be seen from the background art, the existing single-modal and two-modal (pressure and temperature) sensing and feedback stimulation are relatively common, but the multi-modal sensor for detecting more than three kinds of stimulations still faces challenges.

[0023] The related technology provides an array-type flexible electronic skin for robot tactile feedback, including a tactile feedback response system, a plurality of accommodating shells distributed in an array, and a plurality of flexible electronic skin units respectively arranged in the accommodating shells. The plurality of accommodating shells are connected through a splicing part. Each flexible electronic skin unit includes a substrate, an electrochromic pressure display unit located on the substrate, and a triboelectric pressure sensitive unit located on the electrochromic pressure display unit. The tactile feedback response system is respectively connected with the electrochromic pressure display unit and the triboelectric pressure sensitive unit. However, this technology only realizes the tactile response feedback to the robot system and does not further give a stimulation feedback to the human body.

[0024] Another related technology provides a motion electrical stimulation feedback perception system and method for an ontological prosthesis. The motion of the prosthesis is measured by an inertial measurement unit, and the motion information of the prosthesis is transmitted to the main control chip. The main control chip reads and analyzes the motion information of the prosthesis. Through electrical stimulation feedback to the nerve endings of the residual limb, the motion sensory function of the natural limb can be simulated, enabling the amputee to more accurately perceive and control the motion of the prosthesis. However, this technology realizes a single electrical stimulation through inertial measurement feedback and does not have a tactile feedback function.

[0025] In addition, another related technology provides a brain-computer interface method and system for enhancing lower limb motor imagery based on motion illusion. It designs a visual, auditory and motion illusion multi-modal induced lower limb motor imagery paradigm to realize the synchronous operation of electroencephalogram acquisition and multi-modal induction; selects the FBCSP feature extraction algorithm and the SVM classification algorithm to complete the binary classification of electroencephalogram features with small samples and establish a training model to control the FES stimulation feedback in real time. By mechanically vibrating and stimulating the lower limb Achilles tendon position, the lower limb motion illusion is induced to enhance the activation degree of the motor cortex of the brain, improve the lower limb motor imagery ability and the efficiency of lower limb functional rehabilitation training. However, the stimulation of this technology is also a single mechanical vibration, with a single function and unable to realize multi-modal stimulation. But in practice, it is necessary for the multi-modal sensing feedback to correspond to multi-modal stimulations.

[0026] In view of the lack of three - modality perception and stimulation in current electronic skins, an embodiment of the present application provides a two - way electronic skin system and application for multi - modality sensing and feedback stimulation. The system includes: a two - way electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish the corresponding relationships between temperature perception and the hot - cold stimulation of a thermoelectric cooler, force perception and electrical stimulation, and resistance perception and the stimulation of a vibration motor; the two - way electronic skin includes a multi - modality stimulation electronic skin and a multi - modality synchronous perception electronic skin; during use, the two - way electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multi - modality synchronous perception electronic skin to sense external stimuli; the multi - modality stimulation electronic skin is attached to the human body surface to stimulate human touch, and enhances human touch through sensing feedback stimulation, thereby improving the human - machine interaction ability. An embodiment of the present application provides a two - way electronic skin system for multi - modality sensing and feedback stimulation, which feeds back the temperature, resistance, and force of touch perception to the human body in the form of corresponding temperature, vibration, and electrical stimulation, so as to enhance human touch perception and improve the human - machine interaction ability.

[0027] The following will elaborate on each embodiment of the present application with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in each embodiment of the present application, many technical details are provided to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0028] An embodiment of the present application provides a two - way electronic skin system for multi - modality sensing and feedback stimulation, including: a two - way electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish the corresponding relationships between temperature perception and the hot - cold stimulation of a thermoelectric cooler, force perception and electrical stimulation, and resistance perception and the stimulation of a vibration motor; the two - way electronic skin includes a multi - modality stimulation electronic skin and a multi - modality synchronous perception electronic skin; during use, the two - way electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multi - modality synchronous perception electronic skin to sense external stimuli; the multi - modality stimulation electronic skin is attached to the human body surface to stimulate human touch, and enhances human touch through sensing feedback stimulation, thereby improving the human - machine interaction ability. The two - way electronic skin system for multi - modality sensing and feedback stimulation provided by the present application does not involve complex usage operation processes, has strong universality, and shows broad application prospects especially in fields such as prosthetic human - machine interaction.

[0029] The two-way electronic skin system for multi-modal sensing and feedback stimulation provided by this application is used on prosthetic limbs worn by disabled people, and specifically includes the following parts: A person with a missing hand wears a prosthetic hand. A three-modal synchronous sensing electronic skin is integrated on the robotic finger of the prosthetic hand, and a three-modal stimulation electronic skin is integrated on the inner surface of the fixed housing of the prosthetic dexterous hand. Each sensor unit and stimulation unit of the electronic skin are led out through wires and connected to a control circuit. The control circuit is located inside the palm of the prosthetic dexterous hand, which not only controls various movements of the prosthetic dexterous hand but also processes various sensing information.

[0030] See Figure 1 , an embodiment of this application provides a two-way electronic skin system for multi-modal sensing and feedback stimulation, which is applied to a wearable prosthetic hand. The two-way electronic skin is installed on the wearable prosthetic hand. As Figure 1 shown, on the robotic finger 1 of the prosthetic dexterous hand 2, a three-modal synchronous sensing electronic skin is integrated. The control circuit 3 is connected to the two-way electronic skin through a wire 4, and the multi-modal stimulation electronic skin 5 is fixed on the inner surface of the fixed housing 6 of the prosthetic dexterous hand.

[0031] As Figure 2 shown, in some embodiments, the multi-modal stimulation electronic skin 5 is a three-modal stimulation electronic skin with vibration, temperature, and electrical stimulation; the three-modal stimulation electronic skin is composed of a stimulation electrode 51, a thermoelectric cooler 52, a vibration motor 53, and an electromyogram electrode array 54. Among them, the stimulation electrode 51, the thermoelectric cooler 52, the vibration motor 53, and the electromyogram electrode array 54 are encapsulated by a packaging material 55, and the bottoms of the stimulation electrode 51, the thermoelectric cooler 52, the vibration motor 53, and the electromyogram electrode array 54 are all exposed on the surface of the packaging material 55.

[0032] In some embodiments, the three-modal stimulation electronic skin is used to output vibration stimulation by the vibration motor 53, output heat and cold stimulation by the thermoelectric cooler 52, and output electrical stimulation by the electromyogram electrode array 54 under the control of the control circuit.

[0033] As Figure 3 shown, in some embodiments, the preparation method of the three-modal stimulation electronic skin includes the following steps: First, attach double-sided tape to the bottom of the mold; then, attach the vibration motor and the thermoelectric cooler to the surface of the double-sided tape, inject the packaging material into the mold, and heat and cure it; after the packaging material is cured, remove the double-sided tape on the silicone, print or print the electrode circuit through 3D printing, encapsulate the electrode lead circuit and leave out the stimulation site and the electromyogram acquisition site; coat conductive silicone at the stimulation site and coat low-impedance hydrogel at the electromyogram acquisition site to complete the preparation of the three-modal stimulation electronic skin.

[0034] In some embodiments, the bottoms of the vibration motor, the thermoelectric cooler, the stimulation electrode, and the electromyography electrode array are all exposed on the surface of the encapsulating material; the encapsulating material is a flexible and stretchable polymer material; the encapsulating material includes one of polydimethylsiloxane, silica gel, and ionic elastomer.

[0035] In some embodiments, the multi-modal synchronous sensing electronic skin is a three-modal synchronous sensing electronic skin for temperature, resistance, and force sensing; the three-modal synchronous sensing electronic skin is made of copper paste, silver paste, and graphite.

[0036] As Figure 4 shown, in some embodiments, the multi-modal synchronous sensing electronic skin 7 prepares a pressure sensing unit 73, a temperature sensing unit 72, and a resistance sensing unit layer by layer with copper paste, silver paste, and graphite on the flexible substrate 74 to achieve the encapsulation and production of three-modal sensors per unit area. Among them, 71 are the two endpoints for resistance measurement.

[0037] As Figure 5 shown, in some embodiments, the preparation method of the multi-modal synchronous sensing electronic skin includes the following steps: First, use a 3D printer or screen printing to print a silver paste circuit on the bottom flexible substrate to form a graphite sensing layer in the force sensing area, and then coat the graphite sensing layer with encapsulating glue; arrange copper paste temperature sensing lines at the center position on the upper surface of the top flexible substrate, and make resistance measurement contact lines on both sides of the upper surface of the top flexible substrate; arrange symmetric silver paste circuits on the lower surface of the top flexible substrate, bond and encapsulate the lower surface of the top flexible substrate with the surface of the bottom flexible substrate coated with encapsulating glue, and lead out the sensing unit wires.

[0038] In some embodiments, the flexible substrate is a flexible polymer; the flexible substrate includes one of a polydimethylsiloxane film, polyethylene terephthalate, thermoplastic polyurethane, and a polyimide film.

[0039] In the specific implementation process, the bidirectional e-skin for multi-modal stimulation and perception provided by this application includes a three-modal stimulation e-skin and a three-modal synchronous perception e-skin. Among them, the three-modal stimulation e-skin is a three-modal stimulation e-skin with vibration, temperature, and electrical stimulation prepared based on a vibration motor, a thermoelectric cooler, stimulation electrodes, and an electromyogram (EMG) electrode array; the three-modal synchronous perception e-skin is a three-modal synchronous perception e-skin with temperature, resistance, and force perception prepared based on copper paste, silver paste, and graphite; the three-modal stimulation e-skin and the three-modal synchronous perception e-skin are respectively connected to a control circuit through wires to form a bidirectional e-skin system; the bidirectional e-skin can be installed on a wearable prosthetic hand and connected to the control circuit through wires. The control circuit integrates an algorithm processing unit to respectively establish the corresponding relationship between temperature perception and the thermal and cold stimulation of the thermoelectric cooler, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and the stimulation of the vibration motor; the dexterous fingers of the prosthetic integrate the three-modal synchronous perception e-skin to perceive external stimuli, and the three-modal stimulation e-skin is attached to the human body surface to stimulate human touch. By perceiving and feedback stimulating, it enhances human touch and improves the human-computer interaction ability.

[0040] In the three-modal stimulation e-skin, the vibration motor, the thermoelectric cooler, the stimulation electrodes, and the EMG electrode array can be horizontally distributed; the bottoms of the vibration motor, the thermoelectric cooler, the stimulation electrodes, and the EMG electrode array are all exposed on the surface of the encapsulation material; it should be noted that the arrangement and distribution of the components of the three-modal stimulation e-skin can be arranged arbitrarily according to user needs. The three-modal stimulation e-skin fabricated in this application can make the vibration motor output vibration stimulation, the thermoelectric cooler output thermal and cold stimulation, and the EMG electrode array output electrical stimulation under the control of the control circuit. The e-skin encapsulation material is a flexible and stretchable polymer material, such as polydimethylsiloxane, silicone, ion elastomer, etc. The difference between the stimulation electrodes and the EMG electrode array lies in different impedances and functions. The stimulation electrodes are used for electrical stimulation output, and the EMG electrode array is used for collecting EMG signals, which can be used for the recognition of human intentions and the control of the action execution of the prosthetic hand.

[0041] The three-modal synchronous perception electronic skin prepares a pressure perception unit, a temperature perception unit, and a resistance perception unit layer by layer on a flexible substrate with copper paste, silver paste, and graphite, realizing the packaging and production of three-modal sensors per unit area. The bidirectional electronic skin system prepared in this application can not only realize the perception function but also the stimulation function. The electronic skin substrate is prepared with flexible materials and can conformally fit to the human body surface. The embedded perception units and stimulation units are all led out with wires and connected to the control circuit. The control circuit integrates a sensor data acquisition unit, an algorithm processing unit, a stimulation output control unit, and a control unit for the working timing of the prosthetic hand. According to the perception signal and the stimulation module, the corresponding relationships between temperature perception and thermoelectric cooler thermal stimulation, force perception and electrical stimulation, and resistance perception and vibration motor stimulation are established respectively.

[0042] It should be noted that the three-modal synchronous perception electronic skin is installed on the end surface of the dexterous finger of the prosthetic limb. Specifically, it can be installed on a single finger or multiple fingers. When the three-modal synchronous perception electronic skin is installed on multiple fingers, the control circuit uses a neural network algorithm to establish the corresponding relationship between perception and stimulation. Among them, the neural network algorithm includes backpropagation algorithm, convolutional neural network algorithm, recurrent neural network algorithm, Transformer algorithm, etc.

[0043] In some embodiments, the bidirectional electronic skin is installed on a wearable prosthetic dexterous hand. One end of the fixed shell of the prosthetic dexterous hand is connected to the prosthetic robotic hand, and the other end is wrapped around the stump. The three-modal stimulation electronic skin is on the inner surface of the fixed shell of the prosthetic dexterous hand, and the stimulation surface of the three-modal stimulation electronic skin is in contact with the human skin.

[0044] The bidirectional electronic skin system provided in this application is applied to the field of wearable prosthetic hands. The multi-modal stimulation electronic skin is installed on the inner surface of the fixed shell of the wearable prosthetic hand, and the stimulation surface of the multi-modal stimulation electronic skin is in contact with the human skin to stimulate human touch. This application enhances human touch and improves the human-machine interaction ability through perception feedback stimulation. A typical application is that when the prosthetic hand grabs a hot water cup, the tactile sensor detects that the temperature is too high and feeds back that it is not easy to grab due to the overheated temperature of the stump. The stump transmits a stop signal to the control circuit, and the control circuit controls the prosthetic dexterous hand to stop grabbing. When the prosthetic dexterous hand grabs a heavy object, the tactile sensor perceives that the force is too large and the electrical stimulation is strengthened. The stump transmits a stop signal to the control circuit, and the control circuit controls the prosthetic dexterous hand to stop grabbing. When the prosthetic dexterous hand grabs a metal object, the resistance is small, and vibration indicates to the stump that the grabbed object has good conductivity and is not easy to operate in an electrified area.

[0045] According to the above technical solution, the embodiment of the present application provides a two-way electronic skin system with multimodal sensing and feedback stimulation, and the system includes: a two-way electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish the corresponding relationship between temperature sensing and the cold and heat stimulation of a thermoelectric cooler, the relationship between force sensing and electrical stimulation, and the corresponding relationship between resistance sensing and the stimulation of a vibration motor; the two-way electronic skin includes a multimodal stimulation electronic skin and a multimodal synchronous sensing electronic skin; when in use, the two-way electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the human body surface to stimulate human touch, and enhances human touch through sensing feedback stimulation, thereby improving the human-computer interaction ability.

[0046] The embodiment of the present application provides a two-way electronic skin system with multimodal sensing and feedback stimulation. By integrating multimodal sensing and multimodal stimulation into one, the two-way function of sensing and stimulation is realized; moreover, the prosthetic dexterous hand is integrated with a three-modal synchronous sensing electronic skin to sense external stimuli, especially the two basic units of temperature and force, so that the prosthetic dexterous hand has a bionic tactile function and meets the basic daily needs; finally, the three-modal stimulation electronic skin can realize temperature stimulation, vibration stimulation, and electrical stimulation, and through scene simulation, various human-computer interaction scenarios can be realized.

[0047] Those of ordinary skill in the art can understand that the above embodiments are specific embodiments for implementing the present application. In actual applications, various changes can be made to them in form and details without departing from the spirit and scope of the present application. Any person skilled in the art can make their respective changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application should be subject to the scope defined by the claims.

Claims

1. A two-way electronic skin system for multimodal sensing and feedback stimulation, characterized in that, Comprising: A bidirectional electronic skin and a control circuit connected thereto; The control circuit is used to respectively establish the corresponding relationship between temperature perception and the thermal and cold stimulation of a thermoelectric cooler, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and the stimulation of a vibration motor; The bidirectional electronic skin includes a multimodal stimulation electronic skin and a multimodal synchronous perception electronic skin; During use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand integrate the multimodal synchronous perception electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the human body surface to stimulate human touch, and enhances human touch through sensing feedback stimulation, improving the human-computer interaction ability.

2. The bidirectional electronic skin system for multimodal sensing and feedback stimulation according to claim 1, wherein The multimodal stimulation electronic skin is a three-modal stimulation electronic skin with vibration, temperature, and electrical stimulation; The three-modal stimulation electronic skin is composed of a vibration motor, a thermoelectric cooler, a stimulation electrode, and an electromyogram electrode array.

3. The bidirectional electronic skin system for multimodal sensing and feedback stimulation according to claim 2, wherein The three-modal stimulation electronic skin is used to output vibration stimulation by the vibration motor, output thermal and cold stimulation by the thermoelectric cooler, and output electrical stimulation by the electromyogram electrode array under the control of the control circuit.

4. The bidirectional electronic skin system for multimodal sensing and feedback stimulation according to claim 2, wherein The preparation method of the three-modal stimulation electronic skin includes: Attach double-sided tape to the bottom of the mold, attach the vibration motor and the thermoelectric cooler to the surface of the double-sided tape, and inject encapsulating material into the mold and heat-cure it; After the encapsulating material is cured, remove the double-sided tape on the silicone, print the electrode circuit by 3D printing or printing, encapsulate the electrode lead circuit and leave the stimulation site and the electromyogram acquisition site; Coat conductive silicone at the stimulation site and coat low-impedance hydrogel at the electromyogram acquisition site to complete the preparation of the three-modal stimulation electronic skin.

5. The bidirectional electronic skin system for multimodal sensing and feedback stimulation according to claim 4, characterized in that, The bottoms of the vibration motor, the thermoelectric cooler, the stimulation electrode, and the electromyogram electrode array are all exposed on the surface of the encapsulating material; The encapsulating material is a flexible and stretchable polymer material; The encapsulating material includes one of polydimethylsiloxane, silicone, and ion elastomer.

6. The two-way electronic skin system for multimodal sensing and feedback stimulation according to claim 1, characterized in that The multimodal synchronous perception electronic skin is a three-modal synchronous perception electronic skin for temperature, resistance, and force perception; The three-modal synchronous perception electronic skin is made of copper paste, silver paste, and graphite.

7. The two-way electronic skin system for multimodal sensing and feedback stimulation according to claim 6, characterized in that, The multimodal synchronous perception electronic skin prepares a pressure perception unit, a temperature perception unit, and a resistance perception unit layer by layer with copper paste, silver paste, and graphite on a flexible substrate to achieve the packaging and production of three-modal sensors per unit area.

8. The bidirectional electronic skin system for multimodal sensing and feedback stimulation according to claim 1, wherein The preparation method of the multimodal synchronous perception electronic skin includes the following steps: Use a 3D printer or screen printing to print a silver paste circuit on the bottom flexible substrate, form a graphite perception layer in the force perception area, and then coat an encapsulating adhesive on the graphite perception layer; Arrange copper paste temperature sensing lines at the center position of the upward surface of the top flexible substrate, and make resistance measurement contact lines on both sides of the upward surface of the top flexible substrate; Arrange symmetric silver paste circuits on the downward surface of the top flexible substrate, bond and encapsulate the downward surface of the top flexible substrate with the surface of the bottom flexible substrate coated with the encapsulating adhesive, and lead out the sensing unit wires.

9. The bidirectional electronic skin system for multimodal sensing and feedback stimulation according to claim 8, characterized in that, The flexible substrate is a flexible polymer; The flexible substrate includes one of polydimethylsiloxane film, polyethylene terephthalate, thermoplastic polyurethane, and polyimide film.

10. Application of a bidirectional electronic skin system for multimodal sensing and feedback stimulation according to any one of claims 1 to 9 in the field of wearable prosthetic hands.

Images