A bidirectional electronic skin system with multimodal sensing and feedback stimulation and its application

Abstract

The present application relates to the field of sensor technology, and in particular to a bidirectional electronic skin system and application with multimodal sensing and feedback stimulation, the system comprising: a bidirectional electronic skin and a control circuit connected thereto; the control circuit being used to respectively establish a correspondence between temperature perception and hot and cold stimulation of a semiconductor refrigeration plate, a relationship between force perception and electrical stimulation, and a correspondence between resistance perception and stimulation of a vibration motor; the bidirectional electronic skin comprising a multimodal stimulation electronic skin and a multimodal synchronous sensing electronic skin; the bidirectional electronic skin being installed on a wearable prosthetic hand, the flexible fingers of the wearable prosthetic hand being integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; and the multimodal stimulation electronic skin being attached to the surface of the human body to stimulate the human body's sense of touch. The bidirectional electronic skin system provided in the present application converts the tactilely sensed temperature, resistance, and force into corresponding temperature, vibration, and electrical stimulation and feeds it back to the human body, thereby enhancing the human body's tactile perception and improving the human-computer interaction capability.

Description

Technical Field

[0001] The present application relates to the field of sensor technology, and in particular to a bidirectional electronic skin system and application of multimodal sensing and feedback stimulation. Background Art

[0002] Typically, amputees lack tactile perception and feedback due to the use of prosthetic limbs, making it impossible for them to perform many normal behaviors in their daily lives. Although some researchers have developed some simple-function intelligent prosthetic limbs, which mainly assist amputees in completing simple movements such as grasping and moving by collecting simple electromyographic signals, they are unable to handle a large number of sophisticated movements in daily life. By adding multimodal perception feedback stimulation functions, not only can the human body's tactile function be enhanced through feedback stimulation, but also the human-computer interaction ability can be improved, narrowing the gap between prosthetic limbs and normal limbs. Currently, single-modal and two-modal (pressure and temperature) perception and feedback stimulation are more common, but multimodal sensors still face challenges in detecting more than three stimuli. Summary of the Invention

[0003] The embodiments of the present application provide a bidirectional electronic skin system and application with multimodal sensing and feedback stimulation, which converts the tactilely perceived temperature, resistance, and force into corresponding temperature, vibration, and electrical stimulation and feeds them back to the human body, thereby enhancing the human body's tactile perception and improving human-computer interaction capabilities.

[0004] To solve the above technical problems, in the first aspect, an embodiment of the present application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, which includes: a bidirectional electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish a correspondence between temperature perception and hot and cold stimulation of a semiconductor refrigeration plate, a relationship between force perception and electrical stimulation, and a correspondence between resistance perception and vibration motor stimulation; the bidirectional electronic skin includes multimodal stimulation electronic skin and multimodal synchronous sensing electronic skin; when in use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the surface of the human body to stimulate the human body's sense of touch, enhance the human body's sense of touch through perceptual feedback stimulation, and improve human-computer interaction capabilities.

[0005] In some exemplary embodiments, the multimodal stimulation electronic skin is a trimodal stimulation electronic skin with vibration, temperature, and electrical stimulation; the trimodal stimulation electronic skin is composed of a vibration motor, a semiconductor cooling plate, a stimulation electrode, and an electromyography electrode array.

[0006] In some exemplary embodiments, the trimodal stimulation electronic skin is used to enable the vibration motor to output vibration stimulation, the semiconductor cooling plate to output cold and heat stimulation, and the electromyography electrode array to output electrical stimulation under the control of the control circuit.

[0007] In some exemplary embodiments, the preparation method of tri-modal stimulation electronic skin includes: attaching double-sided tape to the bottom of the mold, attaching the vibration motor and the semiconductor cooling sheet to the surface of the double-sided tape, and injecting packaging material into the mold for heating and curing; after the packaging material is cured, removing the double-sided tape on the silicone, and encapsulating the electrode lead circuit through 3D printing or printing the electrode circuit and leaving the stimulation site and the electromyography collection site; coating the stimulation site with conductive silicone, and coating the electromyography collection site with low impedance hydrogel to complete the preparation of the tri-modal stimulation electronic skin.

[0008] In some exemplary embodiments, the bottoms of the vibration motor, semiconductor cooling plate, stimulation electrode and electromyographic electrode array are all exposed on the surface of the packaging material; the packaging material is a flexible and stretchable polymer material; the packaging material includes one of polydimethylsiloxane, silicone, and ion elastomer.

[0009] In some exemplary embodiments, the multimodal synchronous sensing electronic skin is a trimodal synchronous sensing electronic skin that senses temperature, resistance, and force; the trimodal synchronous sensing electronic skin is made of copper paste, silver paste, and graphite.

[0010] In some exemplary embodiments, the multimodal synchronous sensing electronic skin uses copper paste, silver paste and graphite to prepare pressure sensing units, temperature sensing units and resistance sensing units layer by layer on a flexible substrate, thereby realizing the packaging and production of three-modal sensors per unit area.

[0011] In some exemplary embodiments, a method for preparing a multimodal synchronous sensing electronic skin includes the following steps: printing a silver paste circuit on a bottom flexible substrate using a 3D printer or a template, forming a graphite sensing layer in the force sensing area, and then coating the outside of the graphite sensing layer with packaging glue; arranging a copper paste temperature sensing line at the center position of the upper surface of the top flexible substrate, and making resistance measurement contact lines on both sides of the upper surface of the top flexible substrate; arranging a symmetrical silver paste circuit on the lower surface of the top flexible substrate, bonding and encapsulating the lower surface of the top flexible substrate to the surface of the bottom flexible substrate coated with packaging glue, and leading out the sensing unit wires.

[0012] In some exemplary embodiments, the flexible substrate is a flexible polymer; the flexible substrate includes one of polydimethylsiloxane film, polyethylene terephthalate, thermoplastic polyurethane, and polyimide film.

[0013] In a second aspect, the present application also provides an application of a bidirectional electronic skin system with multimodal sensing and feedback stimulation as described in the above embodiment in the field of wearable prosthetic hands.

[0014] The technical solution provided by the embodiments of the present application has at least the following advantages:

[0015] An embodiment of the present application provides a bidirectional electronic skin system and application with multimodal sensing and feedback stimulation, the system comprising: bidirectional electronic skin and a control circuit connected thereto; the control circuit being used to respectively establish a correspondence between temperature perception and hot and cold stimulation of a semiconductor refrigeration plate, a relationship between force perception and electrical stimulation, and a correspondence between resistance perception and stimulation of a vibration motor; the bidirectional electronic skin comprising multimodal stimulation electronic skin and multimodal synchronous sensing electronic skin; when in use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the surface of the human body to stimulate the human body's sense of touch, and the human body's sense of touch is enhanced through perceptual feedback stimulation, thereby improving human-computer interaction capabilities.

[0016] The embodiment of the present application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, which realizes the bidirectional functions of perception and stimulation through the integration of multimodal perception and multimodal stimulation; moreover, the prosthetic dexterous hand integrates three-modal synchronous perception electronic skin to perceive external stimuli, especially the two basic units of temperature and force, so that the prosthetic dexterous hand has bionic tactile function to meet basic daily needs; finally, the three-modal stimulation electronic skin can realize temperature stimulation, vibration stimulation, and electrical stimulation, and through scene simulation, it can realize a variety of human-computer interaction scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] One or more embodiments are exemplarily described by the pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.

[0018] Figure 1 A schematic diagram of a disabled person's prosthetic dexterous hand wearing a bidirectional electronic skin provided in one embodiment of the present application.

[0019] Figure 2 A schematic diagram of the unit structure layout design of the tri-modal stimulation electronic skin provided in one embodiment of the present application.

[0020] Figure 3 A schematic flow chart of a method for preparing a tri-modal stimulation electronic skin according to an embodiment of the present application.

[0021] Figure 4 Schematic diagram of a prosthetic dexterous finger integrated with tri-modal synchronous sensing electronic skin provided in one embodiment of the present application.

[0022] Figure 5 A schematic flow chart of a method for preparing a tri-modal synchronous sensing electronic skin according to an embodiment of the present application.

[0023] Description of reference numerals:

[0024] 1. Robotic finger, 11. Finger tip, 2. Prosthetic dexterous hand, 3. Control circuit, 4. Conductivity, 5. Trimodal stimulation electronic skin, 51. Stimulation electrode, 52. Semiconductor cooling chip, 53. Vibration motor, 54. Myoelectric electrode array, 55. Packaging material, 6. Prosthetic dexterous hand fixed shell, 7. Trimodal synchronous sensing electronic skin, 71. Two endpoints of resistance measurement, 72. Temperature sensing unit, 73. Pressure sensing unit, 74. Flexible substrate. DETAILED DESCRIPTION

[0025] As can be seen from the background technology, existing single-modal and two-modal (pressure and temperature) perception and feedback stimulation are relatively common, but multimodal sensors still face challenges in detecting more than three stimuli.

[0026] Related technology provides an array-type flexible electronic skin for tactile feedback in robots. The system includes a tactile feedback response system, multiple housings arranged in an array, and multiple flexible electronic skin units located within the housings. The housings are connected by joints. Each flexible electronic skin unit includes a substrate, an electrochromic pressure display unit located on the substrate, and a triboelectric pressure-sensitive unit located on the electrochromic pressure display unit. The tactile feedback response system is connected to the electrochromic pressure display unit and the triboelectric pressure-sensitive unit, respectively. However, this technology only provides tactile feedback to the robot system and does not provide further stimulation feedback to the human body.

[0027] Another related technology provides a motion-sensing electrical stimulation feedback system and method for proprioceptive prostheses. This system measures prosthetic limb motion using an inertial measurement unit (IMU) and transmits this information to a main control chip, which reads and analyzes it. By delivering electrical stimulation feedback to the nerve endings of the residual limb, it can simulate the motor sensory function of a natural limb, enabling amputees to more accurately perceive and control prosthetic movement. However, this technology uses only inertial measurement feedback and lacks tactile feedback.

[0028] In addition, a related technology provides a brain-computer interface method and system for enhancing lower limb motor imagery based on motion illusion. It designs a multimodal lower limb motor imagery induction paradigm using vision, hearing, and motion illusion, achieving synchronous EEG acquisition and multimodal induction. It selects the FBCSP feature extraction algorithm and SVM classification algorithm to complete binary classification of a small sample of EEG features and establish a training model to control FES stimulation feedback in real time. Mechanical vibration stimulation of the Achilles tendon of the lower limb induces lower limb motor illusion, enhances the activation level of the brain's motor cortex, improves lower limb motor imagery ability and the efficiency of lower limb functional rehabilitation training. However, the stimulation of this technology is also a single mechanical vibration with a single function, and it cannot achieve multimodal stimulation. However, in practice, multimodal sensor feedback must correspond to multimodal stimulation.

[0029] In view of the fact that current electronic skin lacks trimodal perception and stimulation, the embodiment of the present application provides a bidirectional electronic skin system and application with multimodal sensing and feedback stimulation, which includes: a bidirectional electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish the corresponding relationship between temperature perception and hot and cold stimulation of semiconductor refrigeration plates, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and vibration motor stimulation; the bidirectional electronic skin includes multimodal stimulation electronic skin and multimodal synchronous perception electronic skin; when in use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with multimodal synchronous perception electronic skin to perceive external stimuli; the multimodal stimulation electronic skin is attached to the surface of the human body to stimulate the human body's sense of touch, enhance the human body's sense of touch through perception feedback stimulation, and improve human-computer interaction capabilities. The embodiment of the present application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, which converts the tactile perception of temperature, resistance, and force into corresponding temperature, vibration, and electrical stimulation and feeds it back to the human body, so as to enhance the human body's tactile perception and improve human-computer interaction capabilities.

[0030] The following detailed description of the various embodiments of the present application is provided in conjunction with the accompanying drawings. However, those skilled in the art will appreciate that many technical details are provided in the various embodiments of the present application to facilitate a better understanding of the present application. However, even without these technical details and the various variations and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0031] The embodiment of the present application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, including: a bidirectional electronic skin and a control circuit connected thereto; the control circuit is used to respectively establish a correspondence between temperature perception and hot and cold stimulation of a semiconductor refrigeration plate, a relationship between force perception and electrical stimulation, and a correspondence between resistance perception and stimulation of a vibration motor; the bidirectional electronic skin includes a multimodal stimulation electronic skin and a multimodal synchronous sensing electronic skin; when in use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the surface of the human body to stimulate the human body's sense of touch, enhance the human body's sense of touch through perceptual feedback stimulation, and improve the human-computer interaction ability. The bidirectional electronic skin system with multimodal sensing and feedback stimulation provided by the present application does not involve a complicated operating procedure and has extremely strong universality. It shows broad application prospects in the fields of prosthetic human-computer interaction, etc.

[0032] This application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, designed for use on prosthetic limbs worn by disabled individuals. Specifically, it comprises a prosthetic hand worn by individuals with hand loss, a trimodal synchronous sensing electronic skin integrated into the robotic fingers of the prosthetic hand, and a trimodal stimulation electronic skin integrated into the inner surface of the fixed housing of the prosthetic hand. Wires connect the sensor and stimulation units of the electronic skin to a control circuit. This control circuit, located within the palm of the prosthetic hand, not only controls the various movements of the prosthetic hand but also processes various sensory information.

[0033] See Figure 1 The embodiment of the present application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, which is applied to a wearable prosthetic hand. The bidirectional electronic skin is installed on the wearable prosthetic hand, such as Figure 1 As shown, the robotic finger 1 of the prosthetic dexterous hand 2 is integrated with a trimodal synchronous sensing electronic skin, the control circuit 3 is connected to the bidirectional electronic skin through a wire 4, and the multimodal stimulation electronic skin 5 is fixed on the inner surface of the prosthetic dexterous hand fixed shell 6.

[0034] like Figure 2 As shown, in some embodiments, the multimodal stimulation electronic skin 5 is a trimodal stimulation electronic skin with vibration, temperature and electrical stimulation; the trimodal stimulation electronic skin is composed of a stimulation electrode 51, a semiconductor cooling plate 52, a vibration motor 53 and an electromyographic electrode array 54, wherein the stimulation electrode 51, the semiconductor cooling plate 52, the vibration motor 53 and the electromyographic electrode array 54 are encapsulated by a packaging material 55, and the bottom of the stimulation electrode 51, the semiconductor cooling plate 52, the vibration motor 53 and the electromyographic electrode array 54 are all exposed on the surface of the packaging material 55.

[0035] In some embodiments, the trimodal stimulation electronic skin is used to enable the vibration motor 53 to output vibration stimulation, the semiconductor cooling plate 52 to output cold and heat stimulation, and the electromyography electrode array 54 to output electrical stimulation under the control of the control circuit.

[0036] like Figure 3 As shown, in some embodiments, the preparation method of trimodal stimulation electronic skin includes the following steps: first, attaching double-sided tape to the bottom of the mold; then, attaching the vibration motor and the semiconductor cooling sheet to the surface of the double-sided tape, and injecting the packaging material into the mold, and heating and curing; after the packaging material is cured, the double-sided tape on the silicone is removed, and the electrode circuit is encapsulated by 3D printing or printing, and the stimulation site and the electromyography collection site are reserved; conductive silicone is coated on the stimulation site, and low-impedance hydrogel is coated on the electromyography collection site to complete the preparation of the trimodal stimulation electronic skin.

[0037] In some embodiments, the bottoms of the vibration motor, semiconductor cooling plate, stimulation electrode and electromyography electrode array are all exposed on the surface of the packaging material; the packaging material is a flexible and stretchable polymer material; the packaging material includes one of polydimethylsiloxane, silicone, and ion elastomer.

[0038] In some embodiments, the multimodal synchronous sensing electronic skin is a trimodal synchronous sensing electronic skin that senses temperature, resistance, and force; the trimodal synchronous sensing electronic skin is made of copper paste, silver paste, and graphite.

[0039] like Figure 4 As shown, in some embodiments, multimodal simultaneous sensing electronic skin 7 uses copper paste, silver paste, and graphite to layer-by-layer fabricate pressure sensing units 73, temperature sensing units 72, and resistance sensing units on a flexible substrate 74, achieving the packaging of a trimodal sensor per unit area. Reference numeral 71 represents the two endpoints of resistance measurement.

[0040] like Figure 5 As shown, in some embodiments, a method for preparing a multimodal synchronous sensing electronic skin includes the following steps: first, a silver paste circuit is printed on a bottom flexible substrate using a 3D printer or a template to form a graphite sensing layer in the force sensing area, and then packaging glue is coated on the outside of the graphite sensing layer; a copper paste temperature sensing line is arranged at the center position of the upper surface of the top flexible substrate, and resistance measurement contact lines are made on both sides of the upper surface of the top flexible substrate; a symmetrical silver paste circuit is arranged on the lower surface of the top flexible substrate, the lower surface of the top flexible substrate is bonded and packaged with the surface of the bottom flexible substrate coated with packaging glue, and the sensing unit wires are led out.

[0041] In some embodiments, the flexible substrate is a flexible polymer; the flexible substrate includes one of polydimethylsiloxane film, polyethylene terephthalate, thermoplastic polyurethane, and polyimide film.

[0042] In the specific implementation process, the bidirectional electronic skin with multimodal stimulation and perception provided by the present application includes trimodal stimulation electronic skin and trimodal synchronous perception electronic skin, wherein the trimodal stimulation electronic skin is a trimodal stimulation electronic skin with vibration, temperature and electrical stimulation made based on a vibration motor, a semiconductor cooling plate, a stimulation electrode and an electromyography electrode array; the trimodal synchronous perception electronic skin is a trimodal synchronous perception electronic skin with temperature, resistance and force perception made based on copper paste, silver paste and graphite; the trimodal stimulation electronic skin is connected to the trimodal synchronous perception electronic skin. The two-way electronic skin is connected to the control circuit through wires to form a bidirectional electronic skin system. The bidirectional electronic skin can be installed on the wearable prosthetic hand and connected to the control circuit through wires. The control circuit integrates an algorithm processing unit to establish the corresponding relationship between temperature perception and hot and cold stimulation of the semiconductor refrigeration plate, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and vibration motor stimulation. The dexterous fingers of the prosthetic limb integrate three-modal synchronous sensing electronic skin to perceive external stimuli. The three-modal stimulation electronic skin is attached to the surface of the human body to stimulate the human body's sense of touch, and the human body's sense of touch is enhanced through sensory feedback stimulation, thereby improving human-computer interaction capabilities.

[0043] The vibration motor, semiconductor cooling plate, stimulation electrode and myoelectric electrode array in the trimodal stimulation electronic skin can be distributed horizontally; the bottoms of the vibration motor, semiconductor cooling plate, stimulation electrode and myoelectric electrode array are all exposed on the surface of the packaging material; it should be noted that the arrangement and distribution of the various components of the trimodal stimulation electronic skin can be arbitrarily arranged according to user needs. The trimodal stimulation electronic skin produced in this application can enable the vibration motor to output vibration stimulation, the semiconductor cooling plate to output hot and cold stimulation, and the myoelectric electrode array to output electrical stimulation under the control of the control circuit. The electronic skin packaging material is a flexible and stretchable polymer material, such as polydimethylsiloxane, silicone, ion elastomer, etc. The difference between the stimulation electrode and the myoelectric electrode array is that the impedance is different and the function is different. The stimulation electrode is used for electrical stimulation output, and the myoelectric electrode array is used for collecting myoelectric signals, which can be used to identify human intentions and control the execution of prosthetic hand movements.

[0044] The trimodal synchronous sensing electronic skin uses copper paste, silver paste and graphite to prepare pressure sensing units, temperature sensing units and resistance sensing units layer by layer on a flexible substrate, realizing the packaging and production of trimodal sensors per unit area. The bidirectional electronic skin system prepared by the present application can not only realize the sensing function, but also the stimulation function. The electronic skin substrate is made of flexible materials and can conform to the surface of the human body. The embedded sensing units and stimulation units are both connected to the control circuit with wires. The control circuit integrates a sensor data acquisition unit, an algorithm processing unit, a stimulation output control unit and a control unit for the working sequence of the prosthetic hand; according to the sensing signal and the stimulation module, the corresponding relationship between temperature perception and hot and cold stimulation of the semiconductor refrigeration plate, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and vibration motor stimulation are established respectively.

[0045] It should be noted that the end surface of the prosthetic dexterous finger is installed with a trimodal synchronous sensing electronic skin, which can be installed on a single finger or on multiple fingers; when the trimodal synchronous sensing electronic skin is installed on multiple fingers, the control circuit uses a neural network algorithm to establish a corresponding relationship between perception and stimulation, wherein the neural network algorithm includes backpropagation algorithm, convolutional neural network algorithm, recurrent neural network algorithm, Transformer algorithm, etc.

[0046] In some embodiments, the bidirectional electronic skin is installed on a wearable prosthetic dexterous hand. One end of the prosthetic dexterous hand fixed shell is connected to the prosthetic robotic hand, and the other end is wrapped around the residual limb. The inner surface of the prosthetic dexterous hand fixed shell is a trimodal stimulation electronic skin, and the stimulation surface of the trimodal stimulation electronic skin is in contact with the human skin.

[0047] The bidirectional electronic skin system provided by the present application is applied in the field of wearable prosthetic hands. The multimodal stimulation electronic skin is installed on the inner surface of the fixed shell of the wearable prosthetic hand, and the stimulation surface of the multimodal stimulation electronic skin contacts the human skin to stimulate the human sense of touch. The present application enhances the human sense of touch and improves the human-computer interaction ability through sensory feedback stimulation. Its typical application is that when the prosthetic hand grasps a hot water cup, the tactile sensor detects that the temperature is too hot, and the feedback is given to the residual limb that the temperature is too hot and it is difficult to grasp. The residual limb transmits a stop signal to the control circuit to control the prosthetic dexterous hand to stop grasping; when the prosthetic dexterous hand grasps a heavy object, the tactile sensor's perception is too strong, the electrical stimulation is strengthened, the residual limb transmits a stop signal to the control circuit to control the prosthetic dexterous hand to stop grasping; when the prosthetic dexterous hand grasps metal, the resistance is small, and the vibration prompts the residual limb to grasp the object. The conductivity is good and it is not easy to operate in an electric area.

[0048] Based on the above technical solution, the embodiment of the present application provides a bidirectional electronic skin system and application with multimodal sensing and feedback stimulation, which includes: bidirectional electronic skin and a control circuit connected to it; the control circuit is used to establish the corresponding relationship between temperature perception and hot and cold stimulation of semiconductor refrigeration plates, the relationship between force perception and electrical stimulation, and the corresponding relationship between resistance perception and vibration motor stimulation; the bidirectional electronic skin includes multimodal stimulation electronic skin and multimodal synchronous sensing electronic skin; when in use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the flexible fingers of the wearable prosthetic hand are integrated with multimodal synchronous sensing electronic skin to perceive external stimuli; the multimodal stimulation electronic skin is attached to the human body surface to stimulate the human body's tactile sense, enhance the human body's tactile sense through perceptual feedback stimulation, and improve human-computer interaction capabilities.

[0049] The embodiment of the present application provides a bidirectional electronic skin system with multimodal sensing and feedback stimulation, which realizes the bidirectional functions of perception and stimulation through the integration of multimodal perception and multimodal stimulation; moreover, the prosthetic dexterous hand integrates three-modal synchronous perception electronic skin to perceive external stimuli, especially the two basic units of temperature and force, so that the prosthetic dexterous hand has bionic tactile function to meet basic daily needs; finally, the three-modal stimulation electronic skin can realize temperature stimulation, vibration stimulation, and electrical stimulation, and through scene simulation, it can realize a variety of human-computer interaction scenarios.

[0050] Those skilled in the art will appreciate that the above-described embodiments are specific examples for implementing the present application, and that in actual applications, various changes in form and detail may be made thereto without departing from the spirit and scope of the present application. Any person skilled in the art may make changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined in the claims.

Claims

1. A bidirectional electronic skin system with multimodal sensing and feedback stimulation, characterized in that: include: Bidirectional electronic skin and a control circuit connected thereto; The bidirectional electronic skin includes multimodal stimulation electronic skin and multimodal synchronous perception electronic skin; When in use, the bidirectional electronic skin is installed on a wearable prosthetic hand, and the fingers of the wearable prosthetic hand are integrated with the multimodal synchronous sensing electronic skin to sense external stimuli; the multimodal stimulation electronic skin is attached to the surface of the human residual limb to stimulate the human sense of touch, and the human sense of touch is enhanced through sensory feedback stimulation, thereby improving human-computer interaction capabilities; The multimodal stimulation electronic skin is a trimodal stimulation electronic skin with vibration, temperature, and electrical stimulation; the trimodal stimulation electronic skin is composed of a vibration motor, a semiconductor cooling sheet, a stimulation electrode, and an electromyographic electrode array; double-sided tape is affixed to the bottom of a mold, the vibration motor and the semiconductor cooling sheet are affixed to the surface of the double-sided tape, and packaging material is injected into the mold and heated and cured; after the packaging material is cured, the double-sided tape on the mold is removed, and the electrode circuit is encapsulated by 3D printing or printing, leaving stimulation sites and electromyographic collection sites; Conductive silicone is coated on the stimulation site, and low-impedance hydrogel is coated on the electromyography collection site to complete the preparation of trimodal stimulation electronic skin; The multimodal synchronous sensing electronic skin is a trimodal synchronous sensing electronic skin that senses temperature, resistance, and force. A silver paste circuit is printed on the bottom flexible substrate using a 3D printer or template, a graphite sensing layer is formed in the force sensing area, and then a packaging glue is coated on the outside of the graphite sensing layer. A copper paste temperature sensing line is arranged at the center position of the upper surface of the top flexible substrate, and resistance measurement contact lines are made on both sides of the upper surface of the top flexible substrate. A symmetrical silver paste circuit is arranged on the lower surface of the top flexible substrate, and the lower surface of the top flexible substrate is bonded and encapsulated to the surface of the bottom flexible substrate coated with the packaging glue, and the sensing unit wires are led out to complete the preparation of the trimodal synchronous sensing electronic skin. The control circuit establishes corresponding relationships between temperature perception and hot and cold stimulation of semiconductor refrigeration plates, force perception and electrical stimulation, and resistance perception and vibration motor stimulation. The myoelectric electrode array collects myoelectric signals for controlling the movement of the wearable prosthetic hand.

2. The bidirectional electronic skin system with multimodal sensing and feedback stimulation according to claim 1, characterized in that: The trimodal stimulation electronic skin is used to enable the vibration motor to output vibration stimulation, the semiconductor cooling plate to output cold and heat stimulation, and the stimulation electrode to output electrical stimulation under the control of the control circuit.

3. The bidirectional electronic skin system with multimodal sensing and feedback stimulation according to claim 1, characterized in that: The bottoms of the vibration motor, the semiconductor cooling plate, the stimulation electrode and the myoelectric electrode array are all exposed on the surface of the packaging material; The packaging material is a flexible and stretchable polymer material; The packaging material includes one of polydimethylsiloxane, silica gel, and ion elastomer.

4. The bidirectional electronic skin system with multimodal sensing and feedback stimulation according to claim 1, characterized in that: The flexible substrate is a flexible polymer; The flexible substrate includes one of polydimethylsiloxane film, polyethylene terephthalate, thermoplastic polyurethane, and polyimide film.

5. A wearable prosthetic hand, characterized in that: A bidirectional electronic skin system comprising multimodal sensing and feedback stimulation according to any one of claims 1 to 4.

Images