Flexible electrohaptics immersive interaction method, system, and wearable device
By dynamically adjusting the stimulation waveform and using multi-dimensional electrotactile feedback in VR/AR devices, combined with self-healing coatings and microelectrode arrays, the problem of single tactile feedback in existing technologies is solved, achieving a more refined and comfortable tactile immersion experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2026-02-09
- Publication Date
- 2026-06-02
AI Technical Summary
Existing tactile feedback output methods are limited, resulting in limited fidelity and precision, and failing to provide a near-realistic tactile experience.
By acquiring data streams through a wireless link coupled with VR/AR devices, dynamically adjusting stimulation waveforms, and utilizing a microelectrode array woven from various conductive microfibers, multi-dimensional electrotactile feedback is generated based on virtual scene parameters. Combined with a self-healing coating and detachable mesh encapsulation, flexible electrotactile immersive interaction is achieved.
It achieves a richer and more delicate tactile experience, enhances the user's immersion and interactive realism in the virtual environment, adapts to the individual differences of different users, and provides a comfortable and durable wearing experience.
Smart Images

Figure CN122131913A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of human-computer interaction and wearable device technology, and in particular to a flexible electro-tactile immersive interaction method, system and wearable device. Background Technology
[0002] With the rapid development of technologies such as VR (Virtual Reality) / AR (Augmented Reality) and digital twins, immersive human-computer interaction has become a core direction for breaking through the boundaries of traditional interaction. However, the technical limitations of haptic feedback, as a key support for achieving physical immersion, are becoming increasingly apparent.
[0003] The haptic feedback output methods in related technologies generally use a single vibration source or pressure output. As such, the degree of reproduction and precision are limited. Summary of the Invention
[0004] This application provides an improved flexible electro-tactile immersive interaction method, system, and wearable device.
[0005] This application provides a flexible electrotactile immersive interaction method, including: The data stream of the VR / AR device is acquired via a wireless link coupled to the VR / AR device, and virtual scene parameters are extracted; the data stream includes visual data and auditory data. When it is determined that virtual objects in the virtual space need to output multi-dimensional tactile modalities, the data stream and electro-tactile feedback output are aligned and synchronized, and the stimulation waveform is dynamically adjusted according to the multi-dimensional tactile modalities. Driven by the stimulation waveform, a matching electrotactile feedback output is generated based on the virtual scene parameters; the electrotactile feedback output acts on the final surface of the wearable fabric; the wearable fabric is formed by a microelectrode array, and the microelectrode array is made of a variety of conductive microfibers interwoven together.
[0006] Furthermore, the multi-dimensional tactile modality includes materials; The step of dynamically adjusting the stimulation waveform based on the multi-dimensional tactile modality includes: Determine the first stimulus intensity corresponding to the current material of the current virtual object from the stimulus intensity corresponding to the material learned from history; The stimulation waveform is dynamically adjusted according to the first stimulation intensity.
[0007] Furthermore, the step of matching the virtual scene parameters to generate a matching electrotactile feedback output includes: Based on the virtual scene parameters, find the electrode address of each electrode in the microelectrode array corresponding to the virtual scene parameters; Based on the virtual scene parameters, the phase of the electrode address of each electrode is controlled to adopt a three-dimensional encoding including pulse width, frequency and amplitude in time sequence to simulate different subtype responses of tactile corpuscles and generate an electrotactile feedback output that matches the virtual scene parameters.
[0008] Furthermore, the step of matching the virtual scene parameters to generate a matching electrotactile feedback output includes: When contact is determined to occur in the virtual space based on the virtual scene parameters, the sensor data and virtual scene parameters are integrated to obtain the integrated virtual scene parameters. The integrated virtual scene parameters are mapped to the electrotactile feedback output in real time.
[0009] Furthermore, the microelectrode array composed of multiple interwoven conductive microfibers includes: Multiple conductive microfibers with gradient spacing and preset single electrode diameters are combined with an elastic spandex base fabric through a plain weave process to form the microelectrode array.
[0010] Furthermore, the microelectrode array is a microelectrode array with a gradient distribution in key tactile regions; the key tactile regions include at least one of the torso, limbs, and phalanges; The wearable fabric is formed by a microelectrode array, which is composed of various conductive microfibers interwoven together, including: The microelectrode array is encapsulated using a self-healing coating and a removable mesh to generate the wearable fabric.
[0011] Furthermore, the conductive microfiber comprises gradient micropores ranging from 5 μm to 20 μm; the pore size of the gradient micropores decreases from the outside to the inside.
[0012] Furthermore, the method also includes: If the rate of change of a parameter used to characterize skin comfort is found to meet the cyclic adjustment threshold per unit time, the stimulation parameter is adjusted or the micro-pump of the wearable fabric is activated for cyclic breathability; the comfort parameter includes impedance and / or temperature; the stimulation parameter includes the stimulation waveform; And / or, The data stream includes multimodal data; the method further includes: obtaining multimodal data by receiving multimodal data through a multimodal hot-pluggable first sensing device and / or a second sensing device woven in; the first sensing device includes at least one of an ultrasonic array, a near-field radar module, and a spectral skin sensor; the second sensing device includes an ultrasonic module; the ultrasonic module is woven into the key tactile area of the wearable device using a tunable ultrasonic transducer.
[0013] Furthermore, generating a matching electrotactile feedback output includes periodically inserting a non-stimulation period into the electrotactile feedback output as the final electrotactile feedback output. And / or, The method further includes: rendering the material feel of virtual objects through the microelectrode array in VR mode, and switching between overlaying virtual tactile feedback of the real environment in AR mode; the virtual tactile feedback includes the electrotactile feedback output.
[0014] This application provides a flexible electrotactile immersive interaction system for implementing the flexible electrotactile immersive interaction method as described in any of the preceding claims. The system includes: The signal layer is used to acquire the data stream of the VR / AR device and extract virtual scene parameters via a wireless link coupled to the VR / AR device; the data stream includes visual data and auditory data. The system layer is used to align and synchronize the data stream and electro-haptic feedback output when it is determined that the virtual objects in the virtual space need to output multi-dimensional tactile modalities, and to dynamically adjust the stimulation waveform according to the multi-dimensional tactile modalities. The algorithm layer, driven by the stimulation waveform, matches the virtual scene parameters to generate a matching electrotactile feedback output; the electrotactile feedback output acts on the final surface of the wearable fabric. The physical layer comprises a wearable fabric formed by an array of microelectrodes, which is composed of a variety of conductive microfibers interwoven together.
[0015] This application provides a wearable device for flexible electrotactile immersive interaction, which stores a program that, when executed by a processor, implements the flexible electrotactile immersive interaction method as described in any of the preceding claims.
[0016] Furthermore, the wearable device is connected to the main thread of the wearable fabric via a combination of magnetic attraction and snap fasteners through an external interface of an external device. And / or, The flexible electro-tactile immersive interactive wearable device includes a multimodal sensing fusion interface; wherein, the multimodal sensing fusion interface has an elastic elliptical structure; the multimodal sensing fusion interface receives multimodal data through a multimodal hot-pluggable first sensing device and / or a second sensing device woven in; the first sensing device includes at least one of an ultrasonic array, a near-field radar module, and a spectral skin sensor; the second sensing device includes an ultrasonic module; the ultrasonic module uses a tunable ultrasonic transducer woven into the key tactile area of the wearable device.
[0017] In some embodiments, the flexible electrotactile immersive interaction method of this application dynamically adjusts the stimulation waveform based on multi-dimensional tactile modalities. A wearable fabric formed by a microelectrode array woven from various conductive microfibers allows the final electrotactile feedback output to match the parameters of the virtual scene, achieving a richer and more delicate tactile experience, enabling users to feel a near-realistic touch in a virtual environment. This allows for more refined electrotactile feedback output. Attached Figure Description
[0018] Figure 1 The diagram shown is a flowchart illustrating the flexible electro-tactile immersive interaction method according to an embodiment of this application. Figure 2 As shown Figure 1 A schematic diagram of a wearable fabric for a flexible electro-haptic immersive interaction method; Figure 3a As shown Figure 2 A magnified view of the electrodes of the microelectrode array of the wearable fabric shown. Figure 3b As shown Figure 2 Enlarged view of the hand electrode arrangement of the wearable fabric shown; Figure 4 As shown Figure 1 A schematic diagram of the flexible electro-tactile immersive interaction method shown; Figure 5 The diagram shown is a structural schematic of a flexible electrotactile immersive interactive system according to an embodiment of this application. Detailed Implementation
[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.
[0020] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.
[0021] To address the limitations in the fidelity and precision of related technologies, this application's embodiments utilize a wearable fabric formed by a microelectrode array woven from multiple conductive microfibers. This allows the final electrotactile feedback output to match the virtual scene parameters, resulting in a richer and more nuanced tactile experience, enabling users to perceive near-realistic tactile sensations in a virtual environment. This achieves a more refined electrotactile feedback output.
[0022] Figure 1 The diagram shown is a flowchart of a flexible electrotactile immersive interaction method according to an embodiment of this application.
[0023] like Figure 1 As shown, the flexible electrotactile immersive interaction method may include, but is not limited to, the following steps 110 to 111: Step 110: Obtain the device's data stream and extract virtual scene parameters through the wireless link coupled to the VR / AR device; the data stream includes visual data and auditory data.
[0024] The wireless link coupled to the VR / AR device refers to the link that wirelessly transmits data. This wireless link can capture the user's movement changes in the virtual environment in real time and synchronize the relevant data to the processing unit of this system, thereby ensuring that the haptic feedback is highly consistent with the user's interactive behavior.
[0025] Step 120: If it is determined that the virtual objects in the virtual space need to output multi-dimensional tactile modalities, the data stream and electro-tactile feedback output are aligned and synchronized, and the stimulation waveform is dynamically adjusted according to the multi-dimensional tactile modalities.
[0026] The multi-dimensional tactile modalities are used to describe various tactile features of the virtual object's surface, specifically including multiple perceptual dimensions such as pressure, vibration, roughness, hardness, temperature, and texture. Some of these multi-dimensional tactile modalities, after being parsed by the algorithm, generate corresponding stimulus waveforms to ensure that the output tactile feedback accurately reproduces the physical properties of the virtual object. This ensures that the output electro-tactile feedback precisely matches the characteristics of the virtual object and its interaction with the user. Furthermore, the process of dynamically adjusting the stimulus waveform relies on real-time calculation and optimization, enabling the tactile output to not only match the virtual scene parameters but also respond instantly to the user's interactive behavior. See below for a detailed explanation.
[0027] Step 130: Driven by the stimulation waveform, matching is performed according to the virtual scene parameters to generate a matching electrotactile feedback output; the electrotactile feedback output acts on the final working surface of the wearable fabric; the wearable fabric is formed by a microelectrode array, and the microelectrode array is made of a variety of conductive microfibers interwoven together.
[0028] Figure 2 As shown Figure 1 A schematic diagram of a wearable fabric for a flexible electro-haptic immersive interaction method. Figure 3a As shown Figure 2 A magnified view of the electrodes of the wearable fabric microelectrode array shown.
[0029] like Figure 2 and Figure 3a The final contact surface of the wearable fabric 10 refers to the area in direct contact with the user's skin. This area, through the arrangement of conductive materials and the action of microelectrodes, enables high-resolution tactile feedback transmission. The wearable fabric 10 uses electrical current to stimulate nerve endings in the skin on the contact surface, thereby simulating the tactile characteristics of virtual objects. Furthermore, the final contact surface is flexible, adapting to the curvature changes of different parts of the body, ensuring wearing comfort and consistent feedback. To further improve the accuracy of tactile feedback, each microelectrode on the final contact surface can be independently controlled, thereby achieving differentiated stimulation effects in localized areas.
[0030] The wearable fabric 10 mentioned above may include, but is not limited to, a data acquisition module 12, a data processing module 13, and a safety and comfort closed-loop module 17.
[0031] The data acquisition module 12 is used to acquire the user's physiological signals and environmental information in real time, providing basic data support for subsequent processing. The data processing module 13 analyzes and optimizes the acquired information through built-in algorithms to ensure the accuracy and timeliness of the electrotactile feedback. The safety and comfort closed-loop module 17 is mainly used to monitor the operating status of the entire system and dynamically adjust the output parameters to prevent overstimulation or discomfort. These modules work seamlessly together through an efficient communication protocol, thereby improving overall performance and user experience.
[0032] Continue as Figure 2 and Figure 3a In step 130, the generated stimulation waveform is transmitted to the microelectrode array 11 in the wearable fabric 10, and the electrotactile stimulation is distributed through the interwoven structure of conductive microfibers. The microelectrode array 11 allows for the output of differentiated tactile signals in different areas, thereby simulating complex and diverse tactile experiences. In addition, this wearable fabric 10 is flexible and lightweight, and can provide stable tactile feedback without interfering with the user's natural movements.
[0033] Subsequently, monitoring user physiological feedback and behavioral data further optimizes the accuracy and adaptability of haptic output. By collecting and analyzing data such as user skin conductance and muscle activity, the effectiveness of the current haptic feedback can be determined, and parameters can be automatically adjusted to enhance immersion. This effectively overcomes the problems of monotonous feedback and lack of personalization in traditional haptic technology, providing users with a more realistic interactive experience.
[0034] In this embodiment, the stimulation waveform is dynamically adjusted based on multi-dimensional tactile modalities. The wearable fabric, formed by a microelectrode array woven from various conductive microfibers, ensures that the final electro-tactile feedback output matches the virtual scene parameters, achieving a richer and more nuanced tactile experience, allowing users to feel a near-realistic touch in the virtual environment. This not only enhances the immersiveness of the interaction and enables multi-sensory collaboration in complex scenarios, but also, thanks to the flexibility and adaptability of the wearable fabric, allows the electro-tactile feedback output to be more evenly distributed across the human body surface, further enhancing the user's comfort and experience when using virtual reality and augmented reality.
[0035] Figure 3b As shown Figure 2 A magnified view of the hand electrode arrangement of the wearable fabric shown.
[0036] like Figure 2 , Figure 3a and Figure 3b As shown, wearable fabric can refer to a fully woven flexible electronic bodysuit as the hardware carrier of the system, not covering the entire body, but rather covering key tactile areas. It employs a second skin-like structure composed of multiple interwoven conductive microfibers. Furthermore, the conductive microfibers of the wearable fabric adopt a matrix and conductive bicontinuous phase structure, with a stress dissipation layer at the interface of the conductive microfibers. The surface resistance of the conductive microfibers is less than 5 Ω / cm, and the resistance change rate is less than 10% after 10,000 tensile cycles.
[0037] The matrix refers to the supporting framework of the conductive microfibers, which is the main carrier of the microfibers. It forms a continuous whole with the conductive components, i.e., a bicontinuous phase structure, rather than being scattered and embedded together. It can be imagined that the conductive components are attached to the matrix, and together they form microfibers that are both conductive and flexible.
[0038] The aforementioned matrix not only provides support but also possesses electrical conductivity. This allows it to provide flexible support; as the framework of the microfibers, the matrix exhibits excellent flexibility, enabling it to adapt to the user's body movements, such as stretching and bending, while ensuring comfortable wear without restricting movement. Furthermore, it ensures conductivity by securing the aforementioned conductive components, preventing breakage or decreased conductivity during stretching and bending operations.
[0039] In response, the dual-continuous-phase structure, coupled with a stress-dissipating layer at the interface, gives this structure two main advantages over traditional conductive materials. First, its conductivity hardly diminishes after stretching, making it resistant to breakage and maintaining stable conductivity. Second, the structure balances flexibility and durability. Traditional rigid conductive materials don't conform to the skin and are unsuitable for wear, while ordinary flexible conductive materials lack durability. The structure proposed in this application improves durability, making it adaptable to complex scenarios such as long-term wear and repeated stretching.
[0040] The breathable conductive microfibers achieve one-way moisture conduction (preventing sweat backflow) through 5–20 μm gradient micropores, with a moisture permeability of no less than 2000 g / (m²·24h). The high-density microelectrode array employs a gradient spacing, with a single electrode diameter of 0.5–1.0 mm and a spacing of 1.0–2.0 mm. It is combined with an elastic spandex base fabric using a plain weave process and distributed in key tactile areas such as the torso, limbs, and knuckles. The encapsulation integrates a self-healing coating and a removable mesh. Upon contact with water, the electrode layer softens, allowing for separation and cleaning of the fabric layer. Washing conditions include neutral detergent below 30℃ (machine washing on a gentle cycle in a dedicated laundry bag is acceptable), with a performance retention rate of no less than 85%. The edges are laser-cut with rounded corners (R=0.2 mm) to prevent skin friction damage.
[0041] Compared to conventional robot structures that are overly rigid, relying on heavy gloves or rigid mechanical exoskeletons, resulting in poor comfort, this embodiment employs a "conductive-moisture-permeable dual-function conductive microfiber interwoven structure" for the entire all-fabric flexible bodysuit. Specifically, the conductive microfibers maintain a conductivity of ≥90% at 200% strain and a surface resistance of <5Ω / cm. The moisture-permeable conductive microfibers achieve unidirectional moisture conduction (moisture permeability ≥2000g / (m²·24h)) through gradient micropores (5-20μm). Furthermore, the electrode array is gradient-distributed in key tactile areas such as the torso, limbs, and knuckles, combined with washable encapsulation and a self-healing coating, achieving a balance of comfort, breathability, and durability.
[0042] Furthermore, the signal layer employs a gradient conductive array of conductive microfibers and electrodes to collect information (in conventional scenarios and augmented reality) or is built into the system (in virtual reality scenarios) to perform three-dimensional encoding of pulse width, frequency, and amplitude (pulse width 50-500μs, amplitude 0.1-5mA, frequency 1-200Hz), and superimposes a 100-500kHz high-frequency carrier wave. After further processing, the electro-tactile feedback output is generated, thereby realizing tactile responses such as light touch, vibration, and sustained pressure. In this way, it is possible to perceive fabric material and other factors from multiple dimensions, achieving a more refined electro-tactile feedback output. Moreover, the output includes tactile responses such as light touch, vibration, and sustained pressure, and is not a single vibration source output.
[0043] Furthermore, at the algorithm layer, a cloud-edge collaborative engine is used to map virtual object parameters to electrotactile codes in real time through both physical and data engines. A cross-modal alignment module is employed to eliminate hysteresis artifacts between vision and touch, thereby ensuring that the synchronization error of the three streams is controlled within 20ms. This also improves the precision of the electrotactile feedback output.
[0044] As an optional embodiment of this application, the multi-dimensional tactile modality includes material. Specifically, the multi-dimensional tactile modality includes at least one of pressure, stimulation, temperature, and material texture. Different tactile modalities correspond to different electrotactile encoding methods to achieve accurate simulation of the characteristics of virtual objects. For example, the pressure dimension reflects different degrees of pressure sensation by adjusting the pulse amplitude and frequency; the stimulation dimension uses changes in high-frequency carrier waves to simulate subtle vibrations or point-like tactile sensations; the temperature dimension combines external heat sources with electrical signal feedback to create a realistic experience of hot and cold changes; and the material texture dimension optimizes the gradient conductive array of the signal layer to generate tactile response patterns that match different surface characteristics. Thus, the combination of multi-dimensional tactile modalities not only enhances the user's immersion in the virtual environment but also meets the needs of diverse application scenarios.
[0045] In combination with the above Figure 2 As shown, step 120 above may include, but is not limited to: step 121, determining the first stimulus intensity corresponding to the current material of the current virtual object from the stimulus intensities corresponding to the materials learned in the past.
[0046] By analyzing the correlation between material properties and stimulus intensity in historical learning data through steps 121 above, a mapping relationship between material characteristics and stimulus parameters is established. In this way, the corresponding stimulus intensity value can be quickly matched according to the material characteristics of different virtual objects, thereby ensuring the accuracy and real-time performance of the electrotactile feedback output.
[0047] In practical use, the mapping database between materials and stimulus intensity is automatically updated when a new material is added, ensuring continuous optimization. By introducing machine learning algorithms, the device can dynamically adjust stimulus parameters to adapt to individual differences in tactile perception among different users. These machine learning algorithms include vector machines, neural networks, and deep learning models. Thus, through the analysis of a large amount of historical data, the complex nonlinear relationship between materials and stimulus intensity can be identified, further improving the accuracy of the mapping.
[0048] Step 122: Dynamically adjust the stimulation waveform according to the first stimulation intensity.
[0049] Step 122 further includes selecting a corresponding waveform template based on the numerical range of the first stimulus intensity, and fine-tuning the waveform by combining it with real-time feedback data from the user. During the adjustment process, key parameters such as frequency, amplitude, and duty cycle are comprehensively considered to generate the optimal stimulus waveform scheme.
[0050] Step 122 above further optimizes the adjustment precision of the stimulus waveform by analyzing the material characteristics of the virtual object and combining it with the user's historical interaction data. This not only enhances the realism of the tactile feedback but also generates a more delicate electrotactile output based on different material properties. For example, when the virtual object has a rough surface, the intensity and frequency of the stimulus waveform are automatically increased to simulate the corresponding tactile sensation; while when the material has a smooth surface, the stimulus intensity is reduced to provide a softer feedback experience.
[0051] In this embodiment of the application, by learning the stimulation intensity corresponding to the material in history, it is possible to ensure that the output electrotactile stimulation is more in line with the user's perceptual needs, while improving the realism and comfort of the interactive experience.
[0052] Combination Figure 1 As shown, step 130 above can be performed in at least one of the following optional ways to match the virtual scene parameters and generate a matching electrotactile feedback output: In the first alternative approach, step 130 above involves matching the virtual scene parameters to generate a matching electrotactile feedback output: Step 1311: Based on the virtual scene parameters, find the electrode addresses of each electrode in the microelectrode array corresponding to the virtual scene parameters. This achieves dynamic addressing and hybrid modulation, enabling a "virtual electrode" effect in space through electrode subset switching, and temporally simulating different subtype responses of tactile corpuscles using three-dimensional encoding; a built-in biomimetic waveform library can render delicate tactile sensations such as raindrops falling and cat tongue licking.
[0053] Step 1312: Based on the virtual scene parameters, control the phase of the electrode addresses of each electrode to employ a three-dimensional encoding in timing, including pulse width, frequency, and amplitude, to simulate different subtype responses of the tactile corpuscle, generating an electro-tactile feedback output that matches the virtual scene parameters. With a limited number of electrodes, precise control of different electrodes can achieve a flowing tactile feedback effect. Combined with a built-in biomimetic waveform library, it can render delicate tactile sensations such as raindrops falling or a cat's tongue licking.
[0054] In this embodiment, by using three-dimensional encoding of pulse width, frequency, and amplitude, the subtle differences in different tactile sensations can be simulated more effectively. Thus, by fully utilizing the spatial distribution characteristics of the microelectrode array and combining it with dynamic temporal adjustments, a high degree of fidelity in tactile feedback can be achieved.
[0055] In the second optional approach, in step 1321, when it is determined that contact has occurred in the virtual space based on the virtual scene parameters, the sensor data and the virtual scene parameters are integrated to obtain the integrated virtual scene parameters.
[0056] Sensor data is used to reflect physical contact information during user interaction with the virtual environment. This data includes multi-dimensional metrics such as pressure, speed, and position. By integrating sensor data with virtual scene parameters, the user's real-time operational intentions can be captured more accurately and translated into specific electrotactile feedback commands. This not only enhances the realism of the interaction but also strengthens the user's immersive experience in the virtual environment.
[0057] Step 1322: The integrated virtual scene parameters are mapped to the electrotactile feedback output in real time.
[0058] In this embodiment, the integrated virtual scene parameters are mapped to the electro-haptic feedback output in real time, enabling rapid response to dynamic changes in the virtual scene and ensuring the immediacy and accuracy of the electro-haptic feedback. This not only improves the smoothness of interaction but also enables simultaneous multi-point haptic output in complex virtual environments. Furthermore, the integrated virtual scene parameters are optimized to effectively reduce data redundancy and improve system operating efficiency.
[0059] Combination Figure 2 As shown, the microelectrode array formed by interlacing multiple conductive microfibers includes: combining multiple conductive microfibers with a gradient spacing and a preset single electrode diameter through a plain weave process with an elastic spandex base fabric to form the microelectrode array.
[0060] In this embodiment, the microelectrode array is designed with a balance between flexibility and conductivity. By adjusting the gradient spacing and the diameter of each electrode, wearing comfort can be improved while ensuring tactile feedback sensitivity. The elastic spandex base fabric not only enhances the overall structural ductility but also ensures the stability of electrode performance after repeated stretching and bending.
[0061] As an optional embodiment of this application, the microelectrode array is a gradient-distributed microelectrode array over key tactile regions; the key tactile regions include at least one of the torso, limbs, and knuckles. Thus, the key tactile regions such as the torso, limbs, and knuckles are combined with a washable encapsulation and a self-healing coating.
[0062] The wearable fabric is formed by a microelectrode array, which is composed of various conductive microfibers interwoven together, including: The microelectrode array is encapsulated using a self-healing coating and a removable mesh to create the wearable fabric. The edges of the wearable fabric are laser-cut rounded to prevent skin friction damage.
[0063] Among them, detachable mesh refers to a mesh structure material with separable characteristics, which can facilitate subsequent maintenance or replacement while ensuring the overall sealing effect.
[0064] The aforementioned self-healing coating, through its unique molecular structure, can automatically recover from minor damage, thereby extending the fabric's lifespan and maintaining its functionality. This enhances the device's flexibility and ease of maintenance, allowing users to replace or upgrade specific areas as needed, thus extending the overall lifespan of the device.
[0065] In this embodiment, a dual-functional conductive microfiber and a washable encapsulation are employed, balancing tensile strength and wearing comfort. This addresses the limitation of movement inherent in traditional rigid devices. The washable encapsulation allows for gentle machine washing with neutral detergents at temperatures below 30°C, and can be placed in a dedicated laundry bag. Furthermore, the self-healing coating effectively enhances the durability of the microelectrode array, allowing it to recover some performance even after external damage or prolonged use. This encapsulation method provides excellent breathability and comfort, ensuring that the wearable fabric conforms to the skin without causing significant stuffiness or discomfort, offering users a more natural wearing experience.
[0066] As an optional embodiment of this application, the conductive microfiber comprises gradient micropores ranging from 5 μm to 20 μm; the pore size of the gradient micropores decreases from the outside to the inside. Thus, the outer layer pore size is larger than the inner layer pore size, which facilitates sweat excretion and prevents external sweat backflow.
[0067] Among them, the conductive microfiber maintains a conductivity of ≥90% under 200% strain and has a surface resistance of <5Ω / cm, while the moisture-permeable microfiber achieves unidirectional moisture conduction through gradient micropores (5-20μm) (moisture permeability ≥2000g / (m²·24h)).
[0068] In this embodiment, gradient micropores enhance the performance of conductive microfibers. By optimizing the pore size distribution, they achieve high conductivity in the outer layer while maintaining good mechanical strength and flexibility in the inner layer. This not only improves the signal transmission efficiency of the microelectrode array but also effectively reduces energy loss, thereby improving the overall device's response speed and stability. Furthermore, the presence of gradient micropores provides more contact area for the conductive microfibers.
[0069] In the related technologies, the conductivity of the conductive materials used decreases significantly after stretching, and long-term wear can cause a series of problems such as stuffiness and allergies. Moreover, there is no mention of personalized tactile sensitivity calibration.
[0070] To address the aforementioned issues, this application provides a flexible electrotactile immersive interaction method, continuing as follows: Figure 3a As shown, by monitoring the skin's impedance and temperature in real time, changes such as sudden drops in impedance or increases in temperature can be detected, and the stimulation parameters can be automatically adjusted or the micro-pump 18 of the system can be activated for circulation and ventilation.
[0071] As an optional embodiment of this application, the above method may also include, but is not limited to: if the rate of change of a parameter used to characterize skin comfort is monitored in real time and meets the cyclic adjustment threshold, then the stimulation parameter is adjusted or the micropump 18 of the wearable fabric is activated for cyclic ventilation; the comfort parameter includes impedance and / or temperature; the stimulation parameter includes the stimulation waveform.
[0072] The aforementioned rate of change refers to the change at the current time point exceeding the change at the previous time point. Examples include a sudden drop in impedance or an increase in temperature.
[0073] In this embodiment, by monitoring changes in skin comfort parameters in real time, the user experience can be effectively improved. When the rate of change of parameters such as impedance or temperature per unit time reaches a preset cyclic adjustment threshold, corresponding adjustments are automatically triggered. In this way, not only can the stimulation waveform be dynamically adjusted to adapt to the user's personalized needs, but the micro-pumps inside the wearable fabric can also be activated for cyclic ventilation, thereby alleviating the stuffiness and discomfort that may be caused by prolonged wear.
[0074] Figure 4 As shown Figure 1 The diagram shows a specific schematic of the flexible electro-tactile immersive interaction method.
[0075] like Figure 4 and Figure 3a As shown, the data stream includes multimodal data; the method further includes: obtaining multimodal data by receiving multimodal data through a multimodal hot-pluggable first sensing device and / or a second sensing device woven in; the first sensing device includes at least one of an ultrasonic array, a near-field radar module 15, and a spectral skin sensor; the second sensing device includes an ultrasonic module 14; the ultrasonic module is woven into the key tactile area of the wearable device using a tunable ultrasonic transducer.
[0076] The tunable ultrasonic transducer generates high-frequency vibration signals to simulate different tactile feedbacks. This allows for dynamic adjustment of the output vibration mode and intensity based on user needs or environmental changes, providing a more realistic and nuanced tactile experience.
[0077] The placement and distribution of the aforementioned ultrasound modules ensure maximum coverage in key tactile areas while avoiding interference with the user's normal activities. Thus, the acquisition of multimodal data is not limited to a single perceptual dimension but integrates multiple methods of acquiring multimodal data, improving data accuracy.
[0078] In this embodiment, acquiring the aforementioned diverse multimodal data enables the capture of users' physiological and behavioral information during interaction. This not only enhances the diversity and accuracy of data acquisition but also provides richer input sources for subsequent data processing and analysis. Simultaneously, the tunable ultrasonic transducer can better adapt to different usage scenarios and individual differences. Furthermore, weaving the ultrasonic module into key tactile areas further enhances the device's ability to perceive subtle local changes, achieving more natural and realistic tactile feedback.
[0079] As an optional embodiment of this application, the step of generating a matching electrotactile feedback output includes: periodically inserting a non-stimulation period into the electrotactile feedback output as the final electrotactile feedback output.
[0080] In some examples, the system monitors skin impedance and microtemperature in real time, with impedance measurement ranging from 1kΩ to 100kΩ and temperature measurement accuracy of ±0.1℃. When impedance drops sharply by more than 50% or local temperature exceeds 37℃, the system automatically adjusts stimulation parameters or triggers micro-pump circulation and ventilation within the fabric. Simultaneously, a tactile rest mechanism is implemented, inserting a non-stimulation window every 5 to 10 minutes to prevent adaptive nerve fatigue.
[0081] Furthermore, through learning, personalized calibration and incremental learning can be achieved for different users, thereby dynamically adapting to changes in tactile sensitivity among different individuals.
[0082] In this embodiment, by inserting a period of no stimulation into the electro-haptic feedback output, it is possible to effectively prevent users from developing adaptation or fatigue due to prolonged continuous stimulation. This not only helps maintain the user's sensitivity to haptic feedback but also enhances the comfort and naturalness of the interaction process to some extent.
[0083] Combination Figures 1 to 2 As shown, the above method further includes: rendering the material tactile sensation of virtual objects through the microelectrode array in VR mode, and switching between overlaying virtual tactile feedback of the real environment in AR mode; the virtual tactile feedback includes the electrotactile feedback output.
[0084] In this embodiment, by rendering the tactile texture of virtual objects in VR mode, users can obtain a more realistic immersive experience. Simultaneously, overlaying virtual haptic feedback from the real environment in AR mode not only enhances the realism of the interaction but also provides users with a more natural way of operating.
[0085] Figure 5 The diagram shown is a structural schematic of a flexible electrotactile immersive interactive system according to an embodiment of this application.
[0086] like Figure 5 As shown, this flexible electrotactile immersive interaction system is used to implement the above-mentioned flexible electrotactile immersive interaction method. The system may include the following: Signal layer 31 is used to acquire the data stream of the VR / AR device and extract virtual scene parameters through a wireless link coupled to the VR / AR device; the data stream includes visual data and auditory data; System layer 32 is used to align and synchronize the data stream and electro-haptic feedback output when it is determined that the virtual object in the virtual space needs to output multi-dimensional tactile modalities, and to dynamically adjust the stimulation waveform according to the multi-dimensional tactile modalities. Algorithm layer 33, driven by the stimulation waveform, matches the virtual scene parameters to generate a matching electrotactile feedback output; the electrotactile feedback output acts on the final surface of the wearable fabric. The physical layer 34 is formed by a microelectrode array, which is composed of a variety of conductive microfibers interwoven together.
[0087] Continue as Figure 2 , Figure 3a and Figure 5 As shown, it includes a physical layer of all-fabric flexible electronic bodysuit, a signal layer of distributed neuromorphic driving and acquisition SoC node system, an algorithm layer of cloud-edge collaborative multimodal haptic engine, a system layer of VR / AR multisensory coupling module 17 and multimodal sensing fusion interface 16.
[0088] The related technologies have weak scenario adaptability, have not achieved multi-domain functional integration, and are difficult to support consumer-grade immersive experience, medical-grade assisted perception and industrial-grade skills training at the same time.
[0089] Compared to related technologies, the multimodal sensor fusion interface (an overall elastic elliptical structure distributed at locations such as the clavicle, wristband, and lumbar region) in the flexible electro-tactile immersive interaction method of this application allows for hot-swapping of independent functional modules such as ultrasound and radar modules. Subsequently, through standardized interfaces and SDKs (Software Development Kits), it can be adapted to various application scenarios, including guide vision warning (ultrasound-triggered gradient vibration), industrial training (radar rendering of thermal tingling sensation), and e-commerce tactile interaction (real-time rendering of fabric tactile sensation), thereby achieving multi-purpose functionality for a single garment. This allows for long-term use in multiple scenarios such as guide vision, industrial training, and e-commerce interaction.
[0090] The physical layer is embedded with a high-density microelectrode array and a moisture-permeable conductive microfiber structure.
[0091] The signal layer nodes are networked via a high-speed serial bus with a speed of not less than 50MHz and are connected to the physical layer bus via magnetic attraction and snap-fit.
[0092] The algorithm layer model is deployed at the edge and in the cloud to achieve cross-modal data processing; the system layer connects to the VR / AR headset via a low-latency wireless link with a latency of no more than 20ms; the multimodal sensor fusion interface is distributed in easily accessible areas such as the collarbone, wristband, and lower back of the bodysuit, enabling hot-swappable functional modules. See below for detailed explanation.
[0093] The aforementioned algorithm layer is a cloud-edge collaborative multimodal haptic engine that enables real-time mapping from virtual parameters to haptic codes. It employs a dual-engine driven mode (physical and data dual engines): in the offline stage, finite element simulation generates label pairs containing the geometry, material, dynamic parameters, and electro-haptic codes of virtual objects; in the online stage, a lightweight Transformer distillation model is deployed, with end-to-end latency below 50ms. Personalized calibration is achieved through a meta-learning framework; new users only need 5 minutes of active touch calibration to automatically regress their personal haptic sensitivity curve; the cross-modal alignment module dynamically adjusts the stimulus waveform, eliminating "hysteresis artifacts" caused by the spatiotemporal inconsistency between visual and tactile sensations.
[0094] Each SoC node integrates a high-voltage microcurrent stimulation source, at least 8 channels of sensing interfaces, and an edge AI inference core. It is networked via a high-speed serial bus with a speed of at least 50MHz, with a synchronization error between nodes not exceeding 10μs. The nodes are connected to the fabric busbar using both magnetic and snap-fit connections, allowing for quick assembly and disassembly, and enabling gentle washing. The stimulation source utilizes a three-dimensional encoding method including pulse width, frequency, and amplitude, superimposed with a 100–500kHz high-frequency carrier wave, covering typical tactile responses such as light touch, vibration, and sustained pressure.
[0095] The algorithm layer is mainly used to make decisions and translate instructions, while the SOC system is mainly used to collect data and execute instructions from the algorithm layer. The two do not replace each other. The working principle of the cloud-edge collaborative multimodal haptic engine will be explained in detail below.
[0096] Based on this, the cloud-edge collaborative multimodal model of the algorithm layer takes a pre-trained physics engine and a tactile Transformer as its core. In the offline stage, it generates label pairs of virtual object geometry, material, dynamic parameters and electrotactile codes through finite element simulation. In the online stage, it uses a lightweight distillation model to achieve real-time inverse rendering. The model realizes personalized calibration based on meta-learning, with a new user calibration time of no more than 5 minutes and an incremental learning cycle of no less than 24 hours.
[0097] In response, the work of the cloud-edge collaborative multimodal haptic engine can be mainly divided into three steps: The first step is data preparation before shipment or during system updates, which is similar to establishing a correspondence between some objects and haptic signals in advance, equivalent to preparing a dictionary that records some haptic information. If pre-prepared data is encountered, it can be quickly searched and matched (corresponding to the offline stage).
[0098] The second step is to convert the virtual scene into tactile commands in real time when the user uses it. In this step, the algorithm layer receives two types of data simultaneously. One is the virtual scene parameters obtained from the system layer (VR / AR headset) (such as the material, hardness, shape, temperature, etc. of a cup touched in VR). The other is the real-time data of the user from the signal layer (SOC system) (such as the user's skin impedance and current gesture). After determining the two pieces of information, the pre-trained Transformer distillation model finds the corresponding label pairs from the dictionary matched in the previous step and translates the parameters of the virtual cup into the corresponding point tactile codes in real time (such as a frequency of 200Hz, an amplitude of 8V, and a pulse width of 8ms). Finally, the commands are sent to the SOC system to drive the electrode array of the physical layer to generate tactile sensation (corresponding to the online stage).
[0099] The third step is to use a meta-learning framework to adapt the tactile sensitivity of different users. This step mainly uses information such as the skin's impedance response and subjective feedback after touching to personalize the skin sensitivity of different users (i.e., personal tactile sensitivity curve). In subsequent use, it can also continuously learn incrementally and automatically adjust to ensure that the touch is appropriate every time (corresponding to the personalized calibration stage).
[0100] The system layer is a VR / AR multi-sensory coupling module, enabling multi-sensory collaborative immersion. Furthermore, it couples with the VR / AR headset via a low-latency wireless link with a latency of no more than 20ms, achieving alignment of visual, auditory, and tactile senses and enabling one-click switching between VR / AR modes. VR mode renders purely virtual tactile sensations, while AR mode overlays virtual tactile sensations from the real environment. An integrated elastic capacitor local energy storage module enables 10 minutes of offline operation; in idle state, it switches to sleep mode, where power consumption is below 10μA, and is awakened by a sensor trigger signal (response time less than 5ms). This enables multi-user collaborative interaction; when users make physical contact (such as shaking hands) in virtual space, both users' bodysuits synchronously render tactile sensations in the corresponding areas.
[0101] This application provides a wearable device for flexible electrotactile immersive interaction, which stores a program that, when executed by a processor, implements the flexible electrotactile immersive interaction method as described in any of the above claims.
[0102] In some optional embodiments, the wearable device with flexible electrotactile immersive interaction is connected to the main body of the wearable fabric via a combination of magnetic attraction and snap fasteners through an external interface of an external device. This magnetic attraction and snap fastener structure ensures quick connection while increasing stability, effectively preventing accidental detachment due to movement or external force.
[0103] In some optional embodiments, the wearable device with flexible electrotactile immersive interaction includes a multimodal sensing fusion interface; wherein the multimodal sensing fusion interface has an elastic elliptical structure; the multimodal sensing fusion interface receives multimodal data through a multimodal hot-pluggable first sensing device and / or a second sensing device woven in; the first sensing device includes at least one of an ultrasonic array, a near-field radar module, and a spectral skin sensor; the second sensing device includes an ultrasonic module; the ultrasonic module uses a tunable ultrasonic transducer woven into the key tactile area of the wearable device.
[0104] Standardized expansion slots are flexible oval sockets distributed in easily accessible areas such as the clavicle, scapula, wrist ring, lumbar region, and behind the knee. They integrate power, data, and mechanical positioning interfaces and use a combination of mechanical snap-fit and magnetic attraction for hot-swapping of modules such as ultrasonic arrays and near-field radar modules.
[0105] Scenario-based functional adaptation guides the integration of ultrasound modules in blind scenarios (tunable ultrasound transducers are woven into shoulder, back, and wristbands, with a detection distance of 0.1m to 3m and a resolution of no more than 1cm, with obstacles triggering corresponding skin gradient vibrations; in industrial training, the near-field radar module provides gesture tracking accuracy of no more than 0.5mm, with virtual high-risk areas triggering a thermal tingling sensation; in e-commerce scenarios, the SDK package is open to import material parameters and render the fabric tactile feel in real time).
[0106] Multi-source data fusion refers to the fusion of data such as ultrasound echo and bioimpedance, and the use of algorithms to achieve cross-modal alignment and error self-correction. Fault redundancy refers to the use of a ring bus, where data is automatically redirected when a node fails; when the electrode resistance drifts abnormally, the adjacent backup array is activated and a check is prompted via the APP. In the above system, the four-layer collaborative architecture features structural flexibility, high tactile resolution, and strong scene adaptability, which can improve the efficiency of immersive interaction and assisted perception.
[0107] In the aforementioned AI-driven and multimodal sensing-based flexible electro-tactile immersive interaction system, the algorithm layer achieves personalized and precise tactile feedback through meta-learning and cross-modal calibration. The AI-driven and multimodal sensing-based flexible electro-tactile immersive interaction system provided in this application is an integrated system based on flexible electronics, multimodal sensing, and artificial intelligence, possessing numerous advantages such as seamless wearability, high tactile fidelity, and adaptability to multiple scenarios.
[0108] The flexible electro-tactile immersive interactive system based on AI-driven and multimodal sensing provided in this application can simultaneously support diverse needs such as visually impaired warning, industrial training, and e-commerce interaction through standardized interfaces and scenario-based coding strategies. It improves upon the shortcomings of traditional devices with limited functionality and is suitable for long-term use by different groups of people.
[0109] This application solves the problems of poor breathability and easy nerve fatigue caused by traditional tactile devices by adopting dual-function conductive microfibers and closed-loop safety, through gradient moisture-permeable structure and real-time skin condition monitoring.
[0110] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.
[0111] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element qualified by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A flexible electro-haptic immersive interaction method, characterized in that, include: The data stream of the VR / AR device is acquired and virtual scene parameters are extracted via a wireless link coupled to the VR / AR device. The data stream includes visual data and auditory data; When it is determined that virtual objects in the virtual space need to output multi-dimensional tactile modalities, the data stream and electro-tactile feedback output are aligned and synchronized, and the stimulation waveform is dynamically adjusted according to the multi-dimensional tactile modalities. Driven by the stimulation waveform, a matching electrotactile feedback output is generated based on the virtual scene parameters; the electrotactile feedback output acts on the final surface of the wearable fabric; the wearable fabric is formed by a microelectrode array, and the microelectrode array is made of a variety of conductive microfibers interwoven together.
2. The flexible electro-tactile immersive interaction method as described in claim 1, characterized in that, The multidimensional tactile modality includes materials; The step of dynamically adjusting the stimulation waveform based on the multi-dimensional tactile modality includes: Determine the first stimulus intensity corresponding to the current material of the current virtual object from the stimulus intensity corresponding to the material learned from history; The stimulation waveform is dynamically adjusted according to the first stimulation intensity.
3. The flexible electro-tactile immersive interaction method as described in claim 1, characterized in that, The step of matching based on the virtual scene parameters to generate a matching electrotactile feedback output includes: Based on the virtual scene parameters, find the electrode address of each electrode in the microelectrode array corresponding to the virtual scene parameters; Based on the virtual scene parameters, the phase of the electrode address of each electrode is controlled to adopt a three-dimensional encoding including pulse width, frequency and amplitude in time sequence to simulate different subtype responses of tactile corpuscles and generate an electrotactile feedback output that matches the virtual scene parameters.
4. The flexible electro-tactile immersive interaction method according to any one of claims 1 to 3, characterized in that, The step of matching based on the virtual scene parameters to generate a matching electrotactile feedback output includes: When contact is determined to occur in the virtual space based on the virtual scene parameters, the sensor data and virtual scene parameters are integrated to obtain the integrated virtual scene parameters. The integrated virtual scene parameters are mapped to the electrotactile feedback output in real time.
5. The flexible electrotactile immersive interaction method according to any one of claims 1 to 3, characterized in that, The microelectrode array, composed of various interwoven conductive microfibers, includes: Multiple conductive microfibers with gradient spacing and preset single electrode diameters are combined with an elastic spandex base fabric through a plain weave process to form the microelectrode array.
6. The flexible electro-tactile immersive interaction method as described in claim 5, characterized in that, The microelectrode array is a gradient-distributed microelectrode array located in key tactile regions; the key tactile regions include at least one of the trunk, limbs, and phalanges; The wearable fabric is formed by a microelectrode array, which is composed of various conductive microfibers interwoven together, including: The microelectrode array is encapsulated using a self-healing coating and a removable mesh to generate the wearable fabric.
7. The flexible electro-tactile immersive interaction method as described in claim 6, characterized in that, The conductive microfiber contains gradient micropores ranging from 5 μm to 20 μm; the pore size of the gradient micropores decreases from the outside to the inside.
8. The flexible electro-tactile immersive interaction method according to any one of claims 1 to 3, characterized in that, The method further includes: If the rate of change of a parameter used to characterize skin comfort is found to meet the cyclic adjustment threshold per unit time, the stimulation parameter is adjusted or the micro-pump of the wearable fabric is activated for cyclic breathability; the comfort parameter includes impedance and / or temperature; the stimulation parameter includes the stimulation waveform; And / or, The data stream includes multimodal data; the method further includes: obtaining multimodal data by receiving multimodal data through a multimodal hot-pluggable first sensing device and / or a second sensing device woven in; the first sensing device includes at least one of an ultrasonic array, a near-field radar module, and a spectral skin sensor; the second sensing device includes an ultrasonic module; the ultrasonic module is woven into the key tactile area of the wearable device using a tunable ultrasonic transducer.
9. The flexible electrotactile immersive interaction method according to any one of claims 1 to 3, characterized in that, The process of generating a matching electrotactile feedback output includes periodically inserting a non-stimulation period into the electrotactile feedback output as the final electrotactile feedback output. And / or, The method further includes: rendering the material feel of virtual objects through the microelectrode array in VR mode, and switching between overlaying virtual tactile feedback of the real environment in AR mode; the virtual tactile feedback includes the electro-tactile feedback output.
10. A flexible electrotactile immersive interactive system, characterized in that, The system for implementing the flexible electro-tactile immersive interaction method as described in any one of claims 1 to 9 includes: The signal layer is used to acquire the data stream of the VR / AR device and extract virtual scene parameters via a wireless link coupled to the VR / AR device; the data stream includes visual data and auditory data. The system layer is used to align and synchronize the data stream and electro-haptic feedback output when it is determined that the virtual objects in the virtual space need to output multi-dimensional tactile modalities, and to dynamically adjust the stimulation waveform according to the multi-dimensional tactile modalities. The algorithm layer, driven by the stimulation waveform, matches the virtual scene parameters to generate a matching electrotactile feedback output; the electrotactile feedback output acts on the final surface of the wearable fabric. The physical layer comprises a wearable fabric formed by an array of microelectrodes, which is composed of a variety of conductive microfibers interwoven together.
11. A wearable device for flexible electrotactile immersive interaction, characterized in that, It stores a program that, when executed by a processor, implements the flexible electro-tactile immersive interaction method as described in any one of claims 1 to 9.
12. The wearable device for flexible electrotactile immersive interaction as described in claim 11, characterized in that, The wearable device is connected to the main thread of the wearable fabric via a combination of magnetic attraction and snap fasteners through the external interface of the external device. And / or, The flexible electro-tactile immersive interactive wearable device includes a multimodal sensing fusion interface; wherein, the multimodal sensing fusion interface has an elastic elliptical structure; the multimodal sensing fusion interface receives multimodal data through a multimodal hot-pluggable first sensing device and / or a second sensing device woven in; the first sensing device includes at least one of an ultrasonic array, a near-field radar module, and a spectral skin sensor; the second sensing device includes an ultrasonic module; the ultrasonic module uses a tunable ultrasonic transducer woven into the key tactile area of the wearable device.