Multi-mode tactile feedback method and device based on focused ultrasound and electric combined stimulation

Abstract

The invention discloses a multi-mode tactile feedback method and device based on focused ultrasound and electric combined stimulation, and the method comprises the steps: obtaining a tactile demand of a target object, and determining a to-be-stimulated target part of the target object according to the tactile demand; the frequency, the power and the pulse width of focused ultrasound are adjusted according to tactile requirements, target focused ultrasound is obtained, and the target focused ultrasound acts on deep nerves and muscles of a target part and acts on superficial nerve endings of the target part synchronously or asynchronously through an electrically stimulated surface electrode, so that multi-mode tactile simulation is achieved. And a multi-mode touch simulation result is obtained. According to the invention, the high temporal-spatial resolution and non-invasive deep stimulation characteristics of focused ultrasound and the superficial layer characteristics of electrical stimulation in tactile perception are utilized to realize the triggering of multi-modal sensation, and the invention aims to provide rich, accurate and diversified tactile feedback through the organic combination of two stimulation modes to meet the rehabilitation requirements of patients with sensory disorder.

Description

Technical Field

[0001] The present invention relates to the technical field of tactile feedback, and particularly to a multimodal tactile feedback method, device, terminal and computer-readable storage medium based on combined focused ultrasound and electrical stimulation. Background Art

[0002] With the wide application of tactile feedback technology in fields such as medical rehabilitation, virtual reality (VR), augmented reality (AR), and bionic technology, the research on realizing tactile perception through external stimulation has received increasing attention. The core of tactile stimulation technology lies in applying physical or electrical signals to the skin or nervous system to activate tactile receptors and simulate the natural tactile experience of humans. This technology not only provides a new means of rehabilitation treatment for people with sensory impairments, but also shows great potential in enhancing the immersion and interactivity of virtual environments.

[0003] Tactile stimulation technology is usually based on physical means such as electricity, mechanics, heat, and ultrasound. However, due to the limitations of physical mechanisms and device characteristics, existing single-modal tactile stimulation technologies have certain deficiencies in aspects such as spatial resolution, richness of stimulation types, and depth control ability, making it difficult to achieve natural and accurate tactile feedback.

[0004] Mechanical stimulation applies physical pressure or vibration to the skin surface through a vibration motor or piezoelectric element, which is suitable for simulating surface pressure and texture sensations. However, the stimulation range is limited to the skin surface and cannot act on deep tissues, and it also shows deficiencies in tactile resolution and complex feedback experiences. Thermal stimulation simulates the sense of warmth by heating or cooling the skin. Although it can provide a unique temperature perception, the response speed is slow, making it difficult to achieve a dynamic tactile experience, and there is a risk of user discomfort caused by overheating or overcooling.

[0005] In contrast, electrical stimulation can better simulate surface touch such as pressure, vibration and texture by applying microcurrent to the surface of the skin to activate nerve fibers. The equipment is simple and the technology is mature, but due to the diffusivity of current in the human body, its spatial resolution and tactile accuracy are limited, and the bandwidth of stimulation is insufficient, making it difficult to simulate complex tactile experiences. At the same time, electrical stimulation is difficult to act on deep tissues and it is difficult to meet the rehabilitation needs of some deep sensory disorders. Ultrasound stimulation technology mainly relies on focused mechanical waves to produce local thermal effects in the target tissue. This effect can induce a mechanical response of the tissue, thereby effectively activating deep nerves or muscles and providing accurate deep tactile feedback. Compared with traditional electrical stimulation technology, ultrasound stimulation can provide higher spatial resolution in a non-contact manner and simulate deep tactile perceptions such as pain and heat. However, ultrasound technology also faces some challenges, such as large and expensive equipment, and complex parameter adjustment is required in practical applications. At the same time, when ultrasound technology is used alone, it is difficult to effectively simulate the diversity of surface touch, and focused ultrasound stimulation is difficult and costly when combined with multiple probes for synchronous stimulation, which increases the difficulty of application.

[0006] The current single-modality tactile stimulation technology is difficult to meet the needs of multimodal tactile in complex scenarios in practical applications. Mechanical stimulation and thermal stimulation are limited to surface perception and cannot provide deep stimulation and a rich variety of tactile sensations. Although electrical stimulation and focused ultrasound have unique advantages in tactile feedback, they have obvious shortcomings when used alone. For example, the resolution and depth control capabilities of electrical stimulation are insufficient, and focused ultrasound is not natural enough in simulating surface tactile sensations such as pressure and vibration. Especially in the field of medical rehabilitation for people with sensory impairments, these technologies are not yet sufficient to meet the needs of comprehensive reconstruction of deep and surface sensations at the same time. In addition, most current tactile feedback devices are single-modal designs, lacking effective solutions for multimodal integration, making it difficult to build a more realistic and natural tactile experience.

[0007] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0008] The main purpose of the present invention is to provide a multimodal tactile feedback method, device, terminal and computer-readable storage medium based on focused ultrasound and electrical combined stimulation, aiming to solve the problem that the current single-modal tactile stimulation technology in the prior art is difficult to meet the needs of multimodal tactile in complex scenarios in practical applications.

[0009] To achieve the above object, the present invention provides a multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation, the multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation comprising the following steps: Acquiring a tactile requirement of a target object, and determining a target part of the target object to be stimulated according to the tactile requirement; Adjust the frequency, power, and pulse width of the focused ultrasound according to the tactile requirements to obtain the target focused ultrasound. Apply the target focused ultrasound to the deep nerves and muscles of the target site, and synchronously or asynchronously apply surface electrodes for electrical stimulation to the superficial nerve endings of the target site to achieve multimodal tactile simulation and obtain the multimodal tactile simulation result.

[0010] Optionally, in the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation, the sensations generated by the target focused ultrasound include pain, heat, and deep touch; the tactile feedback generated by the electrical stimulation includes a sense of pressure, a light touch, a vibration sense, and a texture sense.

[0011] Optionally, in the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation, the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation further includes: When performing multimodal tactile simulation through focused ultrasound stimulation and electrical stimulation, control the stimulation method through time regulation or spatial regulation.

[0012] Optionally, in the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation, the time regulation specifically includes: Generate coordinated or independent sensations at the same or different locations through synchronous stimulation control or asynchronous stimulation control; The synchronous stimulation control means performing focused ultrasound stimulation and electrical stimulation simultaneously, and the asynchronous stimulation control means performing focused ultrasound stimulation and electrical stimulation at intervals; The synchronous stimulation control is used to enable the target object to simultaneously feel the deep vibration sense and the superficial pressure sense; The asynchronous stimulation control is used to separately simulate the temporal changes of the heat sense and the texture sense to enhance the dynamic expressiveness of the tactile feedback.

[0013] Optionally, in the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation, the spatial regulation specifically includes: Adjust the arrangement of stimulators and stimulation parameters at different positions to achieve differential tactile perception in multiple regions; Perform combined action through focused ultrasound and electrical stimulation within the same region, and construct sensations through stimulator distribution and stimulation control in different regions; Among them, the stimulator includes a focused ultrasound transducer and an electrical stimulation electrode.

[0014] Optionally, in the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation, the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation further includes: Dynamically adjust the stimulation intensity and range in combination with the stimulator arrangement and parameter optimization algorithm.

[0015] Optionally, in the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation, the parameter optimization algorithm is used to adjust the power, frequency pulse width of the focused ultrasound stimulation, and the amplitude and pulse width of the electrical stimulation.

[0016] In addition, to achieve the above object, the present invention also provides a multimodal tactile feedback device based on combined focused ultrasound and electrical stimulation, wherein the multimodal tactile feedback device based on combined focused ultrasound and electrical stimulation includes: A demand and site determination module, configured to obtain the tactile demand of the target object and determine the target site to be stimulated of the target object according to the tactile demand; A multimodal tactile simulation module, configured to adjust the frequency, power and pulse width of the focused ultrasound according to the tactile demand to obtain a target focused ultrasound, apply the target focused ultrasound to the deep nerves and muscles of the target site, and synchronously or asynchronously apply the surface electrodes of the electrical stimulation to the superficial nerve endings of the target site to achieve multimodal tactile simulation and obtain a multimodal tactile simulation result.

[0017] In addition, to achieve the above object, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a multimodal tactile feedback program based on combined focused ultrasound and electrical stimulation stored on the memory and executable on the processor, and when the multimodal tactile feedback program based on combined focused ultrasound and electrical stimulation is executed by the processor, the steps of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation as described above are implemented.

[0018] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a multimodal tactile feedback program based on combined focused ultrasound and electrical stimulation, and when the multimodal tactile feedback program based on combined focused ultrasound and electrical stimulation is executed by a processor, the steps of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation as described above are implemented.

[0019] In the present invention, the tactile needs of a target object are obtained, and the target part of the target object to be stimulated is determined according to the tactile needs; the frequency, power, and pulse width of focused ultrasound are adjusted according to the tactile needs to obtain target focused ultrasound, and the target focused ultrasound is applied to the deep nerves and muscles of the target part, and the surface electrodes of electrical stimulation are synchronously or asynchronously applied to the superficial nerve endings of the target part to achieve multimodal tactile simulation and obtain a multimodal tactile simulation result. The present invention utilizes the high spatio-temporal resolution and non-invasive deep stimulation characteristics of focused ultrasound, as well as the superficial characteristics of electrical stimulation in tactile perception, to trigger multimodal sensations, aiming to provide rich, accurate, and diverse tactile feedback through the organic combination of the two stimulation modalities to meet the rehabilitation needs of patients with sensory disorders. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flowchart of a preferred embodiment of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation of the present invention; Figure 2 is a schematic diagram of the overall composite stimulation in a preferred embodiment of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation of the present invention; Figure 3 is a schematic diagram of the single-region composite stimulation in a preferred embodiment of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation of the present invention; Figure 4 is a schematic diagram of the multi-region composite stimulation in a preferred embodiment of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer and more explicit, the following further elaborates on the present invention with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] Aiming at the technical problems existing in the current tactile feedback technology, such as single tactile types, insufficient stimulation accuracy, limited coverage of deep perception, and lack of naturalness in tactile experience, the present invention proposes a multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation. Through the high spatio-temporal resolution characteristics of focused ultrasound, the present invention non-invasively achieves precise stimulation of deep nerves and muscles, and can generate rich deep tactile perceptions including pain, deep pressure sensation, temperature sensation, and a feeling similar to a pinprick; at the same time, combined with electrical stimulation technology, taking advantage of its mature advantages in simulating superficial light touch, pressure sensation, vibration, and texture touch, the limitations of single modality in tactile types and stimulation ranges are solved.

[0023] The present invention optimizes the overall performance of tactile feedback through multimodal collaborative control, capable of covering both superficial and deep tactile perceptions simultaneously, and significantly enhancing the richness and realism of tactile feedback. This method can be widely applied in the rehabilitation medicine for patients with sensory disorders, virtual reality tactile interaction, bionic tactile systems, and other fields requiring high-precision, multimodal tactile simulation, providing a comprehensive, accurate, and non-invasive solution for the development of tactile feedback technology.

[0024] The multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation according to a preferred embodiment of the present invention, as Figure 1 shown, the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation includes the following steps: Step S10: Obtain the tactile needs of the target object, and determine the target part to be stimulated of the target object according to the tactile needs.

[0025] Specifically, for the group with sensory disorders, obtain the rehabilitation needs (i.e., tactile needs, such as the comprehensive needs to satisfy both deep and superficial sensations simultaneously) of the patients with sensory disorders (i.e., the target object), and then determine the target part to be stimulated (such as one part or multiple parts) according to the tactile needs of the patients with sensory disorders.

[0026] Step S20: Adjust the frequency, power, and pulse width of the focused ultrasound according to the tactile needs to obtain the target focused ultrasound, apply the target focused ultrasound to the deep nerves and muscles of the target part, and synchronously or asynchronously apply the surface electrodes of electrical stimulation to the superficial nerve endings of the target part to achieve multimodal tactile simulation and obtain the multimodal tactile simulation result.

[0027] Specifically, focused ultrasound (FUS) is a non-invasive treatment technology that uses ultrasonic energy to accurately focus on a specific part of the body to generate a thermal effect or a mechanical effect to achieve the treatment purpose (i.e., generate high-frequency sound waves through a transducer, and the sound waves are focused on the target area through a lens or a phased array to generate high temperature or mechanical vibration at the focal point to stimulate nerves). Adjust the frequency, power, and pulse width of the focused ultrasound according to the tactile needs (i.e., determined in advance through experiments according to the perception of the subject, and the relevant parameters are adjusted within the human acceptable range) to obtain the target focused ultrasound, and then accurately apply the target focused ultrasound to the deep nerves and muscles, which can generate various perceptions including pain, heat, and deep touch.

[0028] Electrical Stimulation (ES) is a technique that stimulates nerves or muscles with an electric current to trigger specific physiological responses. The electric current is transmitted to the target tissue through electrodes, stimulating the nerves or muscles to generate action potentials, and producing effects such as muscle contraction, pain relief, or neuromodulation according to the stimulation parameters. Electrical stimulation acts on superficial nerve endings through surface electrodes, and by adjusting the current waveform, it can simulate various tactile feedbacks such as pressure sensation, light touch sensation, vibration sensation, and texture sensation. The two modalities coordinate under synchronous (electrical stimulation and ultrasound act simultaneously) or asynchronous (act successively in time, such as electrical stimulation for one second and then focused ultrasound stimulation for another second) conditions, achieving complementary advantages through modality division of labor, thereby constructing a rich multi-modal tactile experience, realizing real-time and efficient regulation, and enhancing the realism and comfort of tactile feedback.

[0029] Through the coordination and integration of the two stimulation modalities, by utilizing the high spatio-temporal resolution and non-invasive deep stimulation characteristics of focused ultrasound, as well as the superficial characteristics of electrical stimulation in tactile perception, the triggering of multi-modal sensations is achieved, including tactile, pain, thermal, vibration, and texture perception, etc. Through the above-mentioned combined stimulation method, the present invention can simultaneously produce multiple perception effects at the same site, significantly improving the spatial resolution and perception depth of tactile feedback, and having the characteristics of non-invasiveness, high flexibility, and personalization, etc., and can realize three-dimensional tactile simulation.

[0030] In the present invention, regarding the regulation of the stimulation method, the present invention proposes a regulation method in two dimensions of time and space, that is, when performing multi-modal tactile simulation through focused ultrasound stimulation and electrical stimulation, the stimulation method is controlled through time regulation or space regulation to achieve a richer and more precise tactile simulation.

[0031] (1) In terms of time regulation, coordinated or independent sensations are generated at the same site or different sites through synchronous stimulation control or asynchronous stimulation control; the synchronous stimulation control means that focused ultrasound stimulation and electrical stimulation are carried out simultaneously, and the asynchronous stimulation control means that focused ultrasound stimulation and electrical stimulation are carried out at intervals; the synchronous stimulation control is used to enable the target object to simultaneously feel the deep vibration sensation and the superficial pressure sensation; the asynchronous stimulation control is used to respectively simulate the temporal changes of thermal sensation and texture sensation to enhance the dynamic expressiveness of tactile feedback. That is, through synchronous and asynchronous stimulation control, coordinated or independent sensations can be generated at the same site or different sites. For example, synchronous control can enable the user to simultaneously feel the deep vibration sensation and the superficial pressure sensation, while asynchronous control can respectively simulate the temporal changes of thermal sensation and texture sensation, thereby enhancing the dynamic expressiveness of tactile feedback.

[0032] (2) In terms of spatial regulation, adjust the arrangement of stimulators and stimulation parameters at different positions to achieve differential tactile perception in multiple regions; within the same region, combine the effects of focused ultrasound and electrical stimulation, and construct sensations through stimulator distribution and stimulation control in different regions; among them, the stimulator includes a focused ultrasound transducer and an electrical stimulation electrode. That is, by adjusting the arrangement of stimulators and stimulation parameters at different positions (such as frequency, pulse width, power, etc., which are adjusted through the coding control of the stimulator circuit), differential tactile perception in multiple regions can be achieved. Through the combined action of focused ultrasound and electrical stimulation within the same region, a rich tactile experience can be constructed at the same position; while in different regions, a larger range of sensation construction can be achieved through sensor distribution and stimulation control. For example, while feeling deep pressure at the center of the palm, a superficial vibration sensation can be generated in the fingertip area. In addition, combined with the arrangement of the stimulator network (multiple stimulators are arranged, and the distribution of multiple stimulators on the body surface constitutes a network, that is, a multi-point distributed stimulation network) and the parameter optimization algorithm (the parameter optimization algorithm is an algorithm used to adjust the stimulation parameters of electrodes and ultrasound, such as the adjustment of the amplitude and pulse width of electrical stimulation, the power and frequency pulse width of ultrasound stimulation, etc.), the stimulation intensity and range can be dynamically adjusted (for example, for a patient user with no tactile perception in the unilateral fingertip, a small-range stimulator can be arranged on the contralateral part of the body to achieve perception substitution. If the user needs real sensory interaction in a VR scenario, a large-range stimulator needs to be arranged throughout the body. Similarly, the selection of stimulation intensity also needs to be adjusted according to the user's needs and the characteristics of the user's own touch. For example, for a sensitive user, the stimulation intensity needs to be lowered, and for a user with dull senses, the stimulation intensity needs to be increased to achieve the same subjective perception). A more flexible tactile experience is realized. This regulation scheme significantly improves the fineness and controllability of tactile feedback, providing more comprehensive technical support for virtual reality tactile interaction and sensory disorder rehabilitation.

[0033] As Figure 2 shown, it is a schematic diagram showing the overall composite stimulation. Focused ultrasound stimulation is generated by a focused ultrasound transducer (Focused Ultrasound Transducer, which emits focused ultrasound to stimulate a specific point of a patient's specific muscle or tissue, and can also produce various different sensations and rehabilitation physiotherapy effects), acting on the deep nerves and muscles of the target site, generating multiple perceptions including pain, heat, and deep touch. Electrical stimulation is generated by an electrical stimulation electrode acting on the superficial nerve endings. By adjusting the current waveform, various tactile feedbacks such as pressure sensation, light touch sensation, and texture sensation can be simulated. As Figure 3 shown, it is a schematic diagram showing the composite stimulation in a single region. The focused ultrasound transducer and the electrical stimulation electrode act on the same part of the human tissue synchronously or asynchronously. As Figure 4As shown, it is a schematic diagram representing multi - region composite stimulation. Two focused ultrasound transducers act on the deep nerves and muscles of two parts respectively, and two electrical stimulation electrodes act on the superficial nerve endings of the other two parts respectively, enabling simultaneous stimulation of four parts (or even more parts).

[0034] Beneficial effects brought by the technical solution of the present invention: (1) By combining focused ultrasound and electrical stimulation, the present invention realizes multi - modal tactile feedback, capable of providing a richer and more complex tactile experience, including but not limited to pressure, vibration, texture, pain sensation, and thermal sensation.

[0035] (2) The high spatial resolution and deep - layer stimulation ability of focused ultrasound make up for the deficiencies of electrical stimulation in tactile feedback, capable of stimulating deep tissues and reconstructing complex tactile perception. The electrical stimulation technology is mature and can effectively simulate superficial touch. The combined stimulation with focused ultrasound achieves complementary advantages, forming a more natural and precise tactile reconstruction solution.

[0036] (3) The implementation methods of the present invention are flexible and diverse, adaptable to various application scenarios, such as fixed rehabilitation equipment, wearable devices, and rehabilitation robots, greatly expanding the application scope of the technology.

[0037] The present invention realizes a collaborative working mechanism of multi - modal touch by combining the technical advantages of focused ultrasound and electrical stimulation. Focused ultrasound is responsible for simulating deep tactile perception, including acupuncture sensation, deep pressure sensation, pain sensation, and temperature sensation, etc.; electrical stimulation is responsible for simulating superficial tactile perception, such as light touch sensation, pressure sensation, vibration, and texture sensation. By the combined use of the two modalities, it can dynamically cover various tactile categories from the surface layer to the deep layer, greatly enriching the types and perception range of tactile feedback and realizing three - dimensional tactile feedback. Most of the existing technologies are designed for single - modal tactile feedback and cannot simultaneously simulate superficial and deep touch. The multi - modal collaborative working mechanism of the present invention realizes a natural and coherent tactile transition, enhancing the realism and richness of the tactile experience.

[0038] Due to the high spatial resolution and depth - focusing characteristics of focused ultrasound, the present invention can accurately stimulate the target tissue at the millimeter level, achieving precise positioning of deep tactile points. At the same time, through parameter optimization, electrical stimulation supplements the resolution of superficial touch, making the superficial tactile feedback more natural. The combination of the two technologies effectively solves the defects of a single modality: electrical stimulation causes tactile diffusion due to human body conductivity and is difficult to accurately position, while although focused ultrasound has high resolution, it is insufficient in simulating superficial touch. The present invention realizes the combination of high spatial resolution and multi - level tactile feedback through the combined stimulation technology, bringing more accurate and natural perception ability to the tactile system.

[0039] The present invention designs a real-time dynamic collaborative control mechanism, which can dynamically adjust the intensity, frequency, waveform and stimulation timing of focused ultrasound and electrical stimulation according to tactile requirements. By introducing algorithm optimization and sensor feedback, the system can achieve intelligent adjustment of tactile stimulation, including modality switching (single modality or combined modality), adaptive regulation of stimulation parameters and real-time coordination. This mechanism ensures the naturalness, consistency and personalization of tactile feedback. Most of the existing tactile feedback technologies are static parameter control, which cannot meet the real-time requirements of complex tactile scenarios. The dynamic collaborative mechanism of the present invention can not only flexibly adapt to different tactile scenarios (such as deep rehabilitation or surface light touch simulation), but also dynamically respond to user needs to achieve highly personalized tactile feedback.

[0040] Furthermore, based on the above multi-modal tactile feedback method based on combined focused ultrasound and electrical stimulation, the present invention also correspondingly provides a multi-modal tactile feedback device based on combined focused ultrasound and electrical stimulation, wherein the multi-modal tactile feedback device based on combined focused ultrasound and electrical stimulation includes: A demand and site determination module, configured to obtain the tactile demand of a target object and determine the target site to be stimulated of the target object according to the tactile demand; A multi-modal tactile simulation module, configured to adjust the frequency, power and pulse width of focused ultrasound according to the tactile demand to obtain target focused ultrasound, apply the target focused ultrasound to the deep nerves and muscles of the target site, and synchronously or asynchronously apply surface electrodes of electrical stimulation to the superficial nerve endings of the target site to achieve multi-modal tactile simulation and obtain a multi-modal tactile simulation result.

[0041] The requirement and location determination module and the multi-modal tactile simulation module can be integrated into the terminal, which includes a processor and a memory. In some embodiments, the memory can be an internal storage unit of the terminal, such as the hard disk or memory of the terminal. In other embodiments, the memory can also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the terminal. Further, the memory can also include both the internal storage unit and the external storage device of the terminal. The memory is used to store the application software installed on the terminal and various types of data, such as the program code of the installed terminal. The memory can also be used to temporarily store the data that has been output or will be output. In one embodiment, a multi-modal tactile feedback program based on focused ultrasound and electrical combined stimulation is stored on the memory, and this multi-modal tactile feedback program based on focused ultrasound and electrical combined stimulation can be executed by the processor, so as to implement the multi-modal tactile feedback method based on focused ultrasound and electrical combined stimulation in this application. In some embodiments, the processor can be a central processing unit (CPU), a microprocessor or other data processing chips, and is used to run the program code stored in the memory or process data, such as executing the multi-modal tactile feedback method based on focused ultrasound and electrical combined stimulation, etc.

[0042] In the present invention, the terminal can be applied as a fixed device, a wearable device and a rehabilitation robot, as follows: (1) Fixed device: A fixed device suitable for a rehabilitation treatment room, including a tactile feedback bed or chair, and electric stimulation electrodes and focused ultrasound transducers are preset on the contact surface between the patient's skin and the device.

[0043] (2) Wearable device: A wearable device designed based on flexible materials, such as a tactile glove, an arm sleeve, etc., integrating a micro ultrasonic transducer and an electric stimulation electrode, and is used for portable rehabilitation training or daily assistance.

[0044] (3) Rehabilitation robot (robot combination): Integrate this tactile feedback method into the rehabilitation robot, and combine the mechanical touch of the rehabilitation robot with the combined stimulation technology to improve the rehabilitation effect. For example, for patients with symptoms of sensory loss or sensory decline, use the present invention for stimulation to trigger rich and complex tactile experiences of the patients, so as to continuously stimulate the receptors of the patients' tactile perception and strengthen the repair and establishment of neural pathways, thereby playing a role in the rehabilitation of such symptomatic patients.

[0045] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a multimodal tactile feedback program based on combined focused ultrasound and electrical stimulation. When the multimodal tactile feedback program based on combined focused ultrasound and electrical stimulation is executed by a processor, the steps of the multimodal tactile feedback method based on combined focused ultrasound and electrical stimulation as described above are implemented.

[0046] In summary, the present invention provides a multimodal tactile feedback method, device, terminal, and computer-readable storage medium based on combined focused ultrasound and electrical stimulation. The method includes: obtaining the tactile needs of a target object, and determining the target part of the target object to be stimulated according to the tactile needs; adjusting the frequency, power, and pulse width of the focused ultrasound according to the tactile needs to obtain target focused ultrasound, applying the target focused ultrasound to the deep nerves and muscles of the target part, and synchronously or asynchronously applying surface electrodes of electrical stimulation to the superficial nerve endings of the target part to achieve multimodal tactile simulation and obtain a multimodal tactile simulation result. The present invention utilizes the high spatio-temporal resolution and non-invasive deep stimulation characteristics of focused ultrasound, as well as the superficial characteristics of electrical stimulation in tactile perception, to trigger multimodal sensations, aiming to provide rich, accurate, and diverse tactile feedback through the organic combination of the two stimulation modalities to meet the rehabilitation needs of patients with sensory impairments.

[0047] It should be noted that in this article, the terms "including", "comprising", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or terminal including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or terminal. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article, or terminal including that element.

[0048] Certainly, those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium readable by a computer. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disc, etc.

[0049] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation, characterized in that: The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation includes: Acquiring a tactile requirement of a target object, and determining a target part of the target object to be stimulated according to the tactile requirement; The frequency, power and pulse width of the focused ultrasound are adjusted according to the tactile demand to obtain target focused ultrasound, and the target focused ultrasound is applied to the deep nerves and muscles of the target area, and the surface electrodes of the electrical stimulation are synchronously or asynchronously applied to the superficial nerve endings of the target area to achieve multimodal tactile simulation and obtain multimodal tactile simulation results.

2. The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation according to claim 1, characterized in that: The perceptions generated by the targeted focused ultrasound include pain, heat and deep touch; the tactile feedback generated by the electrical stimulation includes pressure, light touch, vibration and texture.

3. The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation according to claim 1, characterized in that: The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation also includes: When multimodal tactile simulation is performed through focused ultrasound stimulation and electrical stimulation, the stimulation mode is controlled through time regulation or space regulation.

4. The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation according to claim 3, characterized in that: The time control specifically includes: Produce coordinated or independent sensations in the same or different parts through synchronous stimulation control or asynchronous stimulation control; The synchronous stimulation control means that focused ultrasound stimulation and electrical stimulation are performed simultaneously, and the asynchronous stimulation control means that focused ultrasound stimulation and electrical stimulation are performed at intervals; The synchronous stimulation control is used to allow the target object to feel a deep vibration sensation and a shallow pressure sensation at the same time; The asynchronous stimulation control is used to simulate the timing changes of thermal sensation and texture sensation respectively, so as to enhance the dynamic expressiveness of tactile feedback.

5. The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation according to claim 3, characterized in that: The spatial regulation specifically includes: Adjust the placement of stimulators and stimulation parameters at different locations to achieve differentiated tactile perception in multiple areas; Combined action by focused ultrasound and electrical stimulation in the same area, sensory construction by stimulator distribution and stimulation control in different areas; Wherein, the stimulator includes a focused ultrasound transducer and an electrical stimulation electrode.

6. The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation according to claim 4 or 5, characterized in that: The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation also includes: Combined with stimulator placement and parameter optimization algorithms, stimulation intensity and range are dynamically adjusted.

7. The multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation according to claim 6, characterized in that: The parameter optimization algorithm is used to adjust the power and frequency pulse width of focused ultrasound stimulation, as well as the amplitude and pulse width of electrical stimulation.

8. A multimodal tactile feedback device based on focused ultrasound and electrical combined stimulation, characterized in that: The multimodal tactile feedback device based on focused ultrasound and electrical combined stimulation comprises: A demand and part determination module, used for obtaining the tactile demand of the target object and determining the target part of the target object to be stimulated according to the tactile demand; The multimodal tactile simulation module is used to adjust the frequency, power and pulse width of the focused ultrasound according to the tactile demand to obtain target focused ultrasound, apply the target focused ultrasound to the deep nerves and muscles of the target part, and synchronously or asynchronously act on the superficial nerve endings of the target part through the surface electrodes of electrical stimulation to achieve multimodal tactile simulation and obtain multimodal tactile simulation results.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a multimodal tactile feedback program based on focused ultrasound and electrical combined stimulation stored in the memory and executable on the processor. When the multimodal tactile feedback program based on focused ultrasound and electrical combined stimulation is executed by the processor, the steps of the multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a multimodal tactile feedback program based on focused ultrasound and electrical combined stimulation. When the multimodal tactile feedback program based on focused ultrasound and electrical combined stimulation is executed by a processor, the steps of the multimodal tactile feedback method based on focused ultrasound and electrical combined stimulation as described in any one of claims 1 to 7 are implemented.

Images