Design method and wearable system of virtual force feedback rendered by electrical stimulation on fingers

Abstract

The present invention relates to virtual reality technology, specifically to a design method and a wearable system for rendering virtual force feedback of fingers by electrical stimulation, wherein the design method comprises the following steps: establishing a hand muscle-skeletal simulation model based on biomechanics and conforming to the basic physiological structure of the human body, for conducting simulation experiments and calculating the desired electrical stimulation parameters; setting multiple types of electrical stimulation modes and the positions of stimulation electrodes on the hand, for rendering the diversified force perception of fingers and virtual objects in different interaction scenarios. The present invention constructs a hand muscle-skeletal simulation model based on biomechanics and conforming to the physiological structure of the human body, and can generate accurate current signals to stimulate muscles by inputting the desired force feedback data and outputting the corresponding electrical stimulation parameters, so as to achieve real and accurate force feedback in virtual reality using electrical stimulation.

Description

Technical Field

[0001] The present invention relates to virtual reality technology, and in particular to a design method and a wearable system for electrically stimulating and rendering virtual force feedback of fingers. Background Art

[0002] The sense of touch is an important component of immersive virtual reality (VR) systems, making the interaction and processing of VR objects more "intuitive and efficient". Haptic feedback, even passive haptics, have been shown to increase the sense of presence in virtual environments, that is, the feeling of being "in the environment". More specifically, tactile feedback is at least as important as visual feedback for presence, and in some cases even more important. In addition, the sense of presence provided by tactile feedback increases with the fidelity of the feedback system. Moreover, tactile feedback can also increase social presence. Haptic feedback can generally be divided into two categories: tactile feedback and kinesthetic (force) feedback, and the present invention focuses on force feedback.

[0003] Existing methods for achieving force feedback include contact force feedback, non-contact force feedback, and electrical stimulation force feedback. The contact force feedback method uses external mechanical devices such as mechanical arms, exoskeletons, and force feedback handles to directly provide force feedback or vibration feedback to the user; the non-contact force feedback method uses the principles of gas jets and ultrasonic waves to spray air or make ultrasonic waves intersect on the skin at the user's interaction position to produce a certain sense of pressure; electrical stimulation force feedback uses the principle of electrical muscle stimulation (EMS) to drive muscle contraction by applying electric current to the muscles, thereby producing involuntary limb movements to simulate the force feedback generated by virtual objects on the limbs.

[0004] Most contact force feedback devices require large external mechanical devices, which are inconvenient to move and costly. However, lightweight devices such as force feedback handles cannot allow users to interact intuitively and naturally. For non-contact force feedback devices, there are disadvantages of a single force feedback method and a large device size, and the reliability and accuracy are not as good as mechanical feedback devices. Unlike the above two solutions, electrical stimulation force feedback devices use muscles as natural actuators and do not require external mechanical actuators. Therefore, EMS systems usually require less space and are cheaper than the other two types of force feedback solutions.

[0005] At present, the research on virtual reality force feedback based on electrical stimulation has the following problems that need to be solved: (1) Virtual reality force feedback research based on muscle electrical stimulation lacks mathematical theoretical models and simulation systems to calculate the expected electrical stimulation parameters, so as to accurately render electrical stimulation force. (2) There is a lack of multi-modal electrical stimulation methods required for interacting with virtual objects of different properties in order to render diverse force sensations. Summary of the invention

[0006] In order to solve the technical problems existing in the prior art, the present invention provides a design method and a wearable system for electrically stimulating rendering virtual force feedback of fingers, constructs a hand muscle-skeletal simulation model based on biomechanics and conforming to the physiological structure of the human body, and can generate accurate current signals to stimulate muscles by inputting the desired force feedback data and outputting the corresponding electrical stimulation parameters, thereby realizing real and accurate force feedback in virtual reality using electrical stimulation.

[0007] The design method in the embodiment of the present invention is implemented by the following technical solution: A design method for electrically stimulating and rendering finger virtual force feedback comprises the following steps:

[0008] Establish a hand muscle skeleton simulation model based on biomechanics and in line with the basic physiological structure of the human body, which is used to conduct simulation experiments and calculate the desired electrical stimulation parameters;

[0009] Set multiple types of electrical stimulation modes and the positions of stimulation electrodes on the hands to render diverse force perceptions between fingers and virtual objects in different interaction scenarios.

[0010] Preferably, the establishment of a hand muscle skeleton simulation model based on biomechanics and conforming to the basic physiological structure of the human body includes:

[0011] Design a mathematical model of hand muscles, including a muscle activation dynamics sub-model for describing the process of nerve signal activating muscle fibers and a muscle contraction dynamics sub-model for describing the process of activated muscle fibers causing muscle contraction to generate contraction force, so as to determine the mathematical theoretical relationship between electrical stimulation parameters and the contraction force of a single muscle;

[0012] A hand musculoskeletal simulation model is established, and the designed hand muscle mathematical model is embedded in the established hand musculoskeletal simulation model. Then, electrical stimulation parameters are input into the hand musculoskeletal simulation model to conduct dynamic simulation experiments, and the finger movement states under different electrical stimulation parameters are observed. Inverse operations are performed on the hand musculoskeletal simulation model, and the expected finger force feedback data is input to obtain the corresponding electrical stimulation parameters.

[0013] The wearable system in the embodiment of the present invention is implemented based on the design method in the embodiment of the present invention, and includes a host computer, a human-computer interaction interface, a hand muscle skeleton simulation module, an electrical stimulation mode selection module, and a muscle electrical stimulation module;

[0014] The host computer is respectively connected to the human-computer interaction interface, the electrical stimulation mode selection module, and the muscle electrical stimulation module, and the electrical stimulation mode selection module is also connected to the muscle electrical stimulation module; the human-computer interaction interface is connected to a VR head display module and a motion capture glove, the VR head display module is connected to the host computer, and the motion capture glove is connected to the muscle electrical stimulation module;

[0015] The host computer generates a virtual reality scene, and detects and determines the interaction between the virtual hand and the virtual object in real time. According to the determination result of the interaction, the host computer instructs the electrical stimulation mode selection module to select the appropriate electrical stimulation mode for the muscle electrical stimulation module. The host computer calculates the force feedback data according to the interaction between the virtual hand and the virtual object, and transmits it to the hand musculoskeletal simulation module to obtain the electrical stimulation parameters, thereby instructing the muscle electrical stimulation module to output the corresponding current to stimulate the muscles to produce force feedback, thereby realizing accurate force rendering.

[0016] Compared with the prior art, the beneficial effects of the present invention include:

[0017] 1. The present invention constructs a hand muscle-skeletal simulation model based on biomechanics and conforming to the physiological structure of the human body. By inputting the desired force feedback data and outputting the corresponding electrical stimulation parameters, accurate current signals are generated to stimulate the muscles, thereby achieving real and accurate force feedback.

[0018] 2. The present invention proposes multiple types of electrical stimulation modes to render diverse force perceptions according to the interactions between fingers and virtual objects of different properties, thereby improving the immersion and realism of the interactions between fingers and virtual objects.

[0019] 3. The present invention proposes a wearable system for electrically stimulating and rendering virtual force perception of fingers, and applies the hand muscle skeleton simulation model and multiple types of electrical stimulation modes constructed by the present invention to a virtual reality wearable system to achieve real and accurate force feedback.

[0020] 4. The present invention further takes into account the minimum current threshold of electrical stimulation for rendering human force perception and the maximum current intensity that the human body can tolerate, thereby achieving a high-precision virtual force rendering effect and ensuring safety of use and individual adaptation.

[0021] 5. The wearable system of the present invention has a simple structure, low manufacturing cost, and low power consumption, and is suitable for daily use by individual users. The wearable system is applied to virtual scenes to render the force perception of fingers interacting with various virtual objects, thereby improving the user's sense of immersion and reality in the virtual scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic flow chart of a design method for finger virtual force feedback in an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of the Millard tendon model used to describe the muscle contraction dynamics in the present embodiment;

[0024] Figure 3 Schematic diagram of an example of a hand muscle-skeletal simulation model established in an embodiment of the present invention, wherein (a) is a schematic diagram of modeling of hand bones and joints, and (b) is a schematic diagram of modeling of hand muscles and joints;

[0025] Figure 4 Schematic diagrams of reference electrode positions for stimulating finger flexor muscles in an embodiment of the present invention, wherein (a) is a schematic diagram of the back of the hand, and (b) is a schematic diagram of the palm of the hand;

[0026] Figure 5 Reference diagram of electrode positions for stimulating finger extensor muscles;

[0027] Figure 6 This is an architecture diagram of a wearable system for electrically stimulating and rendering virtual force feedback on fingers in an embodiment of the present invention;

[0028] Figure 7 FIG. 4 is a simplified circuit block diagram for generating an EMS signal in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In view of the problems existing in the existing force feedback technology, the present invention focuses on the following aspects in the implementation of the technical solution: (1) a human hand musculoskeletal simulation model based on biomechanical theory is proposed, which can input the desired force feedback value and output the corresponding electrical stimulation parameters; (2) multiple types of electrical stimulation modes are proposed when the fingers interact with virtual objects of different properties to render diverse force perceptions; (3) combining the above-mentioned hand musculoskeletal simulation model and multiple types of electrical stimulation modes, an electrical stimulation force feedback wearable system is proposed to achieve real and accurate force feedback through electrical stimulation.

[0030] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0031] Example

[0032] like Figure 1 As shown, this embodiment provides a design method for electrically stimulating and rendering finger virtual force feedback, comprising the following steps:

[0033] S1. Establish a hand muscle skeleton simulation model based on biomechanics and in line with the basic physiological structure of the human body, which is used to conduct simulation experiments and calculate the desired electrical stimulation parameters, so as to perform realistic and accurate electrical stimulation force rendering.

[0034] In this embodiment, according to the principle of muscle electrical stimulation and biomechanics theory, a hand muscle-skeletal simulation model based on electrical stimulation (referred to as the hand muscle-skeletal model) is established. A series of electrical stimulation parameters can be input into the hand muscle-skeletal simulation model to conduct dynamic simulation experiments of the hand muscle-skeletal model and obtain finger muscle output force data. Similarly, by using the simulation system for inverse calculation, the corresponding muscle electrical stimulation parameters can be calculated by inputting the expected finger muscle output force data; and the calculated electrical stimulation parameters are used to generate current signals to act on the muscles, thereby performing real and accurate virtual force rendering.

[0035] S11. Design a mathematical model of hand muscles.

[0036] The purpose of designing a mathematical model of hand muscles is to study the dynamic characteristics of the musculoskeletal system. Biologically, the hand muscles and bones mainly include three parts: bones, muscles, and joints. The mathematical model of hand muscles designed in this step mainly includes a muscle activation dynamics sub-model and a muscle contraction dynamics sub-model. The muscle activation dynamics sub-model is used to describe the process of nerve signals activating muscle fibers, while the muscle contraction dynamics sub-model is used to describe the process of activated muscle fibers causing muscle contraction to produce contractile force. The above two processes influence each other and jointly determine athletic ability and athletic performance.

[0037] (1) For the muscle activation dynamics sub-model, muscles can be activated by nerve signals sent by the brain or by external electrical stimulation. The degree of muscle activation under electrical stimulation is mainly affected by the pulse width, amplitude and frequency of the electrical stimulation. Generally, a fixed pulse width is selected and the amplitude is changed to adjust the intensity of electrical stimulation.

[0038] The relationship between muscle activation and amplitude amp is determined as the normalized recruitment curve a r :

[0039]

[0040] Among them, amp th is the amplitude threshold, amp sat is the amplitude saturation value.

[0041] The relationship between muscle activation and frequency f is determined as the stimulation frequency function a f :

[0042]

[0043] Where β is the waveform factor and is set to 0.1.

[0044] Therefore, in this embodiment, the muscle activation dynamics sub-model can be determined as:

[0045]

[0046] Where u is the muscle excitability, u = a r a f Determine; a is the muscle activation; τ(a,u) is the time constant, determined as: Among them, t act is the activation time constant (set to 10ms in this embodiment), t deact is the deactivation time constant (set to 40ms in this embodiment).

[0047] (2) For the muscle contraction dynamics sub-model, it can be described by the Millard tendon model (e.g. Figure 2 As shown in FIG. 2 , the tendon model includes an active contraction unit 201, a passive elastic unit 202 and an elastic tendon unit 203. The active contraction unit represents the force generated by muscle contraction, which is mainly derived from the actin-myosin interaction in skeletal muscle fibers. Its form is nonlinear, taking into account the relationship between force and factors such as muscle length, contraction speed, and muscle activation; the passive elastic unit represents the elastic structure between muscle fibers, such as the elastic connection between protein molecules, and its form is linear; the elastic tendon unit represents the elastic structure in tendons and muscles, such as muscle fibers, and its form is usually also linear.

[0048] The muscle contraction dynamics submodel can be determined as: in is the maximum isometric force of the muscle, a is the degree of muscle activation, and α is the pennation angle between the muscle and tendon; It is a muscle contraction force function with normalized muscle length as variable, called "force-length" function; It is a muscle contraction force function with normalized muscle contraction velocity as variable, called “force-velocity” function; The passive force function of muscle with normalized muscle length as variable is called the “passive force-length” function.

[0049] Force-Length Function Determined as:

[0050]

[0051] in is the current muscle length l m Relative to the optimal muscle length l opt Normalized muscle length (the length of the muscle when producing maximum force): ε is a shape factor, and in this embodiment, the value is 0.45.

[0052] Force-Velocity Function Determined as:

[0053]

[0054] in is the current muscle contraction velocity v m Relative to the maximum contraction velocity of the muscle v max Normalized velocity (the speed at which a muscle contracts at maximum force output):

[0055] Passive Force-Length Function Determined as:

[0056]

[0057] where k PE is the shape factor (the value is 5 in this embodiment), is the normalized muscle length, is the passive muscle strain caused by the maximum isometric force (the value in this embodiment is 0.6).

[0058] The above muscle activation dynamics sub-model and muscle contraction dynamics sub-model can determine the mathematical theoretical relationship between electrical stimulation parameters and the contraction force of a single muscle. However, if you want to simulate the force generated by the limbs, you must establish a musculoskeletal model based on biomechanics theory that includes bones, muscles, and joints and conforms to the physiological structure of the human body.

[0059] S12. Establish a hand muscle skeleton simulation model.

[0060] In this embodiment, the hand musculoskeletal simulation model can be established using the Opensim platform. Opensim is an open source biomechanical simulation platform for establishing and analyzing computational models of the human musculoskeletal system. It provides a series of tools and libraries for simulating and analyzing human motion, mechanics, and biomechanical behavior.

[0061] The hand muscle skeleton simulation model to be established in this step needs to basically conform to the physiological structure of the human body. The model is required to include 28 bones of the human hand, 36 muscle drivers including internal and external hand muscles (see Table 1) and 14 finger joints, and the model is required to include at least 20 degrees of freedom of five fingers (4 degrees of freedom for each finger). For the modeling of bones, the geometric shape, connection relationship, mass and inertial characteristics of each bone can be determined through medical imaging data or other data sources for accurate three-dimensional modeling, and the three-dimensional model of the hand skeleton can be imported using the modeling tool of Opensim; for the modeling of joints, the rotation axis and motion constraints of each joint can be determined based on anatomical knowledge and literature research, and personalized adjustments can be made according to the range of motion of specific joints to adapt to the anatomical characteristics of different individuals; for muscle modeling, the starting point and insertion point of each muscle can be determined based on anatomical knowledge, and the geometric shape and mechanical characteristics of the muscle can be added to the model; for the modeling of joint degrees of freedom, the degrees of freedom of each joint can be determined based on the anatomical characteristics and kinematic requirements of the model. In addition, the model is also required to establish the coupling characteristics between joints, such as the passive finger flexion torque will increase as the wrist joint is straightened. The Opensim official website provides a simple hand muscle skeleton model, and the hand muscle skeleton model of the present invention is improved and perfected on the basis of this model. Figure 3 shown.

[0062] Table 1. List of muscle driver names and their English abbreviations for the hand model

[0063]

[0064] Furthermore, the hand musculoskeletal simulation model established using the Opensim platform must also be embedded with the muscle mathematical model designed in the above steps with the electrical stimulation parameters as input, i.e., the muscle activation dynamics sub-model and the muscle contraction dynamics sub-model. The muscle mathematical model can be embedded in the established hand musculoskeletal simulation model using C++, Python, or MATLAB programming languages. Then, the electrical stimulation parameters can be input into the hand musculoskeletal simulation model to conduct dynamic simulation experiments, observe the finger movement state under different electrical stimulation parameters, and also obtain data such as finger output force and angles of each joint. In addition, the hand musculoskeletal simulation model can be inversely operated, the desired finger force feedback data can be input, and the corresponding electrical stimulation parameters can be obtained, and then the electrical stimulation parameters can be used in the electrical stimulation force rendering of the virtual scene.

[0065] S2. Setting multiple types of electrical stimulation modes and the positions of stimulation electrodes on the hands to render diverse force perceptions between fingers and virtual objects in different interaction scenarios.

[0066] In this embodiment, according to the contact between the finger and the virtual object, it can be divided into two scenarios: the finger actively contacts the virtual object and the finger passively contacts the virtual object (such as the virtual object collides with the finger). In addition, according to the properties of the virtual object, the virtual object can be divided into an elastic virtual object and a rigid virtual object. Therefore, the interaction scenarios between the finger and the virtual object mainly include: the finger actively interacts with the elastic virtual object, the finger actively interacts with the rigid virtual object, the finger passively interacts with the virtual object through the back of the finger, and the finger passively interacts with the virtual object through the palm of the finger.

[0067] S21. Set multiple types of electrical stimulation modes.

[0068] This step proposes three types of electrical stimulation modes to render diverse force sensations based on the different force sensations that will be generated when the fingers touch virtual objects of different properties:

[0069] Mode 1, only stimulates the finger extensor muscles; this electrical stimulation mode can provide force feedback from the palm to the back of the finger, and can render force feedback from the elastic ball to the finger when the finger presses the elastic ball;

[0070] Mode 2, only stimulates the finger flexors; this electrical stimulation mode can provide force feedback from the back of the finger to the palm, and can render force feedback when the back of the finger is impacted in a boxing game;

[0071] Mode 3 stimulates the finger extensors and flexors at the same time; this electrical stimulation mode can control the fingers to be fixed at a certain preset angle and has a certain preset stiffness value, which can render force feedback such as when the fingers touch the edge of a rigid object and cannot continue to flex.

[0072] S22. Match corresponding electrical stimulation modes for different interaction scenarios.

[0073] For different interaction scenarios, the following types of electrical stimulation modes are used: when the finger actively contacts the elastic virtual object, Mode 1 electrical stimulation mode is used; when the finger actively contacts the rigid virtual object, Mode 3 electrical stimulation mode is used; when the virtual object collides with the finger from the back of the finger (which is the case where the finger passively contacts the virtual object), Mode 2 electrical stimulation mode is used; when the virtual object collides with the finger from the palm of the finger (which is also the case where the finger passively contacts the virtual object), Mode 1 electrical stimulation mode is used.

[0074] S23. Setting the stimulation electrode at a specific position on the hand.

[0075] For the electrode position of stimulating the finger flexor muscles: the stimulating electrode for controlling the flexion of the thumb is placed on the external skin of the palm end above the opponens pollicis muscle (such as Figure 4Electrode 401); Stimulating electrodes for controlling the flexion of the remaining four fingers are arranged on the external skin of the back of the hand above the lumbrical muscles and interosseous muscles corresponding to the four fingers (such as Figure 4 Electrode 402, electrode 403, electrode 404, electrode 405); a common ground electrode is arranged at the back of the wrist (such as Figure 4 Electrode 406). The above-mentioned stimulation electrode positions are all set to stimulate the internal muscles of the hand to control the flexion of the fingers. Stimulating the internal muscles of the hand to control the flexion of the fingers is more independent and stable than stimulating the external muscles of the hand, and can greatly reduce the oscillation and uncontrollable phenomenon caused by the electric stimulation driving the fingers.

[0076] For electrode placement to stimulate the finger extensors: The stimulating electrodes for controlling thumb extension are placed on the external skin on the side of the arm above the extensor pollicis longus and brevis muscles (eg Figure 5 Electrode 501); the stimulation electrodes for controlling the extension of the remaining four fingers are arranged on the external skin of the arm side above the extensor digitorum common extensor muscle, the extensor digitorum middle finger, the extensor digitorum ring finger, and the extensor digitorum little finger corresponding to the extensor digitorum common. Figure 5 Electrode 502, electrode 503, electrode 504, electrode 505); a common ground electrode is arranged on the skin on the side of the arm above the common extensor tendon (such as Figure 5 Electrode 506).

[0077] It should be noted that due to differences in muscle distribution and muscle size among different individuals, Figure 4 and Figure 5 The position of the stimulation electrode may need to be adjusted appropriately according to the different situations of different people, but the muscle position where each stimulation electrode is located remains unchanged.

[0078] In addition, all stimulation channels share a ground electrode (i.e., a common ground electrode) to reduce the number of electrodes. The electrodes can be fabric-based textile electrodes or gel patch electrodes. In addition, according to the muscle size of the fingers, the stimulation electrode sheet is selected to be a rectangle with a shape of 1×3 cm, and the ground electrode sheet is selected to be a rectangle with a shape of 3×3 cm.

[0079] Preferably, after setting the stimulation electrode position, the electrical stimulation parameters are also set. The electrical stimulation parameters are selected as follows: frequency range 30-50Hz, pulse width range 100-500μs, amplitude range 0-10mA. First, through preliminary experiments, determine the current frequency and pulse width that make the subject feel comfortable and can control the movement of the fingers (the frequency selected in this embodiment is 50Hz and the pulse width is 200μs), and then fix the pulse width and frequency, and adjust the electrical stimulation intensity by changing the amplitude; for the determination of the electrical stimulation threshold and saturation value: after determining the current pulse width and frequency, gradually increase the current amplitude from 0mA and act on the subject. When the subject's fingers have an obvious force response, determine that the current amplitude at this time is the amplitude threshold amp th; Further increase the current amplitude, when the subject feels physiological discomfort or the change in force response is no longer obvious, determine that the current amplitude at this time is the amplitude saturation value amp sat .

[0080] This embodiment also provides a wearable system for rendering virtual force feedback of fingers with electrical stimulation, and further explains how to apply the above-mentioned hand muscle-bone simulation model and multiple types of electrical stimulation modes to a virtual reality wearable system and achieve real and accurate force feedback. Figure 6 As shown, the virtual reality wearable system of this embodiment includes: a host computer, a human-computer interaction interface, a hand musculoskeletal simulation module, an electrical stimulation mode selection module and a muscle electrical stimulation module. The hand musculoskeletal simulation module (referred to as the hand musculoskeletal simulation module) is deployed in the host computer to simplify the equipment. The host computer is respectively connected to the human-computer interaction interface, the electrical stimulation mode selection module, and the muscle electrical stimulation module, and the electrical stimulation mode selection module is also connected to the muscle electrical stimulation module; the human-computer interaction interface is connected to a VR head display module and a motion capture glove, the VR head display module is connected to the host computer, and the motion capture glove is connected to the muscle electrical stimulation module.

[0081] In summary, the motion capture gloves track the user's hand motion information and transmit it to the VR head display module, thereby mapping the user's hand to the virtual hand in the virtual scene, enabling the user to interact with the digital content; the host computer generates the virtual reality scene, and detects and determines the interaction between the virtual hand and the virtual object in real time, and instructs the electrical stimulation mode selection module to select the appropriate electrical stimulation mode for the muscle electrical stimulation module according to the judgment result of the interaction situation; the hand musculoskeletal simulation module is deployed on the host computer, and the host computer calculates the force feedback data according to the interaction between the virtual hand and the virtual object, and transmits it to the hand musculoskeletal simulation module to obtain the electrical stimulation parameters, thereby instructing the muscle electrical stimulation module to output the corresponding current to stimulate the muscle to produce force feedback, and achieve accurate force rendering. The following will describe the various components of the wearable system in detail.

[0082] (1) Host computer: used to generate virtual reality scenes, detect and identify the interaction between the virtual hand and the virtual object in real time, and instruct the electrical stimulation mode selection module to select a suitable electrical stimulation mode for the muscle electrical stimulation module according to the identification result; the host computer further outputs force information data to the human-computer interaction interface according to the interaction between the virtual object and the hand in the virtual reality scene, thereby realizing the interaction between the user and the digital content.

[0083] Among them, force information refers to the force feedback information of the virtual object in the virtual reality scene on the virtual hand. For example, pressing a virtual elastic ball should get 5N force feedback to the finger, and then this 5N force feedback information is transmitted to the electrical stimulation module, which outputs the corresponding current to stimulate the finger muscles to render the 5N force. The motion capture gloves of the human-computer interaction interface are used to capture hand information and map it to the virtual hand, so that the virtual hand in VR is synchronized with the user's hand movements.

[0084] (2) VR head-mounted display module: used to provide an interactive environment in a virtual reality scene and communicate with the motion capture gloves. The hand movement information captured by the motion capture gloves will be synchronously mapped to the virtual hand in the virtual reality scene to facilitate interaction with virtual objects.

[0085] (3) Motion capture gloves: including the glove body and muscle electrical stimulation electrodes placed on the glove body.

[0086] The glove body is used to record and track hand movements. It captures the angle data of each hand joint in real time based on inertial measurement unit and sensitivity sensor technology, realizes the capture and tracking of hand movements and postures, and transmits the captured and tracked data to the VR headset module for synchronous mapping to the virtual hand in the virtual reality scene.

[0087] The muscle electrical stimulation electrode includes a stimulation electrode and a grounding electrode. The stimulation electrode is located at the back of the hand of the glove body, and the grounding electrode is located at the back of the wrist of the glove body. All stimulation channels share a grounding electrode (i.e., a common grounding electrode) to reduce the number of electrodes. The stimulation electrode can be a fabric-based textile electrode or a gel patch electrode. In addition, according to the muscle size of the finger, the stimulation electrode is selected to be a rectangular sheet structure with a shape of 1×3 cm, and the grounding electrode is selected to be a rectangular sheet structure with a shape of 3×3 cm.

[0088] (4) Hand musculoskeletal simulation module: A hand musculoskeletal simulation module based on electrical stimulation is established according to the principle of muscle electrical stimulation and biomechanics. The simulation can be used to perform inverse operations and output the corresponding electrical stimulation parameters by inputting the desired finger force feedback data.

[0089] In this embodiment, the hand musculoskeletal simulation module receives the virtual force feedback data of the virtual object on the finger in the virtual reality scene transmitted by the host computer as input, outputs the corresponding electrical stimulation parameters, and transmits them to the muscle electrical stimulation module to generate microcurrent with certain parameters to act on the hand muscles, thereby performing accurate virtual force rendering.

[0090] (5) Muscle electrical stimulation module: includes a small electrical stimulation device and an EMS electrode array connected to each other. The small electrical stimulation device is connected to the hand muscle-skeletal simulation module and the electrical stimulation mode selection module respectively. The small electrical stimulation device can be implemented by a microcontroller (MCU). According to the electrical stimulation parameters output by the hand muscle-skeletal simulation module and the electrical stimulation mode output by the electrical stimulation mode selection module, the EMS electrode array is activated to output a corresponding current stimulation signal to act on the hand muscles, so that the hand muscles produce force feedback and realize force rendering.

[0091] In this embodiment, a simplified circuit block diagram for generating an EMS signal is shown in FIG. Figure 7 As shown, the miniaturized electrical stimulation device includes a control unit and a gate connected to each other, the control unit is connected to the host computer through a communication module, and the gate is connected to the EMS electrodes in the EMS electrode array. Among them, the control unit includes a printed circuit board (PCB) and a 3.7V lithium battery, and is encapsulated in a 3D printed shell. The PCB circuit includes a microcontroller MCU (ESP32-WROOM32 is optional) and an EMS circuit; the EMS circuit includes a boost converter, a switch circuit and a variable resistor circuit connected in sequence, wherein the switch circuit is also connected to the microcontroller, and the variable resistor circuit is also connected to the gate. The boost converter amplifies the output of the 3.7V lithium polymer battery to 50V, and the output of the boost converter generates an AC signal through the switch circuit. The output of the switch circuit reaches the gate through the resistor circuit, selects the corresponding electrical stimulation channel and generates an electrical stimulation signal to the EMS electrode.

[0092] As an application example, the wearable system for electrically stimulating and rendering virtual force feedback on fingers provided in this embodiment can be applied to fields related to artificial intelligence or the metaverse.

[0093] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A design method for rendering finger virtual force feedback by electrical stimulation, characterized in that: The following steps are involved: Establish a hand muscle skeleton simulation model based on biomechanics and in line with the basic physiological structure of the human body, which is used to conduct simulation experiments and calculate the desired electrical stimulation parameters; Set multiple types of electrical stimulation modes and the positions of stimulation electrodes on the hands to render diverse force perceptions of fingers and virtual objects in different interaction scenarios; The method of establishing a hand muscle skeleton simulation model based on biomechanics and conforming to the basic physiological structure of the human body includes: Design a mathematical model of hand muscles, including a muscle activation dynamics sub-model for describing the process of nerve signal activating muscle fibers and a muscle contraction dynamics sub-model for describing the process of activated muscle fibers causing muscle contraction to generate contraction force, so as to determine the mathematical theoretical relationship between electrical stimulation parameters and the contraction force of a single muscle; A hand musculoskeletal simulation model is established, and the designed hand muscle mathematical model is embedded in the established hand musculoskeletal simulation model, and then the electrical stimulation parameters are input into the hand musculoskeletal simulation model to conduct a dynamic simulation experiment, and the finger movement state under the action of different electrical stimulation parameters is observed; and the hand musculoskeletal simulation model is inversely operated, and the desired finger force feedback data is input to obtain the corresponding electrical stimulation parameters; Interaction scenarios between fingers and virtual objects, including: active interaction between fingers and elastic virtual objects, active interaction between fingers and rigid virtual objects, passive interaction between fingers and virtual objects through the back of the fingers, and passive interaction between fingers and virtual objects through the palm of the fingers; Set multiple types of electrical stimulation modes, including: Mode 1, only stimulates the finger extensor muscles; this electrical stimulation mode provides force feedback from the palm to the back of the finger; Mode 2, only stimulates the finger flexors; this electrical stimulation mode provides force feedback from the back of the finger to the palm of the finger; Mode 3, stimulates the finger extensors and finger flexors simultaneously; this electrical stimulation mode controls the fingers to be fixed at a preset angle and has a preset stiffness value; For different interaction scenarios, the corresponding electrical stimulation mode is matched: when the finger actively contacts the elastic virtual object, the Mode 1 electrical stimulation mode is used; when the finger actively contacts the rigid virtual object, the Mode 3 electrical stimulation mode is used; when the virtual object hits the finger from the back of the finger, the Mode 2 electrical stimulation mode is used; when the virtual object hits the finger from the palm of the finger, the Mode 1 electrical stimulation mode is used; Set the specific location of the stimulation electrodes on the hand, including: As for the electrode positions for stimulating the finger flexors, the stimulating electrode for controlling the flexion of the thumb is arranged on the external skin at the palm end above the opponens pollicis muscle; the stimulating electrodes for controlling the flexion of the remaining four fingers are respectively arranged on the external skin at the dorsal end of the hand above the lumbrical muscles and interosseous muscles corresponding to the four fingers; the common ground electrode is arranged on the back of the wrist. As for the electrode positions for stimulating the finger extensor muscles, the stimulating electrode for controlling thumb extension is arranged on the external skin on the arm side above the extensor pollicis longus and the extensor pollicis brevis; the stimulating electrodes for controlling the extension of the remaining four fingers are arranged on the external skin on the arm side above the extensor digitorum common, the extensor digitorum middle, the extensor digitorum ringis, and the extensor digitorum littleis corresponding to the extensor digitorum common, respectively; the common ground electrode is arranged on the skin on the arm side above the extensor digitorum common tendon.

2. The design method for electrically stimulating and rendering finger virtual force feedback according to claim 1, characterized in that: The muscle activation dynamics sub-model is: Where u is the degree of muscle excitability; a is the degree of muscle activation; τ(a,u) is the time constant, determined as: Among them, t act is the activation time constant, t deact is the deactivation time constant.

3. The design method for rendering finger virtual force feedback by electrical stimulation according to claim 2, characterized in that: The muscle excitability is given by u=a r a f OK, r is the normalized recruitment curve used to characterize the relationship between muscle activation and amplitude amp: Among them, amp th is the amplitude threshold, amp sat is the amplitude saturation value; a f is the stimulation frequency function used to characterize the relationship between muscle activation and frequency f: Where β is the waveform factor.

4. The design method for rendering finger virtual force feedback by electrical stimulation according to claim 1, characterized in that: The muscle contraction dynamics sub-model is: in is the maximum isometric force of the muscle, a is the degree of muscle activation, and α is the pennation angle between the muscle and tendon; It is a muscle contraction force function with normalized muscle length as variable, which is a "force-length" function; It is a muscle contraction force function with normalized muscle contraction velocity as variable, which is a "force-velocity" function; It is a muscle passive force function with normalized muscle length as the variable, which is a "passive force-length" function.

5. The design method for rendering finger virtual force feedback by electrical stimulation according to claim 4, characterized in that: Force-Length Function Determined as: in is the current muscle length l m Relative to the optimal muscle length l opt Normalized muscle length: ε is the shape factor; Force-Velocity Function Determined as: in is the current muscle contraction velocity v m Relative to the maximum contraction velocity of the muscle v max The normalized speed is: "Passive force-length" function Determined as: where k PE is the shape factor, is the normalized muscle length, is the passive muscle strain caused by the maximal isometric force.

6. A wearable system for rendering virtual force feedback of fingers by electrical stimulation, implemented based on the design method described in any one of claims 1 to 5, characterized in that: It includes a host computer, a human-computer interaction interface, a hand muscle-skeletal simulation module, an electrical stimulation mode selection module and a muscle electrical stimulation module; The host computer is respectively connected to the human-computer interaction interface, the electrical stimulation mode selection module, and the muscle electrical stimulation module, and the electrical stimulation mode selection module is also connected to the muscle electrical stimulation module; the human-computer interaction interface is connected to a VR head display module and a motion capture glove, the VR head display module is connected to the host computer, and the motion capture glove is connected to the muscle electrical stimulation module; The host computer generates a virtual reality scene, and detects and determines the interaction between the virtual hand and the virtual object in real time, and instructs the electrical stimulation mode selection module to select a suitable electrical stimulation mode for the muscle electrical stimulation module according to the result of the interaction determination; The host computer calculates force feedback data based on the interaction between the virtual hand and the virtual object, and transmits it to the hand musculoskeletal simulation module to obtain electrical stimulation parameters, thereby instructing the muscle electrical stimulation module to output corresponding current to stimulate the muscles to produce force feedback and achieve accurate force rendering.

7. The wearable system according to claim 6, characterized in that: The motion capture glove comprises a glove body and muscle electrical stimulation electrodes arranged on the glove body; The hand musculoskeletal simulation module is deployed on the host computer.

Images