A multi-modal haptic feedback glove and system for virtual handshakes

Abstract

The application provides a multi-modal tactile feedback glove and system for virtual handshaking, wherein the multi-modal tactile feedback glove specifically comprises: a palm air bag, a plurality of finger air bags, a refrigeration sheet, a linear motor, an inertial sensor and a glove. The application presents four dimensions through three tactile feedback forms of force, temperature and vibration, and solves the problems of single tactile feedback form and limitation to fingertips of current tactile feedback gloves. Moreover, the application uses flexible air bags to improve wearing comfort and skin friendliness, and arranges the above flexible air bags according to the distribution of real handshaking contact force in the finger and palm areas, thereby improving the reality of handshaking experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of haptic feedback, in particular to a multi-modal haptic feedback glove and system for virtual handshaking. BACKGROUND

[0002] The existing handshaking system usually adopts a mechanical arm to provide a handshaking experience, which has defects such as complex mechanical structure and control, high cost, and being heavy and inconvenient to carry, and is not suitable for social activities at any time and any place. In comparison, the haptic feedback glove is more portable, and can provide an immersive interactive experience for users in combination with virtual reality display technology.

[0003] At present, most haptic feedback gloves can only provide fingertip force feedback, lack distributed force feedback effect of finger pads and entire palm area, and are difficult to coordinate the contradiction between the force feedback glove in free space and in constrained space. In addition, in addition to the grip experience, during the real handshaking process, the two parties will experience a variety of other tactile sensations such as temperature, texture, shaking, etc. However, there is currently a lack of haptic gloves that can simulate a variety of tactile experiences during the handshaking process. It is a key link to realize immersive virtual handshaking to develop a multi-modal haptic feedback glove for handshaking to meet the needs of multi-modal tactile fusion presentation during handshaking.

[0004] One of the challenges in developing a multi-modal haptic feedback glove is how to miniaturize the device, that is, how to embed multiple drivers in the compact space of the glove and avoid mutual interference, and how to perform spatial and temporal registration of multi-modal haptic feedback to ensure that it is less than the perception threshold of the human hand. SUMMARY

[0005] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a multi-modal haptic feedback glove and system for virtual handshaking.

[0006] To achieve the above purpose, the present application provides the following solutions:

[0007] A multi-modal haptic feedback glove for virtual handshaking, comprising: a glove, a linear motor, an inertial sensor, a palm air bag, a plurality of finger air bags, and a refrigeration sheet.

[0008] Each finger section of the glove is equipped with a finger airbag, and the palm section of the glove is equipped with a palm airbag. The finger airbags, when inflated, compress the user's finger skin, hindering the bending of the user's finger joints within the glove, thus creating a handshake contact experience. The palm airbags, when inflated, compress the user's palm skin, thus creating a handshake contact experience. Cooling elements are respectively located in the palm and finger sections, and are used to provide temperature stimulation to the user. Both the inertial sensor and the linear motor are connected to a microcontroller. The inertial sensor is located at the wrist section of the glove and is used to collect the user's palm shaking data. The linear motor is located at the base of the palm section and is used to generate vibrations based on the shaking data, allowing the user to experience a handshake.

[0009] Preferably, it also includes multiple heat sinks;

[0010] The heat sink is fixedly disposed on the surface of the cooling plate and is used to dissipate heat from the cooling plate.

[0011] Preferably, it also includes multiple Velcro straps;

[0012] The cooling element is attached to the finger area via Velcro.

[0013] Preferably, the finger airbag includes: a silicone finger sleeve and a first airbag actuator disposed on the fingertip side of the silicone finger sleeve; the silicone finger sleeve is disposed on the finger; the first airbag actuator includes a first enclosed cavity, a first air tube and a first silicone shell; the first enclosed cavity is disposed inside the first silicone shell, the first air tube is connected to an air pump and extends into the first enclosed cavity; the first silicone shell is connected to the fingertip side of the silicone finger sleeve.

[0014] Preferably, the thickness of the first silicone shell in the region near the fingertip is less than the thickness of other regions of the first silicone shell.

[0015] Preferably, the palm airbag comprises: a silicone palm sleeve, an annular second airbag actuator, and a third airbag actuator; the second airbag actuator and the third airbag actuator are connected through the silicone palm sleeve to form a closed annular structure; the annular structure is fitted onto the palm portion; the second airbag actuator comprises a second closed cavity, a second air tube, and an annular second silicone shell; the second closed cavity is disposed within the second silicone shell, and the second air tube is connected to an air pump and extends into the second closed cavity; the third airbag actuator comprises a third closed cavity, a third air tube, and an annular third silicone shell; the third closed cavity is disposed within the third silicone shell, and the third air tube is connected to an air pump and extends into the third closed cavity; a first end of the second silicone shell is connected to a first end of the third silicone shell through the silicone palm sleeve; a second end of the second silicone shell is connected to a second end of the third silicone shell through the silicone palm sleeve.

[0016] Preferably, the second silicone shell is fitted to the web portion of the glove, and the third silicone shell is fitted to the ulnar side portion of the glove; the thickness of the area of ​​the second silicone shell near the palm is less than the thickness of other areas of the second silicone shell; the thickness of the area of ​​the third silicone shell near the palm is less than the thickness of other areas of the third silicone shell.

[0017] Preferably, two cooling plates are provided in the thenar eminence area of ​​the palm.

[0018] A multimodal haptic feedback system for virtual handshakes includes the aforementioned multimodal haptic feedback glove for virtual handshakes, a hand motion tracking module, a virtual reality device, a host computer, a slave computer, an air pump, a motor drive module, a vibration drive module, and a digital-to-analog converter chip.

[0019] The host computer is connected to the hand motion tracking module, the virtual reality device, and the slave computer. The hand motion tracking module is installed on various parts of the glove and is used to acquire hand motion information of the user after wearing the glove. The hand motion information includes the joint angle information of the five fingers and hand posture information. The virtual reality device is used to generate a virtual hand avatar in a virtual scene and to visualize the handshake scene.

[0020] When a user controls the virtual hand avatar to shake hands based on the hand motion tracking module, the host computer calculates handshake information based on the user's hand motion information and the positional relationship between the virtual hand avatars of the two parties shaking hands in the virtual scene. The handshake information includes handshake position information, force information, temperature information, and shaking information. The slave computer is connected to the air pump, the motor drive module, the vibration drive module, the digital-to-analog converter chip, and the inertial sensor. The air pump is connected to the digital-to-analog converter chip, which controls the air pump to inflate the palm airbag and finger airbag. The motor drive module is connected to the cooling chip, which controls the cooling chip to achieve temperature changes. The vibration drive module is connected to the linear motor, which controls the vibration frequency of the linear motor.

[0021] The lower-level machine controls the working state of the digital-to-analog converter chip, the motor drive module, and the vibration drive module respectively based on the handshake information and the shaking data obtained by the inertial sensor, so that the user can obtain the experience of force, temperature, and vibration tactile sensations of different handshake depths.

[0022] Preferably, it further includes at least one proportional servo valve; the proportional servo valve is disposed on the air pump, the proportional servo valve is connected to the digital-to-analog converter chip, and the proportional servo valve is used to adjust the air pressure delivered by the air pump to the palm airbag and / or the finger airbag.

[0023] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0024] This invention provides a multimodal haptic feedback glove and system for virtual handshakes, proposing four dimensions of handshake: depth of handshake, force, temperature (hot or cold), and shaking speed. This invention presents these four dimensions through force, temperature, and vibration haptic feedback, solving the problems of current haptic feedback gloves having only one form of haptic feedback and being limited to the fingertips. Furthermore, the use of flexible airbags made of silicone improves wearing comfort and skin-friendliness, and the placement of these flexible airbags in the fingers and palm areas according to the distribution of contact force in a real handshake enhances the realism of the handshake experience. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of the palm back structure of the multimodal tactile glove provided in an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the palm inner side structure of the multimodal tactile glove provided in an embodiment of the present invention;

[0028] Figure 3 A schematic diagram (front view) of the structure of the multimodal haptic glove finger airbag before inflation in an embodiment of the present invention;

[0029] Figure 4 A schematic diagram (front view) of the structure of the multimodal tactile glove finger airbag after inflation in an embodiment of the present invention;

[0030] Figure 5 A side view of the structure of the multimodal tactile glove finger airbag before inflation in an embodiment of the present invention;

[0031] Figure 6 A side view of the structure of the multimodal tactile glove finger airbag after inflation in an embodiment of the present invention;

[0032] Figure 7 A schematic diagram of the structure of the palm airbag of the multimodal tactile glove before inflation in an embodiment of the present invention;

[0033] Figure 8 A schematic diagram of the structure of the palm airbag of the multimodal tactile glove after inflation in an embodiment of the present invention;

[0034] Figure 9 The diagram shows the framework of the multimodal tactile glove control structure and virtual handshake system provided in the embodiments of the present invention.

[0035] Symbol explanation:

[0036] 1-First airbag actuator, 2-Cooling plate, 3-Second airbag actuator, 4-Silicone finger sleeve, 5-Hook and loop fastener, 6-Glove, 7-Silicone palm sleeve, 8-Linear motor, 9-Third airbag actuator. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0039] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, including a series of steps, processes, methods, etc., is not limited to the steps listed, but may optionally include steps not listed, or may optionally include other steps inherent to these processes, methods, products, or devices.

[0040] The purpose of this invention is to provide a multimodal haptic feedback glove and system for virtual handshakes, which solves the problems of current haptic feedback gloves having a single haptic feedback form and being limited to the fingertips, improves the comfort and skin-friendliness of the gloves, and further enhances the realism of the handshake experience.

[0041] Furthermore, in response to the current situation of limited tactile modalities, difficulty in miniaturizing actuators, and susceptibility to interference in tactile feedback gloves, this embodiment proposes a glove design and manufacturing method with multimodal tactile feedback function to simulate the multimodal tactile experience involved in handshake.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] Figure 1 and Figure 2 These are schematic diagrams of the back of the palm and the inner side of the palm structure of the multimodal tactile glove provided in the embodiments of the present invention, respectively. Figure 1 and Figure 2As shown, this invention provides a multimodal haptic feedback glove for virtual handshakes, comprising: a glove 6, a linear motor 8, an inertial sensor, a palm airbag, multiple finger airbags, and a cooling pad 2; each finger portion of the glove 6 is provided with a finger airbag, and the palm portion of the glove 6 is provided with a palm airbag; the finger airbags, when inflated, compress the user's finger skin, hindering the bending of the user's finger joints within the glove 6, thereby generating a handshake contact force experience on the fingers; the palm airbags, when inflated, compress the user's palm skin, thereby generating a handshake contact force experience on the palm; the cooling pad 2 is respectively disposed on the palm portion and the finger portion, and is used to provide temperature stimulation to the user; the inertial sensor is connected to the linear motor 8, and is disposed on the wrist portion of the glove 6, and is used to collect the shaking data of the user's palm; the linear motor 8 is disposed at the base of the palm portion, and is used to generate vibration based on the shaking data, so that the user experiences a handshake shaking.

[0044] The finger airbag includes: a silicone finger sleeve 4 and a first airbag actuator 1 disposed on the fingertip side of the silicone finger sleeve 4; the silicone finger sleeve 4 is sleeved on the finger; the first airbag actuator 1 includes a first closed cavity, a first air tube and a first silicone shell; the first closed cavity is disposed inside the first silicone shell, the first air tube is connected to an air pump and extends into the first closed cavity; the first silicone shell is connected to the fingertip side of the silicone finger sleeve 4.

[0045] The palm airbag includes: a silicone palm sleeve 7, an annular second airbag actuator 3, and a third airbag actuator 9; the second airbag actuator 3 and the third airbag actuator 9 are connected through the silicone palm sleeve 7 to form a closed annular structure; the annular structure is fitted onto the palm portion; the second airbag actuator 3 includes a second closed cavity, a second air tube, and an annular second silicone shell; the second closed cavity is disposed within the second silicone shell, and the second air tube is connected to an air pump and extends into the second closed cavity; the third airbag actuator 9 includes a third closed cavity, a third air tube, and an annular third silicone shell; the third closed cavity is disposed within the third silicone shell, and the third air tube is connected to an air pump and extends into the third closed cavity; the first end of the second silicone shell is connected to the first end of the third silicone shell through the silicone palm sleeve 7; the second end of the second silicone shell is connected to the second end of the third silicone shell through the silicone palm sleeve 7.

[0046] Optionally, two cooling pads 2 are provided in the thenar eminence area of ​​the palm.

[0047] Specifically, the multimodal tactile feedback glove in this embodiment includes three functional modules: force tactile feedback, temperature tactile feedback, and vibration tactile feedback, which are used to realize four dimensions of handshake: handshake depth, force, temperature (hot or cold), and shaking speed.

[0048] As an optional implementation, this embodiment arranges the actuators to implement force feedback functionality based on the contact areas of both hands during a real handshake, specifically a deep grip and a shallow grip. This embodiment uses pneumatic software actuators to achieve force feedback, adjusting the grip strength. That is, software actuators are arranged on the fingers and palm according to the contact areas of a real handshake, providing the user with a distributed force experience.

[0049] Furthermore, force feedback is achieved by inflating a self-made silicone airbag to generate compressive force. For example... Figure 1 and Figure 2 As shown, the airbags on the haptic feedback glove are divided into two parts: finger airbags and palm airbags. Each finger has one airbag, covering both the proximal interphalangeal point (PIP) and the distal interphalangeal point (DIP). Figures 3 to 6 As shown, for ease of wear, the airbag is fitted with a silicone finger sleeve underneath, which can be directly slipped onto the fingers. When the airbag is inflated via the air tube, it restricts joint flexion, and the expanding air chamber compresses the skin, creating the handshake contact experience. During a handshake, the contact area of ​​the palm is primarily on the sides, with almost no contact between the palm and the back of the hand. Figures 7 to 8 As shown, two air bladders are glued together to form a ring-shaped palm air bladder. During use, the palm air bladder is placed over the palm, with the centers of the two air bladders located on either side of the palm. When the air bladder is inflated through the air tube, the pressure on the sides of the palm air bladder increases, while the palm and the middle of the back of the hand feel almost no pressure, consistent with the distribution of grip strength during a handshake. It is worth noting that, as... Figures 3 to 8 As shown, the silicone in the finger and palm airbags is thinner near the fingers / palms and thicker further away. This design aims to allow the air chambers of the finger and palm airbags to expand towards the fingers / palms during inflation, creating a more noticeable compression on the skin and enhancing the force feedback experience during a handshake.

[0050] Specifically, the flexible silicone airbag actuator (first airbag actuator 1) is divided into two parts, manufactured independently. The upper part connects to the air tube, and the lower part is cast together with the silicone sleeve. They are then bonded together with silicone adhesive to form the central cavity. The manufacturing steps are as follows: First, mix silicone (such as Ecoflex, Dragon Skin, etc.) in equal proportions, stir thoroughly, and then place in a vacuum chamber to remove air. Stop vacuuming when fewer bubbles appear, open the vacuum chamber, and remove the silicone. Slowly pour the silicone into the mold, remove obvious air bubbles in key areas with tweezers, and wait 6 hours. Slowly demold, and cut off any excess material around the edges with scissors. First, bond the air tube and the airbag connecting the air tube. Apply an appropriate amount of silicone adhesive to the outside of the air tube, insert it into the connecting airbag, and continue applying an appropriate amount of adhesive to the outside to prevent leakage. Apply adhesive evenly to the side of the airbag not connected to the air tube, and gently press the side of the airbag connected to the air tube onto the adhesive, pressing gently with your hand to ensure it adheres firmly. Other tools can be used to help hold it in place, and wait at least 1 hour. Apply a suitable amount of adhesive to the outside of the thin silicone tube and insert it into the larger tube. Then, seal the connection with hot melt adhesive. Finally, conduct an air permeability test and repair any leaks with silicone adhesive or hot melt adhesive.

[0051] Figure 3 and Figure 4 These are front views of the multimodal haptic glove's finger airbags before and after inflation, respectively, in embodiments provided by the present invention. Figure 5 and Figure 6 These are side views of the multimodal tactile glove finger airbag before and after inflation, respectively, according to an embodiment of the present invention. The thickness of the first silicone shell in the region near the fingertip is less than the thickness of other regions of the first silicone shell. The "air cavity" indicated by the dashed box in the figures is the first airbag actuator 1 in this embodiment, and the "air tube" in the figures is the first air tube in this embodiment.

[0052] Preferably, it also includes multiple heat sinks;

[0053] The heat sink is fixedly disposed on the surface of the cooling chip 2 and is used to dissipate heat from the cooling chip 2.

[0054] Preferably, it also includes a plurality of Velcro straps 5; the cooling element 2 is attached to the finger area through the Velcro straps 5.

[0055] Optionally, in this embodiment, a temperature actuator is arranged according to the human hand's sensitivity to temperature. The temperature actuator has heating and cooling functions and can adjust the different degrees of hot and cold during the handshake process.

[0056] Specifically, regarding temperature tactile feedback, the cooling pad 2 uses a TEC semiconductor cooling pad with a temperature cycling range of 0-60℃ and dimensions of 10mm*10mm*3mm. A heat sink of the same size is attached to the surface for easy heat dissipation. One cooling pad is placed near the proximal knuckle of each finger, using Velcro 5 to ensure full contact between the cooling pad 2 and the finger, adapting to fingers of different sizes and avoiding gaps that could affect the temperature feedback experience. Two cooling pads 2 are placed on the palm below the thumb on the thenar eminence. This arrangement stimulates temperature-sensitive areas, enhancing the user's temperature experience and preventing individual differences from causing some users to not perceive temperature changes.

[0057] Optionally, in this embodiment, vibration is used to simulate the shaking during a handshake, and different vibration modes are designed to distinguish the number and speed of shaking.

[0058] Furthermore, this invention employs haptic feedback to simulate the shaking during a handshake, designing different vibration modes to differentiate the number and speed of the shaking. For the haptic feedback, a linear motor 8 is used. The linear motor 8 measures 22.6mm*10mm*9mm and is placed at the base of the palm near the wrist. Specifically, an inertial sensor is positioned at the wrist to collect shaking data, which is then used by the linear motor 8 to adjust the vibration mode and present it to the user, allowing the user to perceive the number and speed of the handshake.

[0059] Figure 7 and Figure 8 The following are schematic diagrams of the multimodal tactile glove's palm airbag before and after inflation, as provided in the embodiments of the present invention. Figure 7 and Figure 8 As shown in the figure, the two dashed boxes indicate the "air chambers," which are the second airbag actuator 3 and the third airbag actuator 9 in this embodiment, respectively. The "air tube" in the figure refers to the second or third air tube in this embodiment. The second silicone shell is fitted and connected to the web area of ​​the glove, and the third silicone shell is fitted and connected to the ulnar side of the glove. The thickness of the area of ​​the second silicone shell near the palm is less than the thickness of other areas of the second silicone shell; the thickness of the area of ​​the third silicone shell near the palm is less than the thickness of other areas of the third silicone shell.

[0060] Figure 9 The following is a framework diagram of the multimodal tactile glove control structure and virtual handshake system provided in the embodiments of the present invention, such as... Figure 9As shown, this embodiment also provides a multimodal haptic feedback system for virtual handshakes, including the aforementioned multimodal haptic feedback glove for virtual handshakes, a hand motion tracking module (e.g., a pose tracking module and a Noitom glove), a virtual reality device (VR device), a host computer (host PC), a slave computer, an air pump, a motor drive module, a vibration drive module (vibration motor drive module), and a digital-to-analog converter chip.

[0061] The host computer is connected to the hand motion tracking device, the virtual reality device, and the slave computer respectively; the hand motion module is installed on various parts of the glove, and the hand motion tracking module is used to acquire the hand motion information of the user after wearing the glove; the hand motion information includes the joint angle information of the five fingers and the hand posture information; the virtual reality device is used to generate a virtual hand avatar in a virtual scene and to visualize the handshake scene.

[0062] When a user controls the virtual hand avatar to shake hands based on the hand motion tracking module, the host computer calculates handshake information based on the user's hand motion information and the positional relationship between the virtual hand avatars of the two parties shaking hands in the virtual scene. The handshake information includes handshake position information, force information, temperature information, and shaking information. The slave computer is connected to the air pump, the motor drive module, the vibration drive module, the digital-to-analog converter chip, and the inertial sensor. The air pump is connected to the digital-to-analog converter chip, which controls the air pump to inflate the palm airbag and finger airbags (five-finger airbags). The motor drive module is connected to the cooling element 2 (five-finger cooling element and palm cooling element), which controls the cooling element 2 to achieve temperature changes. The vibration drive module is connected to the linear motor 8 (linear vibration motor), which controls the vibration frequency of the linear motor 8.

[0063] The lower-level machine controls the working state of the digital-to-analog converter chip, the motor drive module, and the vibration drive module respectively based on the handshake information and the shaking data obtained by the inertial sensor, so that the user can experience force, temperature, and vibration tactile sensations of different handshake depths.

[0064] Preferably, it further includes at least one proportional servo valve; the proportional servo valve is disposed on the air pump, the proportional servo valve is connected to the digital-to-analog converter chip, and the proportional servo valve is used to adjust the air pressure delivered by the air pump to the palm airbag and / or the finger airbag.

[0065] Specifically, the control system of the multimodal haptic glove is as follows: Figure 9As shown, the host computer runs a virtual handshake interaction scenario, which can be displayed in a VR headset. The user wears motion capture gloves with trackers that determine the bending angles of the finger joints and the hand posture. This data drives a virtual hand avatar in the virtual scene, allowing the user to see their virtual hand avatar perform various hand movements, such as extending and shaking hands. When the user initiates a handshake with the virtual hand avatar in the virtual scene, the host computer communicates with the slave computer via a serial USB interface. The slave computer is an Arduino Mega2560 microcontroller, which, upon receiving the signal, drives the force haptic, temperature haptic, and vibration haptic components to provide the corresponding tactile sensations.

[0066] As mentioned above, pneumatic control is used for force tactile feedback. An air pump compresses air, which is then stored in an air tank. A pressure regulating valve at the gas outlet allows for manual pressure adjustment, ensuring a stable air pressure is delivered to the proportional valve. The air pump only activates when the air pressure in the tank falls below the threshold set by the pressure regulating valve, thus avoiding frequent start-stop cycles. A digital-to-analog converter chip converts the digital signal from the microcontroller into an analog signal, which is then sent to the proportional valve. The proportional valve dynamically adjusts the output air pressure based on the input voltage, providing controllable air pressure to the airbags. Two proportional valves can be used to control the five finger airbags and the palm airbag respectively, or multiple proportional valves can be used to independently control each airbag, resulting in a richer tactile experience. For temperature tactile feedback, the microcontroller parses and processes the serial port information before sending a signal to the L298N. The L298N's two drives control the five finger cooling elements and the palm cooling element respectively. Similarly, each cooling element 2 can be independently controlled. Positive and negative power supply allows the cooling element 2 to produce heating and cooling effects. In terms of tactile vibration, the signals provided by the DRV2605 vibration motor drive module can generate diverse vibration sensations on the motor to match the frequency and speed of shaking. It's worth noting that when simulating a deep grip, Figure 1 and Figure 2 All components with actuator functions operate; during a simulated shallow grip, the palm airbag and the palm cooling plate 2 will not operate since only the fingers are in contact. The aforementioned deep and shallow grips are common handshake techniques in real handshakes. By independently controlling the airbags in the fingers and palm, as well as the cooling plates 2 in the fingers and palm, the proposed multimodal haptic feedback glove can provide a richer tactile experience and is also within the scope of this invention.

[0067] In addition, the system in this embodiment may also include the aforementioned multimodal haptic feedback glove, hand motion tracking module, virtual scene display module, data acquisition module, and data communication module. The data acquisition module is used to acquire handshake shaking data; the data communication module is used for data transmission between the host computer and the slave computer (including handshake shaking data acquired by inertial sensors) and for data transmission between the two user terminals (i.e., the two parties shaking hands) (including hand position and posture data of the two parties shaking hands).

[0068] Specifically, the multimodal haptic feedback system for virtual handshakes calculates the handshake force based on the hand joint angle information of both parties in the handshake, collected by the hand motion tracking module, and the positional relationship of the virtual hand avatars of both parties in the virtual scene. The system then sends the force information (including the position and magnitude of the actuators), temperature information, and shaking information, along with corresponding control signals, to the lower-level computer via the data communication module. The lower-level computer controls the various actuators on the multimodal haptic glove, enabling the user to obtain the corresponding haptic experience. At the same time, the virtual scene is updated.

[0069] It is worth noting that the actuators on the multimodal haptic gloves that provide force, temperature, and vibration feedback are all adjustable. Combined with the positional layout of the force feedback actuators, they can provide users with 16 or more handshake experiences.

[0070] The beneficial effects of this invention are as follows:

[0071] (1) This invention proposes a multimodal haptic feedback glove for virtual handshakes, filling the gap in current haptic feedback gloves' inability to simulate the tactile experience of handshakes and providing hardware support for virtual handshake systems. Combined with virtual reality technology, this multimodal haptic feedback glove can be used to build a virtual handshake system, providing users with an immersive handshake experience.

[0072] (2) Based on the key elements of a real handshake, this invention proposes four dimensions that a haptic feedback glove for handshakes should possess: the depth of the handshake, the strength of the grip, the temperature (hot or cold), and the speed of the shake. Furthermore, these four dimensions are presented through three forms of haptic feedback: force, temperature, and vibration. It is worth noting that these three haptic experiences are continuously adjustable, allowing for the combination of 16 or more handshake experiences. This solves the problems of current haptic feedback gloves having a single form of haptic feedback and being limited to the fingertips.

[0073] (3) Based on the tactile sensitivity of different areas of the human hand, the handshake contact area and the distribution of contact force, the present invention optimizes the layout of the actuators on the tactile feedback glove, so as to provide users with a specific and distinct tactile experience without interfering with each other.

[0074] (4) Regarding handshake contact force feedback, the present invention uses flexible airbags made of silicone to improve wearing comfort and skin-friendliness. The flexible airbags are arranged in the finger and palm areas according to the actual distribution of handshake contact force. Deep and shallow handshake experiences can be achieved through individual control of the finger and palm airbags. Furthermore, the force provided by the flexible airbags is continuously adjustable, simulating different distributed grip force experiences.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the gloves disclosed in the embodiments, since they correspond to the systems disclosed in the embodiments, the description is relatively simple; relevant details can be found in the glove section.

[0076] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A multimodal haptic feedback glove for virtual handshakes, characterized in that, include: Gloves, linear motors, inertial sensors, palm airbags, multiple finger airbags, and cooling pads; Each finger section of the glove is equipped with a finger airbag, and the palm section of the glove is equipped with a palm airbag. The finger airbags, when inflated, compress the user's finger skin, hindering the bending of the user's finger joints within the glove, thus creating a handshake contact experience. The palm airbags, when inflated, compress the user's palm skin, thus creating a handshake contact experience. Cooling elements are respectively located in the palm and finger sections, and are used to provide temperature stimulation to the user. Both the inertial sensor and the linear motor are connected to a microcontroller. The inertial sensor is located at the wrist section of the glove and is used to collect the user's palm shaking data. The linear motor is located at the base of the palm section and is used to generate vibrations based on the shaking data, allowing the user to experience a handshake. The finger airbag includes: a silicone finger sleeve and a first airbag actuator disposed on the fingertip side of the silicone finger sleeve; The silicone finger sleeve is disposed on the finger; the first airbag actuator includes a first enclosed cavity, a first air tube, and a first silicone shell; the first enclosed cavity is disposed inside the first silicone shell, the first air tube is connected to an air pump and extends into the first enclosed cavity; the first silicone shell is connected to the fingertip side of the silicone finger sleeve.

2. The multimodal haptic feedback glove for virtual handshakes according to claim 1, characterized in that, It also includes multiple heat sinks; The heat sink is fixedly disposed on the surface of the cooling plate and is used to dissipate heat from the cooling plate.

3. The multimodal haptic feedback glove for virtual handshakes according to claim 1, characterized in that, It also includes multiple Velcro straps; The cooling element is attached to the finger area via Velcro.

4. The multimodal haptic feedback glove for virtual handshakes according to claim 1, characterized in that, The thickness of the first silicone shell in the area near the fingertip is less than the thickness of the other areas of the first silicone shell.

5. The multimodal haptic feedback glove for virtual handshakes according to claim 1, characterized in that, The palm airbag includes: a silicone palm sleeve, a ring-shaped second airbag actuator, and a third airbag actuator; the second airbag actuator and the third airbag actuator are connected through the silicone palm sleeve to form a closed ring structure; the ring structure is fitted onto the palm portion; the second airbag actuator includes a second closed cavity, a second air tube, and a ring-shaped second silicone shell; the second closed cavity is disposed within the second silicone shell, and the second air tube is connected to an air pump and extends into the second closed cavity; the third airbag actuator includes a third closed cavity, a third air tube, and a ring-shaped third silicone shell; the third closed cavity is disposed within the third silicone shell, and the third air tube is connected to an air pump and extends into the third closed cavity; a first end of the second silicone shell is connected to the first end of the third silicone shell through the silicone palm sleeve; a second end of the second silicone shell is connected to the second end of the third silicone shell through the silicone palm sleeve.

6. The multimodal haptic feedback glove for virtual handshakes according to claim 5, characterized in that, The second silicone shell is fitted and connected to the web area of ​​the glove, and the third silicone shell is fitted and connected to the ulnar side of the glove; the thickness of the area of ​​the second silicone shell near the palm is less than the thickness of other areas of the second silicone shell; the thickness of the area of ​​the third silicone shell near the palm is less than the thickness of other areas of the third silicone shell.

7. The multimodal haptic feedback glove for virtual handshakes according to claim 1, characterized in that, Two cooling elements are provided in the thenar eminence area of ​​the palm.

8. A multimodal haptic feedback system for virtual handshakes, characterized in that, The invention includes, as described in any one of claims 1 to 7, a multimodal haptic feedback glove for virtual handshakes, a hand motion tracking module, a virtual reality device, a host computer, a slave computer, an air pump, a motor drive module, a vibration drive module, and a digital-to-analog converter chip; The host computer is connected to the hand motion tracking module, the virtual reality device, and the slave computer. The hand motion tracking module is installed on various parts of the glove and is used to acquire hand motion information of the user after wearing the glove. The hand motion information includes the joint angle information of the five fingers and the hand posture information. The virtual reality device is used to generate a virtual hand avatar in a virtual scene and to visualize the handshake scene. When a user controls the virtual hand avatar to shake hands based on the hand motion tracking module, the host computer calculates handshake information based on the user's hand joint angle information and the positional relationship between the virtual hand avatars of the two parties shaking hands in the virtual scene. The handshake information includes handshake position information, force information, temperature information, and shaking information. The slave computer is connected to the air pump, the motor drive module, the vibration drive module, the digital-to-analog converter chip, and the inertial sensor. The air pump is connected to the digital-to-analog converter chip, which controls the air pump to inflate the palm airbag and finger airbag. The motor drive module is connected to the cooling chip, which controls the cooling chip to achieve temperature changes. The vibration drive module is connected to the linear motor, which controls the vibration frequency of the linear motor. The lower-level machine controls the working state of the digital-to-analog converter chip, the motor drive module, and the vibration drive module respectively based on the handshake information and the shaking data obtained by the inertial sensor, so that the user can experience force, temperature, and vibration tactile sensations of different handshake depths.

9. The multimodal haptic feedback system for virtual handshakes according to claim 8, characterized in that, It also includes at least one proportional servo valve; the proportional servo valve is disposed on the air pump, the proportional servo valve is connected to the digital-to-analog converter chip, and the proportional servo valve is used to adjust the air pressure delivered by the air pump to the palm airbag and / or the finger airbag.

Images