Haptic display device fusing tactile and somatosensory displays

Abstract

This invention relates to the field of human-computer interaction technology, providing a haptic display device that integrates tactile and motion-sensing displays. The tactile display unit in this device is adapted to conform to a localized area of ​​the user's skin and generates pressure feedback on the skin based on input signals. The motion-sensing display unit is adapted to be fixed to both sides of a joint on the user's body and generates torque feedback at the joint. A flexible connector passes through the motion-sensing display unit and connects to the tactile display unit. When the user's joints move, the tension of the flexible connector changes, and this tension change is transmitted to the tactile display unit to generate localized pressure, simultaneously generating corresponding torque feedback at the joint. This invention mechanically couples tactile and motion sensations together through a single flexible connector, ensuring a high degree of synchronization between the two sensations in time and intensity, thus enhancing the realism and immersion of the tactile feedback.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a tactile display device that integrates touch and motion sensing. Background Technology

[0002] In fields such as virtual reality (VR), augmented reality (AR), remote operation, and medical rehabilitation, haptic feedback technology is key to enhancing user immersion and the realism of operations. In recent years, with the rise of concepts such as virtual reality (VR) and the metaverse, deeply immersive experiences have become a core pursuit in technological development. Among all the sensory channels used to build immersion, haptic feedback, as a crucial element independent of audiovisual perception, is increasingly important.

[0003] When the human body physically interacts with objects in the real world, the central nervous system receives and processes two different types of tactile stimuli. The first is localized tactile sensation acting on the skin's surface, which allows us to perceive fine information such as the object's shape, size, surface texture, and vibrations. The second is systemic somatosensory or kinematic sensation acting on muscles and joints, which allows us to perceive the object's weight, inertia, resistance to movement, or acceleration, typically manifested as forces or torques acting at joints. Therefore, ideal tactile display technology needs to be able to simulate both of these sensations simultaneously.

[0004] Based on these two sensations, existing haptic feedback devices are generally divided into two main categories: one is haptic display devices, such as those using vibration motor arrays or pneumatic / electric pin arrays, which simulate tactile information by applying pressure, vibration, or temperature changes to localized skin areas such as fingertips and palms. The other is motion or haptic display devices, such as exoskeletons or linkage-type force feedback arms, which simulate large-scale motion resistance or macroscopic force interactions by applying torque to joints such as the user's fingers, wrists, and elbows.

[0005] In real-world physical interactions, the local pressure applied to the skin and the torque applied to the joints are intrinsically related and satisfy a torque balance relationship. For example, when pressing a finger on a table, the pressure felt by the fingertip and the torque felt by the finger joint are synchronous and related. However, most existing haptic display solutions are single-function, either tactile or motion-sensing. Even when the two types of devices are simply combined, the separation of their driving principles and application points makes it difficult for them to form an intrinsic physical coupling. This results in a mismatch between the local tactile sensation and the overall motion-sensing point of application and intensity, affecting the realism of the interaction. Summary of the Invention

[0006] This invention provides a tactile display device that integrates tactile and somatosensory displays. Tactile sensation (skin pressure) and somatosensory sensation (joint torque) are mechanically coupled together through a flexible connector (such as a rope), so that the generation of the two sensations is highly synchronized in time and intensity, thereby enhancing the realism and immersion of tactile feedback.

[0007] This invention provides a tactile display device that integrates tactile and motion-sensing displays, comprising: A haptic display unit, the haptic display unit being adapted to fit against a part of a user's skin and used to generate pressure feedback on the skin based on an input signal; A motion-sensing display unit, the motion-sensing display unit being adapted to be fixed to both sides of a joint on the user's body and used to generate torque feedback at the joint; A flexible connector passes through the motion-sensing display unit and is connected to the touch-sensing display unit; When the user's joints move, the tension of the flexible connector changes, and the tension change is transmitted to the haptic display unit through the flexible connector to generate local pressure, while generating corresponding torque feedback at the joint.

[0008] According to the haptic display device integrating tactile and motion-sensing display provided by the present invention, the haptic display unit is a tension-operated structure, comprising: substrate; At least one drive beam is disposed on the base plate, and each drive beam has at least one movable end; The flexible connector is connected to the movable end of each drive beam. When the flexible connector is tightened, the drive beam is compressed and undergoes out-of-plane buckling deformation, thereby causing its surface to bulge to generate pressure feedback on the skin.

[0009] According to the tactile display device integrating tactile and somatosensory display provided by the present invention, the tactile display unit further includes a constraint mechanism, which is disposed on the layout path of the drive beam and is used to constrain the movable end of the drive beam to move only along its length direction.

[0010] According to the haptic display device integrating tactile and somatosensory display provided by the present invention, the haptic display unit further includes a restoring elastic element, one end of which is connected to the movable end, and the other end of which is fixed to the substrate. The restoring elastic element is used to reset the drive beam when the flexible connector is released.

[0011] According to the haptic display device integrating tactile and somatosensory display provided by the present invention, the flexible connector is configured such that when the user's joint bends in a specified direction, the flexible connector is tightened, thereby increasing the pressure applied to the drive beam and causing a change in the degree of protrusion of the drive beam.

[0012] According to the haptic display device integrating tactile and motion-sensing display provided by the present invention, the haptic display unit is a release-actuated structure, comprising: substrate; At least one drive beam; A pre-tightening elastic element is placed between the base plate and each of the drive beams to apply a pre-set thrust to each of the drive beams and to make it bulge. The flexible connector is connected to each of the drive beams and is used to apply a tensile force to each drive beam that is opposite to the thrust of the pre-tightening elastic member; when the flexible connector is released, the drive beam bulges under the action of the pre-tightening elastic member to generate pressure feedback to the skin.

[0013] According to the haptic display device integrating tactile and somatosensory display provided by the present invention, the flexible connector is configured such that when the user's joint bends in a specified direction, the flexible connector is relaxed, thereby reducing the tension applied to the drive beam and increasing the degree of protrusion of the drive beam under the action of the pre-tightening elastic member.

[0014] According to the haptic display device integrating tactile and somatosensory display provided by the present invention, the somatosensory display unit includes a first hinge and a second hinge that are movably connected, and the first hinge and the second hinge are adapted to be fixed to both sides of a joint. The first hinge is provided with a first guide, and the second hinge is provided with a second guide and a third guide near its two ends, respectively; The tactile display unit is located between the second guide and the third guide, and is connected to the second guide and the third guide respectively; The first guide, the second guide, and the substrate of the touch display unit are all provided with wire holes. The flexible connector passes through each of the wire holes and is connected to the drive beam of the touch display unit.

[0015] The tactile display device integrating tactile and motion-sensing display provided by the present invention further includes: A drive component is disposed on the first hinge member, and the drive component is connected to the flexible connector for actively adjusting the tension or release state of the flexible connector. The tactile display device integrating tactile and motion-sensing display provided by the present invention further includes: A thermal display module is disposed on the tactile display unit and is used to generate a cold or hot sensation. The thermal display module is a semiconductor cooling chip, which is disposed on the substrate or drive beam of the tactile display unit, as well as other structures between the drive beam and the skin.

[0016] The haptic display device integrating tactile and haptic feedback provided by this invention mechanically couples tactile sensation (skin pressure) and haptic sensation (joint torque) together through a flexible connector (such as a cord). This ensures that the two sensations are highly synchronized and coordinated in both time and intensity, enhancing the realism and immersion of the haptic feedback. Users no longer perceive two separate signals, but rather a unified and coherent physical interaction.

[0017] Compared to setting up separate drive and control systems for tactile and motion sensing, this invention utilizes mechanical transmission to enable a single drive component and the user's own movement to simultaneously drive both types of feedback, simplifying the system structure and reducing energy consumption and cost. Furthermore, the design, which utilizes joint movement to naturally change the tension of the flexible connector (cord), can simulate many real-world physical phenomena, such as the torque generated at the joint when lifting a heavy object, or the localized reaction force felt when gripping an object. This passive feedback is very natural and responsive. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the structural schematic diagrams of a tactile display device that integrates tactile and somatosensory display provided in an embodiment of the present invention.

[0020] Figure 2 This is the second schematic diagram of the structure of the tactile display device that integrates touch and body sensation display provided in the embodiment of the present invention.

[0021] Figure 3 This is the third schematic diagram of the structure of the tactile display device that integrates touch and body sensation display provided in the embodiment of the present invention.

[0022] Figure 4 This is a wiring diagram of the tension action type tactile display unit provided in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the tension action type tactile display unit provided in an embodiment of the present invention.

[0024] Figure 6 This is an exploded view of the structure of the tension action type haptic display unit provided in an embodiment of the present invention.

[0025] Figure 7 This is a tensioning effect diagram of the tensioning action-type tactile display unit provided in an embodiment of the present invention.

[0026] Figure 8 This is a wiring diagram of the release action type tactile display unit provided in an embodiment of the present invention.

[0027] Figure 9 This is a schematic diagram of the structure of the release action type tactile display unit provided in an embodiment of the present invention.

[0028] Figure 10 This is a schematic diagram of the drive beam provided in an embodiment of the present invention.

[0029] Figure label: 10. Touch display unit; 11. Substrate; 12. Drive beam; 13. Constraint mechanism; 14. Resilience member; 15. Pre-tightening member; 20. Motion-sensing display unit; 21. First hinge; 22. Second hinge; 23. First extension; 24. Second extension; 25. First guide; 26. Second guide; 27. Third guide; 28. Through hole; 30. Flexible connector; 40. Drive assembly; 50. Thermal display module. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application based on the specific circumstances.

[0032] In the embodiments of this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0033] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0034] Because the human limbs have a wide range of motion, but when interacting with fixed objects (such as walls or tabletops) in a fine manner, the hands often move only within a very small space, while the perceived reaction force changes drastically. This places stringent cross-scale accuracy requirements on systems that rely solely on position tracking to drive force feedback. The system needs to support large-scale motion tracking while simultaneously achieving high-precision positioning and force control within a very small area at the moment of contact. This makes the system implementation extremely complex, costly, and lacking in robustness.

[0035] In highly dynamic applications such as motion-sensing games, users move at high speeds and exert significant force. This requires force feedback devices to not only respond quickly but also possess sufficient mechanical stiffness to provide clear feedback, cushioning impacts and simulating rigid contact. Traditional displacement or force control modes based on motor servos often sacrifice output stiffness in pursuit of high dynamic response, or exhibit sluggish response while maintaining stiffness, making it difficult to provide a natural and powerful feedback experience in virtual reality interactions.

[0036] Therefore, the present invention provides a tactile display device that integrates tactile and motion sensing to solve the problems of separation of tactile and motion sensing, strong dependence on high-precision tracking systems, and insufficient feedback stiffness in the prior art.

[0037] See Figures 1 to 3This invention provides a tactile display device that integrates tactile and motion-sensing displays. The tactile display device includes a tactile display unit 10, a motion-sensing display unit 20, and a flexible connector 30.

[0038] The haptic display unit 10 is a module that generates local skin sensation and comes into direct contact with the user's skin. The haptic display unit 10 is used to generate pressure on local skin areas of the user's body (such as fingertips, palms, and arm surfaces) based on signals, such as normal pressure, which is pressure perpendicular to the skin surface.

[0039] The haptic display unit 10 includes at least one haptic pixel for applying localized pressure to human skin. The haptic pixel is the smallest unit in the haptic display unit 10 that performs a specific haptic stimulus. The haptic pixel can be a retractable pin, a small bump that can change its orientation, or any other mechanism capable of converting driving force into normal pressure on the skin. When the haptic pixel protrudes or the pressure on the skin increases, the user perceives a localized haptic sensation.

[0040] The motion-sensing display unit 20 is a module that generates joint force feedback. Here, "motion-sensing" specifically refers to proprioception or kinesthesia, that is, the user's perception of joint position, movement, and force. The motion-sensing display unit 20 is fixed to both sides of the user's joints (e.g., both sides of the finger joints, wrist joints, or elbow joints) to generate or resist a torque during joint movement, allowing the user to feel resistance or guided force at the joint.

[0041] The motion-sensing display unit 20 includes a first hinge 21 and a second hinge 22 that are movably connected. The first hinge 21 and the second hinge 22 can move relative to each other. The first hinge 21 and the second hinge 22 are located on both sides of the joint and can rotate around the joint in a specified direction, just like the joint movement of the human body to stretch and bend.

[0042] In other words, the first hinge 21 and the second hinge 22 are configured to be detachably fixed to both sides of a specific joint on the user's body. For example, when applied to a finger joint, the first hinge 21 can be fixed to one of the user's phalanges (such as the proximal phalanx), while the second hinge 22 is fixed to an adjacent phalanx (such as the middle phalanx). When the user bends or straightens their finger, the first hinge 21 and the second hinge 22 rotate relative to each other with the movement of the bones, their axes of motion substantially coinciding with or parallel to the natural axis of motion of the human joint, thereby reproducing the joint's movement.

[0043] The first hinge member 21 is provided with a first guide member 25, and the second hinge member 22 is provided with a second guide member 26 and a third guide member 27 near its two ends, respectively. The first guide member 25, the second guide member 26, and the third guide member 27 are all circular or semi-circular structures that serve to support and guide. It should be noted that the structural form of the first guide member 25, the second guide member 26, and the third guide member 27 is not limited. They can be annular parts with multiple through holes. The properties of each guide member are adapted to the shape of the user's position, so that each guide member forms a reliable connection with the flexible connector 30.

[0044] The touch display unit 10 is disposed between the second guide member 26 and the third guide member 27, and is fixedly connected to the second guide member 26 and the third guide member 27 respectively. The first guide member 25, the second guide member 26 and the substrate 11 of the touch display unit 10 are all provided with wire holes 28. The wire holes 28 can be cable holes. The flexible connector 30 passes through each wire hole 28 and is connected to at least one drive beam 12 on the touch display unit 10.

[0045] To effectively convert the driving force into torque acting on the joint, in some embodiments of the present invention, a first extension 23 may be detachably connected to the first hinge 21. The first extension 23 extends outward from the body of the first hinge 21 to form an effective lever arm, and a first guide 25 may be disposed on the first extension 23. A second extension 24 may be detachably connected to the second hinge 22. The second extension 24 extends outward from the body of the second hinge 22 to form an effective lever arm, and a third guide 27 may be disposed at the end of the second extension 24 or at a suitable position. A second guide 26 may be disposed on the second hinge 22 near the hinge point of the first hinge 21. A flexible connector 30 passes through wire holes 28 on the first guide 25 and the second guide 26, and through wire holes 28 on the substrate 11 of the touch display unit 10, and then connects to each drive beam 12 of the touch display unit 10.

[0046] The generation and transmission of torque depend on the coordinated operation of the flexible connector 30 and the haptic display unit 10. A flexible connector 30 (e.g., a high-strength, low-strength cable, fishing line, or special fiber filament) is cleverly positioned, its path passing through or wrapping around the first guide 25 and the second guide in sequence. At least one end of the flexible connector 30 is connected to an active force source, namely the haptic display unit 10. When the flexible connector 30 is tightened, the resulting tension is converted into a torque that attempts to bring the first hinge 21 and the second hinge 22 closer together or further apart, and this torque ultimately acts on the user's joint.

[0047] By controlling the magnitude and direction of this torque, the present invention can achieve two main functions: resisting user joint movement: when the user attempts to bend the joint, a counter-torque is applied to simulate the damping sensation when gripping a heavy object, pushing away an obstacle, or moving in a viscous liquid. Guiding user joint movement: a torque is actively applied to guide the user's joint towards a predetermined angle or along a specific trajectory.

[0048] In a practical implementation, if the display is a wearable device, the first guide 25, the second guide 26, and the third guide 27 can be supports fixed to two adjacent limbs of the human body, respectively. For example, a support fixed to the forearm (first guide 25) and a support fixed to the upper arm (second guide 26 and third guide 27) are connected by a mechanical hinge simulating an elbow joint, such as a joint structure formed by a first hinge 21 and a second hinge 22.

[0049] The flexible connector 30 can be understood as a flexible cable, tendon, or high-strength thin wire. The flexible connector 30 passes through the motion-sensing display unit 20 and connects to the touch-sensing display unit 10, playing a role in mechanical coupling and linking the two separate display units together.

[0050] In other words, changes in the effective length (i.e., tension) of the flexible connector 30 directly alter the physical state (such as height and pressure) of the tactile pixels. For example, when the flexible connector 30 needs to be tightened to generate feedback, it can be wired along the outer side of the joint bend. Thus, when the joint bends in a specified direction, the effective path between the two fixed points of the flexible connector 30 becomes longer, and the flexible connector 30 is passively tightened. Conversely, if the flexible connector 30 needs to be relaxed to generate feedback, it can be wired along the inner side of the joint bend.

[0051] The tactile display device that integrates tactile and somatosensory display provided by the present invention, taking a device worn on the elbow as an example, has a flexible connector 30 wired along the outside of the elbow joint.

[0052] When a user bends their elbow, the flexible connector 30, located on the outside of the joint, stretches its two fixed points, increasing tension. This increased tension, through its lever arm at the joint, generates a torque that resists elbow flexion. The user experiences a resistance, seemingly from the outside, preventing them from bending their arm—this is the haptic feedback.

[0053] Simultaneously, the increased tension of the flexible connector 30 drives the haptic pixels through its connection with the haptic display unit 10. For example, tension might cause a lever action, pressing a pin against the user's forearm skin. The user will feel a clear, localized pressure point, i.e., haptic feedback. The more the joint bends, the greater the tension of the flexible connector 30, and the greater the pressure felt on the skin.

[0054] Both the tactile torque and the tactile pressure feedback originate from the same physical quantity: the tension change of the flexible connector 30. The tactile torque and the tactile pressure occur synchronously, are positively correlated in strength, and are generated by the same simple mechanical structure, achieving an inherent physical coupling.

[0055] In some embodiments of the present invention, the haptic display device integrating touch and motion sensing further includes a drive component 40, which can be independent of the user's joint movement and is connected to the flexible connector 30. The drive component 40 actively adjusts the tension or release of the flexible connector 30, thereby actively adjusting the state of the entire haptic display device integrating touch and motion sensing by tightening or loosening the flexible connector 30.

[0056] For example, in a virtual reality (VR) environment, when a user's hand touches a virtual object, the drive component 40 can instantly tighten the flexible connector 30, while simultaneously generating a tactile sensation (normal pressure) on the user's skin and a blocking force (torque feedback) at the joint, simulating a realistic feeling of touching a physical object.

[0057] It is understood that the haptic display device integrating tactile and haptic feedback provided in this embodiment of the invention mechanically couples tactile sensation (skin pressure) and haptic sensation (joint torque) together through a flexible connector 30 (such as a rope), making the generation of the two sensations highly synchronized and coordinated in time and intensity, thus enhancing the realism and immersion of the haptic feedback. The user no longer perceives two separate signals, but rather a unified and coherent physical interaction.

[0058] Compared to setting up separate drive and control systems for tactile and motion sensing, this invention utilizes mechanical transmission to enable a single drive component 40 and the user's own movement to simultaneously drive both types of feedback, simplifying the system structure and reducing energy consumption and cost. Furthermore, the design of naturally changing the tension of the flexible connector 30 (cord) through joint movement can simulate many real-world physical phenomena, such as the localized pressure and torque on the joint when lifting a heavy object, or the reaction force felt when gripping an object. This passive feedback is very natural and responsive.

[0059] The feedback is directly related to physical changes in joint angles, rather than relying on measurements and complex calculations from external sensors (such as cameras and IMUs). Haptic display devices that integrate haptic and motion-sensing displays have self-sensing and self-feedback characteristics, providing a certain degree of physical pressure feedback even in the event of lost or delayed tracking signals. This pressure feedback can be normal or tangential pressure applied to the skin.

[0060] See Figures 4 to 7 In some embodiments of the present invention, the haptic display unit 10 is a tension-operated structure. The haptic display unit 10 includes a substrate 11 and at least one drive beam 12. The substrate 11 serves as the supporting foundation for the entire haptic display unit 10. The drive beam 12 is a deformable member, similar to a slender beam. One end (fixed end) of the drive beam 12 is connected to the substrate 11, and the other end (movable end) of the drive beam 12 is free or movable. Multiple drive beams 12 can be provided. The embodiments of the present invention are described using a single drive beam 12 as an example.

[0061] The flexible connector 30 is connected to the movable end of each drive beam 12. When the drive assembly 40 tightens the flexible connector 30, the drive beam 12 is compressed and undergoes out-of-plane buckling deformation, causing its surface to bulge and generate normal pressure. Specifically, when the drive assembly 40 tightens the tensioning cable, the cable applies a pressure (axial pressure) pointing towards the fixed end to the movable end of the drive beam 12.

[0062] As this axial pressure increases, when the pressure reaches a critical value, the originally straight drive beam 12 will become unstable and arch or bulge outward in a direction outside its plane (i.e., perpendicular to the substrate 11). The surface of this bulge will directly contact and press against the user's skin, thereby generating the required normal pressure. The greater the tension, the greater the axial pressure on the drive beam 12, the greater its buckling deformation (i.e., the height of the bulge), and the stronger the generated normal pressure.

[0063] It is understood that, in this embodiment of the invention, the axial tension of the flexible connector 30 (rope) is converted into the normal displacement and pressure of the drive beam 12. Through a mechanical amplification mechanism, even minute changes in rope tension can cause a perceptible pressure output. The deformation of the drive beam 12 itself is passive, requiring no independent motor or actuator; it is driven solely by the tension of the rope, resulting in a simple, lightweight structure with low power consumption.

[0064] In some embodiments of the present invention, the tactile display unit 10 further includes a constraint mechanism 13, which is disposed on the layout path of the drive beam 12 and is used to constrain the movable end of the drive beam 12 to move only along its own length direction, such as a slide, guide rail or flexible hinge. Its function is to limit the degree of freedom of movement of the movable end, so that it can only or mainly move along the length direction (axial direction) of the drive beam 12.

[0065] Without constraints, the movable end may undergo lateral torsion or displacement under compression, resulting in unstable and unreliable buckling deformation. The constraint mechanism 13 ensures that the movable end can move stably along the axial direction under compression, making the buckling direction and shape of the drive beam 12 predictable and repeatable.

[0066] It is understood that, based on the above embodiments, the present invention adds a constraint mechanism 13 to optimize the performance of the tension-operated structure. The constraint mechanism 13 ensures that the drive beam 12 deforms in a consistent manner during each actuation, avoiding chaotic motion and failure.

[0067] In some embodiments of the present invention, the tactile display unit 10 further includes a restoring elastic element 14, one end of which is connected to the movable end, and the other end of which is fixed to the substrate 11. The restoring elastic element 14 is used to reset the drive beam 12 when the flexible connector 30 is released. For example, it can be a small tension spring or a piece of elastic material. The restoring elastic element 14 is connected to the movable end, and its elastic force is opposite to the direction of the tension in the cord, i.e., it always attempts to pull the movable end back to its initial position.

[0068] When the drive assembly 40 releases the cord (the tension decreases or disappears), the pressure of the cord on the movable end also decreases. At this time, the elastic force of the restoring elastic element 14 pulls the movable end back to its original position, thereby restoring the buckled drive beam 12 to a straight state and relieving the pressure on the skin.

[0069] It is understood that, based on the above embodiments, this invention further adds a restoring elastic element 14. The restoring elastic element 14 ensures that the tactile unit can quickly and reliably reset after the drive signal is withdrawn, preparing for the next dynamic feedback, thereby improving the dynamic performance of the device. It should be noted that, without the restoring elastic element 14, the drive beam 12 can also rely on the elasticity of the skin itself to reset the mechanism.

[0070] In some embodiments of the present invention, when the user's joint bends in a specified direction, the flexible connector 30 is tightened, thereby increasing the pressure applied to the drive beam 12 and increasing the degree of protrusion of the drive beam 12.

[0071] For example, a haptic display device integrating touch and motion sensing is worn on a finger. The motion sensing display unit 20 is fixed to both sides of the finger joint, and the haptic display unit 10 is placed on the fingertip. A cord passes around the joint from the back of the finger. When the finger is bent, the path on the back of the finger becomes longer, the cord is tightened, and the tension increases. The increased tension is transmitted to the tension-actuated haptic display unit 10, increasing the axial pressure on its drive beam 12. The protrusion of the drive beam 12 increases, and the pressure felt by the user on the fingertip also increases accordingly.

[0072] This design establishes a positive correlation between the degree of joint flexion in a specified direction and the intensity of skin pressure. It aligns with the natural feeling of grasping an object: the tighter the grip (the greater the degree of joint flexion in the specified direction), the greater the pressure the object exerts on the fingers. This feedback is generated naturally without conscious effort, resulting in a highly realistic effect.

[0073] For example, when a haptic display device integrating tactile and motion sensing is worn on the arm, the flexible connector 30 should be positioned along the outer side of the arm. When the user bends their elbow, the effective length of the flexible connector 30 on the outer side of the elbow joint is stretched, resulting in increased tension in the flexible connector 30. This increased tension, on the one hand, causes the drive beam 12 to buckle through the aforementioned mechanism, generating localized tactile pressure; on the other hand, this tension acts at the joint, generating a torque that resists elbow bending. The user thus simultaneously feels the pressure point on the skin of the arm and the resistance to elbow movement, the two feedbacks naturally merging. When the user straightens their arm, the tension in the flexible connector 30 decreases, the elastic element returns the drive beam 12 to its original position, and the tactile and motion sensing feedback weakens or disappears.

[0074] See Figure 8 and Figure 9 In some embodiments of the present invention, the tactile display unit 10 is a release-operated structure, and the tactile display unit 10 includes a substrate 11, at least one drive beam 12, and a pre-tensioning elastic member 15. The substrate 11 and the drive beam 12 are basically the same as those in the foregoing embodiments, and the structure of the drive beam 12 is the same, and will not be described again here.

[0075] The preload elastic element 15 is the core component of the release-actuated structure, such as a compression spring or elastomer. The preload elastic element 15 is disposed between the base plate 11 and each drive beam 12, applying a continuous thrust to each drive beam 12 that causes it to bulge outwards. When no external force is applied, the drive beam 12 is in a bulging state under the action of the preload elastic element 15. In other words, the preload elastic element 15 is positioned below the drive beam 12 (e.g., at the midpoint), always exerting an upward thrust on the beam, attempting to make it bulge. The tactile pixels are linked to the beams, therefore the tactile pixels have a natural tendency to move upwards.

[0076] The flexible connector 30 is connected to the beam and is used to apply a tensile force to the beam in the opposite direction to the action of the preload elastic member 15. That is, this tensile force is opposite to the thrust of the preload elastic member 15, which attempts to flatten the protruding drive beam 12.

[0077] In the default or stress-free state, the drive assembly 40 tightens the cable, using sufficient tension to overcome the thrust of the pre-tensioning elastic element 15, keeping the drive beam 12 flat or nearly flat. When normal pressure is required, the drive assembly 40 releases (relaxes) the cable, reducing the tension on the drive beam 12. As the tension decreases, the thrust of the pre-tensioning elastic element 15 gradually becomes dominant, pushing the drive beam 12 outward, thereby generating normal pressure on the user's skin. The greater the release, the smaller the tension, the higher the drive beam 12 bulges under the action of the spring, and the stronger the pressure.

[0078] In essence, the release-actuated haptic display device integrating tactile and motion sensing provided in this embodiment of the invention operates on the opposite principle to the aforementioned embodiments. Specifically, unlike the tension-actuated control method, the reduction (release) of tension in the flexible connector 30 leads to the protrusion or increased pressure of the haptic pixels. This release-actuated mechanism may be more advantageous in certain applications, such as simulating an object that is present by default but removed by external force. It can be designed so that when the cable breaks or the drive system loses power, the system returns to a specific state of pressure or no pressure, increasing design flexibility.

[0079] In this embodiment of the invention, the flexible connector 30 functions to counteract the pre-tightening elastic member 15. The flexible connector 30 passes through the plane of the substrate 11 and connects to the beam, applying a downward tensile force to the beam. In the initial state or under high tension, the tensile force of the flexible connector 30 is greater than or balances the thrust of the pre-tightening elastic member 15, pressing the beam and haptic pixels into a retracted or low-pressure state.

[0080] When the tension of the flexible connector 30 decreases or is released, its downward pulling force is insufficient to counteract the pre-tensioned elastic element 15. At this time, the energy of the pre-tensioned elastic element 15 is released, pushing the beam upward, thereby causing the tactile pixel to bulge or increase pressure. This structure can typically achieve a larger stroke (bulge height) than the tension-actuated type, and has the advantage of a high stroke-to-thickness ratio.

[0081] In some embodiments of the present invention, the flexible connector 30 is configured such that when the user's joint bends in a specified direction, the flexible connector 30 is relaxed, thereby reducing the tension applied to the drive beam 12 and increasing the degree of protrusion of the drive beam 12 under the action of the pre-tightening elastic member 15.

[0082] For example, a haptic display device that integrates tactile and motion-sensing displays can be worn on a finger, with a cord wrapping around the joint from the fingertip side. When the finger is bent, the path on the fingertip side shortens, and the cord passively relaxes.

[0083] In other words, when the user bends their joint, the cord on one side is passively relaxed, reducing tension. This reduced tension is transmitted to the corresponding release-type haptic display unit 10, weakening the force resisting the pre-tensioning elastic member 15. As a result, the pre-tensioning elastic member 15 can more strongly push the drive beam 12 to protrude, increasing the degree of protrusion and thus increasing the pressure felt by the user on their skin.

[0084] Although the internal mechanical principle of this invention is opposite to that of the above embodiments, by matching different cord arrangement methods, this invention also achieves a positive correlation between the degree of joint bending in a specified direction and the skin pressure intensity. The most suitable actuation structure and arrangement scheme can be selected according to specific needs (such as space constraints and material properties).

[0085] For example, when wearing a haptic display device that integrates tactile and motion sensing on the arm, the flexible connector 30 should be arranged along the inside of the arm (in the elbow crease). When the user bends their elbow, the effective length of the flexible connector 30 on the inside of the joint shortens, causing the tension of the flexible connector 30 to decrease or loosen.

[0086] The relaxation of the flexible connector 30 allows the pre-tensioned elastic element 15 to drive the tactile pixels to produce localized tactile sensation. Simultaneously, since the flexible connector 30 is tensioned in its initial state (arm extended), it generates a torque against straightening at the joint, which decreases when bending. In this way, joint movement is correlated with changes in both localized tactile sensation and overall body sensation.

[0087] It should also be noted that a movable connector can be fixedly connected to a tension spring on the base plate 11. Pulling this tension spring will not affect the local tactile sensation, but it can change the torque at the joint. Since the torque exerted by local pressure on the joint will not be exactly equal to the actual torque generated, this movable connector can play a role in fine-tuning the torque.

[0088] In some embodiments of the present invention, the haptic display device integrating tactile and motion-sensing displays further includes a driving component 40, which is disposed on the first hinge 21 and connected to the flexible connector 30, for actively adjusting the length or tension of the flexible connector 30 to independently control the local pressure generated by the haptic pixels and / or the torque at the joint.

[0089] In other words, this embodiment of the invention provides a haptic display device with active control that integrates haptic and somatosensory display. Based on the embodiment where the feedback is entirely determined by the user's joint movements, by adding a driving component 40, it also provides a hybrid active and passive solution.

[0090] The drive component 40 may be selected from at least one of the following: a linear servo, a rotary servo connected to a winch, a tensioning mechanism that tensions the rope by pressing the middle of the rope, and also includes emerging drive components such as artificial muscles and shape memory alloys.

[0091] A linear servo is a type of motor that can directly output linear motion. Its output end is directly connected to a cable, and by extending or retracting it, the cable can be tightened or loosened directly.

[0092] The rotary servo connected to the winch is a combined structure, that is, a winch or spool is installed on a regular rotary servo (outputting rotary motion). The cable is wound on the winch, and the cable is tightened or released by the forward or reverse rotation of the servo, thereby actively adjusting the length or tension of the flexible connector 30.

[0093] A tensioning mechanism that tightens a rope by pressing down on its center is a different tensioning principle. The ends of the rope are fixed, and the mechanism uses an actuator (such as a small push rod) to press down on the center of the rope from the side, deviating it from a straight path. According to geometry, this increases the total tension of the rope.

[0094] When the user's joints are stationary, the drive component 40 can operate independently. For example, in VR, if the hand is stationary but touches a movable virtual creature, the drive component 40 can adjust the extension and retraction of the flexible connector 30 according to the movement of the virtual object, thereby allowing the tactile pixels to generate pulsating pressure feedback.

[0095] When the user's joints move, the drive component 40 can add or reduce feedback on top of passive feedback. For example, if the user bends their arm (passively generating resistance), and the object in the virtual scene becomes heavier, the servo motor can further tighten the flexible connector 30 to enhance the overall force feedback intensity. Conversely, if the object is lowered, the motor can actively release a section of the flexible connector 30 to counteract the passively generated tension, making the user feel a lightness in their arm.

[0096] It is understood that, through the introduction of the driving component 40, this embodiment of the invention upgrades the haptic display from a purely passive mechanism to a programmable active haptic interface. This configuration not only retains the advantages of a passive solution—simple structure and natural feedback—but also enriches the displayable haptic effects, enabling the simulation of more complex and dynamic interactive scenarios.

[0097] See Figure 10 In some embodiments of the present invention, the tactile display device that integrates tactile and body-sensing displays further includes a thermal display module 50, which is disposed on the tactile display unit 10 and is used to generate a cold or hot sensation.

[0098] Furthermore, the thermal display module 50 is specifically defined as a thermoelectric cooler, and its mounting position is explicitly stated to be on the drive beam 12. A thermoelectric cooler is a device that utilizes the Peltier effect. When direct current passes through it, one side absorbs heat (cooling) while the other side releases heat (heating). By changing the direction and magnitude of the current, the surface temperature can be precisely controlled to achieve cooling or heating. Placing the thermoelectric cooler on the drive beam 12 means that when the protrusion of the drive beam 12 contacts the user's skin, the user can not only feel pressure but also simultaneously feel the cold or hot sensation generated by the cooler.

[0099] Understandably, by introducing a temperature dimension, this embodiment of the invention upgrades the tactile experience from a single force feedback to a composite sensation of force and temperature. This is crucial for simulating touching objects of different materials (such as cold metal or warm skin), sensing liquid flow, or changes in ambient temperature, thereby enhancing the immersion and realism of the virtual environment. Furthermore, placing the thermal module directly on the pressure-generating drive beam 12 ensures that the sensations of pressure and temperature overlap in space. The user experiences temperature at the same point where pressure is felt, conforming to the physical laws of the real world and avoiding the unrealistic feeling caused by sensory separation.

[0100] This invention also provides a fusion haptic display method, applied to the haptic display device that integrates haptic and motion sensing displays provided in the above embodiments. The fusion haptic display method includes the following steps: Step S100: By moving the two sides of the joint relative to each other, the effective length of the flexible connector 30 is passively changed to generate local pressure and joint torque related to the joint angle simultaneously; that is, by moving the two sides of the joint relative to each other, the tension of the flexible connector 30 is passively changed to generate local pressure and joint torque related to the joint angle simultaneously.

[0101] This process is entirely driven by the user's physical movements, requiring no external calculations or commands. When the user wears the device and moves their body joints (e.g., bending or straightening the elbow or fingers), the first hinge 21 and the second hinge 22 connected to both sides of the joint move relative to each other. Depending on how the flexible connector 30 is positioned across the joint (e.g., on the outside or inside of the joint), this movement will inevitably and physically cause the flexible connector 30 to tighten or loosen, thereby changing its tension.

[0102] As mentioned earlier, the change in tension of the flexible connector 30 produces two effects simultaneously: it generates a motion-related torque (resistance or assistance) at the joint, i.e., haptic feedback; and it drives the haptic display unit 10, changing its local pressure on the skin, i.e., haptic feedback.

[0103] This step generates a physical-based feedback. The strength of this step is directly related to the user's joint angle. This layer of feedback has the advantages of zero latency, high fidelity, and absolute stability because it originates from Newtonian mechanics rather than digital signals, providing the user with continuous, natural proprioceptive information about their posture and underlying interaction forces.

[0104] Step S200: The tension of the flexible connector 30 is actively adjusted by the drive component 40 to superimpose actively generated tactile information on the basis of passive feedback.

[0105] This step describes the system's information feedback layer (active feedback), which is key to achieving complex and dynamic tactile effects.

[0106] In a haptic display device integrating tactile and motion sensing, the drive component 40 (such as a servo motor) is driven by instructions from an external controller (e.g., a VR host or computer). The controller calculates in real-time the additional or reduced force required based on events and interaction logic in the virtual environment and commands the drive component 40 to perform corresponding actions (tightening or loosening the flexible connector 30). The total movement of the movable connector at the user's skin tactile points is the sum of passive physical and active adjustment, i.e.: L 总 =L 被动 (Joint angle) + L 主动 (Program instructions) Specific application scenarios are illustrated below: Static active feedback: The user's arm remains still (L) 被动 (This is a constant). At this moment, the virtual hand touches a beating heart. The controller commands the drive component 40 to periodically and rapidly extend and retract the flexible connector 30, thereby generating a pulse-like L... 主动 This causes corresponding pulse-like bumps in the tactile pixels of the user's arm. The user will then feel a pulsating sensation on the skin and joints of their arm, synchronized with the virtual heartbeat.

[0107] Dynamic augmented feedback: The user is bending their arm to lift a virtual bucket (L) 被动 As the bending increases, it generates basic resistance. Meanwhile, in the virtual scene, someone is pouring water into a bucket. The controller commands the drive component 40 to further tighten the flexible connector 30, F. 主动 It is a positive value that increases with time. The total resistance F felt by the user. 总 It will be larger than when simply bending the arm, thus realistically simulating the process of the bucket becoming heavier.

[0108] Dynamic reduction feedback: The user is stretching a virtual rubber band (arm straight, F) 被动(Very large, generating enormous resistance). Suddenly, the virtual rubber band snaps. The controller immediately commands the drive component 40 to rapidly release a large section of the flexible connector 30, generating a momentary, large negative force F. 主动 To counteract F 被动 The user's experience of F 总 It will instantly approach zero, thus experiencing a realistic breaking sensation as the tension disappears instantly.

[0109] The fusion haptic display method provided in this invention, compared to simple passive or active feedback methods, combines the foundation of passive feedback with the details of active feedback to simulate highly complex and dynamic haptic events far exceeding those of traditional devices, such as changing object weight, vibrations at different frequencies, and abrupt changes in material stiffness. Basic posture force feedback is guaranteed by the physical structure, exhibiting zero latency and high fidelity; while rich dynamic information is provided by the flexible drive component 40, which can be arbitrarily programmed by software.

[0110] In many scenarios, the primary force feedback is provided by the user's own motion (passive feedback), and the drive component 40 only needs to provide additional differential force or high-frequency signals. This reduces the power and continuous work requirements of the drive component 40, making the device more energy-efficient and lighter.

[0111] It is understood that the embodiments of the present invention not only provide a tactile display device that integrates tactile and motion-sensing displays, but also a set of matching tactile display methods. By utilizing the dual capabilities of passive feedback and active actuation in the device, an excellent balance is achieved between the realism of force feedback, response speed, and expressiveness.

[0112] It should be noted that the haptic display device integrating tactile and motion sensing in this embodiment of the invention only provides a structural demonstration of the mechanical coupling of tactile and motion sensing for a single haptic feedback unit. It is understood that in actual use, multiple haptic feedback units can be set up, connected to the drive end using multiple cables, and through the reasonable design of the positions of the first guide 25, the second guide 26, and their threaded holes, the unification of torque feedback and haptic feedback can be achieved.

[0113] Meanwhile, the embodiments of the present invention provide Figure 1 and Figure 2 Only single-degree-of-freedom joint force feedback was demonstrated. It is understandable that multi-degree-of-freedom joint rotation can be achieved through ball joints or other structural designs, covering joints such as the shoulder. Correspondingly, bending and torque moments can be generated at the ball joints through cable tension.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tactile display device integrating tactile and motion-sensing display, characterized in that, include: A haptic display unit, the haptic display unit being adapted to fit against a part of a user's skin and used to generate pressure feedback on the skin based on an input signal; A motion-sensing display unit, the motion-sensing display unit being adapted to be fixed to both sides of a joint on the user's body and used to generate torque feedback at the joint; A flexible connector passes through the motion-sensing display unit and is connected to the touch-sensing display unit; When the user's joints move, the tension of the flexible connector changes, and the tension change is transmitted to the haptic display unit through the flexible connector to generate local pressure, while generating corresponding torque feedback at the joint.

2. The tactile display device integrating tactile and motion-sensing display according to claim 1, characterized in that, The tactile display unit is a tension-operated structure, including: substrate; At least one drive beam is disposed on the base plate, and each drive beam has at least one movable end; The flexible connector is connected to the movable end of each drive beam. When the flexible connector is tightened, the drive beam is compressed and undergoes out-of-plane buckling deformation, thereby causing its surface to bulge to generate pressure feedback on the skin.

3. The tactile display device integrating tactile and motion-sensing display according to claim 2, characterized in that, The tactile display unit also includes a constraint mechanism located on the layout path of the drive beam, which is used to constrain the movable end of the drive beam to move only along its own length direction.

4. The tactile display device integrating tactile and motion-sensing display according to claim 3, characterized in that, The haptic display unit also includes: A restoring elastic element is provided, one end of which is connected to the movable end, and the other end of which is fixed to the substrate. The restoring elastic element is used to reset the drive beam when the flexible connector is released.

5. The tactile display device integrating tactile and motion-sensing display according to claim 4, characterized in that, The flexible connector is configured such that when the user's joint bends in a specified direction, the flexible connector is tightened, thereby increasing the pressure applied to the drive beam and causing a change in the degree of bulging of the drive beam.

6. The tactile display device integrating tactile and motion-sensing display according to claim 1, characterized in that, The haptic display unit is a release-operated structure, including: substrate; At least one drive beam; A pre-tightening elastic element is placed between the base plate and each of the drive beams to apply a pre-set thrust to each of the drive beams and to make it bulge. The flexible connector is connected to each of the drive beams and is used to apply a tensile force to each of the drive beams that is opposite to the thrust of the pre-tightening elastic member; when the flexible connector is released, the drive beam bulges under the action of the pre-tightening elastic member to generate pressure feedback on the skin.

7. The tactile display device integrating tactile and motion-sensing display according to claim 6, characterized in that, The flexible connector is configured such that when the user's joint bends in a specified direction, the flexible connector is relaxed, thereby reducing the tension applied to the drive beam and increasing the degree of bulging of the drive beam under the action of the pre-tightening elastic member.

8. The haptic display device integrating tactile and somatosensory display according to any one of claims 1 to 7, characterized in that, The motion-sensing display unit includes a first hinge and a second hinge that are movably connected, and the first hinge and the second hinge are adapted to be fixed to both sides of a joint. The first hinge is provided with a first guide, and the second hinge is provided with a second guide and a third guide near its two ends, respectively; The tactile display unit is located between the second guide and the third guide, and is connected to the second guide and the third guide respectively; The first guide, the second guide, and the substrate of the touch display unit are all provided with wire holes. The flexible connector passes through each of the wire holes and is connected to the drive beam of the touch display unit.

9. The tactile display device integrating tactile and motion-sensing display according to claim 8, characterized in that, Also includes: A drive assembly is disposed on the first hinge member and connected to the flexible connector. The drive assembly is used to actively adjust the tension or release state of the flexible connector, thereby controlling the effective length of the flexible connector.

10. The haptic display device integrating tactile and somatosensory display according to any one of claims 1 to 7, characterized in that, Also includes: A thermal display module is disposed on the tactile display unit and is used to generate a cold or hot sensation; The thermal display module is a semiconductor cooling chip, which is disposed on the substrate or drive beam of the tactile display unit, as well as other structures between the drive beam and the skin.

Images