Multimodal tactile feedback control device for virtual simulation training platform
By combining electrode array sheets and elastic sensing devices on the virtual simulation training platform, providing kinesthetic and tactile composite feedback, the problem of lack of tactile information on the virtual simulation training platform is solved, achieving higher realism and portability, and improving User experience.
Patent Information
- Application Number
- CN202311598645.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-11-28
AI Technical Summary
The existing virtual simulation training platform mainly provides visual and auditory feedback, lacks tactile information, limiting the authenticity and immersion of the user experience. Moreover, the tactile feedback device with mechanical structure is large in size and poor in portability, and cannot take into account both realism and portability.
Design a multimodal haptic feedback control device, including a multimodal haptic feedback operating handle and a virtual simulation training system, and provides kinesthetic and haptic composite feedback through the combination of electrode array sheet and elastic sensing device to simulate the weight and temperature of the object. , roughness and softness.
It significantly improves the user's interactive realism and immersion. Users can feel multimodal haptic feedback through the operating handle, which improves the experience of virtual simulation training, taking into account both realism and portability.
Smart Images

Figure CN117742484B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of virtual simulation technology, and in particular to a multi-modal tactile feedback control device for a virtual simulation training platform. Background Art
[0002] A virtual simulation training system is a technical tool used to simulate and imitate real-world scenarios, designed to provide a highly interactive and realistic environment for training and education. These platforms typically combine virtual reality (VR), augmented reality (AR), computer graphics, physics engines, and simulation technologies to create a variety of training and education scenarios. In a virtual simulation training system, the system typically provides real-time feedback and evaluation to help users improve their skills and behaviors. This feedback can help users learn quickly and correct mistakes. The continuous development of virtual simulation training platforms has provided new education and training opportunities in a variety of fields while improving efficiency and safety. They help improve users' skills and confidence while providing repeatability and realistic experiences.
[0003] However, most of the current virtual simulation training platforms only provide real-time feedback including visual and auditory feedback, lacking feedback on tactile information. This results in the user being unable to feel the tactile information such as the weight, temperature, roughness, and softness of the operating object, limiting the user's sense of reality and immersion when using the virtual simulation training platform. Although some virtual simulation training platforms can provide tactile information feedback, due to the use of mechanical structures to generate tactile feedback, there are disadvantages such as large equipment size and single tactile feedback mode. Therefore, the current virtual simulation training system requires users to choose between operational realism and portability, and cannot achieve both, which limits the development and popularization of virtual simulation training systems. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention proposes a multimodal tactile feedback control device for a virtual simulation training platform, which can not only interact with the platform in terms of vision and hearing, but also improve the realism and immersion of the interaction through tactile information interaction.
[0005] To realize the above technical scheme, the present invention provides a multimodal tactile feedback control device for a virtual simulation training platform, comprising: a multimodal tactile feedback operating handle, the multimodal tactile feedback operating handle comprising a handle column, an operating rod is installed on the top of the handle column, an electrode array sheet is wrapped around the surface of the operating rod, a gating control module is installed in the operating rod and is electrically connected to the electrode array sheet, and an elastic force sensor is installed in the handle column and is electrically connected to the gating control module; a virtual simulation training system, the virtual simulation training system comprises an interactive information acquisition module, a tactile information calculation module, a tactile information feedback module and a display, wherein the interactive The information acquisition module is used to obtain the scene information of the operating object and its environment in the current state of the virtual simulation training platform; the tactile information calculation module is used to determine the type of kinesthetic / tactile composite feedback and composite feedback information such as tactile modality that need to be fed back to the user in the current state based on the scene information and the collision detection algorithm; the tactile information feedback module is used to connect with the multimodal tactile feedback operating handle, and based on the composite feedback information, control the multimodal tactile feedback operating handle of the kinesthetic / tactile composite feedback control end of the virtual simulation training platform to provide the user with feedback corresponding to the composite feedback information; the display is used to display the content of the virtual simulation training.
[0006] Preferably, the virtual simulation training system operates in the following manner: first, the information of the interaction process is obtained through the interaction information acquisition module, and then the corresponding kinesthetic and tactile feedback information amount in the state is calculated through the tactile information calculation module and the tactile information feedback module is controlled to generate tactile information in the corresponding state, and then the multimodal tactile feedback is felt through the multimodal tactile feedback operation handle.
[0007] Preferably, the information of the interaction process acquired by the interaction information acquisition module includes tactile information of the operation object, the degree of interactive contact and scene information.
[0008] Preferably, the specific process of calculating the corresponding kinesthetic and tactile feedback information amount in the state by the tactile information calculation module and controlling the tactile information feedback module to generate the tactile information in the corresponding state is as follows:
[0009] (1) Controlling the tactile information feedback module to sense the spring deformation of the elastic force sensor device in the multi-modal tactile feedback operating handle to generate kinesthetic feedback of different strengths, thereby generating feedback of different weights;
[0010] (2) When the temperature of the object is high, the input current amplitude of the electrode array sheet in the multimodal tactile feedback operating handle is increased through the tactile information feedback module to generate a stronger tactile feedback, thereby generating a high temperature tactile sensation;
[0011] (3) When the object is relatively rough, the tactile information feedback module reduces the gating frequency and input current frequency of the input channel of the electrode array sheet in the multimodal tactile feedback operating handle to generate a slowly changing tactile feedback, thereby generating a tactile feedback with a high degree of roughness;
[0012] (4) When the object is soft, the tactile information feedback module simultaneously controls the spring deformation of the elastic force sensor device in the multimodal tactile feedback operating handle and the current change speed of the input electrode array sheet, thereby reducing the kinesthetic feedback intensity and increasing the tactile feedback intensity change speed, thereby generating soft tactile feedback.
[0013] The beneficial effect of a multimodal tactile feedback control device for a virtual simulation training platform provided by the present invention is that the present invention innovatively combines an electrode array sheet, an operating handle and an elastic force sensing device, the elastic force sensing device is built into the operating handle, and the electrode array sheet covers the surface of the operating handle. When using the virtual simulation training platform through the operating handle, the user can not only interact with the platform in terms of vision and hearing, but also improve the realism and immersion of the interaction through tactile information interaction. In terms of tactile information interaction, the user can feel multimodal tactile feedback, including the weight, temperature, roughness, softness, etc. of the object, only through the operating handle, which greatly improves the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is a system architecture diagram of the present invention.
[0015] Figure 2 It is a structural diagram of the multi-modal tactile feedback operating handle in the present invention.
[0016] Figure 3 It is an operation flow chart of the virtual simulation training system in the present invention.
[0017] In the figure: 1. virtual simulation training system; 2. multimodal tactile feedback operating handle; 21. handle column; 22. operating rod; 23. electrode array sheet; 24. gating control module; 25. elastic force sensing device. DETAILED DESCRIPTION
[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary persons in the art without creative work are within the scope of protection of the present invention.
[0019] Embodiment: A multi-modal tactile feedback control device for a virtual simulation training platform.
[0020] Reference Figures 1 to 3 As shown, a multi-modal tactile feedback control device for a virtual simulation training platform comprises:
[0021] A multimodal tactile feedback operating handle 2, the multimodal tactile feedback operating handle 2 includes a handle column 21, an operating rod 22 is installed on the top of the handle column 21, an electrode array sheet 23 is wrapped around the surface of the operating rod 22, a gating control module 24 is installed in the operating rod 22 and is electrically connected to the electrode array sheet 23, and an elastic force sensor device 25 is installed in the handle column 21 and is electrically connected to the gating control module 24. In actual use, when the user holds the electrode array sheet 23, the gating frequency and input current frequency of the input channel of the electrode array sheet 23 can be controlled by the gating control module 24 to generate a slowly changing tactile feedback, so that when the user holds the electrode array sheet 23, a rougher tactile feedback can be generated; when the input current amplitude of the electrode array sheet 23 increases, a stronger tactile feedback is generated, thereby generating a high temperature tactile sensation; when the input current amplitude of the electrode array sheet 23 decreases, a cold tactile sensation is generated; since there is an approximately linear relationship between the weight of an object and the intensity of the kinesthetic feedback, the spring deformation of the elastic force sensor device 25 in the control handle is used to generate kinesthetic feedback of different intensities, thereby generating feedback of different weights. The greater the compression of the spring, the greater the weight fed back, and the smaller the compression of the spring, the smaller the weight fed back.
[0022] A virtual simulation training system 1, wherein the virtual simulation training system 1 comprises:
[0023] (1) Interactive information acquisition module: obtains scene information of the operating object and its environment in the current state of the virtual simulation training platform;
[0024] (2) a tactile information calculation module: based on the scene information and according to a collision detection algorithm, determining the type of kinesthetic / tactile composite feedback and composite feedback information such as tactile modality that needs to be fed back to the user in the current state;
[0025] (3) a tactile information feedback module: used to connect to the multimodal tactile feedback operating handle 2, and based on the composite feedback information, control the multimodal tactile feedback operating handle 2 of the kinesthetic / tactile composite feedback control terminal of the virtual simulation training platform to provide the user with feedback corresponding to the composite feedback information;
[0026] (4) Display: used to display the content of virtual simulation training.
[0027] The present invention can be used in common virtual simulation training scenarios, including students conducting scientific experiments in virtual laboratories, performing flight operations in flight training, and construction workers simulating construction site operations.
[0028] Take students conducting scientific experiments in a virtual laboratory as an example. In a virtual laboratory, when students use the device to manipulate virtual instruments and equipment, the device first obtains information about the interaction process, including tactile information of the operated object (such as weight, temperature, roughness, softness, etc.), degree of interactive contact, and scene information. Then, the tactile information calculation module of the device calculates the corresponding amount of kinesthetic and tactile feedback information in this state and controls the tactile information feedback module to generate tactile information in the corresponding state. The specific process is briefly as follows:
[0029] (1) Controlling the tactile information feedback module to sense the spring deformation of the elastic force sensor device 25 in the multi-modal tactile feedback operating handle 2 to generate kinesthetic feedback of different strengths, thereby generating feedback of different weights;
[0030] (2) When the object temperature is high, the input current amplitude of the electrode array sheet 23 in the multimodal tactile feedback operating handle 2 is increased through the tactile information feedback module to generate a stronger tactile feedback, thereby generating a high temperature tactile sensation;
[0031] (3) When the object is relatively rough, the tactile information feedback module reduces the gating frequency and input current frequency of the input channel of the electrode array sheet 23 in the multimodal tactile feedback operating handle 2, thereby generating a tactile feedback with a slower change, thereby generating a tactile feedback with a greater degree of roughness;
[0032] (4) When the object is soft, the tactile information feedback module simultaneously controls the spring deformation of the elastic force sensor device 25 in the multimodal tactile feedback operating handle 2 and the current change speed of the input electrode array sheet 23, thereby reducing the kinesthetic feedback intensity and increasing the tactile feedback intensity change speed, thereby generating soft tactile feedback.
[0033] The present invention innovatively combines the electrode array sheet, the operating handle and the elastic force sensor device, the elastic force sensor device is built into the operating handle, and the electrode array sheet covers the surface of the operating handle. When using the virtual simulation training platform through the operating handle, the user can not only interact with the platform in terms of vision and hearing, but also improve the realism and immersion of the interaction through tactile information interaction. In terms of tactile information interaction, the user can feel multimodal tactile feedback, including the weight, temperature, roughness, softness, etc. of the object, only through the operating handle, which greatly improves the user experience.
[0034] The above description is only a preferred embodiment of the present invention, but the present invention should not be limited to the contents disclosed in the embodiment and the drawings, so any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.
Claims
1. A multi-modal tactile feedback control device for a virtual simulation training platform, characterized in that include: A multimodal tactile feedback operating handle, the multimodal tactile feedback operating handle comprising a handle column, an operating rod is installed on the top of the handle column, an electrode array sheet is wrapped around the surface of the operating rod, a gating control module is installed in the operating rod and is electrically connected to the electrode array sheet, and an elastic force sensor is installed in the handle column and is electrically connected to the gating control module; A virtual simulation training system, the virtual simulation training system comprises an interactive information acquisition module, a tactile information calculation module, a tactile information feedback module and a display, wherein the interactive information acquisition module is used to acquire scene information of an operating object and its environment in a current state of a virtual simulation training platform; the tactile information calculation module is used to determine, based on the scene information and according to a collision detection algorithm, kinesthetic / tactile composite feedback information that needs to be fed back to a user in the current state; the tactile information feedback module is used to connect with a multimodal tactile feedback operating handle, and based on the composite feedback information, control the multimodal tactile feedback operating handle of a kinesthetic / tactile composite feedback control end of the virtual simulation training platform to provide feedback corresponding to the composite feedback information to the user; the display is used to display the content of the virtual simulation training; the virtual simulation training system operates in the following manner: first, the information of the interactive process is acquired through the interactive information acquisition module, and then the corresponding kinesthetic and tactile feedback information amount in the state is calculated through the tactile information calculation module and the tactile information feedback module is controlled to generate tactile information in the corresponding state, and then the multimodal tactile feedback is felt through the multimodal tactile feedback operating handle; The specific process of calculating the corresponding kinesthetic and tactile feedback information amount in this state through the tactile information calculation module and controlling the tactile information feedback module to generate the tactile information in the corresponding state is as follows: (1) Controlling the tactile information feedback module to sense the spring deformation of the elastic force sensor device in the multi-modal tactile feedback operating handle to generate kinesthetic feedback of different strengths, thereby generating feedback of different weights; (2) When the temperature of the object is high, the input current amplitude of the electrode array sheet in the multimodal tactile feedback operating handle is increased through the tactile information feedback module to generate a stronger tactile feedback, thereby generating a high temperature tactile sensation; (3) When the object is relatively rough, the tactile information feedback module reduces the gating frequency and input current frequency of the input channel of the electrode array sheet in the multimodal tactile feedback operating handle to generate a slowly changing tactile feedback, thereby generating a tactile feedback with a high degree of roughness; (4) When the object is soft, the tactile information feedback module simultaneously controls the spring deformation of the elastic force sensor device in the multimodal tactile feedback operating handle and the current change speed of the input electrode array sheet, thereby reducing the kinesthetic feedback intensity and increasing the tactile feedback intensity change speed, thereby generating soft tactile feedback.
2. The multimodal tactile feedback control device for a virtual simulation training platform according to claim 1, characterized in that: The information of the interaction process obtained by the interaction information acquisition module includes tactile information of the operation object, the degree of interaction contact and scene information.
Citation Information
Patent Citations
Virtual reality tactile feedback interaction system
CN108874150A
A multi-tactile fusion feedback handle
CN109085922A
Handle capable of adjusting feedback force
CN116510320A
