Virtual touch device, control method and multimedia system

Abstract

The invention relates to the technical field of virtual reality, and discloses a virtual touch device, a control method and a multimedia system.The virtual touch device comprises an outer shell, an air chamber, a pneumatic pressurization system and a gesture detection system, and the air chamber, the pneumatic pressurization system and the gesture detection system are arranged in the outer shell. The air chamber is provided with a hand stretching opening for a hand to stretch in, the hand stretching opening is sleeved with a touch film, the touch film is arranged in the air chamber, and the hand of a user can stretch into the touch film through the hand stretching opening. A plurality of air inlets are uniformly formed in the inner wall of the air chamber, the pneumatic pressurization system inputs air into the air chamber through the air inlets under the control of the control system, and a flexible intermediate medium is arranged between the touch film and the air inlets. Pressure brought by airflow input into the air chamber by the pneumatic pressurization system is transmitted to the touch film through the intermediate medium and then transmitted to the hand of a user through the touch film, and the gesture detection system is used for detecting hand actions of the user. The virtual tactile device can provide more real tactile feedback to a user.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality, and particularly to a virtual tactile device, a control method and a multimedia system. Background Art

[0002] With the development of technology, virtual reality (VR) and augmented reality (AR) technologies have gradually become important development directions for human-computer interaction. Existing virtual devices for virtual touch mostly apply comprehensive or partial pressure to the user's hand by blowing precisely controlled airflows onto the hand, enabling the user to realistically feel the existence and characteristics of virtual objects. However, since gas has no fixed shape, the touch sensation brought by using gas still has a certain gap from the touch sensation of real objects, and it is difficult to simulate the touch sensation of a large area. Therefore, a new virtual touch solution needs to be designed to solve the above technical problems. Summary of the Invention

[0003] In order to overcome the deficiencies of the prior art, the purpose of the present invention is to provide a virtual tactile device that can more realistically simulate the touch sensation of objects and improve the sensitivity and accuracy of touch feedback.

[0004] To solve the above problems, the technical solution adopted by the present invention is as follows: A virtual tactile device includes an outer housing, and an air chamber, a pneumatic pressurization system and a gesture detection system arranged inside the outer housing. The air chamber is provided with a hand insertion opening through which a hand can extend. A tactile film is sleeved on the hand insertion opening, and the tactile film is arranged inside the air chamber. The user's hand can extend into the tactile film through the hand insertion opening. A plurality of air inlets are uniformly arranged on the inner wall of the air chamber. The pneumatic pressurization system inputs air into the air chamber through the air inlets under the control of a control system. A flexible intermediate medium is arranged between the tactile film and the air inlets. The pressure brought by the airflow input into the air chamber by the pneumatic pressurization system is transmitted to the tactile film through the intermediate medium, and then transmitted to the user's hand through the tactile film. The gesture detection system is used to detect the hand movements of the user.

[0005] Compared with the prior art, the beneficial effects of the present invention are as follows: By arranging an intermediate medium between the air inlet and the tactile film, and controlling the flow rate of the air entering the air chamber through the air inlet by controlling the pneumatic pressurization system, so that the entering air generates different magnitudes of pressure on the intermediate medium, and is transmitted to the user's hand through the intermediate medium, improving the sensitivity and accuracy of tactile feedback, and thus bringing a more real tactile virtual effect to the user. The pressure generated by the airflow is transmitted to the user's hand through a tangible intermediate medium, and the pressure is more uniform and closer to the tactile feedback of real objects compared to directly impacting the hand with the airflow.

[0006] The above virtual tactile device, wherein the intermediate medium includes a plurality of medium balls, the epidermis of the medium balls is made of a flexible and lightweight material, the interior of the medium balls is filled with liquid, and the medium balls are evenly distributed between the tactile film and the inner wall of the air chamber.

[0007] The above virtual tactile device, wherein the tactile film includes two layers. The outer layer of the tactile film contacts the medium balls and is used to prevent the medium balls from moving relative to the tactile film under the action of an external force. The inner layer of the tactile film contacts the user's hand and is used to provide the tactile sensation to be simulated to the user's hand.

[0008] The above virtual tactile device, wherein the pneumatic pressurization system includes an air pump, a pressurization chamber, and a plurality of micro-compartments. The pressurization chamber is provided with a pressure relief port communicating with the outside. The pressurization chamber is communicated with the air pump through an air inlet. The pressurization chamber is communicated with the micro-compartments through a plurality of air outlets. The micro-compartments are evenly distributed around the air chamber. A plurality of the micro-compartments are respectively and correspondingly communicated with a plurality of air inlets. Control valves are provided at the pressure relief port, the air inlet, and the air outlets. A pressure sensor is provided in the pressurization chamber.

[0009] The above virtual tactile device, wherein a safety valve is provided on the air chamber.

[0010] The above virtual tactile device, wherein the gesture detection system includes four infrared cameras, and the four infrared cameras are respectively arranged at the upper, lower, left, and right positions of the air chamber.

[0011] The above virtual tactile device, wherein a temperature sensor, a heating device, and a refrigeration device are provided in the air chamber.

[0012] A control method for the above virtual tactile device, comprising the following steps:

[0013] Detect the user's gesture, and calculate the magnitude and time of the pressure to be applied to each position of the user's hand according to the user's gesture and the mechanical model of the object to be simulated;

[0014] Calculate the gas flow rate of each air inlet according to the magnitude of the pressure to be applied to each position of the user's hand;

[0015] Calculate the control parameters for controlling the pneumatic pressurization system according to the gas flow rate of each air inlet and the time of applying pressure at each position, and control the operation of the pneumatic pressurization system according to the control parameters.

[0016] A multimedia system, comprising a control system, an audio-visual system, and the above virtual tactile device, wherein the virtual tactile device and the audio-visual system are both connected to the control system.

[0017] The above multimedia system further includes an aroma device, and the aroma device is electrically connected to the control system.

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Description of the Drawings

[0019] Figure 1 It is a schematic structural diagram of the virtual touch device according to an embodiment of the present invention;

[0020] Figure 2 It is a perspective view of the virtual touch device according to an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram when the pressure chamber of the present invention is pressurized by air intake;

[0022] Figure 4 It is a schematic diagram when the pressure chamber of the present invention discharges air;

[0023] Figure 5 It is a flowchart of the control method of the virtual touch device according to an embodiment of the present invention;

[0024] Figure 6 It is a principle block diagram of the multimedia system according to an embodiment of the present invention.

[0025] Explanation of the reference numerals in the drawings:

[0026] 100 outer housing, 200 air chamber, 210 hand insertion port, 220 air inlet, 300 touch film, 400 intermediate medium, 500 pneumatic pressurization system, 510 pressure chamber, 511 pressure relief port, 512 air inlet, 513 air outlet, 520 air pump, 530 micro chamber, 600 gesture detection system. Specific Embodiments

[0027] The embodiments of the present invention will be described in detail below with reference to Figure 1, embodiments of the present invention provide a virtual tactile device, including a housing 100, an air chamber 200, a pneumatic pressurization system 500, and a gesture detection system 600 disposed within the housing 100. The air chamber 200 is provided with a hand insertion opening 210 through which a user's hand can extend. A tactile film 300 is sleeved on the hand insertion opening 210, and the user's hand can extend into the tactile film 300 through the hand insertion opening 210. The tactile film 300 is disposed within the air chamber 200, and a plurality of air inlets 220 are uniformly provided on the inner wall of the air chamber 200. The air inlets 220 are connected to the pneumatic pressurization system 500. The pneumatic pressurization system 500 is configured to input air into the air chamber 200 through the air inlets 220 to form an air flow under the control of a controller. A flexible intermediate medium 400 is disposed between the tactile film 300 and the air inlets 220. The pressure brought by the air flow input into the air chamber 200 by the pneumatic pressurization system 500 is transmitted to the tactile film 300 through the intermediate medium 400 and is fed back to the user's hand through the tactile film 300.

[0028] In this virtual tactile device, the pressure brought by the air flow is transmitted to the user's hand through the intermediate medium 400. The intermediate medium 400 can transmit the pressure brought by the air flow to the user's hand more evenly, and transmit the pressure to the user's hand through the tangible intermediate medium 400. Compared with directly blowing the air flow onto the user's hand, it is easier to simulate the comprehensive tactile sensation of a large area, so as to feedback a more delicate and real tactile sensation to the user, bringing the user a virtual experience closer to interacting with real objects.

[0029] It can be understood that the intermediate medium 400 can be made of sponge or liquid contained in a flexible container. Referring to Figure 1 , in this embodiment, the intermediate medium 400 includes a plurality of medium balls. The epidermis of the medium balls is made of a flexible lightweight material, such as aerogel, etc. The inside of the medium balls is filled with pure water or other safe liquids. The medium balls are uniformly distributed between the tactile film 300 and the inner wall of the air chamber 200. The medium balls can undergo obvious deformation under the pressure of the air flow to simulate different types of tactile sensations, and can quickly return to their original state after the pressure is released, so as to ensure the response speed of tactile feedback and ensure that tactile changes can be continuously fed back. It can be understood that the mechanical properties of the medium balls can be adjusted by adjusting the epidermis material of the medium balls and the density of the liquid inside, so as to provide different tactile feedback to the user's hand, thereby realizing the simulation of the tactile feedback of different objects. In some embodiments, the size and material of the medium balls can be the same, or medium balls with different sizes and materials can be set in different regions according to the sensitivity of different regions of the hand to better realize the tactile feedback of real objects.

[0030] Correspondingly, in this embodiment, the tactile film 300 includes at least two layers. The outer surface of the outer layer contacts the dielectric sphere, and the inner surface of the inner layer directly contacts the user's hand. The outer layer of the tactile film 300 preferably has a relatively high coefficient of friction with the surface of the dielectric sphere, so as to prevent the dielectric sphere from sliding relative to the tactile film 300 under the action of an external force, ensuring the accuracy of pressure transmission. The inner layer of the tactile film 300 is preferably made of a biocompatible polymer material, such as polycarbonate, polytetrafluoroethylene, and polylactic acid, etc., to provide a comfortable wearing experience for the user. It can be understood that the tactile film 300 is preferably made into a glove shape for the convenience of the user to wear. In some embodiments, the inner layer of the tactile film 300 can also be made of materials such as silk or metal according to the touch of the virtual object.

[0031] Referring to Figure 2 , in this embodiment, the pneumatic pressurization system 500 includes an air pump 520, a pressurization chamber 510, and a plurality of micro-compartments 530. The pressurization chamber 510 is provided with a pressure relief port 511 communicating with the outside. The pressurization chamber 510 is communicated with the air pump 520 through an air inlet 512. The pressurization chamber 510 is communicated with the micro-compartments 530 through a plurality of air outlets 513. The micro-compartments 530 are evenly distributed around the air chamber 200. The plurality of micro-compartments 530 are in one-to-one correspondence and communication with the plurality of air inlets 220. Control valves are provided at the pressure relief port 511, the air inlet 512, and the air outlet 513. A pressure sensor is provided in the pressurization chamber 510. It can be understood that the control valve can be a proportional valve or an ordinary on-off valve. Referring to Figure 3 , when pressurization is required to provide a tactile feedback, first control the control valves of the pressure relief port 511 and the air outlet 513 to close, and open the control valve of the air inlet 512 and the air pump 520 to pump air into the pressurization chamber 510 until the air pressure value detected by the pressure sensor reaches the required threshold; Referring to Figure 4, then close the control valve of the air pump 520 and the air inlet 512, and open the control valve of the air outlet 513. Under the action of air pressure, the gas in the pressurizing chamber 510 enters different areas of the air chamber 200 through the micro chamber 530, forming a single airflow at a certain speed to impact the intermediate medium 400 once to achieve single tactile feedback. When pressurization is required to provide continuous tactile feedback, first control the control valves of the pressure relief port 511 and the air outlet 513 to close, and open the control valve of the air inlet 512 and the air pump 520 to pump air into the pressurizing chamber 510 until the air pressure sensor detects that the air pressure value reaches the required threshold; then open the control valve of the air outlet 513. Under the action of air pressure, the gas in the pressurizing chamber 510 enters different areas of the air chamber 200 through the micro chamber 530, forming an airflow at a certain speed. At the same time, control the pressure relief valve of the pressure relief port 511 to open and close intermittently through the PWM algorithm to maintain the air pressure in the pressurizing chamber at a required constant value, so as to maintain the flow rate of the airflow constant and continuously impact the intermediate medium 400 to apply continuous tactile feedback to the user's hand. It can be understood that the number of pressurizing chambers 510 can be multiple, and the air outlets 513 of the multiple pressurizing chambers 510 are respectively connected to the micro chambers 530 in different areas, so as to independently control the tactile feedback in different areas, and thus perform different types of tactile feedback in different areas according to the mechanical properties of the virtual object and the user's hand movements, so as to further increase the fidelity of tactile virtual reality.

[0032] Referring to Figure 2 , in this embodiment, in order to better provide tactile feedback at various angles of the user's hand, the air chamber 200 is spherical, and a plurality of frustum-shaped micro chambers 530 are evenly distributed radially on the circumferential surface of the air chamber 200. The end face with a larger area of the frustum-shaped micro chamber 530 is connected to the pressurizing chamber 510, and the end face with a smaller area is connected to the inside of the air chamber 200 through the air inlet 220 to facilitate increasing the flow rate of the gas entering the air chamber 200.

[0033] In some possible embodiments, in order to prevent the pressure in the air chamber 200 from being too high and damaging the air chamber 200 or bringing a strong squeezing feeling to the user's hand and ensuring the safe use of the device, a safety valve is provided on the air chamber 200. When it is detected that the air pressure in the air chamber 200 is too high, the safety valve will automatically open to relieve the air pressure in the air chamber 200.

[0034] It can be understood that the gesture detection system 600 can obtain the user's gesture actions by detecting the infrared rays of the hand, or directly capture the actions of the user's hand by using motion capture sensors such as triaxial geomagnetic sensors. In this embodiment, the gesture detection system 600 includes four infrared cameras, which are respectively arranged in the four directions of up, down, left, and right of the air chamber 200. If a motion capture sensor is used, the motion capture sensor can be arranged between the outer layer and the inner layer of the touch film 300.

[0035] It can be understood that in some embodiments, in order to simulate the temperature of the virtual object, a temperature sensor, a heating device, and a refrigeration device can also be arranged in the air chamber 200. The temperature in the air chamber 200 can be adjusted by the heating device and the refrigeration device to simulate the temperature of the virtual object and further improve the virtual reality.

[0036] Referring to Figure 5 , the control method of the virtual touch device according to the embodiment of the present invention includes the following steps:

[0037] Detect the user's gesture, and calculate the magnitude and time of the pressure to be applied to each position of the user's hand according to the user's gesture and the mechanical model of the simulated object;

[0038] Calculate the gas flow rate of each air inlet 220 according to the magnitude of the pressure to be applied to each position of the user's hand;

[0039] Calculate the control parameters for controlling the pneumatic pressurization system 500 according to the gas flow rate of each air inlet 220 and the time of applying pressure at each position, and control the operation of the pneumatic pressurization system 500 according to the control parameters.

[0040] It can be understood that the control system can use algorithms such as Mean Shift to process the infrared image of the user's hand collected by the gesture detection system 600, identify the user's gesture actions, input the user's gesture actions into the physics engine, interact with the virtual objects, and output the reaction forces generated by the objects in response to the user's hand actions. Subsequently, the control system calculates the flow rates of the gases to be blown to different positions of the user's hand according to the position, intensity, and time of the reaction forces, in combination with the parameters of the intermediate medium 400, the Bernoulli equation, and the gas dynamics equation; finally, according to the gas flow rates and the time when tactile feedback is required, in combination with the structural parameters of the pressurizing chamber 510 and the micro-chamber 530, the Bernoulli equation, and the gas dynamics equation, it calculates the air pressure required for the pressurizing chamber 510, the power of the air pump 520, the opening degrees and switching frequencies of each control valve, and other parameters, and controls the pneumatic pressurization system 500 according to the calculated parameters, so as to send airflows with the required flow rates to different areas in the air chamber 200 through each micro-chamber 530, allowing the airflows to impact the intermediate medium 400 and transmit the pressure to the user's hand to provide tactile feedback of the virtual objects. It can be understood that the gesture detection algorithm, the algorithm for calculating the reaction forces using the physics engine, and the algorithms for calculating the required gas flow rates and the pneumatic pressurization system 500 using the Bernoulli equation and the gas dynamics equation are common general knowledge in the art, and their specific processes are not elaborated herein. The physics engine can adopt Genesis, PhysX, or Havok, etc.

[0041] Referring to Figure 6 , the multimedia system of the embodiment of the present invention includes the above virtual tactile device, control system, and audio-visual system. The pneumatic pressurization system 500, gesture detection system 600, temperature sensor, heating device, refrigeration device, etc. in the virtual tactile device are all electrically connected to the control system, and the audio-visual system is also electrically connected to the control system. The control system controls the virtual tactile device to provide tactile feedback to the user's hand according to the user's gesture actions, and controls the audio-visual system to display the actions of the virtual objects in response to the user's gestures, such as rotation, bending, etc., and emit corresponding sound effects. The audio-visual system includes an audio system and a video system. The video system can include a display, a projector, or a stereoscopic projector, etc. The audio system can include headphones or speakers. The audio-visual system can also directly adopt a head-mounted VR device, such as a VR glasses or a VR helmet.

[0042] Referring to Figure 6, in some embodiments, in order to further enhance the immersive experience brought by the multimedia system to users, the multimedia system further includes an aroma device. The aroma device is electrically connected to the control system. A variety of media with different scents are provided inside the aroma device. Under the control of the control system, the aroma device releases the corresponding scent medium according to the interaction actions of the user with the virtual object through the virtual touch device, so as to bring an immersive experience to the user. The multimedia system of the embodiments of the present invention can be used in various different fields such as entertainment, training, medical treatment, and rehabilitation training, such as in scenarios like surgical training, VR games, postoperative recovery, or maintenance training. By bringing a more realistic experience to users, it improves the entertainment, training, or recovery effect.

[0043] It should be noted that in the description of the present invention, if there are descriptions related to orientation, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc., they are all based on the orientation or positional relationship shown in the drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and should not be construed as a limitation to the present invention.

[0044] In the description of the present invention, the meaning of "several" is one or more, the meaning of "multiple" is two or more. Understandings such as "greater than", "less than", and "exceeding" do not include the number itself, and understandings such as "above", "below", and "within" include the number itself. If there are descriptions of "first" or "second", etc., they are only for the purpose of distinguishing technical features and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or the sequence relationship of the indicated technical features.

[0045] In the description of the present invention, unless otherwise clearly defined, words such as "set", "installed", and "connected" should be understood in a broad sense. Those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0046] The above embodiments are only the preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantive changes and substitutions made by those skilled in the art based on the present invention fall within the scope of protection required by the present invention.

Claims

1. A virtual touch device, characterized in that: The invention comprises an outer shell (100), an air chamber (200) arranged in the outer shell (100), a pneumatic pressurization system (500) and a gesture detection system (600), wherein the air chamber (200) is provided with a hand insertion port (210) for inserting a hand, a touch film (300) is sleeved on the hand insertion port (210), and the touch film (300) is arranged in the air chamber (200), and a user's hand can be inserted into the touch film (300) through the hand insertion port (210), and a plurality of air inlets (220) are evenly arranged on the inner wall of the air chamber (200). ), the pneumatic pressurization system (500) inputs air into the air chamber (200) through the air inlet (220) under the control of the control system, a flexible intermediate medium (400) is provided between the touch film (300) and the air inlet (220), the pressure brought by the airflow input into the air chamber (200) by the pneumatic pressurization system (500) is transmitted to the touch film (300) through the intermediate medium (400), and then transmitted to the user's hand through the touch film (300), and the gesture detection system (600) is used to detect the user's hand movements.

2. The virtual touch device according to claim 1, characterized in that: The intermediate medium (400) comprises a plurality of dielectric balls, the surface of the dielectric balls being made of a flexible and lightweight material, the interior of the dielectric balls being filled with liquid, and the dielectric balls being evenly distributed between the touch film (300) and the inner wall of the air chamber (200).

3. The virtual touch device according to claim 2, characterized in that: The touch film (300) comprises two layers, the outer layer of the touch film (300) contacts the dielectric ball to prevent the dielectric ball from moving relative to the touch film (300) under the action of an external force, and the inner layer of the touch film (300) contacts the user's hand to provide the user's hand with a touch sensation that needs to be simulated.

4. The virtual touch device according to claim 1, characterized in that: The pneumatic pressurization system (500) comprises an air pump (520), a pressurization chamber (510) and a plurality of micro-compartments (530); the pressurization chamber (510) is provided with a pressure relief port (511) connected to the outside; the pressurization chamber (510) is connected to the air pump (520) via an air inlet (512); the pressurization chamber (510) is connected to the micro-compartments (530) via a plurality of air outlets (513); the micro-compartments (530) are evenly distributed around the air chamber (200); the plurality of micro-compartments (530) are connected to the plurality of air inlets (220) in a one-to-one correspondence; the pressure relief port (511), the air inlet (512) and the air outlet (513) are all provided with control valves; and an air pressure sensor is provided in the pressurization chamber (510).

5. The virtual touch device according to claim 1, characterized in that: The air chamber (200) is provided with a safety valve.

6. The virtual touch device according to claim 1, characterized in that: The gesture detection system (600) comprises four infrared cameras, which are respectively arranged at four positions of the air chamber (200): up, down, left, and right.

7. The virtual touch device according to claim 1, characterized in that: A temperature sensor, a heating device and a refrigeration device are arranged in the air chamber (200).

8. A control method for a virtual touch device according to any one of claims 1 to 7, characterized in that: The steps include: Detecting the user's gestures, and calculating the amount and time of pressure to be applied to each position of the user's hand according to the user's gestures and the mechanical model of the simulated object; Calculating the gas flow rate of each air inlet (220) according to the pressure required to be applied to each position of the user's hand; The control parameters of the pneumatic pressurizing system (500) are calculated based on the gas flow rate of each air inlet (220) and the time for applying pressure at each position, and the operation of the pneumatic pressurizing system (500) is controlled based on the control parameters.

9. A multimedia system, characterized in that: The method comprises a control system, an audio-visual system and a virtual tactile device according to any one of claims 1 to 7, wherein both the virtual tactile device and the audio-visual system are connected to the control system.

10. The multimedia system according to claim 9, characterized in that It also includes a fragrance device, which is electrically connected to the control system.

Images