A whole body posture tracking and tactile device and virtual reality system

Abstract

The present disclosure provides a whole-body posture tracking and tactile device and a virtual reality system, which belongs to the field of virtual reality enhanced display technology. The whole-body posture tracking and tactile device disclosed in the present disclosure includes: a main body structure, at least one motion detection unit, a control unit, and at least one tactile control unit; the main body structure is configured to be worn by a user on a limb; the motion detection unit is installed on the main body structure and is configured to detect the user's limb movements and generate user motion posture information; the control unit is configured to process the received motion posture information, generate a first control signal, and transmit it to a VR device, so that the VR device can control the limb movements of the virtual user, and after receiving a touch signal of the virtual user being touched fed back by the VR device, generate a second control signal according to the touch signal; the tactile control unit is configured to feedback real tactile perception to the user's limbs through the main body structure according to the second control signal.

Description

Technical Field

[0001] The present invention belongs to the field of virtual reality enhanced display technology, and in particular relates to a full-body posture tracking and tactile device and a virtual reality system. Background Art

[0002] Virtual reality technology (VR) is a new practical technology developed in the 20th century. Virtual reality technology includes computers, electronic information, and simulation technology. Its basic implementation method is to use computers to simulate virtual environments to give people a sense of environmental immersion. With the continuous development of social productivity and science and technology, the demand for VR technology in all walks of life is growing.

[0003] Most mobile VR headsets have rotational tracking (3DoF), so you can look up or down, and tilt left or right. But if you try to tilt or move the position of your head, it won't be tracked. Virtual reality is currently based only on our eyes and ears. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art and provide a whole-body posture tracking and tactile device and a virtual reality system.

[0005] In a first aspect, the embodiments of the present disclosure provide a whole-body posture tracking and tactile device, which includes: a main structure, at least one motion detection unit, a control unit, and at least one tactile control unit; wherein,

[0006] The main structure is configured to be worn on a limb by a user;

[0007] The motion detection unit is mounted on the main structure and is configured to detect the user's body movements and generate user motion posture information;

[0008] The control unit is configured to process the received motion posture information, generate a first control signal, and transmit the first control signal to the VR device so that the VR device can control the body movements of the virtual user, and after receiving a touch signal fed back by the VR device indicating that the virtual user is touched, generate a second control signal according to the touch signal;

[0009] The tactile control unit is configured to feedback real tactile perception to the user's limbs through the main structure according to the second control signal.

[0010] In some examples, the motion posture information includes a rotation angle of a motion joint; the motion detection unit includes a first detection module and a second detection module;

[0011] The first detection module is configured to detect the electromyographic signal generated by the user's skeletal muscles and process it;

[0012] The second detection module is configured to detect and process a skin surface tension signal of the user;

[0013] The control unit is configured to obtain the rotation angle of the user's motion joint based on the processed electromyographic signal and the skin surface tension signal through a first preset algorithm, and to obtain the user's motion posture information based on the rotation angle of the user's motion joint through a second preset algorithm to generate a first control signal.

[0014] In some examples, the first detection module includes an electromyographic signal electrode and a first bandpass amplifier; the second detection module includes a skin surface tension strain gauge and a second bandpass amplifier;

[0015] The electromyographic signal electrode is configured to detect the electromyographic signal generated by the user's skeletal muscle;

[0016] The first bandpass amplifier is configured to amplify the electromyographic signal and transmit it to the control unit;

[0017] The skin surface tension strain gauge is configured to detect a skin surface tension signal of a user;

[0018] The second bandpass amplifier is configured to amplify the skin surface tension signal and transmit the amplified signal to the control unit.

[0019] In some examples, the control unit includes an analog-to-digital conversion module, a first calculation module, and a first control module;

[0020] The analog-to-digital conversion module is configured to convert the processed electromyographic signal and the skin surface tension signal into a first digital signal and a second digital signal respectively;

[0021] The first calculation module is configured to obtain the rotation angle of the user's motion joint according to the first digital signal and the second digital signal through a first preset algorithm, and obtain the user's motion posture information through a second preset algorithm;

[0022] The first control module is configured to generate a first control signal according to the user's motion posture information and transmit it to the VR device so that the VR device can control the virtual user's body movements, and after receiving a touch signal from the VR device indicating that the virtual user is touched, generate a second control signal according to the touch signal.

[0023] In some examples, the first calculation module is specifically configured to obtain the rotation angle of the user's motion joints based on the first digital signal and the second digital signal and through a first preset algorithm, and to calculate the spatial posture coordinates of the current position points of each motion joint relative to the initial point of each motion joint based on a pre-established kinematic model, thereby obtaining the user's motion posture information; wherein the initial point is the angle information of each motion joint of the user's initial posture.

[0024] In some examples, the main structure is composed of a plurality of hollow wire tubes woven together; the tactile control unit includes a first switch module, a second switch module and a touch module;

[0025] The first switch module and the second switch module are both connected to the hollow wire tube, and are used to control the gas to enter and discharge from the hollow wire tube according to the second control signal; and the touch module is arranged on the hollow wire tube defined by the first switch module and the second switch module, and when the user wears the main structure, the touch module is located on the side of the hollow wire tube close to the human body.

[0026] In some examples, the control unit includes a second control module and a second calculation module;

[0027] The second calculation module is configured to calculate the actual touched position information of the user according to the touch signal of the virtual user fed back by the VR device;

[0028] The second control module is configured to generate the second control signal according to the position information where the user is actually touched.

[0029] In some examples, the material of the hollow conduit includes a fiber material.

[0030] In some examples, the first switch module and the second switch module include valves.

[0031] In some examples, the striking module includes a striking hammer.

[0032] In some examples, the haptic control unit further includes a suction component configured to fill the hollow wire tube with gas and discharge the gas in the hollow wire tube.

[0033] In some examples, the suction component is an air pump.

[0034] In a second aspect, the embodiments of the present disclosure further provide a virtual reality system, which includes the above-mentioned full-body posture tracking and haptic device and a VR device; the VR device is communicatively connected to the full-body posture tracking and haptic device.

[0035] In some examples, the full-body posture tracking and haptic device and the VR device are connected via WIFI or Bluetooth.

[0036] In some examples, the VR device includes a VR helmet. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of a human kinematics model;

[0038] Figure 2a This is a rendering of the full-body posture tracking and tactile device usage;

[0039] Figure 2b for Figure 2a Schematic diagram of textile details in Part I;

[0040] Figure 3 Schematic diagram of virtual reaction force in virtual touch;

[0041] Figure 4 A schematic diagram of virtual pain in virtual touch;

[0042] Figure 5 Flowchart for whole body posture tracking;

[0043] Figure 6 It is a virtual touch flow chart;

[0044] Figure 7 A schematic diagram of human elbow joint posture tracking;

[0045] Figure 8 Motion detection block diagram;

[0046] Fig. 9 Schematic diagram of virtual reality system. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0048] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] Existing virtual reality (VR) technology usually requires the use of VR equipment to block people's vision and hearing from the outside world, so as to guide users to feel like they are in a virtual environment. Currently, common VR equipment 2 on the market includes VR glasses, VR helmets, etc.

[0050] Among them, VR helmets generally include lenses, display screens, and cables. Among them, the lens, as one of the most critical components, not only plays the role of connecting the user's eyes and VR content, but also largely determines the user's visual effect;

[0051] In addition, high-performance displays give VR helmets enough pixel density to display clear images and make the motion in VR smooth and smooth. High-end VR helmets also use dual-screen displays to provide stereoscopic 3D effects. Each screen displays a slightly offset image to each eye, and then our brain automatically "glue" them together into one image, creating an illusion of depth in the process.

[0052] In addition, the VR helmet is also provided with a built-in sensor so that the virtual character can follow the user's head to make corresponding changes and obtain a more accurate display screen. However, the tracking method of most VR devices 2 in the prior art still revolves around the head and hands, which makes the way of interaction in the virtual environment single and the virtual reality lacks realism. Even if it can be tracked by a tracker or by using a constellation of a specific shape on the tracking object, it cannot be realized because the system is complicated and it is difficult to track detailed postures. At the same time, how to let users experience the touch of different body parts of the virtual user, integrate virtual touch into virtual reality, and thus improve the real experience of virtual reality is still an important issue in this field.

[0053] In view of this, a full-body posture tracking and tactile device 1 is provided in an embodiment of the present disclosure, which detects the user's limb movements and generates user motion posture information through a motion detection unit 20, thereby realizing the user's full-body posture tracking; virtual touch is realized through a main structure 10 and a tactile control unit 40, and the VR device 2 is responsible for general head position and posture tracking, thereby greatly improving the real experience of virtual reality, and having a certain driving effect on the development of true virtual reality of virtual reality systems.

[0054] The display module of the embodiment of the present disclosure is described below in conjunction with the accompanying drawings and specific embodiments.

[0055] In the first aspect, the embodiment of the present disclosure provides a whole-body posture tracking and haptic device 1, which can be used to interact with a VR device 2 worn on the head of a user. The whole-body posture tracking and haptic device 1 includes: a main structure 10, at least one motion detection unit 20, a control unit 30, and at least one tactile control unit 40. Among them, the main structure 10 is configured to be worn on the limbs of the user; the motion detection unit 20 is installed on the main structure 10, and the motion detection unit 20 is configured to detect the limb movements of the user and generate user motion posture information; the control unit 30 is configured to process the received motion posture information, generate a first control signal, and transmit it to the VR device 2, so that the VR device 2 can control the limb movements of the virtual user, and after receiving the touch signal of the virtual user being touched fed back by the VR device 2, generate a second control signal according to the touch signal; the tactile control unit 40 is configured to feedback real tactile perception to the user's limbs through the main structure 10 according to the second control signal.

[0056] It should be noted that the main structure 10 in the embodiment of the present disclosure can be clothing. When the user uses the whole-body posture tracking and tactile device 1, the user wears the main structure 10. At this time, the main structure 10 can be relatively fixed on the user's limbs (such as tights) to fit the human skin. The motion detection unit 20 is fixed on the main structure 10. When the user wears the main structure 10, the position of the motion detection unit 20 generally corresponds to the user's skeletal muscles and is located on the side of the main structure 10 away from the user's skin.

[0057] In the embodiment of the present disclosure, since the whole-body posture tracking and tactile device 1 includes a main structure 10, at least one motion detection unit 20, a control unit 30, and at least one tactile control unit 40, the control unit 30 can process the real motion posture information of the user detected by the received motion detection unit 20, generate a first control signal, and transmit it to the VR device 2, so that the VR device 2 can control the limb movements of the virtual user, and after receiving the touch signal of the virtual user being touched fed back by the VR device 2, generate a second control signal according to the touch signal, and the tactile control unit 40 can feedback the real tactile perception to the user's limbs through the main structure 10 according to the second control signal. Therefore, the whole-body posture tracking and tactile device 1 of the embodiment of the present disclosure can realize the tracking of the user's whole-body posture, and can give the user virtual tactile perception according to the display of the VR device 2, thereby greatly improving the real experience of the virtual display.

[0058] In some examples, the motion posture information may be the rotation angle of the motion joint. The motion detection unit 20 includes a first detection module 21 and a second detection module 22. Specifically, the first detection module 21 is configured to detect the electromyographic signal generated by the user's skeletal muscle and process it; the second detection module 22 is configured to detect the user's skin surface tension signal and process it; the control unit 30 is configured to obtain the rotation angle of the user's motion joint according to the processed electromyographic signal and skin surface tension signal and through a first preset algorithm, and obtain the user's motion posture information according to the rotation angle of the user's motion joint through a second preset algorithm to generate a first control signal.

[0059] Furthermore, the first detection module 21 includes an electromyographic signal electrode 211 and a first bandpass amplifier 212; the second detection module 22 includes a skin surface tension strain gauge 221 and a second bandpass amplifier 222; the electromyographic signal electrode 211 is configured to detect the electromyographic signal generated by the user's skeletal muscles; the first bandpass amplifier 212 is configured to amplify the electromyographic signal and transmit it to the control unit 30; the skin surface tension strain gauge 221 is configured to detect the user's skin surface tension signal; the second bandpass amplifier 222 is configured to amplify the skin surface tension signal and transmit it to the control unit 30.

[0060] It should be noted that the motion detection unit 20 is installed on the main structure 10, and its projection on the main structure 10 covers the outside of the skeletal muscles of the human body; the number of electromyographic signal electrodes 211, the first bandpass amplifier 212, the skin surface tension strain gauge 221, and the second bandpass amplifier 222 in the motion detection unit 20 can be adjusted according to different body parts.

[0061] Specifically, when the muscle contracts, the electromyographic signal electrode 211 detects the electromyographic current. The greater the degree of muscle contraction, the greater the electromyographic current generated. By detecting the size of the electromyographic current, the degree of muscle contraction can be inferred, and then the joint rotation angle can be calculated. The movements of the virtual user can be controlled by the movement angles of each joint of the human body, thereby completing the whole body posture tracking of the human body; the skin surface tension strain gauge 221 can detect the changes in skin stress during muscle contraction and relaxation. When the muscle contracts, the skin is subjected to squeezing force, and the skin surface tension strain gauge 221 is subjected to pressure. When the muscle relaxes, the skin is subjected to surface tension. The skin surface tension strain detects the tension, and the skin stress change current combined with the electromyographic signal current can more accurately calculate the joint rotation angle.

[0062] It should be noted that the first bandpass amplifier 212 and the second bandpass amplifier 222 are used to filter and amplify the electromyographic signal and the skin surface tension signal, but the signal amplifier used to amplify the signal is not limited to the bandpass amplifier, so the type of signal amplifier is not limited here, that is, as long as the detected signal can be effectively amplified.

[0063] Furthermore, the control unit 30 includes an analog-to-digital conversion module 31, a first calculation module and a first control module; the analog-to-digital conversion module 31 is configured to convert the processed electromyographic signal and the skin surface tension signal into a first digital signal and a second digital signal, respectively. The first calculation module is configured to obtain the rotation angle of the user's motion joint according to the first digital signal and the second digital signal and through a first preset algorithm, and to obtain the user's motion posture information through a second preset algorithm. The first control module is configured to generate a first control signal according to the user's motion posture information, and transmit it to the VR device 2 so that the VR device 2 can control the virtual user's limb movements, and after receiving the touch signal of the virtual user being touched fed back by the VR device 2, generate a second control signal according to the touch signal.

[0064] In some examples, the first calculation module is specifically configured to obtain the rotation angle of the user's motion joints based on the first digital signal and the second digital signal and through a first preset algorithm, and to calculate the spatial posture coordinates of the current position points of each motion joint relative to the initial point of each motion joint based on a pre-established kinematic model, thereby obtaining the user's motion posture information; wherein the initial point is the angle information of each motion joint of the user's initial posture.

[0065] Furthermore, the first computing module includes a multimedia application processor (MAP) and an inertial measurement unit (IMU). The first computing module is configured to first process the first digital signal and the second digital signal through MAP to obtain the rotation angle of the user's motion joints, and then calculate the user's motion posture information according to the rotation angle of the user's motion joints through a kinematic formula.

[0066] Specifically, when using the system for the first time, virtual and real body calibration is required, that is, the user's physical size, initial posture, and the spatial coordinates of the hand and foot terminals must correspond to the virtual user. Among them, the physical size of the body is obtained by taking a full-body photo with the VR device 2; the initial posture is based on the initial posture of the user when he starts using the disclosed system. After booting up, the system will record the initial posture through full-body posture tracking. In the subsequent full-body posture tracking process, the joint angles of the initial posture are used as the starting point; the spatial coordinates of the hand and foot terminals can be calculated by kinematics.

[0067] Establish a fixed point O1 and create an X1O1Y1 coordinate system. The position of the fixed point O1 relative to the VR device 2 remains unchanged, and the relative posture is obtained and calculated by the VR device 2 and the IMU. Take the position that the user needs to calculate as O2 and establish an X2O2Y2 coordinate system. Calculate the coordinate transformation matrix of X2O2Y2 relative to X1O1Y1, and finally get the user's posture by calculating the coordinates of the user's limbs relative to O1.

[0068] Figure 1 is a schematic diagram of a human kinematics model; the process of a user controlling a virtual user through body movements is as follows: the user makes corresponding movements according to the virtual screen, the whole body posture tracking and tactile device 1 performs posture tracking, the control unit 30 sends the body posture coordinates to the VR device 2, and the VR device 2 controls the movement of the virtual user through the user posture. Specifically, Figure 1 As shown, taking the left arm of the human body in the figure as an example, a kinematic model is established to calculate the relative spatial pose coordinates of point O2 relative to point O1; the relative position of point O1 to the VR device 2 is fixed, and the relative pose can be obtained and calculated by the IMU of the VR device 2 and the control unit 30. Now the angle information of each joint of the left arm of the human body in the figure can be obtained by whole body pose tracking, and then the pose coordinates of point O2 relative to point O1 are calculated:

[0069]

[0070]

[0071]

[0072]

[0073] In the above formula:

[0074] 1) is the position coordinate of θ2 relative to O1;

[0075] 2) cθ2 is cosθ2;

[0076] 3)sθ2 is sinθ2;

[0077] In the above calculation process, is the coordinate transformation matrix of the hand coordinate system X2O2Y2 relative to the central coordinate system X1O1Y1 of the control unit 30. By calculating the coordinates of the human limbs relative to O1, the whole body posture information of the human body can be obtained. In this process, the process of the user controlling the virtual user through body movements is as follows: the user makes corresponding movements according to the virtual screen, the whole body posture tracking and tactile device 1 performs posture tracking, and the control unit 30 sends the user posture coordinates to the VR device 2, thereby realizing the user controlling the virtual user's movements.

[0078] It is understandable that in the above calculation process, there is no strict requirement for establishing the hand coordinate system X2O2Y2 and the center coordinate system X1O1Y1 of the control unit 30. Usually, the control unit 30 is located at the center of gravity of the human body, which can make it more convenient to calculate the whole body posture information of the human body, but the control unit 30 may not be located at the center of gravity of the human body, and any position of the human body will not affect the tracking and calculation of the whole body posture.

[0079] In some examples, after receiving a touch signal from the VR device 2 indicating that the virtual user is touched, the control unit 30 generates a second control signal based on the touch signal, and the tactile control unit 40 can feedback real tactile perception to the user's limbs through the main structure 10 based on the second control signal.

[0080] In some examples, the control unit 30 includes a second control module and a second calculation module; the second calculation module is configured to calculate the actual position information of the user being touched based on the touch signal of the virtual user being touched fed back by the VR device 2; the second control module is configured to generate a second control signal based on the actual position information of the user being touched, and the tactile control unit 40 can feedback the real tactile perception to the user's limbs through the main structure 10 based on the second control signal.

[0081] It should be noted that the tactile control unit 40 can exist independently of the control unit 30, that is, the control unit 30 does not directly control the realization of virtual tactile, but the tactile control unit 40 can also be controlled by the control unit 30, and the two are connected in communication to realize the virtual tactile function.

[0082] Furthermore, the tactile control unit 40 includes a first switch module 41, a second switch module 42 and a touch module 43; the first switch module 41 and the second switch module 42 are both connected to the hollow wire tube, and are used to control the gas entering and exhausting the hollow wire tube according to the second control signal; and the touch module 43 is arranged on the hollow wire tube defined by the first switch module 41 and the second switch module 42, and when the user wears the main structure 10, the touch module 43 is located on the side of the hollow wire tube close to the human body.

[0083] Specifically, the main structure 10 is composed of a plurality of hollow wire tubes woven together; Figure 2b for Figure 2a Schematic diagram of textile details in part I, such as Figure 2b As shown, the hollow wire tube has a mesh-like interlaced structure and the internal air pressure is adjustable. The flexibility of the hollow wire tube changes with the internal air pressure, and the overall flexibility of the hollow wire tube can be changed by filling or discharging gas; the local air pressure of the hollow wire tube is controllable, and virtual touch can be achieved by adjusting the local air pressure of the hollow wire tube.

[0084] In some examples, the material of the hollow wire tube includes fiber material, i.e., hollow fiber tube. Fiber material has the characteristics of high strength, light weight, and good air permeability, so the main structure 10 woven from the hollow fiber tube is suitable for human wear. However, the material of the hollow wire tube is not limited to fiber material. Under the condition of meeting the functional requirements of the main structure 10, any type of material can be tried, but light weight and high performance are preferred.

[0085] In some examples, the first switch module 41 and the second switch module 42 include but are not limited to valves. The first switch module 41 and the second switch module 42 are used to control the local pressure of the hollow wire tube, so their forms are not limited.

[0086] In some examples, the touch module 43 includes a striking hammer; wherein, when the tactile control unit 40 receives a second control signal, the first switch module 41 and the second switch module 42 on both sides of the striking hammer are controlled to be closed, so that the local air pressure in the hollow wire tube becomes smaller, and the striking hammer strikes the striking point on the surface of the human skin under the action of pressure, thereby realizing virtual pain.

[0087] It should be noted that, instead of closing the first switch module 41 and the second switch module 42 on both sides of the striking hammer to control the striking hammer to strike the human skin surface, the striking hammer may be directly controlled to rise and fall to achieve virtual touch.

[0088] In some examples, the haptic control unit 40 further includes a suction component 44, which is configured to fill the hollow wire tube with gas and discharge the gas in the hollow wire tube. Virtual haptic can be achieved by adjusting the air pressure inside the hollow wire tube.

[0089] In some examples, the suction assembly 44 is an air pump. The air pump fills the hollow wire tube with gas and discharges the gas in the hollow wire tube. Specifically, the gas is introduced into the gas reservoir through the air guide tube, and then introduced into the hollow wire tube. At the same time, the gas reservoir introduces the gas in the gas reservoir into the pressure regulating valve fixed on the air pump through an air guide tube, thereby controlling the air pressure in the gas reservoir. When the air pressure in the gas reservoir does not reach the pressure set by the pressure regulating valve, the gas entering the pressure regulating valve from the gas reservoir cannot push open the valve of the pressure regulating valve; when the air pressure in the gas reservoir reaches the pressure set by the pressure regulating valve, the gas entering the pressure regulating valve from the gas reservoir pushes open the valve of the pressure regulating valve, enters the airway in the air pump that is connected to the pressure regulating valve, and controls the air inlet of the air pump to be normally open through the airway, so that the air pump operates without load. When the air pressure in the gas reservoir is lower than the pressure set by the pressure regulating valve due to loss, the valve in the pressure regulating valve is returned to its position by the return spring, disconnecting the control air path of the air pump, and the air pump starts to pump air again. When not in use, the air pump will automatically exhaust the gas.

[0090] It should be noted that the form and quantity of the suction component 44 are not limited here when it is used to fill the hollow wire tube with gas and discharge the gas in the hollow wire tube. For example, a blower can also be used as the suction component 44. Of course, considering that the whole body posture tracking and tactile device 1 provided by the present disclosure can be worn on the user's body, the weight and performance of the suction component 44 should be taken into consideration, that is, when the suction component 44 is used, the lighter and smaller the suction component 44, the better.

[0091] In order to make the specific working principle of the whole body posture tracking and tactile device 1 provided by the embodiment of the present disclosure clearer, it is described below with reference to specific examples.

[0092] Figure 2a This is a rendering of the full-body posture tracking and tactile device usage; Figure 3 Schematic diagram of virtual reaction force in virtual touch; Figure 4 A schematic diagram of virtual pain in virtual touch; Figure 5 Flowchart for whole body posture tracking; Figure 6 is a virtual touch flow chart; refer to Figure 2a , 3As shown in Figures 4, 5, and 6, the motion detection unit 20 of the full-body posture tracking and tactile device 1 is installed on the main structure 10 and is configured to detect the user's limb movements. When the user moves, the motion detection unit 20 detects the user's limb movements and generates user movement posture information and transmits it to the control unit 30. The control unit 30 is configured to process the received movement posture information, generate a first control signal, and transmit it to the VR device 2, so that the VR device 2 can control the virtual user's limb movements, thereby realizing full-body posture tracking; when the user moves in the VR environment and is hit by a virtual character, a touch signal is generated and transmitted to the VR device 2. After the VR device 2 feeds back a touch signal that the virtual user is touched, the control unit 30 generates a second control signal according to the touch signal. The tactile control unit 40 is configured to feed back real tactile perception to the user's limbs through the main structure 10 according to the second control signal, thereby realizing virtual tactile sensation.

[0093] Specifically, Figure 5 As shown, when using the system for the first time, virtual and real body calibration is required, that is, the user's body physical size, initial posture, and the spatial coordinates of the hand and foot terminals must correspond to the virtual user. Among them, the body physical size is obtained by taking a full-body photo with the VR device 2; the initial posture is based on the initial posture of the user when he starts using the disclosed system. After booting up, the system will record the initial posture through full-body posture tracking. In the subsequent full-body posture tracking process, the joint angles of the initial posture are used as the starting point; the spatial coordinates of the hand and foot terminals can be calculated by kinematics.

[0094] After the user calibrates the virtual and real body, full body posture tracking begins. Figure 7 is a schematic diagram of human elbow joint posture tracking; Figure 7 As shown, taking the tracking of the elbow joint as an example, the elbow joint motion detection units 20 are attached to the outside of the triceps brachii and the outside of the biceps brachii respectively. Figure 8 It is the principle block diagram of motion detection; Figure 8As shown, each motion detection unit 20 includes a first detection module 21 and a second detection module 22, wherein the first detection module 21 includes an electromyographic signal electrode 211 and a first bandpass amplifier 212, and the second detection module 22 includes a skin surface tension strain gauge 221 and a second bandpass amplifier 222. When the muscle contracts, the electromyographic signal electrode 211 detects the electromyographic current. The greater the degree of muscle contraction, the greater the electromyographic current generated. By detecting the magnitude of the electromyographic current, the degree of muscle contraction can be deduced, and then the joint rotation angle can be calculated. The action of the virtual user can be controlled by the action angle of each joint of the human body, thereby completing the whole body posture tracking of the human body; the skin surface tension strain gauge 221 can detect the skin stress changes during muscle contraction and relaxation. When the muscle contracts, the skin is subjected to the squeezing force, and the skin surface tension strain gauge 221 is subjected to the pressure. When the muscle relaxes, the skin is subjected to the surface tension. The skin surface tension strain detects the tension, and the skin stress change current combined with the electromyographic signal current can more accurately calculate the joint rotation angle, thereby completing the whole body posture tracking.

[0095] like Figure 3 As shown in FIG. 1 , when a user tries to push an object with the virtual user's arm in a virtual scene, the user swings his arm in the real environment, and the virtual user also pushes the object in the virtual scene according to the user's action. In a real scene, when we push an object, we will feel the reaction force of the object on our hand or arm. In the present disclosure, we want to achieve this effect in a virtual scene. Figure 3 When the movement shown pushes an object in the opposite direction, the air pressure in the hollow wire tube on the outside of the upper arm decreases, generating a force F1 that contracts inwardly into the hollow wire tube. When the forearm is swung, under the action of F1, the arm will feel a reaction force F2, that is, the user also feels the reaction force received by the virtual user, thereby realizing virtual touch.

[0096] like Figure 6 As shown, when the virtual user is hit, the process of the user feeling pain is as follows: after the virtual user is hit, the VR device 2 records the body coordinate position of the virtual user being hit, and then transmits the position information to the tactile control unit 40. The tactile control unit 40 adjusts the pressure at the corresponding position of the user's body to produce a slight pain.

[0097] Specifically, Figure 4As shown, when the user is playing a fighting game, the virtual opponent hits the virtual user. In order to make the virtual reality closer to the real scene, the user should also feel pain when the virtual user is hit. When a certain part of the virtual user's body is hit, it corresponds to the hitting point on the surface of the user's skin in reality. At this time, the air pressure of the hollow wire tubes on both sides of the first switch module 41 and the second switch module 42 at the hitting point becomes smaller. Under the action of pressure, the first switch module 41 and the second switch module 42 are subjected to the pulling force of F1 and F2. The touch module 43 hits the hitting point on the skin surface under the action of F3 pressure, and the user feels a slight pain, thereby realizing virtual pain.

[0098] In a second aspect, the embodiments of the present disclosure also provide a virtual reality system. Fig. 9 is a schematic diagram of a virtual reality system; Fig. 9 As shown, it includes the above-mentioned full-body posture tracking and haptic device 1 and VR device 2; the VR device 2 is communicatively connected with the full-body posture tracking and haptic device 1.

[0099] In order to make the specific working principle of the virtual reality system provided by the embodiment of the present disclosure clearer, it is described below with reference to specific examples.

[0100] In one example, referring to Fig. 9The virtual reality system provided by the present disclosure includes a whole-body posture tracking and tactile device 1 and a VR device 2. The whole-body posture tracking and tactile device 1 mainly completes the whole-body posture tracking and virtual tactile function, and the VR device 2 mainly completes the general head six-degree-of-freedom tracking. The whole-body posture tracking is mainly completed by the action detection unit 20 arranged on the outer surface of the whole-body skeletal muscle. The action detection unit 20 is mainly composed of an electromyographic signal electrode 211 and a skin surface tension strain gauge 221. When the human body moves, the electromyographic signal electrode 211 and the skin surface tension strain gauge 221 generate a weak current, which is filtered and amplified by a bandpass amplifier and converted into a digital signal by an analog-to-digital conversion module 31. The MAP processes the current value and converts it into a corresponding joint angle to achieve whole-body posture tracking. Virtual touch is achieved by a tights woven from hollow fiber tubes. Virtual touch is achieved by adjusting the pressure inside the local fiber tube. The tights can achieve controllable local pressure. Each local pressure is controlled by a separate switch module. The local pressure is adjusted by introducing or exhausting gas through the suction component 44, and then virtual touch is achieved. The virtual pain is achieved through the touch module 43. At the same time, the IMU can record the posture information of the main control box. The whole body posture tracking and tactile device 1 control unit 30 is connected to the VR device 2 for communication. The control unit 30 sends the user's body posture information to the VR device 2 for processing, and the VR device 2 sends the tactile point coordinates to the tactile control unit 40, thereby realizing the virtual tactile function. It should be noted that in this example, the first switch module 41 and the second switch module 42 are valves, the touch module 43 is a striking hammer, and the suction component 44 is an air pump.

[0101] In some examples, the full-body posture tracking and tactile device 1 and the VR device 2 are connected via WIFI or Bluetooth to achieve information interaction.

[0102] In some examples, the VR device 2 includes a VR helmet. Of course, the VR device 2 can also be any device such as VR glasses that can be worn on the user's head and has a VR display.

[0103] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A whole body posture tracking and tactile device, comprising: A main structure, at least one motion detection unit, a control unit, and at least one tactile control unit; wherein, The main structure is configured to be worn on a limb by a user; The motion detection unit is mounted on the main structure and is configured to detect the user's body movements and generate user motion posture information; The control unit is configured to process the received motion posture information, generate a first control signal, and transmit the first control signal to the VR device so that the VR device can control the body movements of the virtual user, and after receiving a touch signal fed back by the VR device indicating that the virtual user is touched, generate a second control signal according to the touch signal; The tactile control unit is configured to feedback real tactile perception to the user's limbs through the main structure according to the second control signal; The motion posture information includes the rotation angle of the motion joint; the motion detection unit includes a first detection module and a second detection module; The first detection module is configured to detect the electromyographic signal generated by the user's skeletal muscles and process it; The second detection module is configured to detect and process a skin surface tension signal of the user; The control unit is configured to obtain the rotation angle of the user's motion joint based on the processed electromyographic signal and the skin surface tension signal through a first preset algorithm, and to obtain the user's motion posture information based on the rotation angle of the user's motion joint through a second preset algorithm to generate a first control signal.

2. The whole body posture tracking and tactile device according to claim 1, wherein: The first detection module includes an electromyographic signal electrode and a first bandpass amplifier; the second detection module includes a skin surface tension strain gauge and a second bandpass amplifier; The electromyographic signal electrode is configured to detect the electromyographic signal generated by the user's skeletal muscle; The first bandpass amplifier is configured to amplify the electromyographic signal and transmit it to the control unit; The skin surface tension strain gauge is configured to detect a skin surface tension signal of a user; The second bandpass amplifier is configured to amplify the skin surface tension signal and transmit the amplified signal to the control unit.

3. The whole body posture tracking and tactile device according to claim 1 or 2, wherein: The control unit includes an analog-to-digital conversion module, a first calculation module and a first control module; The analog-to-digital conversion module is configured to convert the processed electromyographic signal and the skin surface tension signal into a first digital signal and a second digital signal respectively; The first calculation module is configured to obtain the rotation angle of the user's motion joint according to the first digital signal and the second digital signal through a first preset algorithm, and obtain the user's motion posture information through a second preset algorithm; The first control module is configured to generate a first control signal according to the user's motion posture information and transmit it to the VR device so that the VR device can control the virtual user's body movements, and after receiving a touch signal from the VR device indicating that the virtual user is touched, generate a second control signal according to the touch signal.

4. The whole body posture tracking and tactile device according to claim 3, wherein: The first calculation module is specifically configured to obtain the rotation angle of the user's motion joints based on the first digital signal and the second digital signal and through a first preset algorithm, and to calculate the spatial posture coordinates of the current position points of each motion joint relative to the initial point of each motion joint based on a pre-established kinematic model, thereby obtaining the user's motion posture information; wherein the initial point is the angle information of each motion joint of the user's initial posture.

5. The whole body posture tracking and tactile device according to claim 1, wherein: The main structure is composed of a plurality of hollow wire tubes woven together; the tactile control unit includes a first switch module, a second switch module and a touch module; The first switch module and the second switch module are both connected to the hollow wire tube, and are used to control the gas to enter and discharge from the hollow wire tube according to the second control signal; and the touch module is arranged on the hollow wire tube defined by the first switch module and the second switch module, and when the user wears the main structure, the touch module is located on the side of the hollow wire tube close to the human body.

6. The whole body posture tracking and haptic device according to claim 5, wherein: The control unit includes a second control module and a second calculation module; The second calculation module is configured to calculate the actual touched position information of the user according to the touch signal of the virtual user fed back by the VR device; The second control module is configured to generate the second control signal according to the position information where the user is actually touched.

7. The whole body posture tracking and tactile device according to claim 5, wherein: The material of the hollow wire tube includes fiber material.

8. The whole body posture tracking and haptic device according to claim 5, wherein: The first switch module and the second switch module include valves.

9. The whole body posture tracking and haptic device according to claim 5, wherein: The touch module includes a striking hammer.

10. The whole body posture tracking and haptic device according to claim 5, wherein: The haptic control unit further includes a suction component configured to fill the hollow wire tube with gas and discharge the gas in the hollow wire tube.

11. The whole body posture tracking and haptic device according to claim 10, wherein: The suction component is an air pump.

12. A virtual reality system, comprising the whole body posture tracking and tactile device and a VR device according to any one of claims 1 to 11; the VR device is communicatively connected to the whole body posture tracking and tactile device.

13. The virtual reality system according to claim 12, wherein: The whole body posture tracking and tactile device is connected to the VR device via WIFI or Bluetooth.

14. The virtual reality system according to claim 12, wherein: The VR device includes a VR helmet.

Images