VR or AR interaction system and method based on wearable flexible sensor

Abstract

The invention discloses a VR or AR interaction system and method based on a wearable flexible sensor, the system comprises a flexible sensor, a flexible circuit board and a virtual reality head-mounted display, the flexible sensor comprises a flexible stretching sensor and a flexible pressure sensor, the flexible sensor detects the movement of a palm and a hand back, the stretching sensor is attached to the hand back, and the flexible pressure sensor is attached to the flexible circuit board. The pressure sensor is attached to a palm, the flexible circuit board comprises an R-V conversion unit, an analog-to-digital conversion unit, a wireless transmission unit, a power management unit, a microprocessor unit and an inertial sensing unit, and the virtual reality head-mounted display comprises a wireless receiving unit, a model attitude calculation unit, a model position control unit, a model attitude control unit and a neural network. The flexible sensor and the flexible circuit board jointly act on the virtual reality head-mounted display, the VR or AR interaction system and method based on the wearable flexible sensor are adopted, natural actions of a user are better adapted, the immersion feeling of the user is enhanced, and a more stable and reliable action capture scheme is provided.

Description

Technical Field

[0001] The present invention relates to the field of interactive systems, and in particular to a VR or AR interactive system and method based on a wearable flexible sensor. Background Art

[0002] Disadvantages or limitations of existing virtual reality interaction methods: Current virtual reality (VR) interactions usually rely on tactile modal devices, such as game controllers, keyboards, mice, or motion controllers, to allow users to interact with the virtual world. However, these technologies limit the user's interactive experience to a certain extent, are unnatural in interaction, and limit the level of immersion that users can achieve. In order to seek a more natural way of interaction, researchers are currently exploring various methods of using technology, such as motion controllers, tactile gloves, and optical-based trackers. At present, the most advanced development is optical-based trackers in achieving free interaction, because this technology enables the user's hands to interact freely with the virtual environment without being restricted by the interactive device. However, this optical technology is greatly affected by the environment, such as light, occlusion, and the size of the viewing angle. When the arm moves out of the range that the lens can monitor, the corresponding gesture can no longer be captured.

[0003] Advantages of flexible wearables: Flexible wearable devices are stretchable, soft and can fit the user's body perfectly, provide a more comfortable wearing experience, and can more accurately capture the user's movements and postures. This highly accurate motion capture enables the virtual reality environment to present the user's movements more finely, improving the realism and naturalness of the interaction. Compared with traditional rigid devices, flexible devices can better adapt to the user's natural movements and enhance the user's sense of immersion.

[0004] Flexible wearable technology also has the characteristics of portability and large-area application. Since flexible devices are usually light and easy to carry, users can interact with virtual reality anytime and anywhere without being restricted to specific places or devices. This portability makes the virtual reality experience more free and flexible.

[0005] The preparation cost of flexible wearable technology is relatively low, and flexible electronic devices can be mass-produced using low-cost manufacturing methods such as screen printing. This makes flexible wearable technology more feasible and popular, providing more users with opportunities for virtual reality interaction.

[0006] In contrast, wearable virtual interactive devices have lower requirements for the use environment. The motion capture device with flexible wearable stress sensors and inertial sensors performs well in avoiding the limitations of optical trackers, and can eliminate the cumulative errors caused by the inertial measurement unit (IMU) through algorithms, providing a more stable and reliable motion capture solution.

[0007] The application of flexible pressure sensor arrays can further enhance virtual reality interaction. By attaching a flexible pressure sensor array to the palm of the hand and combining it with artificial intelligence (AI) deep learning algorithms, it is possible to recognize simple objects that users grasp in the real world and present them in a virtual reality environment, thereby enhancing the sense of interaction between reality and virtuality. Summary of the invention

[0008] The purpose of the present invention is to provide a VR or AR interaction system and method based on wearable flexible sensors to better adapt to the user's natural movements, enhance the user's sense of immersion, better adapt to the user's natural movements, enhance the user's sense of immersion, and provide a more stable and reliable motion capture solution, thereby enhancing the sense of interaction between reality and virtuality.

[0009] To achieve the above-mentioned purpose, the present invention provides a VR or AR interaction system and method based on a wearable flexible sensor, including a flexible sensor, a flexible circuit board and a virtual reality head display, wherein the flexible sensor includes a flexible stretch sensor and a flexible pressure sensor, the flexible stretch sensor is attached to the back of the hand, and the flexible pressure sensor is attached to the palm, the flexible circuit board includes an RV conversion unit, an analog-to-digital conversion unit, a wireless transmission unit, a power management unit, a microprocessor unit and an inertial sensing unit, the virtual reality head display includes a wireless receiving unit, a model attitude solution, a model position control, a model attitude control and a neural network, the flexible sensor acts on the virtual reality head display through the flexible circuit board, the preparation of the flexible sensor includes the preparation of the flexible stretch sensor and the preparation of the flexible pressure sensor, and the preparation of the flexible stretch sensor, the preparation of the flexible pressure sensor and the design of the flexible circuit board interact through the virtual reality head display.

[0010] Preferably, the preparation of the flexible stretch sensor includes the preparation of a substrate of the flexible stretch sensor, the preparation and patterning of a conductive filler of the flexible stretch sensor, and the extraction of electrodes of the flexible stretch sensor.

[0011] Preferably, the preparation of the flexible pressure sensor includes preparing a substrate of the flexible pressure sensor, preparing electrodes of the flexible pressure sensor, and leading out the electrodes of the flexible pressure sensor.

[0012] Preferably, the flexible tensile sensor and the flexible pressure sensor are connected by clamping one end of two wires on two electrodes of the sensor at the same time and connecting the other ends of the two wires to the flexible circuit board collection terminals at the same time.

[0013] Preferably, the flexible printed circuit board consists of three parts: a signal acquisition circuit, a wireless transmission unit and a power management unit; the signal acquisition circuit consists of an RV conversion unit, an analog-to-digital conversion unit, a microprocessor unit and an inertial sensor unit, the RV conversion unit converts the flexible stretch sensor signal and the resistance signal of the flexible pressure sensor into a voltage signal; the analog-to-digital conversion unit converts the analog voltage signal into a digital signal and transmits it to the microprocessor unit; the inertial sensor unit solves the circuit; the wireless transmission unit uses a Bluetooth or WiFi wireless communication circuit, the power management unit uses an IP5306 chip, and is equipped with supporting circuits.

[0014] Preferably, the wireless receiving unit receives information from the flexible stretch sensor and the flexible pressure sensor transmitted by the wireless circuit of the flexible circuit board.

[0015] Preferably, S1, preparing a flexible pressure sensor and a flexible stretch sensor, and the flexible circuit board collects signals from the flexible pressure sensor and the flexible stretch sensor, and transmits the position and overall posture information of the hand calculated by the inertial sensor to the virtual reality head display;

[0016] S2. In the virtual reality headset, the pre-written program will standardize the acquired data. The small change data of the flexible stretch sensor and the position information collected by the inertial sensor will be mapped to the corresponding finger bending angle and joint position. The virtual reality system can control the relative displacement and posture changes of the hand in the virtual environment, and make the same movements as the real hand;

[0017] S3, using the 500 data collected in advance, training through a deep learning algorithm; the algorithm will determine the type of object the user grasps based on the value of the pressure sensor during grasping;

[0018] The application of S4, flexible pressure sensors and flexible stretch sensors enables users to interact with virtual reality in a more intuitive and natural way, and the user's hand movements and pressure perception can be accurately conveyed to the virtual environment.

[0019] Therefore, the present invention adopts the above-mentioned VR or AR interaction system and method based on wearable flexible sensors to better adapt to the user's natural movements and enhance the user's immersion. Better adapt to the user's natural movements and enhance the user's immersion. Provide a more stable and reliable motion capture solution. Thereby enhancing the sense of interaction between reality and virtuality.

[0020] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A schematic diagram of preparing a stretch sensor for a VR or AR interactive system and method based on a wearable flexible sensor according to the present invention;

[0022] Figure 2 A schematic diagram of preparing a pressure sensor for a VR or AR interactive system and method based on a wearable flexible sensor according to the present invention;

[0023] Figure 3 It is an overall schematic diagram of a VR or AR interaction system and method based on a wearable flexible sensor of the present invention;

[0024] Figure 4 A schematic diagram of a hand stretch sensor and a pressure sensor of a VR or AR interaction system and method based on a wearable flexible sensor of the present invention;

[0025] Figure 5 A block diagram of a flexible circuit board of a VR or AR interaction system and method based on a wearable flexible sensor of the present invention;

[0026] Figure 6 This is a flexible circuit board design diagram of a VR or AR interaction system and method based on a wearable flexible sensor of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0028] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0029] Embodiment 1

[0030] like Figure 1-6As shown, the present invention is a VR or AR interaction system and method based on a wearable flexible sensor, characterized in that it includes a flexible sensor, a flexible circuit board and a virtual reality head display, the flexible sensor includes a flexible stretch sensor and a flexible pressure sensor, the flexible stretch sensor is attached to the back of the hand, and the flexible pressure sensor is attached to the palm, the flexible circuit board includes an RV conversion unit, an analog-to-digital conversion unit, a wireless transmission unit, a power management unit, a microprocessor unit and an inertial sensing unit, the virtual reality head display includes a wireless receiving unit, a model attitude solution, a model position control, a model attitude control and a neural network, the flexible sensor acts on the virtual reality head display through the flexible circuit board, the preparation of the flexible sensor includes the preparation of the flexible stretch sensor and the preparation of the flexible pressure sensor, and the preparation of the flexible stretch sensor, the preparation of the flexible pressure sensor and the design of the flexible circuit board interact through the virtual reality head display.

[0031] The preparation of the flexible stretch sensor includes the preparation of the substrate of the flexible stretch sensor, the preparation and patterning of the conductive filler of the flexible stretch sensor, and the lead-out of the electrode of the flexible stretch sensor;

[0032] The elastic reservoir in the preparation of the stretch sensor substrate is made by mixing 6.93g Ecoflex 00-30, 0.07g Slo-Jo platinum silicone rubber retarder and 0.07g Thivex; the filling liquid is made by mixing 6.93g Ecoflex 00-30, 0.07g Slo-Jo platinum silicone rubber retarder and 0.78g Silicone Thinner; the elastic reservoir and the filling liquid are stirred in an ARE-310 stirrer at a speed of 1000-3000rpm for 60-90 seconds, and then defoamed at a speed of 1000-3000rpm for 60-90 seconds; the elastic reservoir is poured into a culture dish coated with a release agent CandleMold Release, ArtMind, and the filling liquid is stacked on top of the elastic reservoir. The thickness of the filling liquid is less than 0.5mm, and the entire culture dish is placed on a vortex stirrer and stirred for 2-3 minutes.

[0033] Carbon black CB particles and silicone oil are dissolved in a mass ratio of 1:3, such as dissolving 2g of carbon black CB particles in 6g of silicone oil to prepare carbon conductive grease; the original carbon conductive grease MG Chemicals is uniformly mixed and stirred at 2000-3000rpm for two minutes using an ARE-310 agitator; the ink is continued to be defoamed in the agitator at a speed of 2000-3000rpm for two minutes; the final ink is loaded into a 3 ml syringe for embedded 3D printing, the printing path is set using CAD, and then converted into G-code using CADFusion Aerotech, and the 3D printer is controlled by G-code to print the specified pattern along the specified printing path, and the conductive filler ink is extruded through a nozzle NordsonEFD with an inner diameter of 410 microns, and is controlled by applying pressure, which is controlled by the Ultimus V control device NordsonEFD.

[0034] The preparation of the flexible pressure sensor includes the preparation of the substrate of the flexible pressure sensor, the preparation of the electrode of the flexible pressure sensor and the lead-out of the electrode of the flexible pressure sensor. The preparation of the substrate of the flexible pressure sensor is to prepare a silicon mold with a concave feature of a pyramid pattern by photolithography technology and wet etching, and then a layer of positive photoresist PR is spin-coated on a silicon wafer with a crystal orientation of 100 that has been cleaned by a sulfuric acid peroxide mixture SPM. Then, under the condition of using a photomask, the silicon wafer is exposed using a light alignment machine EVG 6200. After developing and exposing PR, the silicon wafer is wet-etched using a 25wt% tetramethylammonium hydroxide TMAH solution to form a pyramid etching pit with a slope angle of 54.7°; UV-curable polyurethane acrylate PUA, 311RM, Minuta Tech is used to replicate the silicon mold, a PUA precursor is dripped on the silicon mold, and then a transparent polyethylene terephthalate PET, 188μm film is placed on the liquid precursor as a backplane, and then UV light is used After irradiation for 45 seconds and UV curing, it is peeled off from the mold, leaving the first replicated PUA pattern with a recessed pyramid array; the PDMS elastomer and the cross-linker are mixed in a ratio of 10:1 and degassed under vacuum, the mixture is cast on the PUA replica with a pyramid pattern, and a spacer is used to define a thickness of 300 microns. After a thermal curing process at 85°C and 1 hour, the PDMS film with the pyramid array pattern is peeled off from the PUA replica, the micro-patterned PDMS substrate is cleaned with pressure tape, and UV ozone pretreatment is performed for 15 minutes.

[0035] Preparation of the electrode of the flexible pressure sensor: 1g PUD was diluted to 4wt% with 9g Milli-Q water, and then 9.3g PEDOT:PSS poly 3,4-ethylenedioxythiophene-polystyrene sulfonate solution was premixed with 0.6g dimethyl sulfoxide DMSO and 0.1g Zonyl; during the mixing process, the diluted PUD was stirred at high speed, and 40%, 27% or 14% PEDOT:PSS was added dropwise; 73% PUD polymer mixed solution was dropped on the PDMS substrate, and the excess solution was removed by vertically holding the substrate, and then annealed on a hot plate at 70°C for 30 minutes, and finally a uniform, submicron thick thin film electrode with a micro-pyramid structure was formed on the PDMS substrate, and a flexible electrode with a planar structure was covered on the flexible electrode.

[0036] The connection of the flexible tensile sensor and the flexible pressure sensor is achieved by clamping one end of the two wires on the two electrodes of the sensor at the same time, and then connecting the other ends of the two wires to the flexible circuit board collection terminals at the same time to collect signals.

[0037] The design of the circuit board consists of three parts: a multi-channel sensor signal acquisition circuit, a wireless transmission unit, and a power management unit; the signal acquisition circuit consists of four parts: an RV conversion unit, an analog-to-digital conversion unit, a microprocessor unit, and an inertial sensor unit. The RV conversion unit is used to convert the resistance signals of the 14-channel flexible stretch sensor signals and the 25-channel array flexible pressure sensor signals into voltage signals, and then the analog-to-digital conversion unit is used to convert the analog voltage signal into a digital signal and transmit it to the microprocessor unit for data processing. In addition, the inertial sensor solution circuit uses a commercial IMU-MPU6050, which includes a three-axis accelerometer, a three-axis gyroscope, and a digital motion processor DMP (Digital Motion Processor). The DMP can directly obtain the outputs of the three accelerometers and the three-axis gyroscope, and transmit them to the main control MCU through the IIC interface to solve the current overall rotation angle posture information and movement position information of the hand. The flexible circuit board is equipped with a wireless transmission unit, which can use wireless communication circuits such as Bluetooth or WiFi to transmit all the above sensor data to the virtual reality headset. In addition, the circuit board also has a power management unit that uses the IP5306 chip to realize the lithium battery charging and discharging functions, and is equipped with a supporting circuit to output a stable 5V voltage to power the circuit board. There are also 4 LEDs for displaying the lithium battery power information.

[0038] The virtual reality head display includes a wireless receiving unit, a model attitude solution, a model position control, a model attitude control and a neural network.

[0039] The wireless receiving unit receives information from the flexible stretch sensor and the flexible pressure sensor transmitted by the wireless circuit of the flexible circuit board.

[0040] Model position control: The microprocessor unit integrates the three-axis real-time position (x, y, z) changes collected by the inertial sensor unit to obtain the overall position change information of the hand in three-dimensional space and transmit it to the virtual reality headset. The parameters of the three-dimensional spatial position of the hand model displayed in the virtual reality headset can be directly set, that is, the value of its relative spatial coordinate origin can be added or subtracted according to the acquired change information (Δx, Δy, Δz), and the corresponding value (Δx, Δy, Δz) is assigned to the parameters of the three-dimensional spatial position of the hand model, so that the relative change of the hand model position in the virtual headset is synchronized with the real hand position in real time.

[0041] Model posture calculation: The wireless transmission unit of the flexible circuit board transmits 14 data obtained from the flexible stretch sensor to the virtual reality headset. The data transmission order corresponds to the flexible stretch sensor data collected from the two joints of the thumb, the index finger, the middle finger, the ring finger, and the three joints of the little finger. The program in the virtual reality headset first collects 10 sets of data (voltage values) for the fist state and the finger opening, takes the average value of each data, sets the joint to be fully opened when the maximum value is collected and the joint to be fully bent when the minimum value is collected, and then maps the values ​​in this interval, that is, one voltage value corresponds to one bending angle, thereby realizing the bending angle posture calculation of each joint.

[0042] Neural network: An array of flexible pressure sensors (25 pressure sensors) was used to monitor the pressure distribution of the palm. The array of flexible pressure sensors recorded data when holding a variety of different objects, such as spheres, cuboids, cylinders, etc. Then, the monitoring data of each object was standardized to establish a basic sample database. First, the output values ​​of the array of flexible pressure sensors were collected when the fingers were suspended in the air and when the palm was pressed on a plane. For the data of each sensor, the sensor values ​​in the two cases were taken as the maximum and minimum values, respectively, and standardized: divided by the interval value (i.e., the maximum value minus the minimum value), and the deviation was added or subtracted to standardize it to the range of 0-1. Next, the 25 data channels corresponding to each sample in the sample database were fused to form a 25×500 matrix. This matrix contains the finger bending features and palm pressure distribution features. The sample database was divided into a training set and a test set in a ratio of 8:2. The training set was used as training data and trained by a convolutional neural network to realize a model for object recognition grasped by the hand. The trained model is verified through the test set to ensure that the recognition accuracy reaches the predetermined level, such as more than 95% of the verification accuracy. The recognized 3D object model will be displayed synchronously in the virtual reality headset.

[0043] Model posture control: The microprocessor unit integrates the three-axis posture angle (roll, yaw, pitch) collected by the inertial sensor unit to obtain the overall angle change information of the hand in three-dimensional space and transmit it to the virtual reality headset. The parameters of the three-dimensional spatial angle of the hand model displayed in the headset can be directly set, that is, according to the acquired change information (Δroll, Δyaw, Δpitch), the corresponding value (Δroll, Δyaw, Δpitch) is added or subtracted from the value of its relative spatial coordinates, and then assigned to the parameters of the three-dimensional spatial angle of the hand model, so as to achieve real-time synchronization of the angle of the overall hand model in the virtual headset with the real hand.

[0044] The results of the model posture solution are matched to the joints of the hand model in the virtual reality headset, and the values ​​of each joint relative to the spatial coordinates of the hand are assigned to change, so as to achieve the control of the bending posture of each joint of the hand fingers itself in synchronization with reality.

[0045] First, the information of sensors (flexible stretch sensor, flexible pressure sensor, inertial sensor) transmitted by the wireless circuit of the flexible circuit board is received through the wireless receiving Bluetooth in the virtual reality headset. The inertial sensor obtains three-axis acceleration and three-axis attitude angle, and these data are used to synchronously update the overall attitude position of the virtual hand in VR. The bending of the five finger joints is monitored by 14 flexible stretch sensors, and the pressure distribution of the palm is monitored by array flexible pressure sensors (25 pressure sensors). These sensors record the data of holding a variety of different objects, such as spheres, cuboids, cylinders, etc. Then, we perform numerical standardization on the monitoring data of each object when holding it to establish a basic sample database. First, the output values ​​of the array flexible pressure sensor are collected when the fingers are suspended in the air and the palm is pressed on a plane. For the data of each sensor, we take the sensor values ​​in the two cases as the maximum and minimum values, and perform standardization conversion: divide by the interval value (that is, the maximum value minus the minimum value), and add or subtract the deviation to standardize it to the range of 0-1. Next, the 25 data channels corresponding to each sample in the sample database are fused to form a 25x500 matrix. This matrix contains the finger bending features and palm pressure distribution features. The sample database is divided into a training set and a test set in a ratio of 8:2. The training set is used as training data and trained through a convolutional neural network to realize a model for recognizing objects grasped by the hand. The trained model is verified through the test set to ensure that the recognition accuracy reaches a predetermined level, such as a verification accuracy of more than 95%. The recognized 3D object model will be displayed synchronously in the virtual reality headset. The sample database is divided into a training set and a test set in a ratio of 8:2. The training set is used as training data and trained through a convolutional neural network to realize a model for recognizing objects grasped by the hand. The trained model is verified through the test set to ensure that the recognition accuracy reaches a predetermined level, such as a verification accuracy of more than 95%. The recognized 3D object model will be displayed synchronously in the virtual reality headset. In addition, the standardized flexible stretch sensor data is used to control the bending posture of each finger to achieve synchronization of virtual and real movements.

[0046] S1. Prepare a flexible pressure sensor and a flexible stretch sensor, and use a flexible circuit board to collect signals from the flexible pressure sensor and the flexible stretch sensor, and transmit the position and overall posture information of the hand calculated by the inertial sensor to a virtual reality head display;

[0047] S2. In the virtual reality headset, the pre-written program will standardize the acquired data. The small change data of the flexible stretch sensor and the position information collected by the inertial sensor will be mapped to the corresponding finger bending angle and joint position. The virtual reality system can control the relative displacement and posture changes of the hand in the virtual environment, and make the same movements as the real hand;

[0048] S3. Use the 500 data collected in advance to train through the deep learning algorithm; the algorithm will determine the type of object the user is grabbing based on the value of the pressure sensor during grasping; for example, it can distinguish between objects such as balls, cuboids and cones. In the virtual world, the corresponding objects will be displayed based on the judgment results of the trained neural network.

[0049] The application of S4, flexible pressure sensors and flexible stretch sensors enables users to interact with virtual reality in a more intuitive and natural way. The user's hand movements and pressure perception can be accurately conveyed to the virtual environment, thereby enhancing the realism and immersion of the interaction. At the same time, the application of deep learning algorithms further enhances the virtual reality system's ability to recognize the objects grasped by the user, allowing the virtual world to more realistically reflect the user's actions and intentions.

[0050] Therefore, the present invention adopts the above-mentioned VR or AR interaction system and method based on wearable flexible sensors to better adapt to the user's natural movements and enhance the user's immersion. Better adapt to the user's natural movements and enhance the user's immersion. Provide a more stable and reliable motion capture solution. Thereby enhancing the sense of interaction between reality and virtuality.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A VR or AR interactive system based on a wearable flexible sensor, characterized in that: It includes a flexible sensor, a flexible circuit board and a virtual reality headset. The flexible sensor includes a flexible stretch sensor and a flexible pressure sensor. The flexible stretch sensor is attached to the back of the hand, and the flexible pressure sensor is attached to the palm. The flexible circuit board includes an RV conversion unit, an analog-to-digital conversion unit, a wireless transmission unit, a power management unit, a microprocessor unit and an inertial sensor unit. The virtual reality headset includes a wireless receiving unit, a model attitude solution, a model position control, a model attitude control and a neural network. The flexible sensor acts on the virtual reality headset through a flexible circuit board. The preparation of the flexible sensor includes the preparation of a flexible stretch sensor and the preparation of a flexible pressure sensor. The preparation of the flexible stretch sensor, the preparation of the flexible pressure sensor and the design of the flexible circuit board interact through the virtual reality headset.

2. A VR or AR interactive system based on a wearable flexible sensor according to claim 1, characterized in that: The preparation of the flexible stretch sensor includes the preparation of the substrate of the flexible stretch sensor, the preparation and patterning of the conductive filler of the flexible stretch sensor, and the lead-out of the flexible stretch sensor electrode.

3. A VR or AR interactive system based on a wearable flexible sensor according to claim 1, characterized in that: The preparation of the flexible pressure sensor includes the preparation of the substrate of the flexible pressure sensor, the preparation of the electrode of the flexible pressure sensor and the lead-out of the electrode of the flexible pressure sensor.

4. A VR or AR interactive system based on a wearable flexible sensor according to claim 1, characterized in that: The connection between the flexible tensile sensor and the flexible pressure sensor is achieved by clamping one end of the two wires on the two electrodes of the sensor at the same time and connecting the other ends of the two wires to the flexible circuit board collection terminals at the same time.

5. A VR or AR interactive system based on a wearable flexible sensor according to claim 1, characterized in that: The flexible printed circuit board consists of three parts: signal acquisition circuit, wireless transmission unit and power management unit; the signal acquisition circuit consists of RV conversion unit, analog-to-digital conversion unit, microprocessor unit and inertial sensor unit. The RV conversion unit converts the flexible stretch sensor signal and the resistance signal of the flexible pressure sensor into a voltage signal; the analog-to-digital conversion unit converts the analog voltage signal into a digital signal and transmits it to the microprocessor unit; the inertial sensor unit solves the circuit; the wireless transmission unit uses Bluetooth or WiFi wireless communication circuit, the power management unit uses IP5306 chip, and is equipped with supporting circuits.

6. A VR or AR interactive system based on a wearable flexible sensor according to claim 1, characterized in that: The wireless receiving unit receives information from the flexible stretch sensor and the flexible pressure sensor transmitted by the wireless circuit of the flexible circuit board.

7. A method of a VR or AR interactive system based on a wearable flexible sensor as described in any one of claims 1 to 6, characterized in that: The following steps are involved: S1. Prepare a flexible pressure sensor and a flexible stretch sensor, and use a flexible circuit board to collect signals from the flexible pressure sensor and the flexible stretch sensor, and transmit the position and overall posture information of the hand calculated by the inertial sensor to a virtual reality head display; S2. In the virtual reality headset, the pre-written program will standardize the acquired data. The small change data of the flexible stretch sensor and the position information collected by the inertial sensor will be mapped to the corresponding finger bending angle and joint position. The virtual reality system can control the relative displacement and posture changes of the hand in the virtual environment, and make the same movements as the real hand; S3, using the 500 data collected in advance to train through the deep learning algorithm; The algorithm determines the type of object the user is grasping based on the value of the pressure sensor during grasping; The application of S4, flexible pressure sensors and flexible stretch sensors enables users to interact with virtual reality in a more intuitive and natural way, and the user's hand movements and pressure perception can be accurately conveyed to the virtual environment.

Images