XR environment gesture recognition system based on 3D virtual touch control

Abstract

The invention relates to the field of virtual interaction, and discloses an XR environment gesture recognition system based on 3D virtual touch, and the system comprises an input module which is used for receiving an analog pressure signal applied by a user in a supported XR environment; the processing module is used for identifying the gesture of the user according to the analog pressure signal received by the input module; simulation pressure distribution of pressing of a user is captured through the brush force, the gesture analysis unit is used for conducting synchronous processing on the brush force and the position, simulation pressure grades of different grades are effectively detected, accurate recognition of interaction intentions conveyed by gestures is achieved, different visual feedbacks are provided for the user, and the user experience is improved. According to the method, a user can generate tactile feedback similar to reality, so that the naturalness and intuition of interaction are remarkably improved, the user can interact with the XR environment through natural gestures through different levels of simulation pressure, the immersion and experience of the user are enhanced, the accuracy and response speed of interaction are improved, and the user experience is improved. And the system is ensured to provide corresponding feedback in real time according to the simulated pressure grading, so that the user can obtain better perception and control feeling in the interaction process.

Description

Technical Field

[0001] The present invention relates to the field of virtual interaction, and in particular to an XR environment gesture recognition system based on 3D virtual touch. Background Art

[0002] The XR environment gesture recognition system is a gesture recognition system that integrates virtual reality (VR), augmented reality (AR) and mixed reality (MR) technologies. Through a series of virtual interactions and algorithms, it can recognize information such as user's hand pressing, gestures or movements in the virtual environment, allowing users to use natural gestures to interact with objects in the 3D virtual environment. In the extended reality environment, users can not only experience the convenience similar to 3D touch, but also achieve deep interaction in three-dimensional space that exceeds the traditional two-dimensional touch screen.

[0003] According to the Chinese patent announcement number: CN117420910A, an interactive operating system and method applied to the XR interactive metaverse is disclosed, and the steps of the method include: collecting posture data, gesture data, eye movement data, spatial data and facial images through multiple sensors; preprocessing the data, the preprocessing includes data denoising and data normalization; interactive intention recognition, by fusing multi-channel data to obtain user behavior characteristics and spatial characteristics, and identifying the user's interactive intention; collision detection, by using a collision detection algorithm to determine whether a body part is in contact with a virtual control; performing interactive operations, which include user active operations and preset passive triggers. The present invention achieves accurate identification of user interactive intentions by comprehensively analyzing user behavior characteristics, spatial characteristics and user wearing comfort; According to Chinese patent announcement number: CN115407878A, a method and system for implementing XR gesture input is disclosed, which includes: determining environmental assessment data based on the acquired first auxiliary data; acquiring a user's login gesture and performing gesture recognition to determine whether the user can log in to the virtual conference space; the acquisition of the login gesture is related to the environmental assessment data; in response to the user being able to log in to the virtual conference space, acquiring an interaction gesture; the acquisition of the interaction gesture is related to the environmental assessment data; based on the gesture recognition of the interaction gesture, implementing the interaction between the user and the virtual scene in the virtual conference space, However, the above-mentioned XR gesture recognition method still has shortcomings. The gesture recognition system relies on simple touch or motion sensors and lacks accurate capture of pressure distribution. Due to the lack of in-depth analysis of pressure distribution, the traditional system has low accuracy when recognizing complex gestures or subtle changes in force, and is prone to misrecognition, affecting user experience. Therefore, an XR environment gesture recognition system based on 3D virtual touch is needed. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides an XR environment gesture recognition system based on 3D virtual touch, which captures the virtual pressure distribution of pressing force through a force brush, and uses a gesture analysis unit to synchronously process the brush force and position, effectively detects different levels of virtual pressure, and realizes accurate recognition of the interaction intention conveyed by gestures. It not only significantly improves the naturalness and intuitiveness of the interaction, but also enables users to interact with the XR environment with natural gestures by applying different levels of virtual pressure, thereby enhancing the user's immersion and experience.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: an XR environment gesture recognition system based on 3D virtual touch, comprising: Force brush module: Force brush uses the distance between the finger and the virtual interface as the third-dimensional input, and realizes the depth interaction between the user and the main screen through distance. The input module is used to receive the simulated pressure signal applied by the user in the XR environment. The simulated pressure signal is the different interaction distance information between the user's finger and the interface to identify the interaction intention. The role of the force brush is to provide users with different visual prompts and feedback, so that users have virtual touch and produce tactile feedback similar to real ones; A processing module, used to recognize the user's gesture according to the pressure signal received by the input module, the processing module performs real-time analysis on the pressure signal transmitted by the input module, including time domain and frequency domain feature extraction of the signal. Before performing time domain and frequency domain feature extraction, the processing module first pre-processes the signal, including denoising, filtering and normalization, to eliminate the influence of environmental noise and sensor deviation on gesture recognition; The processing module includes a force recognition unit, a position recognition unit and a gesture analysis unit; The force recognition unit is used to recognize the force of the user's gesture according to the detected pressure data; The position recognition unit is used to recognize the position of the user's gesture according to the touch position data detected by the position sensor; The gesture analysis unit is used to synchronously process the brush force and brush position results, generate corresponding interaction instructions according to the force and position of the gesture, render different visual prompt effects, and detect different levels of pressure sensitivity to recognize different gestures by applying different levels of pressure to convey corresponding interaction intentions; An output module for applying the gestures recognized by the processing module to the interaction in the XR environment; The output module includes a signal converter and a transmission interface; The signal converter is used to convert the interaction instruction generated by the processing module into a signal recognizable by the XR environment; The transmission interface is used to transmit the signal output by the signal converter to the XR environment. It can also use wireless technologies such as Bluetooth and Wi-Fi to transmit the signal to the XR device. Using the USB (Universal Serial Bus) interface for transmission can provide high-speed data transmission rate and stable connection. HDMI (High-Definition Multimedia Interface) is often used to transmit video and audio signals, ensuring high-definition picture quality and smooth sound experience. In addition to wired transmission, the transmission interface also supports wireless technologies such as Bluetooth and Wi-Fi. Bluetooth is suitable for scenarios with short transmission distance and small data volume, such as the connection between game controllers and head-mounted display devices. Wi-Fi technology provides higher transmission speeds and longer transmission distances, and is suitable for XR applications that require high bandwidth, such as panoramic video streaming transmission.

[0006] Preferably, the force identification unit includes the following formula for calculating the brush force: ; in is the brush strength at time t, is the simulation pressure weight coefficient, is the total length of the force brush, for The distance between the top of the brush and the contact surface.

[0007] Preferably, the position identification unit includes the following formula for calculating the brush position: ; in, Indicates at time The brush position coordinates, and Respectively at time of Axis and The velocity component on the axis, and The width and height of the XR environment respectively.

[0008] Preferably, the gesture analysis unit synchronously processes the brush force and brush position results, and detects different levels of virtual pressure grading, and the specific calculation method is as follows: ; in, Indicates the current brush strength. Indicates the initial brush strength, represents the position weight factor, Indicates virtual pressure classification; The position weight factor is calculated as follows: ; in, Indicates the current brush position. represents the initial brush position, Indicates the maximum brush position. Substituting into the virtual pressure classification calculation formula, we get: ; .

[0009] Preferably, the gesture analysis unit further comprises a virtual pressure grading unit for detecting different levels of virtual pressure grading. The scope of this application.

[0010] Preferably, the basic virtual pressure corresponds to activating menu items and selecting options, the light touch virtual pressure corresponds to scaling content, the medium virtual pressure corresponds to magnifying content, and the heavy pressure virtual pressure corresponds to quickly selecting multiple options. When the user applies basic virtual pressure, the system activates menu items or makes option selections. This is the most basic way for users to interact with the XR environment. Light touch virtual pressure is usually used to scale content. Users can adjust the viewing angle or object size through slight changes in pressure to achieve delicate operations. Medium virtual pressure is used to magnify content, allowing users to view objects or scenes in more detail and enhance the exploration experience. Heavy pressure allows users to quickly select multiple options and is suitable for batch operations. Different virtual pressure levels correspond to different functions. Under the same virtual pressure level, different sliding directions (or different operations) can also correspond to different functions, thereby greatly expanding the interactive content.

[0011] Preferably, the virtual signal converter includes a gesture instruction parsing unit for parsing interaction instructions and converting them into signals recognizable by the XR environment. For example, if the gesture instruction is "click", the parsing unit converts it into a "click" signal recognizable by the XR environment; if the gesture instruction is "zoom", the parsing unit converts it into a "zoom" signal recognizable by the XR environment.

[0012] Preferably, the transmission interface includes a data transmission unit for transmitting the signal output by the signal converter to the XR environment and realizing data interaction with the XR environment. For example, the data transmission unit transmits the "click" signal output by the gesture instruction parsing unit to the XR environment to realize a click operation on a menu item or option in the XR environment.

[0013] Preferably, the virtual signal converter also includes a signal formatting unit, which is used to format the signal according to the requirements of the XR environment. For example, the signal formatting unit formats the "click" signal output by the gesture instruction parsing unit into a "click" signal format recognizable by the XR environment, so as to facilitate processing and response by the XR environment.

[0014] Preferably, when the force brush presses the virtual interface, the force brush appears to bend, and the fingertip is , a ray is emitted from the finger tip, and the intersection of the ray and the virtual interface is , the brush tip is ,To ensure that the brush length remains unchanged, the binary search algorithm is used to determine Position, and finally determine the Bezier curve formula: Brush length , , initialize the binary search auxiliary point: , , . The brush length is calculated from this , compared with The size of , thereby updating the binary search auxiliary point.

[0015] Beneficial Effects The present invention provides an XR environment gesture recognition system based on 3D virtual touch. Compared with the prior art, it has the following beneficial effects: In the present invention, the virtual pressure distribution of the pressing force is captured by the force brush, and visual prompts are provided to let the user have tactile feedback. The brush force and position are synchronously processed by the gesture analysis unit, and different levels of virtual pressure are effectively detected, so as to realize the accurate recognition of the interaction intention conveyed by the gesture. It not only significantly improves the naturalness and intuitiveness of the interaction, but also enables the user to interact with the XR environment with natural gestures by applying different levels of virtual pressure, thereby enhancing the user's immersion and experience, and improves the accuracy and response speed of the interaction, ensuring that the system provides corresponding feedback in real time according to the virtual pressure level, so that the user can obtain better perception and control during the interaction process. By detecting different levels of virtual pressure, the system not only adapts to various usage scenarios and user needs, and enhances the adaptability of the system, but also provides a flexible, scalable and personalized interaction method as a whole, greatly enhancing the user experience, and making the interaction in the XR environment richer, more natural and more efficient. The force brush uses the relative distance between the finger and the virtual interface as the input of the third dimension to achieve deep interaction. By adjusting this distance variable, the force brush can convey a variety of interaction instructions, thereby expanding the diversity of gesture interaction. With the help of the force brush, users can not only experience the convenience similar to 3D touch in the extended reality environment, but also achieve deep interaction in three-dimensional space that exceeds traditional two-dimensional touch screens, making up for the lack of tactile feedback in traditional three-dimensional bare-hand interaction, and realizing the simulation of pressure in the XR environment, and realizing various functions through virtual pressure level interaction and feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a system framework diagram of an XR environment press recognition system based on 3D virtual touch proposed by the present invention; Figure 2 A schematic diagram of the bending effect of a force brush in an XR environment pressing recognition system based on 3D virtual touch proposed by the present invention; Figure 3 A virtual pressure level diagram of a force brush in an XR environment press recognition system based on 3D virtual touch proposed by the present invention; Figure 4 This is a brush force effect diagram of an XR environment pressing recognition system based on 3D virtual touch proposed by the present invention; Figure 5 A schematic diagram of a force brush operation in a scene of an XR environment press recognition system based on 3D virtual touch proposed by the present invention; Figure 6 A schematic diagram of force brush operation in scene 2 of an XR environment press recognition system based on 3D virtual touch proposed by the present invention; Figure 7 This is a graph showing the error rate of an XR environment press recognition system based on 3D virtual touch proposed by the present invention; Figure 8 This is a graph showing the change in reaction time of a 3D virtual touch-based XR environment press recognition system proposed in the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1-8 The present invention provides two technical solutions, which specifically include the following embodiments: Embodiment 1: A 3D virtual touch-based XR environment gesture recognition system, comprising: Force brush module: Force brush uses the distance between the finger and the virtual interface as the third dimension input, and realizes the depth interaction input module between the user and the main screen through distance, which is used to receive the simulated pressure signal applied by the user in the XR environment; A processing module, used to recognize the user's gesture according to the pressure signal received by the input module, the processing module performs real-time analysis on the pressure signal transmitted by the input module, including time domain and frequency domain feature extraction of the signal. Before performing time domain and frequency domain feature extraction, the processing module first pre-processes the signal, including denoising, filtering and normalization, to eliminate the influence of sensor deviation on gesture recognition; The processing module includes a force recognition unit, a position recognition unit and a gesture analysis unit; The force recognition unit is used to recognize the force of the user's gesture according to the pressure data detected by the force sensor. The force recognition unit includes the following formula for calculating the brush force: ; in is the brush strength at time t, is the simulation pressure weight coefficient, is the total length of the force brush, for The distance between the top of the moment force brush and the contact surface; The position recognition unit is used to recognize the position of the user's gesture according to the touch position data detected by the position sensor. The position recognition unit includes the following formula for calculating the brush position: ; in, Indicates at time The brush position coordinates, and Respectively at time of Axis and The velocity component on the axis, and are the width and height of the XR environment respectively; The gesture analysis unit is used to synchronously process the brush force and brush position results, generate corresponding gesture instructions according to the force and position of the gesture, detect different levels of virtual pressure grading, and convey the corresponding interaction intention by applying different levels of pressure to identify different gestures. The gesture analysis unit synchronously processes the brush force and brush position results, detects different levels of pressure sensitivity, and the specific calculation method of pressure sensitivity is as follows: ; in, Indicates the current brush strength. Indicates the initial brush strength, represents the position weight factor, Indicates virtual pressure classification; The position weight factor is calculated as follows: ; in, Indicates the current brush position. represents the initial brush position, Indicates the maximum brush position. Substituting into the virtual pressure classification calculation formula, we get: ; .

[0019] An output module for applying the gestures recognized by the processing module to the interaction in the XR environment; The output module includes a signal converter and a transmission interface; The virtual signal converter is used to convert the gesture instructions generated by the processing module into signals recognizable by the XR environment. The virtual signal converter first parses the received gesture instructions to understand the specific meaning of the instructions. According to the meaning of the gesture instructions, the virtual signal converter maps them to the corresponding operation signals in the XR environment, and converts the mapped signals into formats recognizable by the XR device, such as digital signals, analog signals, etc. After completing the mapping of gesture instructions to operation signals, the work of the virtual signal converter is not over. It also needs to convert the mapped signals into a format recognizable by the XR device. Digital signals are widely used in XR devices due to their high precision and strong anti-interference capabilities, while analog signals are more applicable in some cases due to their continuity. The virtual signal converter will select the most appropriate signal format for conversion based on the hardware characteristics and software requirements of the XR device; The transmission interface is used to pass the signal output by the virtual signal converter to the XR environment, and transmit the signal to the XR device through a data cable, such as USB, HDMI and other interfaces. It can also use wireless technologies such as Bluetooth and Wi-Fi to transmit the signal to the XR device. Using the USB (Universal Serial Bus) interface for transmission can provide high-speed data transmission rate and stable connection. HDMI (High-Definition Multimedia Interface) is often used to transmit video and audio signals, ensuring high-definition picture quality and smooth sound experience. In addition to wired transmission, the transmission interface also supports wireless technologies such as Bluetooth and Wi-Fi. Bluetooth is suitable for scenarios with short transmission distance and small data volume, such as the connection between game controllers and head-mounted display devices. Wi-Fi technology provides higher transmission speeds and longer transmission distances, and is suitable for XR applications that require high bandwidth, such as panoramic video streaming.

[0020] By capturing the virtual pressure distribution of pressing intensity through the force brush and using the gesture analysis unit to synchronously process the brush force and position, different levels of virtual pressure can be effectively detected, thus achieving accurate recognition of the interaction intention conveyed by gestures. This not only significantly improves the naturalness and intuitiveness of the interaction, but also enables users to interact with the XR environment with natural gestures by applying different levels of virtual pressure, thereby enhancing the user's immersion and experience.

[0021] Embodiment 2: Based on the first embodiment, the gesture analysis unit further includes a virtual pressure grading unit for detecting different levels of virtual pressure grading. The basic virtual pressure corresponds to activating menu items and selecting options, lightly touching the virtual pressure corresponds to zooming the content, medium virtual pressure corresponds to magnifying the content, and heavy virtual pressure corresponds to quickly selecting multiple options. The virtual signal converter includes a gesture instruction parsing unit for parsing the interaction instruction and converting it into a signal recognizable by the XR environment. For example, if the gesture instruction is "click", the parsing unit converts it into a "click" signal recognizable by the XR environment; if the gesture instruction is "zoom", the parsing unit converts it into a "zoom" signal recognizable by the XR environment. The transmission interface includes a data transmission unit for transmitting the virtual signal converter to the XR environment. The signal output by the gesture command parsing unit is transmitted to the XR environment, and data interaction with the XR environment is realized. For example, the data transmission unit transmits the "click" signal output by the gesture command parsing unit to the XR environment, so as to realize the click operation on the menu item or option in the XR environment. The virtual signal converter also includes a signal formatting unit for formatting the signal according to the requirements of the XR environment. For example, the signal formatting unit formats the "click" signal output by the gesture command parsing unit into a "click" signal format recognizable by the XR environment, so as to facilitate the XR environment to process and respond. When the force brush presses the virtual interface, the force brush has a bending effect, and the fingertip is , a ray is emitted from the finger tip, and the intersection of the ray and the virtual interface is , the brush tip is ,To ensure that the brush length remains unchanged, the binary search algorithm is used to determine Position, and finally determine the Bezier curve formula: Brush length , , initialize the binary search auxiliary point: , , . The brush length is calculated from this , compared with The size of the force brush is used to update the binary search auxiliary point. The force brush is composed of 50 small cylinders connected end to end. When the force brush is pressed on the virtual interface, it produces bending and diverging effects at the same time. The bending effect is achieved by rotating each small cylinder according to the tangent of the Bezier curve position, and the diverging effect is achieved by scaling the axial size of each small cylinder. The points on the force brush With from The distance of the emitted ray is , let the axis perpendicular to the tangent line at this point and parallel to the virtual plane be X, then the size of the cylinder at this point is scaled to: The force brush indicates the current pressing depth and the corresponding operation according to the progress bar. When the pressing depth of the brush is less than the threshold t, it corresponds to one pressing force. When the pressing depth of the brush is greater than t, it corresponds to another pressing force. According to experimental research, when t=0.9, the accuracy and time of the operation can be well balanced. The proportion of the blue part in the progress bar changes: , 3D touch adds pressure as the third dimension to the existing input x and y position information, and uses pressure as the input method for deep interaction. Similarly, the force brush uses the distance between the finger and the virtual interface as the third dimension input to achieve deep interaction through distance. Therefore, the force brush can realize diversified interaction intentions through the changes in the third dimension input and enrich gesture interaction. The three interaction methods of the force brush are: light press and heavy press. Touching the interaction plane and detaching before the progress bar is full is a light press. Lightly pressing the application icon can open the application interface. Touching the interaction plane and detaching after the progress bar is full is a heavy press. Heavy pressing the application icon can call out the application shortcut for subsequent operations; calling out the hidden page: after touching the interaction plane, sliding after the progress bar is full is a heavy slide. On the interaction plane, swiping down on the left half calls out the notification page, swiping down on the right half calls out the control center page, and swiping up at any position calls out the desktop page; switching applications: first, swipe left or right to call out the application background list. Tap the interaction plane and the background list starts to slide. If the background list is called up by swiping hard to the left, the list slides to the left; if it is swiped hard to the right, the list slides to the right. When the application you want to switch to reaches the center of the plane and leaves the plane, switch to that application. The force brush uses the relative distance between the finger and the virtual interface as the input of the third dimension to achieve deep interaction. By adjusting this distance variable, the force brush can convey a variety of interaction instructions, thereby expanding the diversity of gesture interaction. With the help of the force brush, users can not only experience the convenience similar to 3D touch in the extended reality environment, but also achieve deep interaction in three-dimensional space that exceeds traditional two-dimensional touch screens, making up for the lack of tactile feedback in traditional three-dimensional bare-hand interaction, and realizing that the XR environment can simulate pressure and realize various functions through virtual pressure level interaction and feedback.

[0022] In order to better understand the technical content, the following application scenarios are given: Scenario 1: Calling out the hidden page To call out the hidden page on a mobile phone, you need to swipe from the border of the screen (such as the upper border) to the middle. In the large-screen interaction of XR, if the interaction form of the existing technology is used, the hand needs to reach a very high position to call out the hidden page, which is very inconvenient. To this end, the extra third dimension of depth in the three-dimensional space is used to assist this process. First, press the force brush at any position on the interactive plane in XR (usually this position is located in the lower middle position, the arm is slightly raised, and it will not be too tiring). Press until the color progress bar on the brush is full, that is, the pressing depth reaches a certain threshold. At this time, swipe down to pull out the hidden page hidden above, just like the operation of a mobile phone, but it is actually completed in the virtual world. After the page is pulled out, the force brush is pressed until the progress bar is full, and then it can also be swiped up to call out the desktop.

[0023] Scenario 2: Switching between background apps When we want to switch background apps in the XR environment, press any position with the force brush until the progress bar is full, and slide it to the left and right. At this time, the background app page will be called out. Press the blue circle with the force brush, and the app list will start to slide. The force brush will leave the interface, and the app list will stop sliding. Use the force brush to click on the app you want to switch to complete the app switching.

[0024] In order to better reflect the beneficial effects of the force brush in actual use, we conducted relevant data experiments, the content is as follows: users perform menu selection tasks under the force brush technology, and different virtual pressure levels represent different menus in the task. Figure 7 and Figure 8 Represents the user's learning curve for velocity brush technique. Figure 7 Indicates the change of error rate with the number of days of use. Figure 8 These two learning curves show that the force brush users can learn to use this technique in a short time and maintain a high level of proficiency in this technique for a long time, indicating that this technique is easy to learn and use.

[0025] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the application should be included in the protection scope of the present application.

Claims

1. An XR environment gesture recognition system based on 3D virtual touch, characterized in that: include: Force brush module: Force brush uses the distance between the finger and the virtual interface as the third dimension input, and realizes the depth interaction input module between the user and the main screen through distance, which is used to receive the simulated pressure signal applied by the user in the XR environment; A processing module, used for recognizing a user's gesture according to the simulated pressure signal received by the input module; The processing module includes a force recognition unit, a position recognition unit and a gesture analysis unit; The force recognition unit is used to recognize the force of the user's pressing according to the detected simulated pressure data; The position recognition unit is used to recognize the position of the user's gesture based on the touch position data detected by the position; The gesture analysis unit is used to synchronously process the brush force and brush position results, generate corresponding interaction instructions according to the force and position of the pressing, and recognize different gestures through different levels of simulated pressure to convey corresponding interaction intentions according to different levels of simulated pressure detection; An output module for applying the gestures recognized by the processing module to the interaction in the XR environment; The output module includes a virtual signal converter and a transmission interface; The virtual signal converter is used to convert the interaction instruction generated by the processing module into a signal recognizable by the XR environment; The transmission interface is used to transmit the signal output by the virtual signal converter to the XR environment.

2. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The force recognition unit includes the following formula for calculating the brush force: ； in is the brush strength at time t, is the simulation pressure weight coefficient, is the total length of the force brush, for The distance between the top of the brush and the contact surface.

3. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The position recognition unit includes the following formula for calculating the brush position: ； in, Indicates at time The brush position coordinates, and Respectively at time of Axis and The velocity component on the axis, and The width and height of the XR environment respectively.

4. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The gesture analysis unit synchronously processes the brush force and brush position results, and detects different levels of virtual pressure grading. The specific calculation method is as follows: ； in, Indicates the current brush strength. Indicates the initial brush strength, represents the position weight factor, Indicates virtual pressure classification; The position weight factor is calculated as follows: ； in, Indicates the current brush position. represents the initial brush position, Indicates the maximum brush position. Substituting into the virtual pressure classification calculation formula, we get: ； 。 5. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The gesture analysis unit also includes a virtual pressure grading unit for detecting different levels of virtual pressure grading The scope of this application.

6. The XR environment gesture recognition system based on 3D virtual touch according to claim 4, characterized in that: The basic virtual pressure corresponds to activating menu items and selecting options, the light touch virtual pressure corresponds to zooming content, the medium virtual pressure corresponds to magnifying content, and the heavy press virtual pressure corresponds to quickly selecting multiple options.

7. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The virtual signal converter includes a gesture instruction parsing unit, which is used to parse the interaction instructions and convert them into signals recognizable by the XR environment.

8. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The transmission interface includes a data transmission unit, which is used to transmit the signal output by the virtual signal converter to the XR environment and realize data interaction with the XR environment.

9. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: The virtual signal converter further includes a signal formatting unit for formatting the signal according to the requirements of the XR environment.

10. The XR environment gesture recognition system based on 3D virtual touch according to claim 1, characterized in that: When the force brush presses the virtual interface, the force brush appears to bend, and the fingertip is , a ray is emitted from the finger tip, and the intersection of the ray and the virtual interface is , the brush tip is ,To ensure that the brush length remains unchanged, the binary search algorithm is used to determine Position, and finally determine the Bezier curve formula: Brush length , , initialize the binary search auxiliary point: , , ; Calculate the brush length from this , compared with , thereby updating the binary search auxiliary point.

Images