Interaction control method and system based on virtual reality

Abstract

The invention discloses an interaction control method and system based on virtual reality, and relates to the technical field of monitoring analysis, and the system comprises a multi-mode motion capture module which is used for capturing limb space coordinate data, joint rotation angle data and muscle activation signals corresponding to a target user; the action intention analysis module comprises a time sequence convolutional neural network model, the time sequence convolutional neural network model comprises an input layer and an output layer, the input layer is used for receiving the skeleton joint point sequence, the angular velocity data and the myoelectricity feature vector of the multi-modal action capture module, and the output layer is used for extracting spatial and temporal features through three-layer expansion causal convolution. Connecting a classifier to generate initial action data corresponding to the target user; the correction module is used for constructing a human body joint angle constraint library, judging the initial action data corresponding to the target user and outputting a correction instruction; and the environment adaptation module is used for adjusting the environment in the virtual scene. The method has the effect of improving the interaction efficiency.

Description

Technical Field

[0001] The present application relates to the field of monitoring and analysis technology, and in particular to an interactive control method and system based on virtual reality. Background Art

[0002] Virtual reality technology is an advanced, digital human-computer interface technology. Its characteristic is that the computer generates an artificial virtual environment, creating an artificial environment with visual perception as the main feature, including comprehensive perception of hearing and touch. People can perceive the virtual world created by the computer through multiple sensory channels such as vision, hearing, touch and acceleration, and can also interact with the virtual world in the most natural ways such as movement, voice, and action, thus creating an immersive experience.

[0003] In the related art, in the existing virtual reality interactive control system, the target user's actions are often not recognized successfully, which leads to a decrease in the interaction efficiency of the target user. There is room for improvement. Summary of the Invention

[0004] In response to the deficiencies of the existing technology, the present application provides an interactive control method and system based on virtual reality.

[0005] In a first aspect, the present application provides an interactive control system based on virtual reality, comprising: A multimodal motion capture module is used to capture the target user's corresponding limb spatial coordinate data, joint rotation angle data, and muscle activation signals through a monitoring device; An action intention parsing module includes a temporal convolutional neural network model and a dynamic weighted fusion unit. The temporal convolutional neural network model includes an input layer and an output layer. The input layer is used to receive the skeletal joint point sequence, angular velocity data, and myoelectric feature vectors from the multimodal motion capture module. The output layer is used to extract spatiotemporal features through a three-layer dilated causal convolution and connect to a Softmax classifier to generate initial action data corresponding to the target user. The dynamic weighted fusion unit is used to adjust the fusion weight of the data from each monitoring device in real time according to the environmental complexity parameter. The correction module is used to build a human joint angle constraint library, judge the initial motion data corresponding to the target user, and output correction instructions based on the judgment results; A multi-channel feedback module comprising a visually guided projection unit, a tactile vibration array, and a bone conduction audio unit. The visually guided projection unit is used to generate a semi-transparent reference motion trajectory in a virtual scene. The tactile vibration array is used to arrange linear resonant actuators on the target user and generate differentiated vibration patterns based on motion error vectors. The bone conduction audio unit is used to select feedback content based on a preset voice command library and error type. Preferably, the multimodal motion capture module includes a binocular infrared depth camera array, an inertial measurement unit sensor group and a surface electromyography sensor; The binocular infrared depth camera array is used to collect a depth image of the virtual scene, extract the contour data corresponding to the target user based on the depth image recognition, and use the joint point detection algorithm to locate the key joint points corresponding to the target user, confirm the joint point coordinates corresponding to the key joint points, and convert the detected joint point coordinates into the world coordinate system to generate a skeletal joint point sequence; The inertial measurement unit sensor group is used to obtain real-time posture data corresponding to each joint of the target user, and extract angular velocity data from the real-time posture data; The surface electromyography sensor is used to collect muscle activation signals, perform feature extraction, and normalize the extracted features to determine the electromyography feature vector.

[0006] Preferably, the process of adjusting the fusion weight of the data of each monitoring device in real time according to the environmental complexity parameter specifically includes: By formula , confirm the complexity of the environment ; in, They are respectively represented as the density of virtual objects and the variance of illumination changes in the virtual scene. Represents the movement speed corresponding to the target user, Expressed as a preset weight coefficient; The complexity of the environment Compare with the preset environment complexity threshold range; If the environment complexity in When the weight distribution is between , the weight distribution is performed according to the preset first distribution strategy; If the environment complexity in When the weight distribution is between , the weight distribution is performed according to the preset second distribution strategy; If the environment complexity , the weight distribution is performed according to the preset third distribution strategy.

[0007] Preferably, the correction module specifically includes: Within a preset time period, a human joint angle constraint library is established, wherein the human joint angle constraint library stores data on the range of motion of each joint and sets kinematic chain verification rules; Defining action recognition confidence , ,in, Indicates the number corresponding to each sensor, They are respectively represented as the weight and recognition probability corresponding to each sensor; Setting dynamic thresholds ,in, Represented as the preset initial confidence, Expressed as a preset correction factor; When the recognition confidence , then the action is determined to be a valid action.

[0008] Preferably, the correction module further includes: If the initial action recognition fails, the state of the virtual object remains unchanged, and prompts and guidance are provided through the multi-channel feedback module; If the second action recognition fails, the joint angle constraints are automatically relaxed; If the third action recognition fails, the environment adaptation module is triggered to adjust the difficulty of the environment in the virtual scene.

[0009] Preferably, the environment adaptation module includes a parameter dynamic adjustment unit and a load evaluation unit; The parameter dynamic adjustment unit is used to modify the mass and friction coefficient of the virtual object in the virtual scene in real time; The load evaluation unit is used to determine a load evaluation index through gaze point data and heart rate data.

[0010] Preferably, the process of modifying the mass and friction coefficient of a virtual object in a virtual scene specifically includes: Construct the mass and friction coefficient parameter adjustment formula of the virtual object in the virtual scene: The quality of the virtual object: ; The coefficient of friction of the virtual object: ; in, It is expressed as the number of consecutive action recognition failures. Expressed as preset standard mass and friction coefficient; And when When the emergency assist mode is activated, the emergency assist mode is used to automatically complete the remaining movement range.

[0011] Preferably, the process of determining the load assessment index through gaze point data and heart rate data specifically includes: Within a preset time window, the gaze point data and heart rate data corresponding to the target user are collected through a preset device; Preprocess the gaze point data and heart rate data corresponding to the target user, and calculate the gaze point dispersion corresponding to the target user through the preprocessed gaze point data and heart rate data and heart rate variability index ; By formula , confirm the load assessment index corresponding to the target user ,in, They are respectively expressed as the weight coefficients corresponding to the gaze point dispersion and heart rate variability indicators; The load assessment index corresponding to the target user With the preset load assessment threshold Make a comparison; If the load assessment index corresponding to the target user , the scene elements in the virtual scene are dynamically simplified.

[0012] In a second aspect, the present application provides a virtual reality-based interactive control method, comprising the following steps: Capturing the target user's corresponding limb space coordinate data, joint rotation angle data, and muscle activation signals through a monitoring device; Motion intention parsing, including a temporal convolutional neural network model and a dynamic weighted fusion unit. The temporal convolutional neural network model includes an input layer and an output layer. The input layer is used to receive the skeletal joint point sequence, angular velocity data, and electromyographic feature vector of the multimodal motion capture module. The output layer is used to extract spatiotemporal features through a three-layer dilated causal convolution and connect the Softmax classifier to generate initial motion data corresponding to the target user. The dynamic weighted fusion unit is used to adjust the fusion weight of the data of each monitoring device in real time according to the environmental complexity parameter; Build a human joint angle constraint library, judge the initial motion data corresponding to the target user, and output correction instructions based on the judgment results; Multi-channel feedback, including a visually guided projection unit, a tactile vibration array, and a bone conduction audio unit. The visually guided projection unit is used to generate a semi-transparent reference motion trajectory in a virtual scene. The tactile vibration array is used to arrange linear resonant actuators on the target user and generate differentiated vibration patterns based on motion error vectors. The bone conduction audio unit is used to select feedback content based on a preset voice command library and error type. Adjust the environment in the virtual scene.

[0013] In a third aspect, the present application provides a computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute any one of the above-mentioned virtual reality-based interactive control systems.

[0014] In summary, this application includes at least one of the following beneficial technical effects: 1. This application provides an interactive control system based on virtual reality. Through the fusion of multimodal data and a dynamic weighting mechanism, the accuracy and environmental adaptability of motion capture are significantly improved. A temporal convolutional neural network is used to extract spatiotemporal features and combined with a human joint constraint library to achieve the dual guarantee of accurate analysis of motion intention and physiological rationality. Multi-channel feedback constructs a three-dimensional correction system, enhancing the naturalness and guidance efficiency of human-computer interaction. The environmental adaptation module ensures the dynamic matching of virtual scenes with real training needs, which can not only ensure the standardization of movements, but also improve the user's training compliance and movement learning effect through a highly immersive experience, thereby effectively reducing the occurrence of unsuccessful recognition due to non-standard movements of the target user, thereby effectively improving the interaction efficiency of the target user. 2. By evaluating the load assessment index of the target user in real time, it can accurately capture the user's physiological and behavioral status in the virtual reality environment. When the load assessment index exceeds the preset threshold, the scene elements are automatically simplified, thereby effectively reducing the cognitive burden of the target user. The dynamic adaptive optimization not only enhances the target user's immersion and comfort, but also significantly improves the interaction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 It is a system diagram of interactive control based on virtual reality according to an embodiment of the present application.

[0017] Figure 2 This is a flow chart of a method for interactive control based on virtual reality according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following is combined with Figure 1-2 This application is described in further detail.

[0019] Example 1 The embodiment of the present application discloses an interactive control system based on virtual reality.

[0020] Reference Figure 1 , an interactive control system based on virtual reality, comprising: A multimodal motion capture module is used to capture the target user's corresponding limb spatial coordinate data, joint rotation angle data, and muscle activation signals through a monitoring device; An action intention parsing module includes a temporal convolutional neural network model and a dynamic weighted fusion unit. The temporal convolutional neural network model includes an input layer and an output layer. The input layer is used to receive the skeletal joint point sequence, angular velocity data, and myoelectric feature vectors from the multimodal motion capture module. The output layer is used to extract spatiotemporal features through a three-layer dilated causal convolution and connect to a Softmax classifier to generate initial action data corresponding to the target user. The dynamic weighted fusion unit is used to adjust the fusion weight of the data from each monitoring device in real time according to the environmental complexity parameter. The correction module is used to build a human joint angle constraint library, judge the initial motion data corresponding to the target user, and output correction instructions based on the judgment results; A multi-channel feedback module comprising a visually guided projection unit, a tactile vibration array, and a bone conduction audio unit. The visually guided projection unit is used to generate a semi-transparent reference motion trajectory in a virtual scene. The tactile vibration array is used to arrange linear resonant actuators on the target user and generate differentiated vibration patterns based on motion error vectors. The bone conduction audio unit is used to select feedback content based on a preset voice command library and error type. The environment adaptation module is used to adjust the environment in the virtual scene.

[0021] Furthermore, the multimodal motion capture module includes a binocular infrared depth camera array, an inertial measurement unit sensor group and a surface electromyography sensor; The binocular infrared depth camera array is used to collect a depth image of the virtual scene, extract the contour data corresponding to the target user based on the depth image recognition, and use the joint point detection algorithm to locate the key joint points corresponding to the target user, confirm the joint point coordinates corresponding to the key joint points, and convert the detected joint point coordinates into the world coordinate system to generate a skeletal joint point sequence; The inertial measurement unit sensor group is used to obtain real-time posture data corresponding to each joint of the target user, and extract angular velocity data from the real-time posture data; The surface electromyography sensor is used to collect muscle activation signals, perform feature extraction, and normalize the extracted features to determine the electromyography feature vector.

[0022] Specifically, when collecting the above data, multi-sensor time synchronization must be achieved to ensure data timestamp alignment, and a unified coordinate system must be established. The skeletal joint point sequence, angular velocity data, and electromyographic feature vector corresponding to the target user must be input into the temporal convolutional neural network model for multimodal feature fusion, thereby confirming the initial motion data after fusion.

[0023] Through the above technical solution, it is possible to accurately obtain skeletal joint point sequences, angular velocity data and myoelectric feature vectors, providing a reliable data basis for subsequent motion recognition and interactive control.

[0024] It should be noted that the process of adjusting the fusion weight of the data of each monitoring device in real time according to the environmental complexity parameter specifically includes: By formula , confirm the complexity of the environment ; in, They are respectively represented as the density of virtual objects and the variance of illumination changes in the virtual scene. Represents the movement speed corresponding to the target user, Expressed as a preset weight coefficient; The complexity of the environment Compare with the preset environment complexity threshold range; If the environment complexity in When the weight distribution is between , the weight distribution is performed according to the preset first distribution strategy; If the environment complexity in When the weight distribution is between , the weight distribution is performed according to the preset second distribution strategy; If the environment complexity , the weight distribution is performed according to the preset third distribution strategy.

[0025] Specifically, different allocation strategies result in different weight coefficients corresponding to the data collected by the binocular infrared depth camera array, inertial measurement unit sensor group and surface electromyography sensor, thereby providing a reliable data basis for the subsequent calculation of recognition confidence.

[0026] It should be noted that the correction module specifically includes: Within a preset time period, a human joint angle constraint library is established, wherein the human joint angle constraint library stores data on the range of motion of each joint and sets kinematic chain verification rules; For example, the range of shoulder abduction range is 0-180°, and the range of knee flexion range is 0-135°. The kinematic chain verification rule for the grasping action is: [shoulder flexion > 30° → elbow flexion > 90° → wrist dorsiflexion > 20°], and an error of 15 degrees is allowed in the intermediate links. Defining action recognition confidence , ,in, Indicates the number corresponding to each sensor, They are respectively represented as the weight and recognition probability corresponding to each sensor, where the recognition probability can be obtained by fitting historical data; Setting dynamic thresholds ,in, Represented as the preset initial confidence, Expressed as a preset correction factor; When the recognition confidence , then the action is determined to be a valid action.

[0027] Furthermore, the correction module specifically includes: If the initial action recognition fails, the state of the virtual object remains unchanged, and prompts and guidance are provided through the multi-channel feedback module; If the second action recognition fails, the joint angle constraints are automatically relaxed; If the third action recognition fails, the environment adaptation module is triggered to adjust the difficulty of the environment in the virtual scene.

[0028] It should be noted that the environment adaptation module includes a parameter dynamic adjustment unit and a load evaluation unit; The parameter dynamic adjustment unit is used to modify the mass and friction coefficient of the virtual object in the virtual scene in real time; The load evaluation unit is used to determine a load evaluation index through gaze point data and heart rate data.

[0029] Furthermore, the process of modifying the mass and friction coefficient of the virtual object in the virtual scene specifically includes: Construct the mass and friction coefficient parameter adjustment formula of the virtual object in the virtual scene: The quality of the virtual object: ; The coefficient of friction of the virtual object: ; in, It is expressed as the number of consecutive action recognition failures. Expressed as preset standard mass and friction coefficient; And when When the emergency assist mode is activated, the emergency assist mode is used to automatically complete the remaining movement range.

[0030] Specifically, dynamically adjusting the mass and friction coefficient of objects in a virtual scene can significantly enhance the realism and application value of simulations. First, by optimizing mass parameters, objects' motion inertia, collision feedback, and energy transfer can be more accurately simulated. For example, in game development, increasing the mass of a metal box will produce a stronger impact and displacement trajectory upon collision, while reducing the weight of a balloon will create a lighter floating effect. Second, adjusting the friction coefficient effectively controls the contact behavior between objects. For example, in a racing game, reducing the friction coefficient between tires and ice can realistically reproduce vehicle skidding, while increasing the friction of rubber can achieve an emergency stop. This parameter adjustment not only enhances the user experience but also has practical implications for industrial simulations. In robotic gripping operations, precise friction coefficient settings can predict the risk of workpiece slippage. In the construction industry, combining friction parameters for components made of different materials can simulate structural displacement during earthquakes. Furthermore, dynamic parameter optimization can balance computing resources, improving system efficiency by reducing the accuracy of physical calculations for non-critical objects. In general, the flexible adjustment of mass and friction coefficient is the core technical means to build a highly realistic virtual environment. It can not only meet the needs of artistic expression, but also provide a reliable digital twin platform for professional fields such as scientific research experiments and product testing.

[0031] Furthermore, the process of determining the load assessment index through gaze point data and heart rate data specifically includes: Within a preset time window, the gaze point data and heart rate data corresponding to the target user are collected through a preset device; Preprocess the gaze point data and heart rate data corresponding to the target user, and calculate the gaze point dispersion corresponding to the target user through the preprocessed gaze point data and heart rate data and heart rate variability index ; By formula , confirm the load assessment index corresponding to the target user ,in, They are respectively expressed as the weight coefficients corresponding to the gaze point dispersion and heart rate variability indicators; The load assessment index corresponding to the target user With the preset load assessment threshold Make a comparison; If the load assessment index corresponding to the target user , the scene elements in the virtual scene are dynamically simplified.

[0032] Specifically, by evaluating the user's load assessment index in real time, the user's physiological and behavioral state in the virtual reality environment can be accurately captured. When the load assessment index exceeds the preset threshold, the system automatically simplifies the scene elements, such as reducing the number of non-critical objects, reducing the complexity of light and shadow, or extending the task time limit, thereby effectively reducing the user's cognitive burden. Dynamic adaptive optimization not only enhances the user's immersion and comfort, but also significantly improves the efficiency of interaction. For example, in virtual driving training, when the user feels stressed by a complex scene, the system automatically reduces the number of background vehicles and simplifies the road texture, allowing the user to focus more on the core task and avoid misoperation or fatigue caused by information overload.

[0033] Example 2 The embodiment of the present application also discloses an interactive control method based on virtual reality.

[0034] Reference Figure 2 , an interactive control method based on virtual reality, comprising the following steps: Capturing the target user's corresponding limb space coordinate data, joint rotation angle data, and muscle activation signals through a monitoring device; Motion intention parsing, including a temporal convolutional neural network model and a dynamic weighted fusion unit. The temporal convolutional neural network model includes an input layer and an output layer. The input layer is used to receive the skeletal joint point sequence, angular velocity data, and electromyographic feature vector of the multimodal motion capture module. The output layer is used to extract spatiotemporal features through a three-layer dilated causal convolution and connect the Softmax classifier to generate initial motion data corresponding to the target user. The dynamic weighted fusion unit is used to adjust the fusion weight of the data of each monitoring device in real time according to the environmental complexity parameter; Build a human joint angle constraint library, judge the initial motion data corresponding to the target user, and output correction instructions based on the judgment results; Multi-channel feedback, including a visually guided projection unit, a tactile vibration array, and a bone conduction audio unit. The visually guided projection unit is used to generate a semi-transparent reference motion trajectory in a virtual scene. The tactile vibration array is used to arrange linear resonant actuators on the target user and generate differentiated vibration patterns based on motion error vectors. The bone conduction audio unit is used to select feedback content based on a preset voice command library and error type. Adjust the environment in the virtual scene.

[0035] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in similar ways. As long as they do not deviate from the concept of the invention, they should all fall within the scope of protection of the present invention.

[0036] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0037] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention.

Claims

1. An interactive control system based on virtual reality, characterized in that: include: A multimodal motion capture module is used to capture the target user's corresponding limb spatial coordinate data, joint rotation angle data, and muscle activation signals through a monitoring device; An action intention parsing module includes a temporal convolutional neural network model and a dynamic weighted fusion unit. The temporal convolutional neural network model includes an input layer and an output layer. The input layer is used to receive the skeletal joint point sequence, angular velocity data, and myoelectric feature vectors from the multimodal motion capture module. The output layer is used to extract spatiotemporal features through a three-layer dilated causal convolution and connect to a Softmax classifier to generate initial action data corresponding to the target user. The dynamic weighted fusion unit is used to adjust the fusion weight of the data from each monitoring device in real time according to the environmental complexity parameter. The correction module is used to build a human joint angle constraint library, judge the initial motion data corresponding to the target user, and output correction instructions based on the judgment results; A multi-channel feedback module comprising a visually guided projection unit, a tactile vibration array, and a bone conduction audio unit. The visually guided projection unit is used to generate a semi-transparent reference motion trajectory in a virtual scene. The tactile vibration array is used to arrange linear resonant actuators on the target user and generate differentiated vibration patterns based on motion error vectors. The bone conduction audio unit is used to select feedback content based on a preset voice command library and error type. The environment adaptation module is used to adjust the environment in the virtual scene.

2. The interactive control system based on virtual reality according to claim 1, characterized in that: The multimodal motion capture module includes a binocular infrared depth camera array, an inertial measurement unit sensor group and a surface electromyography sensor; The binocular infrared depth camera array is used to collect a depth image of the virtual scene, extract the contour data corresponding to the target user based on the depth image recognition, and use the joint point detection algorithm to locate the key joint points corresponding to the target user, confirm the joint point coordinates corresponding to the key joint points, and convert the detected joint point coordinates into the world coordinate system to generate a skeletal joint point sequence; The inertial measurement unit sensor group is used to obtain real-time posture data corresponding to each joint of the target user, and extract angular velocity data from the real-time posture data; The surface electromyography sensor is used to collect muscle activation signals, perform feature extraction, and normalize the extracted features to determine the electromyography feature vector.

3. The interactive control system based on virtual reality according to claim 1, characterized in that: The process of adjusting the fusion weight of each monitoring device data in real time according to the environmental complexity parameters specifically includes: By formula , confirm the complexity of the environment ; in, They are respectively represented as the density of virtual objects and the variance of illumination changes in the virtual scene. Represents the movement speed corresponding to the target user, Expressed as a preset weight coefficient; The complexity of the environment Compare with the preset environment complexity threshold range; If the environment complexity in When the weight distribution is between , the weight distribution is performed according to the preset first distribution strategy; If the environment complexity in When the weight distribution is between , the weight distribution is performed according to the preset second distribution strategy; If the environment complexity , the weight distribution is performed according to the preset third distribution strategy.

4. The interactive control system based on virtual reality according to claim 3, characterized in that: The correction module specifically includes: Within a preset time period, a human joint angle constraint library is established, wherein the human joint angle constraint library stores data on the range of motion of each joint and sets kinematic chain verification rules; Defining action recognition confidence , ,in, Indicates the number corresponding to each sensor, They are respectively represented as the weight and recognition probability corresponding to each sensor; Setting dynamic thresholds ,in, Represented as the preset initial confidence, Expressed as a preset correction factor; When the recognition confidence , then the action is determined to be a valid action.

5. The interactive control system based on virtual reality according to claim 4, characterized in that: The correction module specifically further includes: If the initial action recognition fails, the state of the virtual object remains unchanged, and prompts and guidance are provided through the multi-channel feedback module; If the second action recognition fails, the joint angle constraints are automatically relaxed; If the third action recognition fails, the environment adaptation module is triggered to adjust the difficulty of the environment in the virtual scene.

6. The interactive control system based on virtual reality according to claim 1, characterized in that: The environmental adaptation module includes a parameter dynamic adjustment unit and a load evaluation unit; The parameter dynamic adjustment unit is used to modify the mass and friction coefficient of the virtual object in the virtual scene in real time; The load evaluation unit is used to determine a load evaluation index through gaze point data and heart rate data.

7. The interactive control system based on virtual reality according to claim 6, characterized in that: The process of modifying the mass and friction coefficient of virtual objects in the virtual scene includes: Construct the mass and friction coefficient parameter adjustment formula of the virtual object in the virtual scene: The quality of the virtual object: ; The coefficient of friction of the virtual object: ; in, It is expressed as the number of consecutive action recognition failures. Expressed as preset standard mass and friction coefficient; And when When the emergency assist mode is activated, the emergency assist mode is used to automatically complete the remaining movement range.

8. The interactive control system based on virtual reality according to claim 6, characterized in that: The process of determining the load assessment index using gaze point data and heart rate data includes: Within a preset time window, the gaze point data and heart rate data corresponding to the target user are collected through a preset device; Preprocess the gaze point data and heart rate data corresponding to the target user, and calculate the gaze point dispersion corresponding to the target user through the preprocessed gaze point data and heart rate data and heart rate variability index ; By formula , confirm the load assessment index corresponding to the target user ,in, They are respectively expressed as the weight coefficients corresponding to the gaze point dispersion and heart rate variability indicators; The load assessment index corresponding to the target user With the preset load assessment threshold Make a comparison; If the load assessment index corresponding to the target user , the scene elements in the virtual scene are dynamically simplified.

9. A virtual reality-based interactive control method, applied to a virtual reality-based interactive control system according to any one of claims 1 to 8, characterized in that: The following steps are involved: Capturing the target user's corresponding limb space coordinate data, joint rotation angle data, and muscle activation signals through a monitoring device; Motion intention parsing, including a temporal convolutional neural network model and a dynamic weighted fusion unit. The temporal convolutional neural network model includes an input layer and an output layer. The input layer is used to receive the skeletal joint point sequence, angular velocity data, and electromyographic feature vector of the multimodal motion capture module. The output layer is used to extract spatiotemporal features through a three-layer dilated causal convolution and connect the Softmax classifier to generate initial motion data corresponding to the target user. The dynamic weighted fusion unit is used to adjust the fusion weight of the data of each monitoring device in real time according to the environmental complexity parameter; Build a human joint angle constraint library, judge the initial motion data corresponding to the target user, and output correction instructions based on the judgment results; Multi-channel feedback, including a visually guided projection unit, a tactile vibration array, and a bone conduction audio unit. The visually guided projection unit is used to generate a semi-transparent reference motion trajectory in a virtual scene. The tactile vibration array is used to arrange linear resonant actuators on the target user and generate differentiated vibration patterns based on motion error vectors. The bone conduction audio unit is used to select feedback content based on a preset voice command library and error type. Adjust the environment in the virtual scene.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are run on a computer, the computer is caused to execute the virtual reality-based interactive control system according to any one of claims 1 to 8.

Images