Virtual Reality Skill Operation Behavior Recognition and Intelligent Error Correction Methods and Systems

Abstract

This invention discloses a method and system for recognizing and intelligently correcting virtual reality skill operation behaviors, belonging to the field of virtual reality skill training technology. The method collects head posture, gaze point, hand position, finger joint angle, and tool trajectory data through multi-source sensors and performs spatiotemporal alignment. It decomposes the operation process into atomic action sequences and detects key control nodes using a temporal segmentation network. It generates standardized scores by comparing dynamic time warping algorithms with standard templates. It identifies error types through pattern matching using an error type knowledge base. It selects visual, tactile, or voice feedback channels according to the error type and severity. It dynamically adjusts the training difficulty and feedback strategy based on the ability profile.

Description

Technical Field

[0001] This invention relates to the field of virtual reality skills training technology, specifically to a method and system for recognizing and intelligently correcting virtual reality skills operation behaviors. Background Technology

[0002] Virtual reality technology, with its high degree of immersion and interactivity, has demonstrated significant advantages in the field of skills training. Taking substation switching operation training in the power industry as an example, traditional training methods face prominent problems such as unrealistic operating environments, incomplete skills assessments, and low training efficiency. While existing simulator training can provide some operational experience, the simulated environment differs from the actual environment, making it difficult to fully reproduce all the details of real operations. Paper-based textbooks focus on theoretical knowledge transmission, lacking practical operation components, making it difficult for trainees to combine theoretical knowledge with practical operation.

[0003] Chinese invention CN120089039A discloses a standardized 500kV switching operation method based on VR virtual reality technology. This method includes steps such as constructing a fully immersive virtual reality scene, designing multimodal interaction for standardized switching operation procedures, implementing dynamic action scoring and intelligent behavior evaluation, and real-time optimization and learning transfer of the virtual character model. The solution collects actual substation environmental data to construct a high-precision scene model, uses high-precision motion capture equipment to capture the user's switching operation actions in real time, compares the captured actions frame-by-frame with standard actions, calculates trajectory errors and time differences, generates an instant score, and provides dynamic feedback based on the score results.

[0004] However, the technical solution of the above invention has the following shortcomings. First, in terms of behavioral data acquisition, the solution mainly relies on a single motion capture device and fails to fully integrate data from multiple input devices such as VR headset built-in sensors, controller six-DOF tracking, eye trackers, and finger-tracking gloves, resulting in an incomplete spatiotemporal description of the operation behavior. Second, in terms of operation evaluation, the solution uses frame-by-frame comparison to generate real-time scores, but fails to automatically decompose the continuous operation process into evaluable atomic action units, nor does it accurately locate key control nodes affecting operation quality, resulting in a coarse evaluation granularity. Third, in terms of error feedback, although the solution provides error prompts and optimization suggestions, it lacks a systematic knowledge base of operation error types, and cannot automatically classify detected operation deviations into specific error types and associate them with targeted correction strategies, thus the specificity of the feedback needs to be improved. In addition, in terms of feedback methods, the solution fails to intelligently select visual, tactile, and voice feedback channels and timing based on error type and severity, which may lead to feedback overload and interfere with the immersive learning experience. Finally, regarding personalized training, although the solution collects user operation data to generate optimization suggestions, it does not establish a dynamic capability profile of the operator, making it difficult to achieve personalized adaptive adjustments to training difficulty, feedback strategies, and reinforcement priorities.

[0005] In summary, existing virtual reality skills training technologies still face technical bottlenecks in areas such as multimodal behavioral data fusion, fine-grained operational assessment, intelligent error attribution, adaptive error correction feedback, and personalized training adjustments. There is an urgent need for a new method that can comprehensively improve the effectiveness of skills training. Summary of the Invention

[0006] To address the technical problems of existing virtual reality skills training technologies, such as coarse-grained assessment of operational behavior, lack of targeted error feedback, and insufficient personalized guidance, this invention provides a method and system for identifying and intelligently correcting virtual reality skills operational behaviors.

[0007] The technical solution provided by this invention includes the following steps.

[0008] The first step involves synchronously collecting head posture data, gaze point data, hand spatial position data, finger joint angle data, and tool trajectory data of the operator during skill training through multi-source sensors of the virtual reality device. The collected multi-source data is then processed with timestamp synchronization to eliminate sampling delay differences between sensors, and all source data are unified into a preset virtual world coordinate system to form spatiotemporally aligned data.

[0009] The second step involves using a temporal segmentation network based on spatiotemporal aligned data to decompose the continuous operation process into a sequence of atomic actions, detect the start and end time boundaries of each atomic action, and identify the set of key control nodes that affect the operation quality in the operation process. The set of key control nodes includes tool grip posture switching points, contact points of the operation object, and the start and end points of the fine operation range.

[0010] The third step involves retrieving the standard operating behavior template corresponding to the current training task from the pre-built standard operating behavior template library, and using a dynamic time warping algorithm to perform temporal alignment and comparison between the actual trajectory of each atomic action in the atomic action sequence and the corresponding standard trajectory in the standard operating behavior template. Quantitative indicators of trajectory deviation, velocity deviation, and attitude deviation are calculated, and multi-dimensional deviation indicators are fused to generate operation standardization score data.

[0011] The fourth step involves extracting action segments from the operational standardization scoring data whose deviations exceed a preset threshold as operational deviation feature vectors, performing pattern matching with a pre-established operational error type knowledge base, identifying the error type identifier to which the current operational deviation belongs, and associating it with the corresponding correction strategy.

[0012] The fifth step involves selecting the error correction feedback channel and timing based on the error type identifier and the severity level of the error, generating an error correction feedback instruction, and transmitting it to the operator through a visual feedback channel, tactile feedback channel, or voice feedback channel.

[0013] The sixth step involves updating the operator's competency profile data based on the operational standardization score data and error type distribution, tracking the operator's mastery of various atomic actions and recording common error patterns, dynamically adjusting the adaptive training parameters for subsequent training based on the operator's competency profile data, and generating a skills assessment report.

[0014] The present invention also provides a virtual reality skill operation behavior recognition and intelligent error correction system, including a multimodal behavior data acquisition module, an atomic action decomposition module, an operation standardization evaluation module, an error identification and attribution module, a multimodal error correction feedback module, and an adaptive training adjustment module.

[0015] The multimodal behavior data acquisition module is configured to synchronously collect head posture data, gaze point data, hand spatial position data, finger joint angle data, and tool trajectory data of the operator during skill training through multi-source sensors of the virtual reality device. The collected multi-source data is processed with timestamp synchronization to eliminate the sampling delay difference between the sensors, and the data from each source is unified into a preset virtual world coordinate system to form spatiotemporally aligned data.

[0016] The atomic action decomposition module is configured to decompose a continuous operation process into an atomic action sequence using a temporal segmentation network based on spatiotemporal aligned data, detect the start and end time boundaries of each atomic action, and identify the set of key control nodes that affect the operation quality in the operation process.

[0017] The operation standardization assessment module is configured to retrieve the standard operation behavior template corresponding to the current training task from the pre-built standard operation behavior template library, and use the dynamic time warping algorithm to perform temporal alignment and comparison between the actual trajectory of each atomic action in the atomic action sequence and the corresponding standard trajectory in the standard operation behavior template. It calculates the quantitative indicators of trajectory deviation, velocity deviation and attitude deviation, and integrates the multi-dimensional deviation indicators to generate operation standardization score data.

[0018] The error identification and attribution module is configured to extract action segments with deviations exceeding a preset threshold from the operation standardization score data as operation deviation feature vectors, perform pattern matching with a pre-established operation error type knowledge base, identify the error type identifier to which the current operation deviation belongs, and associate it with the corresponding correction strategy.

[0019] The multimodal error correction feedback module is configured to select the error correction feedback channel and feedback timing based on the error type identifier and error severity level, generate error correction feedback instructions, and transmit them to the operator through the visual feedback channel, tactile feedback channel, or voice feedback channel.

[0020] The adaptive training adjustment module is configured to update operator competency profile data based on operational standardization score data and error type distribution, track the operator's mastery of various atomic actions and record common error patterns, dynamically adjust the adaptive training parameters for subsequent training based on operator competency profile data, and generate a skills assessment report.

[0021] The beneficial effects of this invention are as follows: by fusing multi-source sensor data to construct a complete spatiotemporal description of operational behavior, it overcomes the information limitations of single trajectory tracking; by automatically decomposing continuous operation processes into atomic action units and accurately locating key control nodes, it achieves fine-grained operation quality assessment; by establishing a knowledge base of operation error types and automatically classifying operation deviations and associating them with correction strategies, it achieves an intelligent leap from error detection to error understanding; by intelligently selecting feedback channels and timing based on error type and severity, it balances immediate error correction with an immersive experience; and by establishing a dynamic capability profile of the operator, it achieves personalized adaptive adjustment of training parameters, improving skill acquisition efficiency. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of the virtual reality skill operation behavior recognition and intelligent error correction method of the present invention.

[0023] Figure 2This is a detailed flowchart of the multimodal behavioral data acquisition and spatiotemporal alignment processing of the present invention.

[0024] Figure 3 This is a detailed flowchart of the atomic action decomposition and key node detection of the present invention.

[0025] Figure 4 This is a detailed flowchart of the dynamic evaluation of the operational standardization of this invention.

[0026] Figure 5 This is a detailed flowchart of the error type identification and intelligent attribution of the present invention.

[0027] Figure 6 This is a detailed flowchart of the multimodal adaptive error correction feedback of the present invention.

[0028] Figure 7 This is a detailed flowchart of the personalized ability profile and adaptive training of this invention. Detailed Implementation

[0029] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] This invention provides a method for recognizing and intelligently correcting virtual reality skill operation behaviors. This method integrates a multimodal behavior capture, real-time operation evaluation, and adaptive error correction feedback framework to solve the problems of operation behavior evaluation relying on manual observation, delayed error feedback, and lack of personalized guidance in virtual reality skill training.

[0031] Step S1: Multimodal behavioral data acquisition and spatiotemporal alignment.

[0032] Reference Figure 2 Multimodal behavioral data acquisition and spatiotemporal alignment is the first core step of this invention, the purpose of which is to obtain a complete spatiotemporal description of the operator's behavior from multiple source sensors.

[0033] During the data acquisition phase, this invention integrates multiple input devices, including the VR headset's built-in sensors, the controller's six-DOF tracker, eye trackers, and finger-tracking gloves, to simultaneously collect multi-dimensional behavioral data of the operator. Preferably, the VR headset's built-in sensors collect head posture data at a sampling frequency of 90Hz to 120Hz. The head posture data includes the head's three-dimensional position coordinates in the virtual world coordinate system and its three-dimensional rotational posture represented by quaternions, forming a six-DOF representation. In one embodiment of this invention, the accuracy of the head position coordinates can reach the 0.1mm level, and the angular resolution of the rotational posture can reach 0.01 degrees.

[0034] The six-DOF trackers for the handles are respectively installed on the operator's hands, and collect the spatial position data of the hands at the same sampling frequency as the head-mounted display. The spatial position data of the hands also includes three-dimensional position coordinates and three-dimensional rotational posture. Preferably, when the operator holds the virtual tool to perform an operation, the system also collects the trajectory data of the tool's end effector in virtual space. The trajectory data of the tool can be obtained by correlation calculation between the handle posture and the geometric model of the virtual tool.

[0035] An eye tracker is integrated into the VR headset, collecting gaze point data at a sampling frequency of 60Hz to 120Hz. The gaze point data includes parameters such as the operator's gaze position coordinates in the virtual scene, pupil diameter, and blink frequency. In one embodiment of the invention, the spatial resolution of the gaze position can reach 0.5 degrees of visual angle, enabling the system to accurately determine the device component or operating area currently being focused on by the operator.

[0036] Finger-tracking gloves use flexion sensors or optical markers distributed at each finger joint to collect finger joint angle data at a sampling frequency of 30Hz to 60Hz. For each hand, the system collects flexion angle values ​​for 15 joints across 5 fingers, forming a 30-dimensional finger posture vector. This data is crucial for assessing the standardization of fine motor operations such as adjusting knobs, pressing buttons, and holding tools.

[0037] Because each sensor has a different sampling frequency and operates independently, the raw data collected suffers from timestamp synchronization issues. This invention employs timestamp synchronization processing to eliminate the sampling delay differences between sensors. Specifically, the system uses the rendering frame clock of the VR headset as a reference clock to calculate the timestamp synchronization offset of each sensor's data stream relative to the reference clock. In one embodiment of the present invention, the value range of the timestamp synchronization offset is typically from -50ms to 50ms. After calculating the offset, the system performs timestamp correction on the data from each sensor and resamples the data to a uniform sampling frequency using linear interpolation or cubic spline interpolation. Preferably, the uniform sampling frequency is set to 90Hz.

[0038] Multi-source sensors typically use their own local coordinate systems, such as the head-mounted display coordinate system, the controller coordinate system, and the eye-tracking coordinate system. To fuse and analyze the data from these sources, this invention unifies the data into a preset virtual world coordinate system. For the position data collected by each sensor... and attitude data Through spatial coordinate transformation matrix Transform to virtual world coordinate system:

[0039] ,

[0040] in: For sensors The data is represented in the virtual world coordinate system, with the unit being meters (m). for The homogeneous transformation matrix, containing rotation components ( Translation components ( The transformation matrix is ​​obtained through the calibration process during system initialization; For sensors The raw location data collected is represented in homogeneous coordinates.

[0041] After timestamp synchronization and spatial coordinate unification, the system aligns and stitches together the source data along the timeline to form spatiotemporally aligned data. Preferably, the spatiotemporally aligned data is organized in a time series format, with each moment... Corresponding to a multimodal behavioral feature vector :

[0042] ,

[0043] in: for The head pose vector at any time includes 3D position and 3D rotation (quaternion converted to Euler angles). for The coordinates of the point of constant eye gaze; The left and right handles represent six degrees of freedom in their respective orientations; These are the angle vectors of the finger joints of the left and right hands, respectively; This represents the six-DOF pose of the tool's end effector. Multimodal behavioral feature vector. The total dimensions can reach 57 or higher, depending on the specific sensor configuration.

[0044] In one embodiment of the present invention, for a duration of seconds, sampling frequency is The training task of Hz produces spatiotemporally aligned data with dimensions of . The matrix can fully describe the spatiotemporal characteristics of the operator's behavior throughout the entire training process.

[0045] It's worth noting that multimodal feature fusion is not simply data splicing, but a carefully considered feature engineering design. Each modality's data undergoes normalization before fusion to ensure comparability across different units. Location data is normalized to scene scale, mapping coordinate values ​​to a unified reference range; angle data is represented in radians and normalized to a given interval; and velocity data is normalized using task-standard velocity. This normalization process effectively prevents any single modality's excessively large numerical magnitude from dominating subsequent analysis results.

[0046] Step S2: Atomic action decomposition and key node detection.

[0047] Reference Figure 3 Atomic action decomposition and critical node detection is the second core step of this invention. Its purpose is to decompose the continuous operation process into atomic action units that can be independently evaluated and to identify the critical control nodes that affect the operation quality.

[0048] In the atomic action decomposition stage, this invention employs a temporal segmentation network to automatically identify action start and end boundaries and action types. The temporal segmentation network takes spatiotemporally aligned data as input and outputs action category labels and action boundary confidence scores at each time step. Preferably, the temporal segmentation network adopts a temporal convolutional network (TCN) architecture, containing multiple layers of causal convolutional layers and dilated convolutional layers, which can capture temporal dependencies over long periods of time.

[0049] Specifically, the first... The layer causal convolution operation is defined as follows:

[0050] ,

[0051] in: For the first Layer at time The hidden layer feature vectors have a dimension of ; For the first Layer The weight matrix of each convolutional kernel has dimensions of . ; For the first Layer bias vector; The kernel size is set to [value] in this embodiment. ; The expansion factor increases exponentially with the number of layers. Layer expansion factor ; For the activation function, this embodiment uses the ReLU activation function.

[0052] The output layer of the temporal segmentation network contains two branches: an action category branch and a boundary detection branch. The action category branch uses softmax activation and outputs the probability distribution of each action category at each time step. ,in The number of predefined atomic action categories. The boundary detection branch uses sigmoid activation and outputs the action boundary confidence score at each time step that belongs to the action boundary. The range of values ​​is .

[0053] Preferably, the boundary detection branch is trained using a boundary-sensitive loss function:

[0054] ,

[0055] in: For a moment Boundary labels are used, with boundary moments marked as 1 and non-boundary moments marked as 0; These are negative sample weighting coefficients, used to alleviate the class imbalance problem between boundary samples and non-boundary samples. In this embodiment, they are set as follows: ; This represents the total length of the sequence.

[0056] During the inference phase, the system uses boundary detection thresholds. Determine the action boundary. Preferably, the boundary detection threshold... The value range is from 0.6 to 0.85, and in this embodiment it is set to... When continuous Action boundary confidence of a frame All exceeded the boundary detection threshold At that time, the system determines that there is an action boundary during that period. The value ranges from 3 to 5 frames. Through boundary detection, the system divides the continuous operation process into atomic action sequences. ,in Indicates the first Each atomic action, including the start time. End time and action category tags .

[0057] In the critical node detection phase, this invention identifies a set of critical control nodes in the operation process that affect the quality of the operation. Critical control nodes refer to key moments or critical intervals in the operation process that have a decisive impact on the final quality of the operation, including tool grip posture switching points, contact points of the operation object, and the start and end points of fine operation intervals, etc.

[0058] Preferably, key node detection employs a multi-head attention mechanism to identify critical moments within and between atomic actions. For atomic actions... Temporal feature sequences within The system calculates the critical scores at each time point. :

[0059] ,

[0060] in: These are the query matrix, key matrix, and value matrix, respectively, composed of multimodal behavioral feature vectors. Obtained through linear transformation; In this embodiment, the key vector dimension is set to... Key scores Higher times are marked as critical control nodes.

[0061] In one embodiment of the present invention, a set of key control nodes The following types are included: tool grip posture switching point, which is the moment when the operator changes the way the tool is held, identified by detecting abrupt changes in the angle vector of the finger joints; object contact point, which is the moment when the operating tool first comes into contact with the virtual device, identified by collision detection in the virtual scene; and the start and end points of the fine operation range, which are the boundaries of the range that require high-precision control, identified by changes in hand movement speed and trajectory curvature.

[0062] Step S3: Dynamic evaluation of operational standardization.

[0063] Reference Figure 4 The third core step of this invention is the dynamic evaluation of operational standardization. Its purpose is to quantitatively compare the operator's actual operation with the standard operation to generate a standardization score that reflects the quality of operation.

[0064] During the standard template retrieval phase, the system retrieves the standard operation behavior template corresponding to the current training task from a pre-built standard operation behavior template library. The standard operation behavior template library uses standard operations recorded by domain experts as a reference benchmark, and each standard operation behavior template contains a standard trajectory sequence. Standard velocity sequence and standard attitude sequences ,in The sequence length of the standard template.

[0065] In the temporal alignment and comparison stage, this invention employs the Dynamic Time Warping (DTW) algorithm to temporally align and compare the actual trajectories of each atomic action in the atomic action sequence with the corresponding standard trajectories in the standard operation behavior template. The DTW algorithm can handle the problem of inconsistent operation speeds between operators and experts, finding the optimal alignment path between the two sequences.

[0066] For atomic actions Actual trajectory sequence ( (the actual sequence length) and the corresponding standard trajectory sequence The DTW algorithm calculates the cumulative distance matrix using dynamic programming. :

[0067] ,

[0068] in: For the actual sequence number The point and the standard sequence The cumulative distance between points, in meters; The local distance between two points is calculated using Euclidean distance: The unit is m; initial conditions Boundary conditions .

[0069] DTW distance Defined as the normalized cumulative distance:

[0070] ,

[0071] in: The bottom right element of the cumulative distance matrix represents the total distance of the globally optimal alignment path, in meters. and These are the lengths of the actual sequence and the standard sequence, respectively; The unit is meters (m), which represents the average distance deviation for each pair of alignment points.

[0072] In addition to trajectory deviation, this invention also calculates velocity deviation and attitude deviation. The optimal alignment path is obtained based on the DTW algorithm. The system calculates the ratio of velocity deviations between alignment point pairs. and attitude deviation angle value :

[0073] ,

[0074] in: For the actual trajectory The velocity vector of each point is obtained by calculating the difference between adjacent points, and the unit is m / s; For the standard trajectory The velocity vector of each point, in m / s; Alignment path length; It is a dimensionless quantity, representing the proportion of average velocity deviation.

[0075] .

[0076] in: For the actual posture Quaternion representation of points; For the standard posture Quaternion representation of points; It is a quaternion inner product; The unit is radians (rad), representing the average attitude angle deviation.

[0077] Preferably, the trajectory deviation distance value Speed ​​deviation ratio and attitude deviation angle value This invention constructs a multi-dimensional deviation index vector, integrating these indicators to generate operational standardization scoring data. (Standardization scoring) Calculated using a weighted fusion method:

[0078] .

[0079] in: The score is for operational standardization, and the range of values ​​is: point; Here are the weighting coefficients for each deviation, satisfying... In this embodiment, the following settings are provided. ; The trajectory deviation normalization parameter represents the maximum acceptable trajectory deviation. In this embodiment, it is set to... m; Pi is used to normalize attitude deviations to π. scope.

[0080] In one embodiment of the present invention, when the operator's operation is completely consistent with the standard operation, all deviations are zero, and the standardization score is 100 points; when the operator has a large deviation, the standardization score is reduced accordingly. The system calculates the standardization score for each atomic action in the atomic action sequence, forming operation standardization score data.

[0081] Step S4: Error type identification and intelligent attribution.

[0082] Reference Figure 5 Error type identification and intelligent attribution are the fourth core steps of this invention. Their purpose is to automatically classify the detected operational deviations into specific error types and associate them with targeted correction strategies.

[0083] In the operational deviation extraction stage, the system extracts action segments from the operational standardization scoring data whose deviations exceed a preset threshold as operational deviation feature vectors. Preferably, the preset threshold includes a trajectory deviation threshold. m, velocity deviation threshold and attitude deviation threshold rad. When a certain deviation index exceeds the corresponding threshold, the system extracts the multimodal behavioral feature sequence of that action segment, and encodes it into a fixed-dimensional operational deviation feature vector through a feature extraction network. .

[0084] In the error type matching stage, this invention performs pattern matching between the operational deviation feature vector and a pre-established operational error type knowledge base. The operational error type knowledge base covers various common operational error types, including sequence errors, omission errors, posture errors, force errors, and timing errors.

[0085] Preferably, the error type knowledge base is organized in the form of a knowledge graph, including error type nodes, error feature nodes, and correction strategy nodes. Error type nodes record the name, code, and description of each type of error; error feature nodes record typical feature patterns of each type of error, including feature prototype vectors. ( (Encoding error types); the correction strategy node records the correction methods and suggestions corresponding to various errors.

[0086] Pattern matching uses cosine similarity to calculate the operational bias feature vector. With the prototype vectors of each error type Similarity between :

[0087] ,

[0088] in: For the operational deviation eigenvector and the first The cosine similarity of the class error prototype vectors, with values ​​ranging from 1 to 2. ; It is the dot product of vectors; For vectors Norm.

[0089] The system selects the error type with the highest similarity as the recognition result. Exceeding the matching threshold When the error occurs, output the corresponding error type identifier. When the highest similarity does not exceed the matching threshold, the deviation is marked as an unknown type and included in the subsequent manual review process.

[0090] In one embodiment of the present invention, the detailed definitions of error types are as follows: Sequence error indicates that the operator performs atomic actions in a sequence inconsistent with the standard procedure, for example, performing step B before step A, while the standard procedure requires performing step A before step B. Omission error indicates that the operator skips a necessary atomic action, resulting in an incomplete operation procedure. Posture error indicates that the operator's body posture, hand grip posture, or tool operation posture deviates from the standard requirements when performing the action. Force error indicates that the operator applies too much or too little force, which may damage the equipment or cause operational failure. Timing error indicates that the operator performs the action at an inappropriate time, such as performing an operation too early or too late.

[0091] During the correction strategy association phase, the system uses the identified error type identifiers... The system retrieves corresponding correction strategies from the error type knowledge base. These strategies include text descriptions, demonstration animation paths, and voice guidance scripts, providing material support for subsequent error correction and feedback.

[0092] This invention also calculates the severity level of the error. Error severity levels are categorized into five levels based on the correlation between the operational deviation and critical control nodes. The specific classification rules are as follows: Level 1: Minor deviation not involving critical control nodes, with a deviation amplitude not exceeding 1.5 times the threshold; Level 2: Minor deviation occurring near a critical control node, or moderate deviation not involving a critical control node; Level 3: Moderate deviation occurring at a critical control node, with a deviation amplitude of 1.5 to 2.5 times the threshold; Level 4: Significant deviation occurring at a critical control node, with a deviation amplitude of 2.5 to 4 times the threshold; Level 5: Severe deviation occurring at a critical control node, with a deviation amplitude exceeding 4 times the threshold or causing the operational process to be unable to continue.

[0093] Step S5: Multimodal adaptive error correction feedback.

[0094] Reference Figure 6 Multimodal adaptive error correction feedback is the fifth core step of this invention. Its purpose is to intelligently select appropriate feedback methods and timing based on the error type and severity, balancing immediate error correction with an immersive experience.

[0095] During the feedback channel selection phase, the system identifies the error type. and error severity level Choose the error correction feedback channel. This invention provides three feedback methods: visual feedback channel, tactile feedback channel, and voice feedback channel, each with different information transmission characteristics.

[0096] The visual feedback channel overlays visual cues into the virtual scene, including correct trajectory guide lines, highlighted areas, and error location markers. Correct trajectory guide lines display the standard operating trajectory as semi-transparent curves, guiding the operator along the correct path; highlighted areas use glowing effects to emphasize the equipment components or operating areas that require attention; and error location markers use red marks to indicate the specific location where the operator has deviated from the correct path.

[0097] The haptic feedback channel transmits operational force and direction correction information through the vibration motor of the VR controller or the force feedback glove. Preferably, the haptic feedback employs differentiated vibration modes, with different combinations of vibration frequencies and intensities representing different correction information. In one embodiment of the invention, low-frequency long vibration (frequency 50Hz, duration 300ms) prompts the operator to slow down, high-frequency short vibration (frequency 200Hz, duration 100ms) prompts the operator to apply greater force, and directional vibration (vibration of one controller side) prompts the operator to adjust the direction of movement.

[0098] The voice feedback channel generates natural language error correction instructions, explaining the cause of the error and the correction method. Voice feedback uses text-to-speech technology to convert the preset correction strategy text into speech output. Preferably, the voice feedback content is concise and clear, with each feedback session lasting no more than 3 to 8 seconds, avoiding lengthy explanations that could interfere with operation.

[0099] The feedback channel selection strategy is implemented using a rule engine. Preferably, the selection strategy is defined as follows: when the error severity level... When the error severity level is Level 1 or 2, only the visual feedback channel is selected. Visual cues guide the operator to correct the error independently, avoiding excessive intervention that could negatively impact the immersive experience. When the error severity level... At level three, a combination of visual and tactile feedback channels is selected, adding tactile reminders to the visual guidance to reinforce the corrective signal; when the error severity level... For Level 4 or 5, a combination of visual feedback channels, tactile feedback channels, and voice feedback channels can be selected to provide comprehensive error correction guidance.

[0100] This invention also adjusts the feedback content according to the type of error. For posture errors, visual feedback focuses on displaying a demonstration animation of the correct posture; for trajectory errors, visual feedback focuses on displaying the correct trajectory guide line; for timing errors, voice feedback focuses on explaining the correct timing conditions for operation.

[0101] During the feedback timing decision-making phase, the system distinguishes between two feedback timing modes: immediate error correction mode and delayed review mode. The immediate error correction mode triggers feedback immediately upon detecting an operational deviation, suitable for fine-tuning minor deviations; the delayed review mode displays the deviation analysis results in a concentrated manner after completing the current atomic action or the entire operation process, suitable for complex errors that require in-depth explanation.

[0102] The strategy for selecting feedback timing is defined as follows: For error severity levels... For minor deviations of level one or two, an instant error correction mode is used, displaying only concise visual cues without interrupting the operation process; for error severity levels... For a moderate deviation at level three, an instant error correction mode is used, displaying visual cues and triggering tactile feedback without interrupting the operation process; for error severity levels... For severe deviations of level four or five, a delayed review mode is used, pausing the operation process and displaying detailed error analysis and corrective instructions. The operator can then re-execute the action after confirming their understanding.

[0103] The generation of error correction feedback instructions integrates the decision results of feedback channels and feedback timing to form a structured feedback instruction data package. Error correction feedback instructions include the following fields: a list of feedback channels, visual feedback content, haptic feedback parameters, voice feedback text, and feedback triggering timing.

[0104] Step S6: Personalized capability profile and adaptive training.

[0105] Reference Figure 7 Personalized ability profiling and adaptive training is the sixth core step of this invention. Its purpose is to establish a dynamic ability profile of the operator, realize personalized adaptive adjustment of training parameters, and form a closed-loop feedback to the previous steps.

[0106] During the competency profile update phase, the system updates the operator competency profile data based on operational standardization score data and error type distribution. The operator competency profile data includes a skill mastery vector. and common error mode distribution ,in For the number of atomic action categories, This represents the number of error types.

[0107] Components of the skill mastery vector Indicates the operator's position on the first... The degree of mastery of atomic-like actions, with a value range of [value range missing]. 0 represents no mastery, and 1 represents complete mastery. The mastery level value is calculated iteratively using a Bayesian update method.

[0108] ,

[0109] in: This is the updated mastery level value; This is the mastery level value before the update; The first in this training The normative scoring of atomic-like actions, with a range of values. point; The learning rate has a range of values. In this embodiment, the following settings are provided. .

[0110] Common error mode distribution Record the frequency of various errors during the operator's historical training. Each time an error type is identified... When, corresponding component Increase the count: The system identifies operator weaknesses and common error patterns by analyzing error pattern distribution.

[0111] During the adaptive training parameter adjustment phase, the system dynamically adjusts the adaptive training parameters for subsequent training based on the operator's ability profile data. The adaptive training parameters include the training difficulty coefficient. List of key practice sessions and the level of detail in the feedback .

[0112] Training difficulty level The adjustment strategy is defined as follows: when the operator's average normative score for the current difficulty task... continuous The second time it exceeds the upper limit threshold At the same time, increase the difficulty level of training. When the average normative score continuous The second time below the lower limit threshold At the same time, reduce the training difficulty level. In this embodiment, Next, upper limit threshold Score, lower threshold Divide and adjust step size Training difficulty level The range of values ​​is .

[0113] Training difficulty level Several factors affect the training task: the operation time limit is the standard time divided by... higher The value requires a faster operating speed; the allowable deviation range is the standard threshold divided by [value missing]. higher The value requires more precise operation; the intensity of interference factors and Proportional, higher This introduces more environmental interference, such as visual obstruction and noise interference.

[0114] List of key practice sessions Based on the vector of skill mastery Confirmed. The system extracts the level of mastery value. Below the preset mastery threshold The atomic movement categories are included in the list of key practice sessions. Preferably, a mastery threshold is preset. The value range is from 0.6 to 0.75. In this embodiment, it is set to... The frequency and duration of key practice sessions will be increased in subsequent training.

[0115] Feedback detail level It is determined based on the operator's overall mastery level. Overall mastery level is defined as a vector of skill mastery. mean .when hour, Provide detailed error correction feedback; when hour, Provides feedback with moderate detail; when hour, It only provides concise feedback prompts.

[0116] This invention also includes the step of generating specific training scenarios for atomic action types that operators repeatedly make mistakes in. The system extracts the mastery level value from the operator's ability profile data. Below the preset mastery threshold The atomic action types are grouped together with their corresponding preceding and following actions into independent specialized training units. These specialized training scenarios focus on the operator's weak points, improving mastery of that skill through repeated practice until a preset mastery threshold is reached.

[0117] During the skills assessment report generation phase, the system generates a multi-dimensional skills assessment report. The report includes the following: standardized scores and trend charts for each atomic action, showcasing the operator's current skill level and historical progress; an error type distribution chart, displaying the types and frequencies of common operator errors to help identify areas requiring improvement; a skills growth curve, showing the overall trend of the operator's skills over time; and a gap analysis with standard operating procedures, quantifying the gap between the operator's and expert levels and indicating specific directions for improvement.

[0118] The sixth step of this invention forms a closed-loop feedback mechanism to the preceding steps. Specifically, the operator's competency profile data influences the weighting of various deviations in the dynamic assessment of operational standardization in the third step, giving higher assessment weight to the operator's weaknesses; it influences the attribution strategy in error type identification and intelligent attribution in the fourth step, prioritizing the matching of common operator error patterns; and it influences the feedback intensity and detail in the multimodal adaptive error correction feedback in the fifth step, providing more detailed error correction guidance for the operator's weaknesses. Through this closed-loop feedback mechanism, this invention achieves continuous optimization and personalized adjustment of skills training.

[0119] The technical effects of this invention can be verified through the following experiments. In a 500kV substation switching operation training scenario in the power industry, the method of this invention was used to conduct a two-week training experiment on 30 trainees. The experimental results show that, compared with traditional VR training methods, the method of this invention improved the trainees' operational standardization scores by an average of 23.5%, achieved an error recognition accuracy rate of 92.7%, and shortened the average skill acquisition time by 31.2%. The completeness of multimodal behavioral data collection increased the dimension of operational behavior description from 12 dimensions in traditional methods to 57 dimensions, increasing the information content by 375%. The boundary detection F1-score for atomic action decomposition reached 0.89, and the recall rate for key node detection reached 94.3%.

[0120] In medical surgical training scenarios, the method of this invention also demonstrates significant advantages. For laparoscopic surgery training, the system successfully identified five main error types, including postural errors and force errors, with an error attribution accuracy rate of 88.9%. Multimodal adaptive error correction feedback effectively balances immediate error correction with an immersive experience; trainees' average satisfaction rating for feedback was 4.3 out of 5, significantly higher than the 3.1 rating of traditional single visual feedback. Personalized ability profile-driven adaptive training ensures that trainees of different skill levels receive appropriate training difficulty and feedback intensity, significantly improving training efficiency.

[0121] The technical solution of this invention has good scalability and versatility. Besides the power and medical industries, the method of this invention can also be applied to various skills training scenarios such as aerospace operation training, industrial equipment operation training, and precision instrument assembly training. By changing the standard operation behavior template library and error type knowledge base, the method of this invention can quickly adapt to new training fields without major modifications to the core algorithm framework. Preferably, this invention adopts a modular design architecture, with data interaction between functional modules through standardized interfaces, facilitating system maintenance, upgrades, and functional expansion, significantly enhancing its engineering application value.

[0122] In summary, this invention achieves accurate identification and intelligent error correction of virtual reality skill operation behaviors through the deep coupling and synergy of six core steps. Multimodal behavior data acquisition and spatiotemporal alignment construct a complete spatiotemporal description of the operation behavior; atomic action decomposition and key node detection enable fine-grained operation analysis; dynamic evaluation of operation standardization quantifies operation quality deviations; error type identification and intelligent attribution enable automatic error classification and attribution; multimodal adaptive error correction feedback balances immediate error correction and immersive experience; and personalized ability profiling and adaptive training improve skill acquisition efficiency. Each step forms a forward data flow and a backward closed-loop feedback loop, collectively constituting a complete intelligent framework for virtual reality skills training.

[0123] This invention also provides a virtual reality skill operation behavior recognition and intelligent error correction system. Corresponding to the aforementioned method, this system adopts a modular architecture design, including a multimodal behavior data acquisition module, an atomic action decomposition module, an operation standardization evaluation module, an error identification and attribution module, a multimodal error correction feedback module, and an adaptive training adjustment module. The modules communicate with each other through standardized data interfaces, forming a complete data processing pipeline.

[0124] The multimodal behavior data acquisition module, serving as the system's data input layer, is responsible for establishing communication connections with the multi-source sensors of the virtual reality device. This module includes a sensor interface submodule, a timestamp synchronization submodule, and a coordinate transformation submodule. The sensor interface submodule supports data input from various sensing devices, including VR headset built-in sensors, controller six-DOF trackers, eye trackers, and finger-tracking gloves, employing an asynchronous data acquisition mechanism to acquire data streams from each sensor in parallel. The timestamp synchronization submodule uses the VR headset's rendering frame clock as the reference clock, calculates the timestamp synchronization offset of each sensor's data stream, and performs correction and resampling processing. The coordinate transformation submodule maintains the transformation matrix between the local coordinate system of each sensor and the virtual world coordinate system, unifying the source data into the virtual world coordinate system. The output of the multimodal behavior data acquisition module is spatiotemporally aligned data, transmitted to downstream modules in the form of a multimodal behavior feature vector time series.

[0125] The atomic action decomposition module, serving as the system's behavior analysis layer, is responsible for decomposing continuous operation processes into independently evaluable atomic action units. This module includes a temporal segmentation network submodule and a key node detection submodule. The temporal segmentation network submodule deploys a deep learning model based on a temporal convolutional network architecture, taking spatiotemporally aligned data as input and outputting action category labels and action boundary confidence scores at each time step. The network parameters of the temporal segmentation network submodule are obtained through offline training, with training data including expert-annotated standard operation sequences and their action boundary labels. The key node detection submodule employs a multi-head attention mechanism to identify key moments within and between atomic actions, outputting a set of key control nodes. The output of the atomic action decomposition module, including the atomic action sequence and the set of key control nodes, is passed to the operation standardization evaluation module and the error identification and attribution module.

[0126] The operational standardization assessment module, serving as the system's quality evaluation layer, is responsible for quantitatively comparing the operator's actual actions with standard actions. This module includes a template management submodule, a temporal alignment submodule, and a score calculation submodule. The template management submodule maintains a standard operation behavior template library, supporting indexing and retrieval of corresponding standard operation behavior templates by training task type. The temporal alignment submodule implements a dynamic time warping algorithm to align the actual trajectory with the standard trajectory using the optimal path, outputting a sequence of aligned point pairs. The score calculation submodule calculates quantitative indicators of trajectory deviation, velocity deviation, and attitude deviation based on the alignment results, and generates operational standardization score data using a weighted fusion method. The output of the operational standardization assessment module includes the standardization score for each atomic action and a multi-dimensional deviation index vector, which are passed to the error identification and attribution module and the adaptive training adjustment module.

[0127] The error identification and attribution module, serving as the system's intelligent diagnostic layer, is responsible for automatically classifying detected operational deviations into specific error types. This module includes a deviation extraction submodule, a knowledge base management submodule, and a pattern matching submodule. The deviation extraction submodule performs threshold filtering on the operational compliance scoring data, extracts action segments exceeding the deviation threshold, and encodes them as operational deviation feature vectors. The knowledge base management submodule maintains an operational error type knowledge base, storing error type nodes, error feature prototype vectors, and correction strategy nodes in the form of a knowledge graph. The pattern matching submodule uses cosine similarity calculation to match operational deviation feature vectors with the prototype vectors of each error type, outputting error type identifiers and corresponding correction strategies. The error identification and attribution module also calculates the error severity level, comprehensively considering the correlation between the operational deviation magnitude and key control nodes. The output of this module is passed to the multimodal error correction feedback module and the adaptive training adjustment module.

[0128] The multimodal error correction feedback module, serving as the system's interactive output layer, is responsible for conveying error correction guidance information to the operator. This module includes a feedback decision submodule, a visual feedback submodule, a tactile feedback submodule, and a voice feedback submodule. The feedback decision submodule, based on a rule engine, selects feedback channels and determines the timing of feedback, deciding on the combination of activated feedback channels and whether to use an immediate correction mode or a delayed review mode based on the error type identifier and error severity level. The visual feedback submodule uses a virtual reality rendering engine to overlay visual elements such as correct trajectory guidance lines, highlighted areas, and error location markings into the virtual scene. The tactile feedback submodule outputs differentiated vibration modes through the VR controller's vibration motor drive interface, conveying force and direction correction information. The voice feedback submodule uses a text-to-speech engine to convert the correction strategy text into speech output. All feedback submodules work collaboratively to generate error correction feedback instructions and transmit them to the operator through the corresponding channels.

[0129] The adaptive training and adjustment module, serving as the system's closed-loop optimization layer, is responsible for establishing a dynamic competency profile of the operator and personalizing training parameters. This module includes a competency profile management submodule, a parameter adjustment submodule, and a report generation submodule. The competency profile management submodule maintains operator competency profile data, including skill mastery vectors and common error pattern distributions, and iteratively updates the competency profile using a Bayesian update method based on operational standardization score data. The parameter adjustment submodule dynamically adjusts adaptive training parameters based on the operator competency profile data, including training difficulty coefficients, a list of key practice steps, and feedback detail levels. The report generation submodule summarizes all evaluation data and generates a skill evaluation report containing standardization scores for each action, error type distribution, and skill growth curves. The output of the adaptive training and adjustment module is fed back to the operational standardization evaluation module, the error identification and attribution module, and the multimodal error correction feedback module, forming the system's closed-loop optimization mechanism.

[0130] Each module of this invention's system can be deployed on a single computing node or a distributed computing cluster. The specific deployment method can be flexibly configured according to the computing resources and real-time requirements of the actual application scenario. The multimodal behavior data acquisition module and the multimodal error correction feedback module are typically deployed locally on the VR terminal device to ensure the real-time nature of data acquisition and feedback output. The atomic action decomposition module, operation standardization evaluation module, error identification and attribution module, and adaptive training adjustment module can be deployed on edge computing nodes or cloud servers, communicating with the VR terminal device through a low-latency network. The system uses a message queue mechanism to achieve asynchronous data transmission between modules, ensuring the efficient operation of the data processing pipeline.

[0131] It should be understood that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for recognizing and intelligently correcting virtual reality skill operation behaviors, characterized in that, include: The virtual reality device synchronously collects head posture data, gaze point data, hand spatial position data, finger joint angle data, and tool trajectory data of the operator during skill training through multiple source sensors. The collected multi-source data is time-stamped and synchronized to eliminate sampling delay differences between sensors, and the data from each source is unified to a preset virtual world coordinate system to form spatiotemporally aligned data. Based on spatiotemporal aligned data, a temporal segmentation network is used to decompose a continuous operation process into a sequence of atomic actions. The start and end time boundaries of each atomic action are detected, and the set of key control nodes that affect the operation quality in the operation process is identified. The set of key control nodes includes tool grip posture switching points, contact points of the operation object, and the start and end points of the fine operation range. The standard operation behavior template corresponding to the current training task is retrieved from the pre-built standard operation behavior template library. The dynamic time warping algorithm is used to perform temporal alignment and comparison between the actual trajectory of each atomic action in the atomic action sequence and the corresponding standard trajectory in the standard operation behavior template. The quantitative indicators of trajectory deviation, velocity deviation and attitude deviation are calculated, and the multi-dimensional deviation indicators are fused to generate operation standardization score data. Action segments with deviations exceeding a preset threshold in the operational standardization scoring data are extracted as operational deviation feature vectors. Pattern matching is performed with a pre-established operational error type knowledge base to identify the error type identifier to which the current operational deviation belongs. Error types include sequence errors, omission errors, posture errors, force errors, and timing errors, and corresponding correction strategies are associated with them.

2. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 1, characterized in that, The timestamp synchronization process includes: calculating the timestamp synchronization offset of each sensor data stream relative to the reference clock, with the timestamp synchronization offset ranging from -50ms to 50ms; and interpolating and resampling each sensor data to a uniform sampling frequency, which is from 90Hz to 120Hz.

3. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 1, characterized in that, The temporal segmentation network outputs the action boundary confidence score at each time step. When the action boundary confidence scores of multiple consecutive frames exceed the boundary detection threshold, it is determined to be an action boundary. The boundary detection threshold ranges from 0.6 to 0.

85.

4. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 1, characterized in that, The calculated quantitative indicators include: the trajectory deviation distance value is the average Euclidean distance between the actual trajectory point and the corresponding point of the standard trajectory; the velocity deviation ratio value is the ratio of the actual motion velocity to the standard motion velocity; and the attitude deviation angle value is the angle difference between the actual attitude quaternion and the standard attitude quaternion.

5. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 1, characterized in that, Based on the error type identifier and the severity level of the error, the error correction feedback channel and feedback timing are selected. Error correction feedback instructions are generated and transmitted to the operator through visual feedback channel, tactile feedback channel, or voice feedback channel. The visual feedback channel overlays the correct trajectory guide line and the error location mark in the virtual scene. The tactile feedback channel transmits force and direction correction information through the vibration of the handle. The voice feedback channel generates voice instructions explaining the cause of the error and the correction method. The error severity level is divided into five levels according to the correlation between the operation deviation and the critical control node. Level 1 is a slight deviation that does not involve the critical control node, and Level 5 is a serious deviation that occurs at the critical control node.

6. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 5, characterized in that, The strategy for selecting error feedback channels is as follows: when the error severity level is level one or two, visual feedback channels are preferred; when the error severity level is level three, visual and tactile feedback channels are selected in combination; and when the error severity level is level four or five, visual, tactile, and voice feedback channels are selected in combination.

7. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 6, characterized in that, The operator competency profile data is updated based on the operational standardization score data and error type distribution. The operator's mastery of various atomic actions is tracked and common error patterns are recorded. The adaptive training parameters for subsequent training are dynamically adjusted based on the operator competency profile data. The adaptive training parameters include the training difficulty coefficient, key practice links, and the level of feedback detail. A skill assessment report containing the standardization score of each action, error type distribution, and skill growth curve is generated. The operator competency profile data includes a skill mastery vector. Each component of the skill mastery vector corresponds to the mastery value of various atomic actions. The mastery value ranges from 0 to 1. The mastery value is iteratively calculated using a Bayesian update method based on the standardization score of the corresponding atomic action in historical training.

8. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 7, characterized in that, The training difficulty coefficient adjustment strategy is as follows: when the operator's average standardization score for the current difficulty task is higher than the upper limit threshold for multiple consecutive times, the training difficulty coefficient is increased; when the average standardization score is lower than the lower limit threshold for multiple consecutive times, the training difficulty coefficient is decreased. The upper limit threshold ranges from 80 to 90 points, and the lower limit threshold ranges from 50 to 65 points.

9. The virtual reality skill operation behavior recognition and intelligent error correction method according to claim 8, characterized in that, Feedback timing is divided into immediate error correction mode and delayed review mode. Immediate error correction mode triggers feedback immediately when an operational deviation is detected, while delayed review mode displays the deviation analysis results after the current atomic action or the entire operation process is completed. It also includes the step of generating special training scenarios for atomic action types that the operator repeatedly makes mistakes in. The special training scenarios extract atomic action types with a mastery level value lower than a preset mastery threshold from the operator's ability profile data, and combine the corresponding atomic actions and their related actions into independent special training units. The preset mastery threshold ranges from 0.6 to 0.

75.

10. A virtual reality skill operation behavior recognition and intelligent error correction system, used to implement the virtual reality skill operation behavior recognition and intelligent error correction method as described in claim 9, characterized in that, include: The multimodal behavior data acquisition module is configured to synchronously acquire head posture data, gaze point data, hand spatial position data, finger joint angle data, and tool trajectory data of the operator during skill training through multi-source sensors of the virtual reality device. It performs timestamp synchronization processing on the acquired multi-source data to eliminate sampling delay differences between sensors and unifies the data from each source into a preset virtual world coordinate system to form spatiotemporally aligned data. The atomic action decomposition module is configured to decompose a continuous operation process into an atomic action sequence based on the spatiotemporal alignment data using a temporal segmentation network, detect the start and end time boundaries of each atomic action, and identify a set of key control nodes that affect the operation quality in the operation process. The set of key control nodes includes tool grip posture switching points, operation object contact points, and the start and end points of fine operation intervals. The operation standardization assessment module is configured to retrieve the standard operation behavior template corresponding to the current training task from the pre-built standard operation behavior template library, use the dynamic time warping algorithm to perform time-series alignment and comparison between the actual trajectory of each atomic action in the atomic action sequence and the corresponding standard trajectory in the standard operation behavior template, calculate the quantitative indicators of trajectory deviation, velocity deviation and attitude deviation, and integrate the multi-dimensional deviation indicators to generate operation standardization score data. The error identification and attribution module is configured to extract action segments with deviations exceeding a preset threshold from the operation standardization scoring data as operation deviation feature vectors, perform pattern matching with a pre-established operation error type knowledge base, identify the error type identifier to which the current operation deviation belongs, the error type identifier includes sequence error, omission error, posture error, force error and timing error, and associate the corresponding correction strategy; The multimodal error correction feedback module is configured to select the error correction feedback channel and feedback timing based on the error type identifier and error severity level, generate error correction feedback instructions and transmit them to the operator through a visual feedback channel, a tactile feedback channel or a voice feedback channel. The visual feedback channel overlays the correct trajectory guide line and the error position mark in the virtual scene. The tactile feedback channel transmits force and direction correction information through the vibration of the handle. The voice feedback channel generates voice instructions explaining the cause of the error and the correction method. The adaptive training adjustment module is configured to update the operator's ability profile data based on the operation standardization score data and error type distribution, track the operator's mastery of various atomic movements and record common error patterns, and dynamically adjust the adaptive training parameters for subsequent training based on the operator's ability profile data. The adaptive training parameters include training difficulty coefficient, key practice links and feedback detail, and generate a skill assessment report containing the standardization score of each movement, error type distribution and skill growth curve.

Images