Non-power-cut operation VR training and operation behavior correction method and system

Abstract

The invention discloses a non-power-cut operation VR training and operation behavior correction method and system. The method comprises the following steps: constructing a VR operation scene; in response to simulation operation of a student in the VR operation scene, synchronously acquiring whole body action data, tool operation data, physiological data and voice interaction data of the student; comparing the parameters with pre-stored parameters in a standard action library so as to identify the operation deviation and the type; calculating a real-time comprehensive risk level; according to the type of the operation deviation and the real-time comprehensive risk level, real-time correction guidance fused with at least one of visual prompt, auditory warning and force sense feedback is generated and output to the trainee; and recording the operation deviation, the real-time comprehensive risk level and the correction guidance, thereby generating an evaluation report and updating the skill archive of the student. According to the invention, real-time capture, accurate comparison and dynamic correction of operation actions of trainees can be realized, so that the pertinence, effectiveness and safety of training are improved.

Description

Technical Field

[0001] This invention relates to the field of VR (Virtual Reality)-based power system operation training technology, specifically to a VR training and operational behavior correction method and system for live-line operation. Background Technology

[0002] Live-line working is a key technology for ensuring reliable power supply and improving service quality. However, its working environment is complex and the safety risks are extremely high, placing strict demands on trainees' skill levels, operational standards, and psychological resilience. Traditional training methods mainly rely on theoretical teaching, physical simulation training, and on-site observation by mentors, which have the following significant drawbacks: 1) Building real training routes and equipment requires huge investment, not only in terms of high cost, but also in terms of limited training venues, equipment and time, making it impossible to conduct high-risk project training frequently.

[0003] 2) Safety risks are uncontrollable. Even when training on simulation equipment, there is still a risk of personal injury or equipment damage due to misoperation.

[0004] 3) The training effect relies heavily on the coach's subjective observation and experience judgment, lacking objective and quantitative evaluation methods, making it difficult to accurately locate the trainees' subtle operational flaws and bad habits.

[0005] With the development of virtual reality (VR) technology, its advantages of strong immersion, repeatability, and low cost have provided new ideas for power training. However, existing VR training systems mostly focus on scene simulation and process drills, and cannot monitor and guide the standardization of operational actions, the standardization of posture, and the accuracy of tool use in real time. Summary of the Invention

[0006] To address the problem that existing technologies cannot monitor and guide the standardization of operational actions, the standardization of postures, and the accuracy of tool use in real time, this invention provides a VR training and operational behavior correction method and system for uninterrupted power supply operations. This system can achieve real-time capture, accurate comparison, and dynamic correction of trainees' operational actions, thereby improving the relevance, effectiveness, and safety of training.

[0007] On the one hand, the present invention provides a VR training and operational behavior correction method for live-line work, comprising the following steps: S1: Constructing a VR work scenario, including: loading a device model, calling a physics simulation engine to configure physical properties for the device model to simulate key physical states set during the work process; configuring dynamic environment variables and fault condition scripts to drive the scenario to run; S2: Respond to the student's simulated operation in the VR work scene, and simultaneously collect the student's full-body motion data, tool operation data, physiological data and voice interaction data; S3: Compare the full-body motion data and tool operation data with parameters in the pre-stored standard motion library to identify operation deviations and types; calculate the real-time comprehensive risk level based on the physiological data, voice interaction data, and physical state parameters obtained in real time from the scene; S4: Based on the type of operational deviation and the real-time comprehensive risk level, generate and output real-time corrective guidance to the trainee that integrates at least one of visual cues, auditory warnings and force feedback; S5: Record the operational deviations, real-time comprehensive risk levels, and corrective guidance, and generate an assessment report accordingly to update the trainee's skill profile.

[0008] Furthermore, in step S1, the physical states simulated by the physical simulation engine include: conductor deformation under stress, electric field distribution and breakdown effect, and collision interaction between tools and equipment; the dynamic environmental variables include adjustable weather conditions and light intensity; the fault script is used to simulate at least one unexpected working condition, which includes at least one of abnormal equipment discharge, insulation tool failure, or sudden displacement of adjacent equipment.

[0009] Furthermore, before step S1, a personalized training program generation step is included: based on the skill weaknesses identified in the trainee's skill profile, a targeted initial VR job scenario configuration and training task sequence are automatically generated from a preset knowledge point database and risk point database; wherein, the training task sequence is used to guide the simulation operation and correction process in steps S2 to S4.

[0010] Furthermore, in step S3, the comparison includes: comparing the spatial position deviation of the operation, comparing the joint angle deviation, comparing the operation timing deviation, and comparing the tool movement trajectory deviation; the calculation of the real-time comprehensive risk level includes: combining the severity of the operation deviation, the psychological load level and attention state of the trainee obtained from the physiological data and voice interaction data, and the physical state parameters obtained from the scene in real time, and performing a weighted fusion calculation.

[0011] Furthermore, the force feedback in step S4 specifically includes: dynamically adapting the corresponding force feedback mode according to the type of the operation deviation, wherein the force feedback mode includes simulating tool jamming, abnormal vibration, electric arc repulsion, or safe distance warning touch; the intensity of the force feedback is positively correlated with the real-time comprehensive risk level.

[0012] Furthermore, the recording in step S5 is achieved in the following way: The operational deviations, real-time comprehensive risk levels, and corrective guidance events, along with the configuration information of the VR work scenario, are aligned and correlated along a unified timeline to form a structured copy of the training process data.

[0013] Furthermore, the method also includes a collaborative training mode, implemented through the following steps: The VR work scenario is extended to a networked collaborative scenario instance, deployed on a collaborative host, and different work roles and scene object operation permissions are assigned to multiple connected student terminals. In the collaborative scenario example, operation data, voice interaction data, and interaction event data based on role relationships are collected synchronously from each student's terminal; The collected interaction event data and operation data are subjected to time sequence and logic analysis to evaluate the compliance with the team collaboration procedures. Based on the evaluation results, team collaboration optimization tips are generated and output to the corresponding student terminals or all terminals.

[0014] Furthermore, in the collaborative training mode, step S3 also includes: using speech recognition technology to identify the voice commands of each role in real time, comparing their content with a preset standardized command library, and automatically verifying and recording the logical order and repetition standardization of the commands.

[0015] Furthermore, the method also includes virtual-real interaction and model optimization steps, implemented in the following ways: The standardized operation instruction sequence generated based on the evaluation report will be sent out through a preset communication interface to drive the physical training device to perform the corresponding operation or to provide operation guidance for the operation assistance system. Through the data acquisition interface, we receive operation process data uploaded from physical training devices or real work environment monitoring equipment; The operation process data is processed, and the processed data is used to iteratively optimize the physical simulation parameters configured for the device model in step S1 and the parameters in the standard action library.

[0016] On the other hand, the present invention also provides a VR training and operational behavior correction system for live-line work, used to implement the above-described VR training and operational behavior correction method for live-line work, the system comprising: The scene simulation and management unit is configured to build, load, and run VR job scenes, and manage environment variables and fault scripts; The standard library and rule management unit is configured to store the standard action library, the security rule library, and the risk assessment model. The multi-source data sensing and fusion unit integrates motion capture equipment, tool posture sensors, biosignal acquisition modules and audio acquisition equipment, and is configured to collect trainees' full-body motion data, tool operation data, physiological data and voice interaction data in real time. The intelligent behavior analysis engine is connected to the multi-source data perception and fusion unit and the standard library and rule management unit. It includes a real-time parsing module, a multi-dimensional comparison module, a physiological cognition analysis module and a risk assessment module. An immersive interaction and feedback unit is connected to the intelligent behavior analysis engine, which drives the VR display device, spatial audio device, and force feedback device. The digital twin assessment and archiving unit, connected to the intelligent behavior analysis engine and the multi-source data perception and fusion unit, is configured to record training process data and generate assessment reports and update trainees' skill profiles.

[0017] Furthermore, the intelligent behavior analysis engine adopts an edge-cloud collaborative architecture, including a lightweight edge computing module deployed on the VR local device for processing action comparison and generating basic feedback instructions with extremely high real-time requirements; and a cloud analysis server for performing complex machine learning model inference, long-term data mining, and global optimization and updating of the models in the standard library and rule management unit.

[0018] Furthermore, the system also includes a distributed collaborative training management unit, which comprises: A scene synchronization server is configured to maintain the consistency of the VR job scene state among multiple students; The role and permission allocation module is configured to assign different job roles and corresponding operation permissions to multiple students who have accessed the system. The team behavior analytics sub-engine is configured to analyze collaborative work data among roles and evaluate team processes.

[0019] Furthermore, in the digital twin assessment and archive unit, the digital skills archive is visualized in the form of a time-series-based skills map, which dynamically reflects the historical performance trends and current level of trainees in multiple preset skills dimensions.

[0020] Furthermore, the system also includes an automated lesson plan generation and deduction module, which is connected to the standard library and rule management unit and the digital twin assessment and archive unit. This module can automatically combine knowledge points, risk points and scenario parameters to generate personalized training lesson plans based on the trainees' skill weaknesses and training objectives.

[0021] Compared with the prior art, the present invention has at least the following beneficial effects: 1) By constructing high-fidelity VR work scenarios and configuring dynamic fault scripts, real high-risk operations and various unexpected working conditions (such as equipment discharge and tool failure) can be simulated in a safe environment. The training is safer and the simulated risk operations cover a wider range, solving the problems of uncontrollable safety risks and the inability to carry out high-risk project training in traditional training.

[0022] 2) By synchronously collecting multi-source data, comparing it with the standard movement library in real time from multiple dimensions, and combining it with comprehensive risk assessment based on physiological data, it can identify subtle operational deviations and changes in psychological state of trainees more quickly (e.g., at the millisecond level). It can also provide immediate and accurate corrective guidance through adaptive multi-sensory feedback (visual, auditory, and force-sensory). Operational assessment and corrective guidance are more accurate and timely, overcoming the shortcomings of traditional methods that rely on the coach's subjective judgment and have delayed feedback.

[0023] 3) By constructing a full-process multimodal data package and training process data copies, the training process was fully digitally recorded and accurately replayed, making the training process traceable. Data-driven assessment reports and skill maps allow trainees' skill mastery and weaknesses to be presented objectively, quantitatively, and visually, greatly improving the scientific nature of the assessment.

[0024] 4) The generation of personalized training programs based on trainees' digital skill profiles, and the iterative optimization of standard action libraries and physical simulation parameters using virtual and real interactive data, enable the training system to teach according to individual needs and continuously improve and evolve with the accumulation of data. The system has strong self-evolution capabilities, further enhancing the long-term benefits of training.

[0025] Through the above technical solutions, this invention achieves safer training, easier quantification of training effectiveness, higher real-time correction and guidance, more personalized solutions, and better self-evolution of the system, completely revolutionizing the traditional live-line work training model and possessing extremely high application value and promotion prospects. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating the steps of the VR training and operational behavior correction method for live-line work according to the present invention.

[0028] Figure 2 This is a flowchart illustrating the VR training and operational behavior correction method for live-line work according to the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the principle of multi-dimensional comparison and risk assessment for the intelligent behavior analysis engine of this invention.

[0030] Figure 4 This is a schematic diagram illustrating the data interaction of the method of the present invention in a distributed collaborative training scenario.

[0031] Figure 5 This is a system architecture diagram of the VR training and operational behavior correction system for live-line work as described in an embodiment of the present invention.

[0032] Figure 6 This is a schematic diagram of the system deployment and virtual-real integration of the VR training and operational behavior correction system for live-line work as described in an embodiment of the present invention. Detailed Implementation

[0033] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0034] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Method Implementation Examples

[0035] refer to Figure 1 and Figure 2 This invention provides a VR training and operational behavior correction method for live-line work, comprising the following steps: S1: Constructing VR work scenarios, including: loading equipment models, calling the physics simulation engine to configure physical properties for the equipment models to simulate key physical states set during the work process; configuring dynamic environment variables and fault condition scripts to drive the scene operation; S2: Responds to the simulated operation of the trainee in the VR operation scene, and synchronously collects the trainee's whole body motion data, tool operation data, physiological data and voice interaction data; S3: Compare the full-body motion data and tool operation data with the parameters in the pre-stored standard motion library to identify operation deviations and types; calculate the real-time comprehensive risk level based on physiological data, voice interaction data and physical state parameters obtained from the scene in real time; S4: Based on the type of operational deviation and the real-time comprehensive risk level, generate and output real-time corrective guidance to trainees that integrates at least one of visual cues, auditory warnings, and force feedback; S5: Record operational deviations, real-time comprehensive risk levels, and corrective guidance, thereby generating assessment reports and updating trainees' skill profiles.

[0036] The core of step S1 is to construct a highly realistic and dynamic VR work scenario. This is the foundation for training assignments and student training. Constructing a highly realistic and dynamic VR work scenario involves creating an interactive virtual environment based on detailed 3D models of real electrical equipment (such as wires, insulators, crossarms, and switches) and combining them with a physics engine (such as NVIDIA PhysX or Unity Physics).

[0037] In embodiments of the present invention, a VR system refers to a complete hardware and basic software framework. A VR system includes: 1) hardware devices such as VR headsets (HTC Vive, Meta Quest, etc.), motion capture devices (such as inertial sensor suits), force feedback controllers, and biosensors; 2) a basic software framework, such as a computer / server equipped with a graphics rendering engine (such as Unity3D, Unreal Engine) and a physics simulation engine (such as NVIDIA PhysX, Havok), and a basic software layer for data acquisition, communication, and logic control. The VR system provides basic and general functions for creating and running VR content, such as 3D graphics rendering, spatial positioning, sensor data input, and physics calculations.

[0038] VR work scenarios are virtual work environments with specific functions and content, created specifically for the purpose of live-line work training using the aforementioned VR system. A VR work scenario includes concrete power grid equipment models (such as towers, conductors, and insulators), pre-programmed work process logic, interactive virtual tools, and dynamic environmental variable parameters and fault scripts. VR work scenarios are built based on real power grid parameters. Developers utilize the modeling and programming capabilities of the VR system to transform knowledge from power grid drawings, equipment parameters, safety regulations, and other fields into a runnable virtual world. In other words, the VR system is the technical platform and hardware foundation for creating and running VR work scenarios; VR work scenarios are specific application examples and training content formed by relying on the VR system and injecting specific domain knowledge (power work) into it.

[0039] Preferably, in step S1, the physical states simulated by the physics simulation engine include: conductor deformation under stress, electric field distribution and breakdown effect, and collision interaction between tools and equipment. The VR work scene, by integrating a high-fidelity physics engine, not only possesses visual realism but also simulates basic physical states such as gravity, collision, and friction, ensuring that the interaction between virtual tools and equipment conforms to real physical laws and the trainee's intuition. For example, when a virtual insulating rod approaches a charged body, the high-fidelity physics engine can calculate and present the changes in the electric field distribution; when a wrench tightens a bolt, it can simulate torque transmission and bolt deformation.

[0040] Furthermore, the system allows for flexible configuration of dynamic environmental variables and preset fault scripts for VR work scenarios. These dynamic environmental variables include adjustable voltage levels, weather conditions, and light intensity. Fault scripts simulate at least one unexpected working condition, including abnormal equipment discharge, insulation tool failure, or sudden displacement of nearby equipment, such as sudden insulator breakage or tool slippage. By simulating various extreme and sudden working conditions, a fault script library composed of preset fault scripts is integrated on top of the basic work scenarios. These fault scripts can trigger emergencies such as "abnormal equipment discharge" (simulating the sound and light effects of an electric arc), "surface flashover simulation of insulation tools," and "sudden changes in weather conditions" (such as sudden wind causing increased conductor sway), covering a wide range of training needs and comprehensively training trainees' adaptability and risk management capabilities. For example, simulating a tool suddenly falling during insulator replacement trains trainees to execute emergency evacuation and reporting procedures.

[0041] The core of step S2 is to achieve real-time synchronous acquisition of trainees' multimodal data, and to perform real-time preprocessing and feature extraction on the acquired raw multimodal data to form a standardized real-time data stream that can be used for immediate comparison. For example... Figure 3 As shown, during VR training, trainees wear motion capture equipment (such as an inertial measurement unit (IMU) suit) and force feedback VR controllers. The motion capture equipment, such as a wearable suit based on an IMU or an optical motion capture system, is used to collect spatial position and angle data of the trainee's joints. Tool posture sensors, typically integrated into the force feedback controllers or dedicated simulation tools, are used to collect the pose (position, orientation), movement trajectory, and applied force / torque data of the handheld tool. The system acquires the trainee's full-body motion data (joint position, rotation) in real time through the motion capture equipment, and acquires the handheld tool's operation data (position, direction, button status) in real time through the tool posture sensors integrated into the force feedback VR controllers. Preferably, to more comprehensively assess the trainee's state, the system also integrates biosensors and audio acquisition devices. The biosensors include at least a heart rate sensor and an eye tracker. The heart rate sensor, such as a wristband or ECG patch, is used to collect heart rate and heart rate variability data. The eye tracker is typically integrated into a VR headset (head-mounted display device) to collect visual attention data. The heart rate variability data is used to assess the trainees' work-related psychological workload level, and the gaze focus data is used to assess their compliance with safety observation procedures and attention allocation patterns. The audio acquisition device, i.e., the microphone, is used to collect the trainees' voice interaction data.

[0042] Therefore, in the embodiments of the present invention, the system synchronously collects the following four types of data: 1) Movement and posture data. The spatial position, angle and movement trajectory of all joints of the trainee's body (especially the hands, wrists, elbows and waist) are obtained through wearable inertial motion capture systems or high-precision optical motion capture systems.

[0043] 2) Tool operation data. Measure the tool's grip posture, applied force / torque, movement speed, and path using a force feedback handle with built-in sensors or a customized simulation tool.

[0044] 3) Physiological and cognitive data. For example, heart rate sensors and eye trackers integrated into VR headsets can be used to collect trainees' heart rate variability (HRV) and visual attention points. Heart rate variability (HRV) is an effective indicator for assessing autonomic nervous system activity and reflecting psychological stress and fatigue levels. Eye tracking data can objectively reflect whether trainees' attention allocation follows safety procedures (SOPs), such as whether they are continuously focusing on hazards or critical work points.

[0045] 4) Voice interaction data. Voice commands and team communication during the training process are collected via microphone.

[0046] The above four types of data provide key inputs for subsequent comprehensive risk assessment.

[0047] Example 1: Individual Basic Skills Training

[0048] Take, for example, the scenario of a newly hired trainee, Xiao Wang, undergoing his first basic skills training in "replacing insulators on a 10kV power distribution line".

[0049] The system loads the standard training scenario based on Xiao Wang's file (no history): a 10kV straight pole in clear weather. The standard operation action library has preset standard action parameters for "using an insulated operating rod to pick up an old insulator", such as the arm extension angle range and tool movement trajectory.

[0050] Xiao Wang enters the scene wearing VR equipment. When he uses the virtual control lever to pick up an insulator, the motion capture device collects the angle data of his upper limb joints in real time, and the tool posture sensor collects the position and posture data of the control lever.

[0051] The intelligent behavior analysis engine compares the real-time collected motion data with a standard library. It detects that during Xiao Wang's swing, his elbow angle exceeds the optimal range by 5 degrees, which is identified as "joint angle deviation." Since the current training is low-risk and his physiological data is stable, the system calculates the real-time overall risk level as "low."

[0052] Based on the "low risk" and "angle deviation" categories, the system triggers adaptive feedback: In Xiao Wang's VR view, a semi-transparent green highlight appears on his elbow joint, accompanied by a gentle voice prompt: "Please pay attention to the angle of your arm extension." Xiao Wang immediately adjusts his posture according to the prompt, restoring the angle to the standard range, and the visual prompt disappears.

[0053] The deviation and correction process were recorded and integrated into the multimodal data package of the entire training process. After training, the system-generated report indicated that Xiao Wang had one deviation in "tool operation stability" which had been corrected, and the overall operation standardization score was 88 points. In Xiao Wang's personal skill profile, one training data point was recorded in the "basic operation accuracy" dimension.

[0054] In addition, the collected raw multimodal data undergoes real-time preprocessing and feature extraction, including filtering and spatial coordinate unification of motion data (e.g., noise reduction and feature value calculation (e.g., heart rate variability) of physiological data), and endpoint detection and keyword extraction of speech data. The processed data is then encapsulated into a standardized real-time data stream with a unified timestamp for immediate comparison in subsequent steps.

[0055] A better approach is to construct a real-time data preprocessing and feature extraction pipeline when performing real-time preprocessing and feature extraction on the collected raw multimodal data. This pipeline is used to process multi-source heterogeneous raw data in parallel and to spatially align and temporally synchronize the trainee's body movements, tool postures, and visual attention points through a sensor fusion algorithm. This generates a low-latency, standardized feature vector sequence that represents the trainee's overall operational state and serves as the sole input for real-time behavior comparison.

[0056] The above preprocessing steps for raw multimodal data are a crucial bridge connecting the raw sensors (physical world) and the intelligent analysis engine (digital world). The goal is to transform high-bandwidth, noisy, and heterogeneous raw multimodal data into a clean, compact, and semantically clear feature data stream. Specifically, this includes the following sub-steps: I) Data Cleaning. For motion data, such as raw acceleration and angular velocity data from an inertial measurement unit (IMU), Kalman filtering or complementary filtering is performed to eliminate noise and drift. Inverse kinematics (IK) calculations are then used to uniformly transform the data into three-dimensional spatial joint coordinate data with the human pelvis or work tool as the origin. For physiological data, such as raw photoplethysmography (PPG) signals, filtering is performed to remove motion artifacts, and the time-domain / frequency-domain features of heart rate (HR) and heart rate variability (HRV) (such as RMSSD, LF / HF) are calculated in real time. Raw eye-tracking coordinates are smoothed and transformed into features such as fixation duration and saccade path for a preset region of interest. For tool data, such as raw readings from force / torque sensors, calibration and filtering are performed. II) Data Synchronization. Timestamp all data streams using the same clock source (e.g., microsecond precision) to ensure strict alignment of cross-modal data on the timeline. This is fundamental for subsequent precise analysis, such as identifying the physiological responses accompanying actions at specific moments. III) Spatial and Temporal Feature Extraction. Calculate key safety and ergonomic features in real-time from the unified joint coordinates, such as the Euclidean distance between the right hand and the high-voltage conductor, the elbow flexion angle, and the angle between the tool axis and the conductor tangent. Calculate dynamic features of the action, such as velocity, acceleration, and smoothness of motion, using a sliding window. IV) Multimodal Feature Fusion. Fusion is performed at the feature level. For example, a composite feature vector is generated: [timestamp, hand-to-line distance, elbow angle, tool angle, instantaneous heart rate, whether the risk source is being gazed at]. This composite feature vector integrates spatial, dynamic, and cognitive states, providing rich analytical dimensions for the intelligent engine. V) Forming Standardized Real-Time Data Streams. Encapsulate the processed composite feature vector into a fixed-format data packet (e.g., using Protocol Buffers or a custom binary format). The standardized real-time data stream is published at a stable high frame rate (e.g., 50-100Hz) via a low-latency communication protocol (e.g., UDP or RTI DDS). This standardized real-time data stream is characterized by: low latency (e.g., meeting millisecond-level feedback requirements), high semantics (directly containing multi-dimensional feature values ​​required for comparison), and lightweight nature (significantly reduced data volume compared to the original data, alleviating the burden on subsequent analysis engines). This processing step standardizes complex signal processing problems, allowing the subsequent intelligent behavior analysis engine to focus on high-level logical judgments and decisions without dealing with underlying signal noise and synchronization issues, greatly improving the system's real-time performance, reliability, and maintainability.

[0057] Step S3 mainly includes the establishment of a standard action library, multi-dimensional comparison and identification of operational deviations, and calculation of real-time comprehensive risk level. Its core is multi-dimensional intelligent comparison and comprehensive risk assessment.

[0058] First, before conducting multi-dimensional intelligent comparison, a standard action library needs to be established. The establishment of this standard action library includes: 1) Based on national and industry safety regulations, standardized operating instructions (cards), and integrating the operational experience of experts in the field, a complete operation task (such as "live disconnection of leads") is broken down into a series of continuous key operation steps; 2) Then, each step is further broken down into a more granular, quantifiable sequence of standard actions; 3) Define precise judgment parameters for each action, including but not limited to: the spatial coordinate range of the end effector (such as the hand), the angle threshold of key body joints (such as the elbow and waist), and the operation sequence and time threshold between actions (for example, after attaching the seat belt, the inspection time must be greater than 3 seconds). This constitutes the standard for the system to perform behavior comparison.

[0059] Secondly, it involves multi-dimensional comparison and identification of operational deviations. For example... Figure 3 As shown, the real-time motion data stream acquired (and preprocessed) in step S2 is compared with the preset parameters of the current step in the standard motion library established in step S3. Preferably, the multi-dimensional comparison includes spatial position deviation comparison, joint angle deviation comparison, operation timing deviation comparison, and tool motion trajectory deviation comparison. For example, this multi-dimensional comparison includes: 1) Spatial position deviation comparison to determine whether the distance between the operator and the live conductor is less than the safe distance.

[0060] 2) Compare joint angle deviations to determine whether the waist bending exceeds the safe limit and whether the arm extension angle is optimal.

[0061] 3) Compare the operation timing deviations to determine whether the execution order of each step is correct and whether the dwell time of key steps is sufficient.

[0062] 4) Compare tool path deviations to determine whether the movement trajectory of the insulated operating rod is smooth and efficient.

[0063] Furthermore, the system incorporates physiological data into the analytical framework for a more comprehensive analysis of potential risks. The potential risks posed by a trainee operating correctly but with wandering eyes may be underestimated, while the method of this invention can reveal these potential risks more comprehensively. For example, analyzing heart rate variability can assess a trainee's level of psychological stress or fatigue; analyzing gaze focus data can determine whether their attention is consistently focused on safety-critical points (such as the contact point between tools and live parts, backup protection devices), and assess their adherence to Standard Operating Procedures (SOPs).

[0064] Next, the real-time comprehensive risk level is calculated. In this invention, the system does not simply identify movement deviations, but instead calculates the real-time comprehensive risk level using a pre-set risk assessment model after identifying the deviations. This risk assessment model takes multiple factors as input, including: the type of operational deviation and its quantified severity, a psychological load index calculated based on physiological data (such as heart rate variability), an attention distraction index calculated based on eye-tracking data, and a baseline risk value determined according to the current work scenario type and environmental variables. For example, a small angular deviation may have low risk under low psychological load, but the risk may be amplified when the trainee is stressed. Therefore, the system assigns different weights to these input factors, performs weighted fusion calculations, and finally outputs a quantified risk coefficient, which is mapped to discrete real-time comprehensive risk levels such as "low," "medium," and "high."

[0065] Finally, based on the overall risk level, the system triggers timely feedback through multiple sensory channels (visual, auditory, and tactile). For example, visual highlighting prompts can be used to directly overlay warning boxes at deviation locations; flashing guide arrows can dynamically indicate the correct direction of movement; voice warnings provide clear voice commands; and vibrations from the control handle provide immediate tactile alerts.

[0066] Preferably, the feedback is adaptive and hierarchical, with the system selecting the combination and intensity of feedback based on the overall risk level. The adaptive feedback means that the system adaptively selects and integrates visual cues, spatial audio warnings, and hierarchical force feedback according to a preset feedback strategy matrix. This feedback strategy matrix defines the mapping relationship between the real-time overall risk level, the type of operational deviation, and the feedback method and intensity. For example, when the real-time overall risk level is "low" and the deviation is a minor posture problem, non-invasive visual cues (such as highlighting) are prioritized. When the risk level is "medium" or the deviation involves insufficient safety distance, spatial audio warnings (such as directional voice reminders) are superimposed. When the risk level is "high" or the deviation is a misoperation that could cause a serious virtual accident, high-intensity hierarchical force feedback (such as simulating strong vibrations or repulsive forces) is forcibly triggered, and the scene can be selectively paused, displaying reinforced teaching information. Based on real-time analysis results, the system queries this strategy matrix to dynamically determine the combination and intensity of feedback to be provided at the current moment.

[0067] Preferably, the intensity of the timely feedback is positively correlated with the overall risk level. For example, in step S4, when dynamically adapting the corresponding force feedback mode according to the type of operational deviation, graded force feedback can also be provided, which is implemented through a force feedback device. Specifically, based on the identified operational deviation type, if it is identified as a "tool operation stuck" deviation, a tool stuck feeling is simulated; if it is identified as a "device abnormal vibration" deviation (combined with the scenario fault script), abnormal vibration is simulated; if it is identified as an "insufficient safety distance" deviation, an arc repulsion feeling or a safety distance warning tactile sensation is simulated (e.g., the closer the distance, the stronger the feedback force). Meanwhile, the intensity of the force feedback is not fixed, but positively correlated with the real-time overall risk level calculated in step S3. The higher the risk level, the greater the simulated stuck feeling, vibration amplitude, or repulsion intensity, providing the trainee with a tactile warning commensurate with the risk level.

[0068] Example 2: Emergency Response Training Including Sudden Failures

[0069] Taking the scenario of repairing a live conductor on a power line as an example, Xiao Wang, a trainee with some experience, performs live conductor repair work on a 10kV power line under simulated "strong wind" weather conditions to train his psychological resilience and emergency response capabilities in complex environments. The main steps include: The system loads a high-wind weather scenario (swaying power lines) and includes pre-set dynamic fault scripts. The standard library contains standard actions for the entire repair process. Xiao Wang is focused on repairing the equipment. The system simultaneously collects his movements, tool data, heart rate data collected via biosensors, and gaze data collected via eye trackers. During a critical phase of the operation, the system triggered a preset fault: simulating a "sudden jamming of the repair tool." Almost simultaneously, the physiological cognitive analysis module detected a sudden spike in Xiao Wang's heart rate, but eye-tracking data showed that his gaze quickly locked onto the faulty tool and backup protection device, without any panicked scanning. In terms of action: Xiao Wang did not pull incorrectly, but immediately stopped applying force.

[0070] Cognitive level: The system determined that he had "high psychological load", but his "attention allocation" was reasonable.

[0071] Comprehensive assessment: Based on high-risk sudden failure scenarios, the risk assessment model calculates the real-time comprehensive risk level as "medium-high".

[0072] Based on the "medium to high risk" level, the system triggers enhanced composite feedback: the force feedback handle simulates a strong tool jamming sensation, enhancing the tactile sense of the malfunction; a prominent red warning box pops up in the center of the screen, accompanied by an audio message: "Tool is abnormally jammed, please follow the emergency response procedure!"; the side of the interface simultaneously displays the key steps of the first step in the emergency response, "reporting the malfunction," with text and images.

[0073] Xiao Wang then successfully handled the situation according to the procedures. The system fully recorded the entire chain of data from fault triggering, physiological reaction, operational response to successful handling. The evaluation report highly praised the "correctness of his emergency response" and analyzed his "psychological stress recovery curve" in detail, providing a precise basis for his subsequent psychological stress resistance training.

[0074] In the method embodiment of the present invention, the VR training and operational behavior correction method for live-line work of the present invention further includes a personalized training program generation step. That is, before step S1, based on the skill weaknesses identified in the trainee's digital skill profile, a targeted initial training scenario and task objective sequence are automatically generated from a preset knowledge point database and risk point database.

[0075] Preferably, the live-line operation VR training and operational behavior correction method of the present invention also includes a digital precision assessment step: after a single or phased training session, the system automatically generates a multi-dimensional operational behavior assessment report. This report not only includes the traditional pass / fail indicators, but also provides an operational standardization score, a risk point statistical chart, a deviation heatmap from standard actions (intuitively showing which body parts or tool paths deviate the most), a specific list of improvement suggestions, and a full-process retrospective video recording of the operation that can be played back for analysis.

[0076] Preferably, the live-line work VR training and operational behavior correction method of the present invention also includes a personalized growth management step: the system establishes and maintains individual skill files for trainees, and tracks and records all historical assessment data over a long period. Through data analysis, the mastery curves of each skill and the evolution trend of weak points are presented in chart form. Based on this file, the system can intelligently recommend personalized training subjects and difficulty levels for subsequent training, realizing dynamic optimization of the training plan.

[0077] Preferably, the VR training and operational behavior correction method for live-line work of the present invention further includes personalized recommended action steps: based on a database of historical excellent operation records, operation patterns are analyzed through machine learning algorithms; optimized operation features that comply with safety regulations and are more efficient than the current standard actions are identified; and the action parameters in the standard action library are dynamically updated according to the optimized operation features. More preferably, the standard action parameters are adaptively adjusted based on the trainee's personalized physiological characteristic data to generate personalized recommended actions. For example, different operation actions are recommended for different trainees based on conditions such as height.

[0078] Preferably, step S5 further includes a digital twin construction step: Synchronized with the training process, the real-time data stream generated in step S2, the analysis events and status data generated in step S4, and the training scenario metadata are aligned, correlated, and integrated along a unified timeline to construct and store a multimodal data package of the entire training process, forming a digital twin copy of the training. This digital twin platform allows for further advance simulation of different risk scenarios.

[0079] Preferably, the VR training and operational behavior correction method for live-line work of the present invention also includes a team collaborative work training mode. The system supports collaborative training mode, in which multiple trainees are connected in the same VR work scene, playing roles such as trainees, supervisors, and ground support personnel. Preferably, in the collaborative training mode, it also includes: using speech recognition technology to recognize the voice commands of each role in real time, comparing their content with a preset standardized command library, and automatically verifying and recording the logical order and repetition standardization of the commands.

[0080] In collaborative training mode, real-time multimodal data acquisition is conducted, simultaneously collecting action, voice communication, and operational command data from each role. During behavior analysis, not only individual operational behavior analysis is performed, but a team collaboration analysis module is also added to simultaneously analyze and evaluate the standardization of team collaborative operations, the accuracy and completeness of command transmission, and the fulfillment of safety supervision responsibilities. Correspondingly, real-time corrective feedback generated based on the analysis and evaluation results includes not only feedback on individual operations but also feedback on team collaboration issues (such as unrepeated commands or lack of supervision).

[0081] Example 3: Team Collaboration Training like Figure 4 The example shown is a collaborative operation training session for a 110kV transmission line equipotential bonding work team. The training involves a three-person team: A (equipotential bonding worker), B (ground electrician), and C (work supervisor) conducting collaborative work training for replacing vibration dampers on a 110kV transmission line. The training includes the following steps: The system constructs a collaborative virtual scene and assigns different roles and perspective permissions to the three terminals. In addition to individual action standards, the standard library also includes team instruction specifications (such as repetition requirements). The system simultaneously collects the actions and tool data of the three individuals and records team conversations through audio acquisition devices. Person A, on the conductor, requests: "Ground, pass me an M16 wrench." Person B repeats: "Understood, pass me an M16 wrench." The voice command recognition and logic verification module automatically compares the responses to confirm that the repetition is correct and complete. Meanwhile, the team behavior analysis sub-engine analyzed the eye movement data of C (the person in charge of the work) and found that his gaze was off the work point for 3 seconds during the critical stage of the work, which was marked as "brief interruption of monitoring attention". The system assessment indicates that individual operations pose no risk and the team's processes are generally standardized, but there are shortcomings in supervision. The overall risk level of the team is assessed as "low". During real-time feedback and correction, the system's feedback is more targeted: for users A and B, there are no additional prompts, and the work continues as usual; for user C, a gentle yellow text prompt is displayed in a non-central area of ​​their field of vision: "Please continue to monitor the work point." This is intended to remind user C without interfering with the main work process.

[0082] After the training, in addition to generating individual evaluation reports, the system generates a personalized "Team Collaboration Evaluation Report." For example, the "Team Collaboration Evaluation Report" uses charts to show the "Instruction Transmission Efficiency" (average response time 2.1 seconds), "Instruction Repetition Accuracy" (100%), and "Effective Monitoring Coverage" (98%, deducted due to one interruption) of this task. It also clearly points out the strengths and areas for improvement in team collaboration, allowing the team to collectively review the performance.

[0083] Preferably, the VR training and operation behavior correction method for live-line work of the present invention further includes virtual-real integration and skill transfer steps: the standard operation sequence verified and optimized in VR training is output through a standardized interface to drive the physical training device or operation assistance system; operation data from the physical training device or real operation scenario is received, processed and used to iteratively optimize the core model and database in the VR training system.

[0084] Through the aforementioned virtual-to-physical skill transfer flow and physical-to-virtual data feedback flow, this bidirectional data flow constructs a collaborative evolutionary training system across virtual and real environments.

[0085] System Implementation Examples

[0086] like Figure 5 As shown, this invention also provides a VR training and operational behavior correction system for live-line work, used to implement the VR training and operational behavior correction method for live-line work as described above. The system of this invention mainly includes: a scene simulation and management unit, a standard library and rule management unit, a multi-source data perception and fusion unit, an intelligent behavior analysis engine, an immersive interaction and feedback unit, and a digital twin assessment and archiving unit.

[0087] The scenario simulation and management unit is configured to build, load, and run VR work scenarios, and manage environment variables and fault scripts. This unit manages different voltage levels, weather conditions, and lighting environments through parameterized configuration, and integrates a dynamic fault script library. It is configured to trigger preset emergency conditions such as equipment malfunction, tool failure, or sudden weather changes during training to simulate the complexity and uncertainty of real-world operations.

[0088] The Standards Library and Rules Management Unit is configured to store, maintain, and dynamically optimize the standard action library, safety rule library, and risk assessment model. The core of this unit lies in its dynamic optimization capability: by analyzing historical best practice records, it continuously optimizes standard action parameters using machine learning algorithms, and can generate adaptive action suggestions based on trainees' personalized physiological characteristics, enabling the standard library to evolve from static procedures to a dynamic set of best practices.

[0089] The multi-source data sensing and fusion unit integrates motion capture equipment, tool posture sensors, biosignal acquisition modules, and audio acquisition equipment, and is configured to perform multimodal data real-time synchronous acquisition steps.

[0090] The intelligent behavior analysis engine, connected to the multi-source data perception and fusion unit and the standard library and rule management unit, is configured to execute intelligent behavior comparison and comprehensive risk assessment steps. It includes a real-time parsing module, a multi-dimensional comparison module, a physiological cognitive analysis module, and a risk assessment module. This unit is responsible for real-time, synchronously collecting trainees' whole-body motion data, tool operation data, physiological indicator data, and voice interaction data. Through a built-in real-time preprocessing and feature extraction pipeline, it transforms the raw heterogeneous data stream into a unified, low-latency, standardized feature data stream.

[0091] The immersive interaction and feedback unit is connected to the intelligent behavior analysis engine. Based on the received decision instructions, the unit adaptively drives the VR display device, spatial audio system and force feedback device to generate and output multi-sensory corrective feedback that integrates visual highlighting, spatial voice and graded tactile sensation. Among them, the force feedback device can simulate various contextual tactile sensations such as tool jamming, electric arc repulsion, and safe distance warning, and the feedback intensity is dynamically correlated with the risk level.

[0092] The digital twin assessment and archiving unit is connected to the intelligent behavior analysis engine and the multi-source data perception and fusion unit. This unit fully records and integrates multimodal data streams, analysis events and scenario metadata during the training process to construct a digital twin record of the entire training process. Based on this, it automatically generates quantitative assessment reports and visualized skills heatmaps, and establishes and maintains digital skills archives for trainees with time-series skills maps as the core, dynamically presenting their ability evolution trajectory and weaknesses.

[0093] Preferred, such as Figure 6As shown, the intelligent behavior analysis engine adopts an edge-cloud collaborative architecture: it includes a lightweight edge computing module deployed on the VR local device, configured to handle action comparison and generate basic feedback instructions with extremely high real-time requirements, ensuring millisecond-level latency for correction feedback; simultaneously, the system collaborates with a cloud analysis server, which is configured to perform complex machine learning model training, deep mining of massive historical data, and global optimization and updates of the core models in the standard library and rule management unit. These core models include risk assessment models, among others.

[0094] Preferably, the system of the present invention further includes a distributed collaborative training management unit, configured to support multi-student remote collaborative training; this distributed collaborative training management unit mainly includes: a scene synchronization server, a role and permission allocation module, and a team behavior analysis sub-engine. The scene synchronization server is configured to maintain the consistency of the virtual scene state among all connected students, ensuring a shared sense of presence in collaborative work. The role and permission allocation module is configured to assign specific roles such as operators, monitors, and ground support personnel to different students, and configure corresponding operation permissions and perspectives. The team behavior analysis sub-engine is configured to synchronously analyze the instruction transmission links, work sequence logic, and cross-safety monitoring behaviors between roles, evaluating the standardization and collaborative efficiency of the team's overall work process.

[0095] Preferably, the distributed collaborative training management unit also integrates a voice command recognition and logic verification module. This module is configured to recognize the voice commands of trainees in each role in real time, automatically compare the recognized content with a standardized command library, and verify and record the logical order of command issuance and the standardization of the recipient's recitation, thereby realizing the automated assessment of the team communication process.

[0096] Preferred, such as Figure 6As shown, the system of the present invention also includes a virtual-real linkage and data interface unit. This unit constructs a data bridge connecting the virtual training system and the physical operation environment, specifically including: a skill transfer output interface and a data feedback acquisition interface. The skill transfer output interface is configured to convert the standard operation sequence verified and optimized in VR training into executable motion control commands for the physical robot operation platform, or into visual operation guidance animations in the augmented reality guidance system. The data feedback acquisition interface is configured to receive real operation record data from the physical simulation operation platform, the robot teaching system, or de-identified real operation records. This data serves as training samples and verification basis for optimizing the action models and calibrating physical simulation parameters in the standard library and rule management unit, forming a closed-loop system evolution that complements the virtual and real worlds. Preferably, in the digital twin assessment and archive unit, the digital skill archive is visualized in the form of a time-series-based skill map. This skill map dynamically reflects the historical performance trends and current levels of trainees across multiple preset skill dimensions.

[0097] Preferably, in the digital twin assessment and archiving unit, trainees' digital skills profiles are visualized through a time-series-based multi-dimensional skills map. This multi-dimensional skills map, presented in the form of a radar chart or line graph, dynamically and intuitively reflects the trainees' mastery trends, progress curves, and areas needing improvement across multiple preset skills dimensions, such as operational standardization, risk identification, emergency response, psychological qualities, and teamwork, as the training progresses.

[0098] Preferably, the system further includes an automated lesson plan generation and deduction module, which is intelligently connected to the standard library and rule management unit and the digital twin assessment and archive unit. This module can automatically retrieve and combine information from the knowledge point library, risk point library and scenario parameter library based on the specific weaknesses and phased training objectives identified in the trainee's personal skill file, and generate personalized training lesson plans that include targeted training scenarios, task sequences and assessment points, so as to achieve precise training planning for "one plan per person".

[0099] It should be understood that, apart from the common implementation parts, the above methods or system embodiments may include specific embodiments for multiple different application scenarios (such as Embodiment 1, Embodiment 2, etc.).

[0100] It should be noted that in this paper, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply these relationships. There is no such actual relationship or order between entities or operations. Furthermore, the terms "including" and "package" do not apply. The word "comprise" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0102] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0103] In particular, the device embodiments are basically similar to the method embodiments, so they are described in a simpler way. For relevant details, please refer to the description of the method embodiments.

[0104] For ease of description, the above apparatus is described by dividing it into various functional units / modules. Of course, in implementing this invention, the functions of each unit / module can be implemented in one or more software and / or hardware.

[0105] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0106] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A VR training and operational behavior correction method for live-line work, characterized in that, Includes the following steps: S1: Constructing a VR work scenario, including: loading a device model, calling a physics simulation engine to configure physical properties for the device model to simulate key physical states set during the work process; configuring dynamic environment variables and fault condition scripts to drive the scenario to run; S2: Respond to the student's simulated operation in the VR work scene, and simultaneously collect the student's full-body motion data, tool operation data, physiological data and voice interaction data; S3: Compare the full-body motion data and tool operation data with parameters in the pre-stored standard motion library to identify operation deviations and types; calculate the real-time comprehensive risk level based on the physiological data, voice interaction data, and physical state parameters obtained in real time from the scene; S4: Based on the type of operational deviation and the real-time comprehensive risk level, generate and output real-time corrective guidance to the trainee that integrates at least one of visual cues, auditory warnings and force feedback; S5: Record the operational deviations, real-time comprehensive risk levels, and corrective guidance, and generate an assessment report accordingly to update the trainee's skill profile.

2. The method according to claim 1, characterized in that, In step S1, the physical states simulated by the physical simulation engine include: conductor deformation under stress, electric field distribution and breakdown effect, and collision interaction between tools and equipment; the dynamic environmental variables include adjustable weather conditions and light intensity; the fault script is used to simulate at least one unexpected working condition, which includes at least one of abnormal equipment discharge, insulation tool failure, or sudden displacement of adjacent equipment.

3. The method according to claim 1, characterized in that, Before step S1, a personalized training program generation step is also included: based on the skill weaknesses identified in the trainee's skill profile, a targeted initial VR operation scenario configuration and training task sequence are automatically generated from a preset knowledge point database and risk point database; wherein, the training task sequence is used to guide the simulation operation and correction process in steps S2 to S4.

4. The method according to claim 1, characterized in that, In step S3, the comparison includes: comparing the spatial position deviation of the operation, comparing the joint angle deviation, comparing the operation timing deviation, and comparing the tool movement trajectory deviation; the calculation of the real-time comprehensive risk level includes: combining the severity of the operation deviation, the psychological load level and attention state of the trainee obtained from the physiological data and voice interaction data, and the physical state parameters obtained from the scene in real time, and performing a weighted fusion calculation.

5. The method according to claim 1, characterized in that, The force feedback in step S4 specifically includes: dynamically adapting the corresponding force feedback mode according to the type of operation deviation. The force feedback mode includes simulating tool jamming, abnormal vibration, electric arc repulsion, or safe distance warning touch. The intensity of the force feedback is positively correlated with the real-time comprehensive risk level.

6. The method according to claim 1, characterized in that, The recording in step S5 is achieved in the following way: The operational deviations, real-time comprehensive risk levels, and corrective guidance events, along with the configuration information of the VR work scenario, are aligned and correlated along a unified timeline to form a structured copy of the training process data.

7. The method according to claim 1, characterized in that, The method also includes a collaborative training mode, implemented through the following steps: The VR work scenario is extended to a networked collaborative scenario instance, deployed on a collaborative host, and different work roles and scene object operation permissions are assigned to multiple connected student terminals. In the collaborative scenario example, operation data, voice interaction data, and interaction event data based on role relationships of each student's terminal are collected synchronously. The collected interaction event data and operation data are subjected to time sequence and logic analysis to evaluate the compliance with the team collaboration procedures. Based on the evaluation results, team collaboration optimization tips are generated and output to the corresponding student terminals or all terminals.

8. The method according to claim 1, characterized in that, The method also includes virtual-real interaction and model optimization steps, which are implemented in the following ways: The standardized sequence of operation instructions generated based on the evaluation report will be sent out through a preset communication interface to drive the physical training device to perform corresponding operations or to provide operation guidance for the operation assistance system. Through the data acquisition interface, we receive operation process data uploaded from physical training devices or real work environment monitoring equipment; The operation process data is processed, and the processed data is used to iteratively optimize the physical simulation parameters configured for the device model in step S1 and the parameters in the standard action library.

9. A VR training and operational behavior correction system for live-line work, used to implement the VR training and operational behavior correction method for live-line work as described in any one of claims 1-8, characterized in that, include: The scene simulation and management unit is configured to build, load, and run VR job scenes, and manage environment variables and fault scripts; The standard library and rule management unit is configured to store the standard action library, the security rule library, and the risk assessment model. The multi-source data sensing and fusion unit integrates motion capture equipment, tool posture sensors, biosignal acquisition modules and audio acquisition equipment, and is configured to collect trainees' full-body motion data, tool operation data, physiological data and voice interaction data in real time. The intelligent behavior analysis engine is connected to the multi-source data perception and fusion unit and the standard library and rule management unit. It includes a real-time parsing module, a multi-dimensional comparison module, a physiological cognition analysis module and a risk assessment module. An immersive interaction and feedback unit is connected to the intelligent behavior analysis engine, which drives the VR display device, spatial audio device, and force feedback device. The digital twin assessment and archiving unit, connected to the intelligent behavior analysis engine and the multi-source data perception and fusion unit, is configured to record training process data and generate assessment reports and update trainees' skill profiles.

10. The system according to claim 9, characterized in that, The system also includes a distributed collaborative training management unit, which comprises: A scene synchronization server is configured to maintain the consistency of the VR job scene state among multiple students; The role and permission allocation module is configured to assign different job roles and corresponding operation permissions to multiple students who have accessed the system. The team behavior analytics sub-engine is configured to analyze collaborative work data among roles and evaluate team processes.

Images