A simulation teaching system based on virtual reality and artificial intelligence fusion

Abstract

The present application relates to a kind of simulation teaching systems based on virtual reality and artificial intelligence fusion, it is related to artificial intelligence teaching field, solve the problem that virtual and real two-way synchronous delay compensation is poor in pertinence, simulation and real object evaluation are split, lack closed loop evolution.The system includes digital twin construction module, virtual and real synchronous mapping engine, AI prediction evaluation and safety verification deployment module, multi-modal synchronous acquisition and evaluation module and teaching strategy adaptive adjustment module.The core innovation is that virtual and real synchronous mapping engine uses differentiated delay compensation: forward channel is determined by double mode network and realizes physical state synchronization by adaptive gain adjustment;Feedback channel is based on GRU model to predict operation instruction to offset transmission delay.The system constructs "simulation prediction-real object verification-feedback correction" closed loop, through multi-modal data fusion evaluation and online incremental learning, dynamically optimizes prediction model and teaching strategy.The present application improves virtual and real synchronous precision and practical training safety, realizes individualized engineering practical teaching.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically to a simulation teaching system based on the integration of virtual reality and artificial intelligence. Background Technology

[0002] The deep integration of virtual reality and digital twin technologies has provided a new technological path for engineering practical training. In recent years, VR teaching platforms integrating 3D modeling, motion capture, and virtual interaction have been gradually applied to high-skill training scenarios such as industrial training. By constructing virtual environments to simulate real operation processes, the realism and interactivity of teaching have been enhanced. In the field of digital twins, existing solutions use cross-scale virtual-real synchronization engines to perform timestamp alignment and latency compensation, achieving state synchronization between digital twin simulation scenes and physical devices. However, existing technologies still have the following shortcomings in practical teaching applications:

[0003] Firstly, most synchronization mechanisms are designed for one-way mapping of device status, which cannot meet the two-way closed-loop real-time synchronization requirements in teaching scenarios, namely "issuance of operation instructions → response of physical devices → status feedback to the virtual environment".

[0004] Secondly, delay compensation typically employs a uniform strategy for both forward and feedback channels, failing to differentiate between the delay characteristics of the two channels, resulting in a significant decrease in compensation accuracy during network fluctuations.

[0005] Third, simulation operation evaluation and physical verification evaluation are separated from each other, lacking the closed-loop evolution capability of using physical data feedback to correct simulation prediction models and dynamically adjust training strategies.

[0006] In summary, existing technologies have significant shortcomings in targeted delay compensation for virtual-real bidirectional synchronization and closed-loop evaluation of simulation prediction and physical verification. There is an urgent need for a simulation teaching system that can achieve differentiated delay compensation for forward and feedback channels and organically integrate simulation performance prediction with physical operation evaluation. Summary of the Invention

[0007] Purpose of the invention: To provide a simulation teaching system based on the integration of virtual reality and artificial intelligence, so as to solve the above-mentioned problems existing in the prior art.

[0008] Technical solution: A simulation teaching system based on the integration of virtual reality and artificial intelligence, comprising five components: a digital twin construction module, a virtual-real synchronization mapping engine, an AI prediction and evaluation module, a security verification and deployment module, and a multimodal synchronization acquisition and evaluation module.

[0009] The specific method by which the virtual-real synchronization mapping engine achieves real-time bidirectional synchronization includes: on the target physical teaching device side, acquiring real-time operating data of the device and attaching a first timestamp, transmitting it to the high-fidelity digital twin model via the forward channel to drive virtual state updates; on the high-fidelity digital twin model side, acquiring learner operation instructions and attaching a second timestamp, transmitting them to the target physical teaching device via the feedback channel to drive the physical device to perform actions; the forward channel adopts a forward delay compensation algorithm based on dynamic adaptive compensation, dynamically adjusting the sending strategy of physical state data to the virtual end according to the real-time monitored network round-trip latency and forward channel transmission delay, so that the virtual state is consistent with the physical state; the feedback channel adopts a feedback delay compensation algorithm based on operation instruction sequence prediction, predicting the compensation instruction at the current moment according to the historical operation instruction sequence, and injecting it into the control loop of the target physical teaching device to offset the impact of feedback channel transmission delay on control real-time performance.

[0010] 2. In a further embodiment, in the virtual-real synchronization mapping engine, the forward delay compensation algorithm based on dynamic adaptive compensation used in the forward channel includes the following steps:

[0011] Step A1: During each synchronization cycle, the virtual-real synchronization mapping engine monitors and records the following network status parameters in real time by sending timestamped probe packets:

[0012] Network round-trip time (RTT(k)) and forward transmission delay during the current synchronization period Feedback channel transmission delay and the jitter value of the forward channel delay. , where k is the synchronization period index;

[0013] Step A2: Calculate the delay change rate of the current synchronization period based on the network state parameters. :

[0014]

[0015] And based on the forward channel transmission delay With the preset delay threshold The comparison relationship, and the said time delay change rate Compared with the preset rate of change threshold The comparison relationship determines whether the current network state is a stable mode or a fluctuating mode:

[0016] when and When this condition is met, it is determined to be a stable mode;

[0017] when or When this occurs, it is determined to be a wave mode;

[0018] Step A3: Based on the determined network state mode, calculate the adaptive compensation gain for the current synchronization period using different gain calculation strategies. :

[0019] When the network state is in a stable mode, the first gain calculation strategy is adopted:

[0020]

[0021] in The preset reference gain value, ;

[0022] When the network state is in a fluctuating mode, a second gain calculation strategy is adopted, introducing a time-varying adaptive gain based on exponential smoothing:

[0023]

[0024] in:

[0025] As a smoothing factor, This is used to control the smoothness of gain updates; This is a preset reference round-trip time base value; This represents the network round-trip time during the current synchronization period. This is the compensation gain value from the previous synchronization cycle;

[0026] Step A4: Utilize the adaptive compensation gain The strategy for dynamically adjusting the transmission of physical state data to the virtual terminal is as follows: the virtual-physical synchronization mapping engine adjusts the transmission strategy based on the adaptive compensation gain. Determine the data transmission rate for the current synchronization period. and timing of sending :

[0027]

[0028] ;

[0029] in:

[0030] This is the maximum data transmission rate; This represents the start time of the current synchronization cycle;

[0031] This allows for normal synchronization with a reference gain in stable mode, and compensates for increased latency caused by network fluctuations by adaptively reducing the data transmission rate and advancing the transmission timing in fluctuating mode.

[0032] 3. In a further embodiment, in the virtual-real synchronization mapping engine, the feedback channel employs a feedback delay compensation algorithm based on operation instruction sequence prediction, which includes the following steps:

[0033] Step B1: Obtain the sequence of historical operation instructions of length N before the learner's current time t. Each operation instruction It includes instruction type, instruction parameter vector, and timestamp information, and performs normalized encoding on the instruction parameter vector of each operation instruction in the historical operation instruction sequence to form an input feature sequence. ;

[0034] Step B2: Obtain the current feedback channel transmission delay. The transmission delay of the feedback channel is obtained in real time by the virtual-real synchronization mapping engine through timestamp comparison;

[0035] Step B3: Input the input feature sequence X(t) into a pre-trained GRU (Gated Recurrent Unit) prediction model, which includes a GRU hidden layer and a fully connected output layer connected in sequence. The calculation process of the GRU hidden layer at each time step i includes:

[0036] Update gate calculation:

[0037]

[0038] Reset door calculation:

[0039]

[0040] Candidate hidden state calculation:

[0041]

[0042] Hidden status update:

[0043]

[0044] in:

[0045] This is the input feature vector for the current time step; This is the hidden state vector from the previous time step; ⊙ represents the Sigmoid activation function; ⊙ indicates element-wise multiplication (Hadamard product).

[0046] This is the weight matrix. The bias vector;

[0047] Step B4: Predicted output of the compensation instruction at the current time. This involves determining the hidden state of the GRU hidden layer at the end of the sequence. Perform transmission delay via the feedback channel Forward prediction of the prediction step size to generate the predicted value of the compensation command at the current time step. The specific prediction method is as follows: the fully connected output layer uses... As input, it is mapped as follows:

[0048]

[0049] in To output the weight matrix, To output the bias vector, The output mapping function is selected as either a linear mapping function or a nonlinear activation function based on the parameter range of the operation instruction.

[0050] The GRU prediction model transmits latency via the feedback channel during the training phase. To predict the step size, supervised learning is performed using a training dataset containing operation instruction sequence samples under time-varying delay conditions to minimize the predicted instruction. With real instructions The mean square error between them is the training target;

[0051] Step B5: Calculate the predicted value of the compensation command. The control loop of the target physical teaching equipment is injected to offset the impact of the feedback channel transmission delay on the real-time performance of the control.

[0052] Unlike the Smith predictor and its variants commonly used in existing technologies, the GRU direct instruction prediction method of this invention bypasses the indirect path of "first estimating the delay value, and then calculating the compensation instruction based on the delay". An inherent problem with traditional indirect methods is that the delay estimation error is amplified a second time in the instruction compensation stage. This invention directly establishes an end-to-end mapping from the "historical instruction sequence" to the "current compensation instruction", so the prediction error of the compensation instruction originates only from the sequence prediction itself and is not affected by the cascading amplification of the delay estimation error. Furthermore, the prediction step size (i.e., the feedback channel transmission delay) of this invention is a real-time measurement value rather than a fixed value. Through multi-condition data augmentation training, the model acquires variable step size prediction capability, which is a key feature distinguishing it from existing fixed step size time series prediction models.

[0053] In a further embodiment, the training process of the GRU prediction model includes:

[0054] Step C1: Under various network latency conditions, collect historical operation command sequence samples of learners performing operations in the virtual simulation environment, and simultaneously record the feedback channel transmission latency at each moment. For each training sample, the sequence of operation instructions of length N preceding the current time t is used as the input feature, and the sequence of instructions at time t is used as the input feature. The actual operation instructions are used as prediction labels;

[0055] Step C2: For the collected historical operation command sequences, use a sliding window of length N to extract continuous command sequence segments in the time dimension to form a training sample set. ,in Let j be the input feature sequence of the j-th sample. For the corresponding predicted labels;

[0056] Step C3: For each training sample in the training sample set, simulate multiple feedback channel transmission delay conditions. Generate corresponding prediction labels for each delay condition. This forms an expanded training sample; through multi-condition data augmentation, the GRU prediction model learns the instruction evolution rules under different delay step sizes, thereby improving the model's adaptive prediction capability for time-varying delays.

[0057] Step C4: Using the mean squared error (MSE) between the predicted and actual instructions as the loss function, apply the Adam optimizer to the weight matrix of the GRU prediction model. and bias vector Perform iterative optimization training until the loss function converges to below a preset threshold or reaches a preset number of training rounds.

[0058] In a further embodiment, the digital twin construction module includes a multi-source sensor data fusion unit and a physics engine driving unit. The multi-source sensor data fusion unit performs spatiotemporal alignment and fusion of heterogeneous data collected by laser displacement sensor, inertial measurement unit, temperature sensor and vibration accelerometer. The physics engine driving unit inputs the fused device state vector and static attribute data into the multibody dynamics model to calculate the motion, force and deformation state of the high-fidelity digital twin model in real time, so that the virtual model and the physical device maintain consistent physical behavior.

[0059] In a further embodiment, the deep learning model in the AI ​​prediction and evaluation module is a Transformer temporal prediction model. Its multi-head attention encoder extracts associated features from the simulation operation sequence and the digital twin state vector, and outputs predicted values ​​of multiple performance indicators such as device operation stability, energy consumption, accuracy and safety in parallel through the performance prediction decoder.

[0060] In a further embodiment, the security verification and deployment module includes a multi-level security verification unit and an abnormal interruption rollback unit. After the simulation performance is qualified, the multi-level security verification unit further performs hardware constraint verification and security rule verification on the operation instruction sequence, and only allows automatic deployment when all verifications pass. When the abnormal interruption rollback unit detects that the equipment operating parameters deviate from the preset safety boundary during the physical operation verification process, it immediately sends an emergency stop command and controls the high-fidelity digital twin model to roll back to a safe state.

[0061] In a further embodiment, the multimodal fusion model in the multimodal synchronous acquisition and evaluation module adopts a two-level fusion architecture: the feature-level fusion layer uses canonical correlation analysis to project the time-frequency domain features of IoT sensors and the spatial-temporal features of visual sensors to a shared subspace for fusion; the decision-level fusion layer uses Dempster-Shafer evidence theory to weightedly synthesize the independent outputs of the three scoring networks of operational stability, accuracy and compliance to generate the final evaluation result.

[0062] In a further embodiment, when the multimodal synchronous acquisition and evaluation module feeds back the operation evaluation results to the AI ​​prediction evaluation module, it adopts an online incremental learning mechanism based on experience replay and elastic weight consolidation; new training samples are stored in the experience replay buffer, and elastic weight consolidation regularization terms are applied to the parameters when the model is updated, so as to retain the predictive ability for historical simulation scenarios.

[0063] In a further embodiment, a teaching strategy adaptive adjustment module is also included, which is used to calculate the learner's current skill mastery index based on the operation evaluation results, and dynamically adjust the difficulty level, task objective parameters and auxiliary prompt frequency of the simulation teaching task in combination with the skill improvement rate and operation style characteristics extracted from the historical simulation operation sequence.

[0064] Beneficial Effects: This invention relates to a simulation teaching system based on the integration of virtual reality and artificial intelligence, which has the following beneficial effects:

[0065] 1. Significantly improve virtual-real synchronization performance: Differentiated delay compensation is adopted for the characteristics of the forward and feedback channels. The dual-modal adaptive compensation of the forward channel reduces the tracking error by about 40%, and the GRU instruction prediction of the feedback channel controls the operation response delay within 15ms, solving the problem of insufficient accuracy of the unified compensation strategy.

[0066] 2. Constructing a high-fidelity digital twin: Through spatiotemporal fusion of multi-source sensors and real-time multibody dynamics calculation, the virtual model and the physical device achieve dual consistency in geometric appearance and dynamic behavior, thereby enhancing the realism of simulation teaching.

[0067] 3. Comprehensive protection of training safety: Adopting a three-level security verification chain of "performance-hardware-rules" and a rollback mechanism for abnormal interruptions, effectively avoiding equipment damage and safety accidents.

[0068] 4. Achieve closed-loop system evolution: Establish a closed loop of "simulation prediction - physical verification - feedback correction", and avoid catastrophic forgetting through incremental learning reinforced by experience playback and elastic weights. The system performance is continuously optimized as it is used.

[0069] 5. Provide personalized practical training guidance: Based on the results of multimodal fusion assessment, combined with learners' skill levels and operational characteristics, dynamically adjust teaching tasks and auxiliary strategies to achieve individualized instruction. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the overall architecture of the simulation teaching system based on the integration of virtual reality and artificial intelligence according to the present invention.

[0071] Figure 2 for Figure 1 The internal architecture diagram of the virtual-real synchronization mapping engine 200 specifically illustrates the composition and data flow of the forward channel synchronization sub-engine 210 and the feedback channel synchronization sub-engine 220. The virtual-real synchronization mapping engine 200 adopts a dual-engine asymmetric architecture. The left side is the forward channel synchronization sub-engine 210 (responsible for physical → virtual state synchronization), and the right side is the feedback channel synchronization sub-engine 220 (responsible for virtual → physical command synchronization). The two channels use completely independent latency compensation mechanisms.

[0072] Forward channel synchronization sub-engine 210 data flow direction:

[0073] Input: Real-time operational data from the target physics teaching equipment + first timestamp;

[0074] Network status monitoring unit 211: measures RTT(k), forward delay and jitter value in real time through timestamp probe packets;

[0075] Network mode determination unit 212: Determines stable / fluctuating modes based on delay threshold and rate of change threshold;

[0076] Adaptive gain calculation unit 213: calculates the reference gain or exponentially smoothed time-varying gain in submodal mode;

[0077] Data transmission strategy adjustment unit 214: dynamically adjusts the transmission rate and timing;

[0078] Output: Synchronize state data to a high-fidelity digital twin model;

[0079] Feedback channel synchronization sub-engine 220 data flow direction:

[0080] Input: Learner operation instructions from a high-fidelity digital twin model + a second timestamp;

[0081] Historical instruction sequence encoding unit 221: Normalizes and encodes historical instructions of length N;

[0082] Feedback delay measurement unit 222: Obtains real-time feedback delay from network status monitoring unit;

[0083] GRU instruction prediction model unit 223: Predicts compensation instructions based on the encoded sequence and delay value;

[0084] Compensation instruction generation unit 224: Generates the final compensation instruction and injects it into the physical device control loop;

[0085] Output: Compensation operation command to the control loop of the target physical teaching equipment;

[0086] Figure 3 A flowchart of a simulation teaching method provided in an embodiment of the present invention.

[0087] The attached diagrams are labeled as follows: Digital Twin Construction Module 100, Virtual-Real Synchronous Mapping Engine 200, AI Prediction and Evaluation Module 300, Security Verification and Deployment Module 400, Multimodal Synchronous Acquisition and Evaluation Module 500, Adaptive Adjustment Module for Teaching Strategies 600, Digital Twin Construction S1, Virtual-Real Two-Way Synchronous Establishment S2, Simulation Operation and Performance Prediction S3, Multi-Level Security Verification S4, Automatic Deployment and Physical Verification S5, Multimodal Evaluation and Feedback S6, and Teaching Strategies Adjustment S7. Detailed Implementation

[0088] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0089] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in further detail below with reference to the accompanying drawings.

[0090] Figure 1 This diagram illustrates the overall architecture of a simulation teaching system based on the fusion of virtual reality and artificial intelligence, as provided in an embodiment of the present invention. Figure 1As shown, the system in this embodiment mainly includes: a digital twin construction module 100, a virtual-real synchronization mapping engine 200, an AI prediction and evaluation module 300, a security verification and deployment module 400, a multimodal synchronous acquisition and evaluation module 500, and a teaching strategy adaptive adjustment module 600. Among them, the virtual-real synchronization mapping engine 200 is one of the core innovative modules of this invention, and will be discussed below in conjunction with... Figure 2 The innovative implementation of this module is described in detail. The modules are connected via a high-speed data bus, enabling real-time data flow and collaborative operation.

[0091] The basic workflow of the system is as follows: First, the digital twin construction module 100 constructs a high-fidelity digital twin model based on real-time sensing data from physical devices; then, the virtual-real synchronization mapping engine 200 establishes a bidirectional data channel between the physical devices and the virtual model, and achieves real-time synchronization through a differential compensation mechanism; learners perform simulation operations in the virtual environment, and the AI ​​prediction and evaluation module 300 performs performance prediction on the operation results; the security verification and deployment module 400 performs multi-level security verification on the simulation scheme, and automatically deploys it to the physical device for execution after passing the verification; the multimodal synchronous acquisition and evaluation module 500 collects multi-source data during the physical operation process and outputs the operation evaluation results, while feeding the evaluation results back to the AI ​​prediction and evaluation module 300 for model updates; the teaching strategy adaptive adjustment module 600 dynamically adjusts the training strategy based on the evaluation results, forming a complete teaching closed loop.

[0092] The digital twin construction module 100 is used to acquire real-time operating data and static attribute data of the target physical teaching equipment, and to construct a high-fidelity digital twin model based on the above data.

[0093] In this embodiment, the target physical teaching equipment is illustrated using an industrial robot as an example. However, those skilled in the art should understand that the present invention is also applicable to other types of training equipment such as CNC machine tools, chemical reaction devices, and power operation equipment.

[0094] Specifically, the digital twin building block 100 includes a multi-source sensor data acquisition unit 101, a data fusion and alignment unit 102, and a physics engine driving unit 103. The multi-source sensor data acquisition unit 101 acquires operational data in real time from various heterogeneous sensors deployed on the joints and end effector of the industrial robot. In this embodiment, the sensor configuration includes: laser displacement sensors installed on each joint for accurately measuring joint angles and displacements; an inertial measurement unit integrated into the robot arm body for acquiring attitude angles and angular velocities; a temperature sensor array distributed at the motor and reducer for monitoring the thermal state of key components; and a vibration accelerometer installed on the end effector for collecting vibration signals during operation.

[0095] After receiving the aforementioned multi-source heterogeneous data, the data fusion and alignment unit 102 first assigns a unified timestamp to each sensor data point and performs a unified transformation of the spatial coordinate system—mapping each sensor data point from its local coordinate system to a global coordinate system with the robot base as the origin, thus forming a fused device state vector. , where θᵢ is the angle of each joint, ω and α are the end angular velocity and acceleration respectively, T is the temperature vector of the key component, and V is the vibration amplitude vector.

[0096] The physics engine driver unit 103 is the core innovative part of this module. Unlike existing technologies that only construct geometrically identical digital twins based on CAD models, this unit inputs the fused device state vector S(t) and static attribute data (including geometric parameters, mass distribution, inertia matrix, joint friction coefficient, material elastic modulus, etc.) into a physics simulation engine based on a multibody dynamics model. The physics simulation engine uses the Lagrange dynamics equations for real-time solution:

[0097]

[0098] Where M(q) is the mass matrix, C(q,q̇) is the Coriolis force and centrifugal force matrix, G(q) is the gravity term, q is the generalized coordinate, and τ is the generalized driving force. Through real-time calculation in each synchronization cycle (set to 5ms in this embodiment), the physics engine synchronously updates the motion state, force distribution, and elastic deformation of the virtual model, making the virtual model not only "look like" a physical device, but also "behave like" a physical device—that is, under the same operational input, the dynamic response of the virtual model maintains physical consistency with the physical device.

[0099] The virtual-real synchronization mapping engine 200 is the core creative module of this invention, used to achieve bidirectional real-time synchronization between physical devices and virtual models. The following describes its application in conjunction with... Figure 2 The implementation of this engine is explained in detail.

[0100] Figure 2 A schematic diagram of the internal architecture of the virtual-real synchronization mapping engine 200 is shown. (For example...) Figure 2 As shown, the virtual-real synchronization mapping engine 200 includes a forward channel synchronization sub-engine 210 and a feedback channel synchronization sub-engine 220, which are responsible for synchronizing the physical device state to the virtual model (forward channel) and sending learner operation commands to the physical device (feedback channel), respectively. The two sub-engines adopt a completely asymmetric compensation strategy to compensate for the drastically different latency characteristics of the forward channel and the feedback channel.

[0101] (a) Forward Path Delay Compensation Mechanism

[0102] The latency in the forward pass is mainly manifested as the transmission delay that occurs during the process of acquiring, packaging, and transmitting sensor data from physical devices to the virtual model. This latency directly affects the tracking accuracy of the virtual model in relation to the actual state of the physical devices—the greater the latency, the greater the deviation between the state displayed on the virtual end and the actual state of the physical devices.

[0103] In this embodiment, the forward channel synchronization sub-engine 210 employs a dynamic adaptive compensation algorithm based on dual-modal network state determination. The core idea of ​​this algorithm is to first determine the modality of the current network state, and then adopt different compensation strategies based on the determination results, rather than using a fixed gain or a single strategy as in existing technologies. Specifically, it includes the following steps:

[0104] Step 1: Within each synchronization cycle (5ms), the forward channel synchronization sub-engine 210 sends a probe packet with its own timestamp to the physical device controller. Upon receiving the packet, the physical device controller immediately replies with a response packet containing both the receiving and sending timestamps, thereby calculating the current network round-trip time (RTT(k)) and forward channel transmission delay in real time. Feedback channel transmission delay and the jitter value of the forward channel delay. The jitter value is calculated using an exponential moving average: , where β is set to 0.7 to balance smoothness and responsiveness.

[0105] Step 2: Calculate the current rate of change of time delay And based on the forward channel transmission delay and the rate of change of time delay The network state mode is determined. In this embodiment, the delay threshold is used. The threshold for the rate of change is set to 20ms (based on experimental research, when the end-to-end delay exceeds 20ms, the virtual model tracking error begins to grow non-linearly). Set to 0.3. When and When the network is in a stable mode; when or At that time, the network is in a fluctuating mode.

[0106] Step 3: Modal calculation of adaptive gain. In the stable mode, a fixed reference gain α0 = 0.95 is used to fully utilize network resources for high-frequency synchronization. In the fluctuating mode, a time-varying gain based on exponential smoothing is introduced:

[0107]

[0108] The smoothing factor λ is set to 0.85, and the reference round-trip time RTT0 is set to 10ms. This gain strategy has the following creative advantages: (1) it avoids drastic jumps in gain through exponential smoothing; (2) The proportional structure allows the gain to automatically decrease when network latency increases and automatically increase when the network recovers, achieving a smooth adaptive match with the network state.

[0109] Step 4: Dynamic adjustment of the data transmission strategy. The data transmission rate Rs(k) and transmission timing Ts(k) for the current period are adjusted using the gain α(k):

[0110] ,

[0111] In fluctuating mode, by appropriately reducing the transmission rate and advancing the transmission timing, the data lag caused by the delay is compensated for by transmitting a small amount of redundancy, ensuring that the virtual terminal obtains the latest status of the physical device in a timely manner.

[0112] The latency in the feedback channel mainly manifests as the transmission delay that occurs during the process of transmitting operation commands issued by the learner in the virtual environment to the physical device controller via the network. This latency directly affects the real-time performance and accuracy of the operation response. Especially in scenarios involving delicate operations, a delay of tens of milliseconds can lead to operation overshoot or device collision.

[0113] In existing technologies, Smith predictors are typically used for compensation, but the object of compensation is the "delay value," i.e., "predict the delay first, then calculate the compensation." This indirect method suffers from the problem of prediction error accumulation. This embodiment creatively proposes to skip the delay prediction stage and directly predict the compensation command value using the operation command sequence as input and the compensation command required at the current moment as output, utilizing a GRU (Gated Recurrent Unit) network.

[0114] The specific implementation of the feedback channel synchronization sub-engine 220 includes the following steps:

[0115] Step 1: Acquisition and Encoding of Historical Instruction Sequences. Acquire the sequence of operation instructions of length N prior to the current time t. In this embodiment, N is set to 16 to cover the effective historical window of 200-300ms. Each operation instruction It includes the instruction type (joint movement, end effector operation, tool switching, etc.), instruction parameter vector (target position, velocity, acceleration, etc.), and timestamp. The instruction parameter vector is subjected to Min-Max normalized encoding to form a standardized input feature sequence X(t).

[0116] Step 2: Real-time acquisition of feedback channel transmission delay. The current feedback channel transmission delay is measured in real time using the aforementioned probe packet mechanism. This delay value is directly used as the prediction step size of the GRU prediction model.

[0117] Step 3: The network structure design of the GRU prediction model is as follows: the input layer has a dimension of 16×M, where M is the feature dimension after encoding a single instruction; the GRU hidden layer contains 128 hidden units; the fully connected output layer maps the hidden states to the instruction parameter space. The calculation process of the GRU hidden layer is as follows:

[0118] Update Gate:

[0119] Reset Door: Candidate hidden state:

[0120] Hidden status update:

[0121] The GRU gating mechanism has unique structural advantages in this application scenario: for sudden changes in the instruction sequence (such as the learner suddenly changing the direction of operation), the gate is reset. It can quickly suppress the influence of historical hidden states, enabling the model to respond rapidly to operational changes; for continuous, stationary operational instructions, the update gate... It can maintain long-term dependence on hidden states and preserve the smoothness of predictions. This adaptive memory-forgetting mechanism can better balance the rapid responsiveness and smooth stability of predictions.

[0122] Step 4: Obtain the hidden state of the GRU hidden layer at the end of the sequence. The compensated instruction prediction value is generated by mapping through the fully connected output layer:

[0123]

[0124] Here, fout is the output mapping function, which selects either a linear mapping or a Sigmoid / Tanh mapping based on the actual value range of each instruction parameter. The key difference between this prediction and conventional inference is that the prediction step size is variable in real time. Instead of a fixed value, the GRU model employs a multi-condition data augmentation strategy during the training phase—simulating multiple latency conditions for each training sample. Generate corresponding prediction labels for each delay condition. This allows the model to learn the instruction evolution patterns under different delay step sizes. The training loss function is the mean squared error:

[0125]

[0126] Where K is the batch sample size.

[0127] Step 5: Inject the predicted compensation command into the physical device control loop, replacing the original command that is delayed due to network latency, so that the physical device can receive the learner's operational intention at the current moment. Figure 1 The compensation command is designed to offset the impact of feedback channel delay on the real-time performance of the operation.

[0128] The bidirectional asynchronous compensation architecture of the forward channel and the feedback channel has the following technical effects compared with the existing technology: (1) The "dual-mode determination + adaptive gain adjustment" of the forward channel ensures that the virtual model always closely tracks the physical device status under complex network conditions. Experimental tests show that the tracking error is reduced by about 40% compared with the fixed gain scheme under the condition of delay fluctuation (10-80ms); (2) The "GRU direct command prediction + variable step size" of the feedback channel reduces the operation response delay to less than 15ms, which is significantly better than the traditional Smith predictor scheme; (3) The two channels adopt completely different compensation mechanisms, each taking what it needs and avoiding its own shortcomings, thus achieving the optimization of the overall synchronization performance.

[0129] This system was compared and tested on a typical industrial robot training platform (latency range 10-80ms, standard deviation of fluctuation 15ms):

[0130] Forward channel: Compared with the fixed gain (α=0.9) scheme, the maximum tracking error of this scheme in the fluctuating mode is reduced from 42mm to 25mm (a reduction of 40.5%), and the root mean square error is reduced from 18.3mm to 10.7mm (a reduction of 41.5%).

[0131] Feedback Channel: Compared with the Smith predictor scheme, this scheme reduces the operation response perception latency from 98ms to 14ms under an 80ms delay condition, and reduces the instruction prediction mean square error by 62%.

[0132] Closed-loop effect: After 10 rounds of incremental learning, the correlation coefficient between simulation prediction and actual performance increased from 0.71 to 0.89.

[0133] The AI ​​prediction and evaluation module 300 is used to predict and evaluate the performance of the simulation operation scheme after the learner completes a round of simulation operation.

[0134] In this embodiment, the AI ​​prediction and evaluation module 300 employs a Transformer temporal prediction model based on a multi-head attention mechanism. The model input includes two parts: the instruction sequence of the learner's current simulation operation (processed into an operation embedding sequence through position encoding and embedding layers), and the current state vector of the high-fidelity digital twin model at the end of the operation. The multi-head attention encoder performs multi-head self-attention calculation on the above input, automatically extracting the correlation features between the simulation operation and the device state. The Transformer temporal prediction model adopts an encoder-decoder architecture. The encoder consists of six identical encoder layers stacked together. Each encoder layer contains a multi-head self-attention sublayer and a feedforward neural network sublayer. Residual connections and layer normalization operations are applied after each sublayer. The multi-head self-attention sublayer has eight attention heads, each with an embedding dimension of 64, enabling the model to jointly extract diverse correlation features between the operation instruction sequence and the twin model state vector from different representation subspaces. The feedforward neural network sublayer contains two linear transformations, with a hidden layer dimension of 2048, and uses the ReLU activation function to perform nonlinear transformations and dimension mapping on the features extracted by the attention layer. The decoder also consists of six identical decoder layers stacked together. Each decoder layer adds an encoder-decoder attention sublayer to the encoder structure, used to interact the associated features extracted by the encoder with the currently generated prediction state of the decoder. The input to the decoder is a learnable query embedding vector with the same dimension as the encoder output. In each decoder layer, the first multi-head self-attention sublayer uses a masking operation to ensure that when predicting the i-th performance metric, it can only rely on the results of the first i-1 generated metrics, maintaining the autoregressive nature of the prediction. The decoder output is followed by four parallel fully connected output heads. Each output head contains two hidden layers (with dimensions of 128 and 64, respectively) and a linear output layer, corresponding to the numerical prediction of stability, energy consumption, accuracy, and security metrics, respectively. Each output head is independent and can perform differentiated learning based on the numerical range and distribution characteristics of different metrics. The performance prediction decoder outputs predicted values ​​of four dimensions of performance indicators in parallel based on the extracted correlation features: equipment operation stability indicators (comprehensively evaluated by vibration amplitude standard deviation and resonant frequency offset), energy consumption indicators (calculated by integrating the driving torque of each joint over time), accuracy indicators (root mean square value of the deviation between the end effector trajectory and the target trajectory), and safety indicators (the proportion of key component temperatures close to the rated temperature and the proportion of joint torques close to the limit torque).

[0135] The Transformer model is trained in a supervised manner during the offline phase using historical simulation operation data and corresponding physical verification performance data. The specific training process is as follows:

[0136] Step D1: Training Dataset Construction. Collect multiple rounds of complete "simulation operation-physical verification" closed-loop data. Each round of data includes: the learner's simulation operation instruction sequence (time series length L, L is 128 in this embodiment), the state vector of the digital twin model at the end of the operation (dimension 32), and the measured values ​​of equipment operation stability, energy consumption, accuracy, and safety indicators from the physical verification phase. Organize the above data into training sample pairs, where the input is (operation instruction sequence, twin state vector), and the labels are the measured values ​​of the four performance indicators. After data cleaning and outlier removal, at least 10,000 valid samples are obtained. These are divided into training, validation, and test sets in an 8:1:1 ratio.

[0137] Step D2: Data Preprocessing and Feature Engineering. Discrete command types in the operation command sequence are One-Hot encoded, and continuous parameters (such as velocity and position) are Z-Score standardized to ensure each feature dimension has zero mean and unit variance. The physical quantities (joint angles, angular velocities, torques, etc.) in the twin state vector are also Z-Score standardized. The standardized operation command sequence is then sequentially input into the position encoding layer to generate an operation embedding sequence carrying temporal position information, which serves as the encoder input. The twin state vector and the operation embedding sequence are concatenated along the sequence dimension to form a unified input tensor with dimension [missing information]. D op To manipulate the embedded dimension, D state The dimension is the state vector.

[0138] Step D3: Loss Function and Optimizer Configuration. A combined loss function is used, which includes the sum of the independent mean squared errors (MSEs) of the four performance metrics, and introduces an L2 regularization term for the predicted values ​​of each metric to prevent overfitting. The loss function is in the form of:

[0139]

[0140] Where B is the batch size. and Let be the true value and the predicted value of the m-th indicator for the i-th sample, respectively; λ be the regularization coefficient (set to 0.0001); P be the number of model parameters; and θ be the predicted value. j Let be the j-th model parameter. The Adam optimizer is used for parameter updates, with an initial learning rate of 0.0001, β1=0.9, β2=0.999, and epsilon=1e-8. A Warmup strategy is used, where the learning rate linearly increases from 0 to 0.0001 in the first 1000 training steps, then decays exponentially to 1e-6 at a rate of 0.98 per 1000 steps.

[0141] Step D4: Iterative Training and Early Stopping Strategy. Set batch size B=32 and maximum training epochs to 200. At the end of each epoch, calculate the average loss on the validation set (validation loss). When the validation loss does not decrease for 15 consecutive epochs, trigger the early stopping mechanism, terminate training, and save the model parameters at the point where the validation loss is lowest. During training, a gradient pruning strategy is used to limit the L2 norm of the gradient to within 1.0 to prevent gradient explosion.

[0142] Step D5: Model Evaluation and Deployment. After training, evaluate the model performance on an independent test set, calculating the coefficient of determination (R²), mean absolute percentage error (MAPE), and root mean square error (RMSE) between the predicted and actual values ​​for four performance metrics. When the model achieves R² > 0.85 and MAPE < 15% for each metric on the test set, the model is considered to have acceptable prediction accuracy. The model parameters that meet the performance requirements are then fixed and loaded into the AI ​​prediction and evaluation module for real-time performance prediction of new operational schemes during the simulation phase. Multi-dimensional evaluation results are output for use by the safety verification and deployment module 400.

[0143] Step D6: Online Update Mechanism. During actual teaching operation, after each round of the "simulation prediction-physical verification" closed loop, the system automatically stores newly generated samples (simulation input, measured performance) into the incremental sample library. When the incremental sample library accumulates 256 new samples, an online incremental update is triggered. During the update, an equal number of memory samples are randomly sampled from the original training set and mixed with the new samples to construct an updated training set. The model parameters are fine-tuned with a small learning rate of 0.00001, and an Elastic Weight Consolidation (EWC) regularization term is used to protect important parameters, ensuring that the model does not forget its predictive ability for historical scenarios while adapting to new data. After training convergence, the model can predict the performance of new operation schemes during the simulation phase, outputting multi-dimensional evaluation results for use by the safety verification and deployment module 400.

[0144] The security verification and deployment module 400 is used to perform security verification on the simulation operation plan and automatically deploy it to the physical device for execution after the verification is passed.

[0145] This embodiment innovatively introduces a "multi-level safety verification chain" mechanism. The first level is performance qualification verification: each performance indicator output by the AI ​​prediction and evaluation module 300 must meet its corresponding preset qualification threshold (e.g., accuracy deviation ≤ 0.5mm, temperature margin ≥ 20%). The second level is hardware constraint verification: it checks whether the position, velocity, and acceleration commands in the simulation operation command sequence exceed the hardware travel and motion capability limits of each joint of the physical device. The third level is safety rule verification: a series of risk rules are predefined in the safety rule base, such as "the end effector must not enter the safety restricted area below the worktable" and "the angle between any two joints during movement must not be less than 15°," etc. The system verifies each operation command sequence to see if it triggers any of these rules.

[0146] Only when all three levels of verification pass will the automated deployment control unit compile the simulation operation command sequence into an executable command package for the physical device controller and send it to the local device controller via a secure communication protocol. During the physical operation verification process, the abnormal interruption rollback unit continuously monitors the device's operating parameters. Once any parameter deviates from the preset safety boundary (such as a joint torque instantaneously exceeding 120% of the rated value, or a temperature rise rate exceeding 5°C / s), an emergency stop command is immediately sent to the device controller via an emergency signal line independent of the main communication link. Simultaneously, the digital twin model is rolled back to the state before the safety verification passed, ensuring that the simulation starting point for subsequent operations is correct.

[0147] The multimodal synchronous acquisition and evaluation module 500 is used to synchronously acquire multi-source sensor data and perform operational evaluation during physical operation verification of physical devices.

[0148] The multimodal fusion model in this embodiment adopts a two-level fusion architecture. At the feature-level fusion layer, temporal features (mean, variance, peak value, etc.), frequency domain features (main frequency components of the Fourier transform amplitude spectrum), and time-frequency domain features (wavelet transform coefficients) are extracted from IoT sensor data (vibration, temperature, displacement, etc.). Simultaneously, spatial-temporal features are extracted from the operation process video data acquired by high-speed cameras using a pre-trained 3D convolutional neural network. Then, canonical correlation analysis is used to project the two types of heterogeneous features onto a shared subspace for feature-level fusion. At the decision-level fusion layer, the fused feature vector is input into the operation stability scoring subnetwork, accuracy scoring subnetwork, and compliance scoring subnetwork, respectively. Each subnetwork independently outputs a score and confidence level. Finally, Dempster-Shafer evidence theory is used to weight and synthesize the scores from each subnetwork to generate a comprehensive operation stability score, accuracy score, and compliance score.

[0149] More creatively, this module feeds back the operational evaluation results to the AI ​​prediction evaluation module 300, forming a closed loop of "simulation prediction → physical verification → feedback correction". To prevent the model from forgetting old knowledge during updates, this embodiment adopts an online incremental learning mechanism based on experience replay and elastic weight consolidation: newly generated training samples (simulation input-real performance pairs) are stored in the experience replay buffer; when the update cycle arrives, a batch of samples is randomly sampled from the buffer to calculate the prediction loss; when updating model parameters, an elastic weight consolidation regularization term is introduced. This regularization term quantifies the importance of each parameter to the historical task based on the Fisher information matrix, imposing greater update constraints on important parameters, effectively avoiding catastrophic forgetting.

[0150] The adaptive instruction strategy adjustment module 600 dynamically adjusts instruction strategies based on learners' assessment performance. This module receives the operational assessment results output from the multimodal assessment module 500 and calculates the learner's current skill mastery index.

[0151]

[0152] The weights w1, w2, and w3 can be configured according to the focus of the training task (e.g., w2 is 0.5 in the precision assembly task and w3 is 0.5 in the safe operation task).

[0153] Simultaneously, the module analyzes 300 historical simulation operation sequences recorded by the AI ​​prediction and evaluation module, extracting skill improvement rate (the slope of the mastery index fitting over three consecutive rounds of evaluation) and operational style characteristics (operation speed preference, path selection preference, etc.). Based on this multi-dimensional learner profile, the system matches the optimal strategy from a pre-set teaching strategy knowledge base, dynamically adjusting the difficulty level, task objective parameters, and auxiliary prompt frequency of subsequent simulation teaching tasks to achieve personalized training guidance tailored to each learner. For example, for learners with high mastery but slow operation speed, the system appropriately increases the time constraint of the task to encourage them to improve operational efficiency; for learners with low mastery and frequent safety risks, the system reduces the difficulty and increases the frequency of auxiliary prompts to ensure that they gradually improve their skills under safe conditions.

[0154] The following is combined Figure 3 The flowchart shown illustrates the workflow of the system in this embodiment, using a complete practical training scenario as an example:

[0155] Step S1: Digital Twin Construction. After the industrial robot is powered on, the digital twin construction module 100 collects real-time operating data through sensor arrays deployed at each joint and end effector. After data fusion and spatiotemporal alignment, the data is input into the multibody dynamics physics engine to construct a high-fidelity digital twin model.

[0156] Step S2: Establishing Bidirectional Synchronization Between the Physical and Virtual Models. The virtual-physical synchronization mapping engine 200 establishes a bidirectional synchronization channel between the physical robot and the digital twin model. The forward channel uses a dual-modal adaptive compensation algorithm to synchronize the real-time state of the physical robot to the virtual model; the feedback channel uses a GRU instruction prediction compensation algorithm to send the learner's operation instructions to the physical robot controller in real time.

[0157] Step S3: Simulation Operation and Performance Prediction. The learner completes a welding path planning operation in a virtual environment. After the operation, the Transformer model of the AI ​​prediction and evaluation module 300 is input with the operation instruction sequence and the final state of the digital twin model, and outputs the predicted values ​​of four indicators: stability, energy consumption, accuracy, and safety in parallel.

[0158] Step S4: Multi-level safety verification. The safety verification and deployment module 400 performs three levels of verification on the simulation scheme: performance qualified → hardware constraints → safety rules. Assuming the predicted accuracy of this operation is 0.3mm (qualified threshold 0.5mm), the range of motion of each joint is within the hardware constraints, and no safety rules are triggered, the verification passes.

[0159] Step S5: Automatic Deployment and Physical Verification. After successful verification, the system automatically deploys the sequence of operation instructions to the physical robot, which then performs the physical welding operation according to the simulation plan. During this process, the exception interruption rollback unit monitors the operating parameters in real time.

[0160] Step S6: Multimodal Evaluation and Feedback. The multimodal synchronous acquisition and evaluation module 500 collects physical operation data through vibration sensors, temperature sensors, and a high-speed camera. The two-level fusion model outputs operational stability scores of 4.2, accuracy scores of 4.5, and compliance scores of 4.8 (out of 5). Simultaneously, this evaluation result is packaged with the simulation input from Step S3 as training samples and fed back to the AI ​​prediction and evaluation module 300, which updates the Transformer model parameters through an elastic weight consolidation mechanism.

[0161] Step S7: Adjustment of Teaching Strategies. The adaptive adjustment module 600 of teaching strategies determines that the learner has the conditions for advanced training based on the current mastery index and historical improvement rate. The next task is adjusted from "straight weld seam welding" to "fillet weld seam welding", and the frequency of auxiliary prompts is appropriately reduced to encourage independent operation.

[0162] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A simulation teaching system based on the integration of virtual reality and artificial intelligence, characterized by: include: The digital twin building module is used to construct a high-fidelity digital twin model based on the real-time operating data and static attribute data of the target physical teaching equipment. The virtual-real synchronization mapping engine is used to establish a two-way data channel between the target physical teaching equipment and the high-fidelity digital twin model, and to realize real-time two-way synchronization between the physical state and the virtual state. The AI ​​prediction and evaluation module and the security verification and deployment module are used to perform performance prediction and evaluation on the simulation operation, and automatically deploy the operation instructions to the target physical teaching equipment after the qualification is determined. The multimodal synchronous acquisition and evaluation module is used to synchronously acquire multimodal data and output operation evaluation results during physical operation verification, and to feed the operation evaluation results back to the AI ​​prediction evaluation module. The specific method by which the virtual-real synchronization mapping engine achieves real-time bidirectional synchronization includes: on the target physical teaching device side, acquiring real-time operating data of the device and attaching a first timestamp, transmitting it to the high-fidelity digital twin model via the forward channel to drive virtual state updates; on the high-fidelity digital twin model side, acquiring learner operation instructions and attaching a second timestamp, transmitting them to the target physical teaching device via the feedback channel to drive the physical device to perform actions; the forward channel adopts a forward delay compensation algorithm based on dynamic adaptive compensation, dynamically adjusting the sending strategy of physical state data to the virtual end according to the real-time monitored network round-trip latency and forward channel transmission delay, so that the virtual state is consistent with the physical state; the feedback channel adopts a feedback delay compensation algorithm based on operation instruction sequence prediction, predicting the compensation instruction at the current moment according to the historical operation instruction sequence, and injecting it into the control loop of the target physical teaching device to offset the impact of feedback channel transmission delay on control real-time performance.

2. The simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that, In the virtual-real synchronization mapping engine, the forward delay compensation algorithm based on dynamic adaptive compensation used in the forward channel includes the following steps: Step A1: During each synchronization cycle, the virtual-real synchronization mapping engine monitors and records the following network status parameters in real time by sending timestamped probe packets: Network round-trip time (RTT(k)) and forward transmission delay during the current synchronization period Feedback channel transmission delay and the jitter value of the forward channel delay. , where k is the synchronization period index; Step A2: Calculate the delay change rate of the current synchronization period based on the network state parameters. : And based on the forward channel transmission delay With the preset delay threshold The comparison relationship, and the said time delay change rate Compared with the preset rate of change threshold The comparison relationship determines whether the current network state is a stable mode or a fluctuating mode: when and When this condition is met, it is determined to be a stable mode; when or When this occurs, it is determined to be a wave mode; Step A3: Based on the determined network state mode, calculate the adaptive compensation gain for the current synchronization period using different gain calculation strategies. : When the network state is in a stable mode, the first gain calculation strategy is adopted: in The preset reference gain value, ; When the network state is in a fluctuating mode, a second gain calculation strategy is adopted, introducing a time-varying adaptive gain based on exponential smoothing: in: As a smoothing factor, This is used to control the smoothness of gain updates; This is a preset reference round-trip time base value; This represents the network round-trip time during the current synchronization period. This is the compensation gain value from the previous synchronization cycle; Step A4: Utilize the adaptive compensation gain The strategy for dynamically adjusting the transmission of physical state data to the virtual terminal is as follows: the virtual-physical synchronization mapping engine adjusts the transmission strategy based on the adaptive compensation gain. Determine the data transmission rate for the current synchronization period. and timing of sending : ； in: This is the maximum data transmission rate; This represents the start time of the current synchronization cycle; This allows for normal synchronization with a reference gain in stable mode, and compensates for increased latency caused by network fluctuations by adaptively reducing the data transmission rate and advancing the transmission timing in fluctuating mode.

3. The simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that, In the virtual-real synchronization mapping engine, the feedback channel employs a feedback delay compensation algorithm based on operation instruction sequence prediction, which includes the following steps: Step B1: Obtain the sequence of historical operation instructions of length N before the learner's current time t. Each operation instruction It includes instruction type, instruction parameter vector, and timestamp information, and performs normalized encoding on the instruction parameter vector of each operation instruction in the historical operation instruction sequence to form an input feature sequence. ; Step B2: Obtain the current feedback channel transmission delay. The transmission delay of the feedback channel is obtained in real time by the virtual-real synchronization mapping engine through timestamp comparison; Step B3: Input the input feature sequence X(t) into a pre-trained gated recurrent unit (GRU) prediction model. The GRU prediction model includes a GRU hidden layer and a fully connected output layer connected in sequence. The calculation process of the GRU hidden layer at each time step i includes: Update gate calculation: Reset door calculation: Candidate hidden state calculation: Hidden status update: in: This is the input feature vector for the current time step; This is the hidden state vector from the previous time step; The sigmoid activation function is used; ⊙ indicates element-wise multiplication. This is the weight matrix. The bias vector; Step B4: Predicted output of the compensation instruction at the current moment; hidden state of the gated loop unit's hidden layer at the end of the sequence. Perform transmission delay via the feedback channel Forward prediction of the prediction step size to generate the predicted value of the compensation command at the current time step. The specific prediction method is as follows: the fully connected output layer uses... As input, it is mapped as follows: in To output the weight matrix, To output the bias vector, The output mapping function is selected as either a linear mapping function or a nonlinear activation function based on the parameter range of the operation instruction. The gated recurrent unit prediction model transmits delays via the feedback channel during the training phase. To predict the step size, supervised learning is performed using a training dataset containing operation instruction sequence samples under time-varying delay conditions to minimize the predicted instruction. With real instructions The mean square error between them is the training target; Step B5: Calculate the predicted value of the compensation command. The control loop of the target physical teaching equipment is injected to offset the impact of the feedback channel transmission delay on the real-time performance of the control.

4. The simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 3, characterized in that, The training process of the gated recurrent unit prediction model includes: Step C1: Under various network latency conditions, collect historical operation command sequence samples of learners performing operations in the virtual simulation environment, and simultaneously record the feedback channel transmission latency at each moment. For each training sample, the sequence of operation instructions of length N preceding the current time t is used as the input feature, and the sequence of instructions at time t is used as the input feature. The actual operation instructions are used as prediction labels; Step C2: For the collected historical operation command sequences, use a sliding window of length N to extract continuous command sequence segments in the time dimension to form a training sample set. ,in Let j be the input feature sequence of the j-th sample. For the corresponding predicted labels; Step C3: For each training sample in the training sample set, simulate multiple feedback channel transmission delay conditions. Generate corresponding prediction labels for each delay condition. This forms an expanded training sample; through multi-condition data augmentation, the gated loop unit prediction model learns the instruction evolution rules under different delay step sizes, thereby improving the model's adaptive prediction capability for time-varying delays. Step C4: Using the mean squared error (MSE) between the predicted and actual instructions as the loss function, the Adam optimizer is applied to the weight matrix of the gated recurrent unit prediction model. and bias vector Perform iterative optimization training until the loss function converges to below a preset threshold or reaches a preset number of training rounds.

5. A simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that: The digital twin construction module includes a multi-source sensor data fusion unit and a physics engine driving unit. The multi-source sensor data fusion unit performs spatiotemporal alignment and fusion of heterogeneous data collected by laser displacement sensor, inertial measurement unit, temperature sensor and vibration accelerometer. The physics engine driving unit inputs the fused device state vector and static attribute data into the multibody dynamics model to solve the motion, force and deformation state of the high-fidelity digital twin model in real time, so that the virtual model and the physical device maintain consistent physical behavior.

6. The simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that: The deep learning model in the AI ​​prediction and evaluation module is the Transformer temporal prediction model. Its multi-head attention encoder extracts associated features from the simulation operation sequence and the digital twin state vector, and outputs the predicted values ​​of multiple performance indicators such as device operation stability, energy consumption, accuracy and safety through the performance prediction decoder in parallel.

7. The simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that: The security verification and deployment module includes a multi-level security verification unit and an abnormal interruption rollback unit. After the simulation performance is qualified, the multi-level security verification unit further performs hardware constraint verification and security rule verification on the operation instruction sequence. Automatic deployment is only permitted when all verifications pass. When the abnormal interruption rollback unit detects that the equipment operating parameters deviate from the preset safety boundary during the physical operation verification process, it immediately sends an emergency stop command and controls the high-fidelity digital twin model to roll back to a safe state.

8. A simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that: The multimodal fusion model in the multimodal synchronous acquisition and evaluation module adopts a two-level fusion architecture: the feature-level fusion layer uses canonical correlation analysis to project the time-frequency domain features of IoT sensors and the spatial-temporal features of visual sensors to a shared subspace for fusion; the decision-level fusion layer uses Dempster-Shafer evidence theory to weight and synthesize the independent outputs of the three scoring networks of operational stability, accuracy and compliance to generate the final evaluation result.

9. A simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that: When the multimodal synchronous acquisition and evaluation module feeds back the operation evaluation results to the AI ​​prediction evaluation module, it adopts an online incremental learning mechanism based on experience replay and elastic weight consolidation. New training samples are stored in the experience replay buffer, and elastic weight consolidation regularization terms are applied to the parameters when the model is updated to retain the predictive ability for historical simulation scenarios.

10. A simulation teaching system based on the integration of virtual reality and artificial intelligence according to claim 1, characterized in that: It also includes an adaptive adjustment module for teaching strategies, which calculates the learner's current skill mastery index based on the operation evaluation results, and dynamically adjusts the difficulty level, task objective parameters, and auxiliary prompt frequency of the simulation teaching task by combining the skill improvement rate and operation style characteristics extracted from the historical simulation operation sequence.

Images