Virtual experiment environment guidance information generation method and system, terminal and medium

Abstract

The invention relates to the field of experiment guidance, and particularly provides a virtual experiment environment guidance information generation method and system, a terminal and a medium, and the method comprises the steps: capturing operation data of a student in a virtual experiment environment in real time to extract operation intention features, carrying out the compliance check of parameter setting, and carrying out the recognition and judgment of an equipment state and an experiment result, the method comprises the following steps: firstly, carrying out scene analysis on a scene, forming a uniform situation representation vector, carrying out rule matching through a predefined rule based on the vector, predicting and identifying operation errors, logic errors, safety violations and potential experiment risks through a machine learning model, and outputting a final diagnosis result by adopting a conflict resolution mechanism; and according to the error type, the risk level, the historical performance of the student and the learning preference, dynamically generating and outputting guidance information in a text, vision, voice or animation form. According to the invention, the accuracy and comprehensiveness of error recognition are improved, the safety protection capability of the experiment process is enhanced, and real-time and accurate guidance of student operation is realized.

Description

Technical Field

[0001] This application relates to the field of experimental guidance, specifically to a method, system, terminal, and medium for generating guidance information for a virtual experimental environment. Background Technology

[0002] While existing virtual experiment systems primarily focus on simulating experimental operations and visualizing results, achieving some degree of automated management of the experimental process, they still fall short in real-time analysis and intelligent guidance of student behavior. These systems rely on pre-set static rule bases for error detection, lacking the ability to identify hidden errors in complex operational sequences, particularly limiting their effectiveness in detecting non-explicit issues such as logical errors and incorrect operation sequences. Secondly, most solutions employ a single judgment mechanism, either entirely based on rule-based reasoning or entirely reliant on machine learning models, making it difficult to balance rule clarity and model generalization, resulting in a trade-off between error detection coverage and accuracy. Furthermore, feedback mechanisms are rigid, typically providing only simple correct / incorrect prompts or standard answer displays, failing to generate explanatory and targeted guidance based on individual student differences, and lacking proactive warnings of operational risks. Finally, these systems generally lack effective multimodal data fusion capabilities, making it difficult to construct a complete understanding of the experimental context from multi-dimensional data such as operational behavior, parameter settings, and equipment status, thus limiting the accuracy of their analysis and judgment. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides a method, system, terminal, and medium for generating guidance information in a virtual experimental environment. By combining rules and machine learning with multimodal personalized feedback, it improves the accuracy and comprehensiveness of error identification, enhances the safety protection capabilities of the experimental process, and enables real-time and precise guidance for student operations.

[0004] In a first aspect, the technical solution of the present invention provides a method for generating guidance information for a virtual experimental environment, comprising the following steps: Real-time capture of students' operational behavior sequences, experimental parameter settings, device status data, and interaction events in the virtual experimental environment, along with timestamp information; The operation behavior sequence is modeled, operation intention features are extracted, the parameter settings are checked for compliance, the equipment status and experimental results are identified and judged, and multi-source data are integrated to form a unified context representation vector. Based on a unified contextual representation vector, rule matching is performed using predefined rules in a rule knowledge base. Simultaneously, prediction is performed using a machine learning model to identify operational errors, logical errors, safety violations, and potential experimental risks. Finally, a conflict resolution mechanism is used to output the diagnostic results. Based on error type, risk level, student's historical performance, and learning preferences, guidance information in text, visual, audio, or animation formats is dynamically generated and output.

[0005] Secondly, the technical solution of the present invention provides a virtual experimental environment guidance information generation system, comprising: The operation data capture module is used to capture in real time the sequence of students' operation behaviors, experimental parameter settings, equipment status data and interaction events in the virtual experimental environment, and to include timestamp information. The scenario representation vector generation module is used to model the operation behavior sequence, extract operation intention features, perform compliance checks on parameter settings, identify and judge the device status and experimental results, and integrate multi-source data to form a unified scenario representation vector. The diagnostic result generation module is used to perform rule matching based on a unified context representation vector and predefined rules in a rule knowledge base. At the same time, it uses a machine learning model to make predictions, identify operational errors, logical errors, safety violations and potential experimental risks, and output the final diagnostic results using a conflict resolution mechanism. The guidance information display module is used to dynamically generate and output guidance information in the form of text, visuals, voice, or animation based on error type, risk level, student's historical performance, and learning preferences.

[0006] Thirdly, the technical solution of the present invention provides a terminal, comprising: The memory is used to store the program that generates guidance information for the virtual experimental environment; A processor, configured to implement the steps of the virtual experimental environment guidance information generation method as described above when executing the virtual experimental environment guidance information generation program.

[0007] Fourthly, the present invention provides a computer-readable storage medium storing a virtual experimental environment guidance information generation program, wherein the virtual experimental environment guidance information generation program, when executed by a processor, implements the steps of the virtual experimental environment guidance information generation method as described in any of the above claims.

[0008] As can be seen from the above technical solutions, this application has the following advantages: First, by combining rule matching and machine learning prediction mechanisms with conflict resolution algorithms, it can effectively identify explicit operational errors and implicit logical risks, improving the comprehensiveness and accuracy of error diagnosis; second, based on dynamically generated multimodal guidance information, it can provide real-time, specific, and actionable personalized feedback according to error type, risk level, student's historical performance, and learning preferences; third, by integrating multi-source data and conducting in-depth analysis, it can identify potential experimental risks and safety violations in advance and issue timely warnings, ensuring the safety and reliability of the virtual experiment process. Attached Figure Description

[0009] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of a method for generating virtual experimental environment guidance information according to an embodiment of the present invention.

[0011] Figure 2 This is a schematic block diagram of a virtual experimental environment guidance information generation system provided in an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0015] Figure 1 This is a schematic flowchart illustrating a method for generating guidance information for a virtual experimental environment, as provided in an embodiment of the present invention. Figure 1 The executing entity can be a virtual experimental environment guidance information generation system. The virtual experimental environment guidance information generation method provided in this embodiment of the invention is executed by a computer device, and correspondingly, the virtual experimental environment guidance information generation system runs on the computer device. Depending on different needs, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0016] like Figure 1 As shown, the method includes the following steps.

[0017] S1 captures in real time the sequence of students' operational behaviors, experimental parameter settings, device status data, and interactive events in the virtual experimental environment, along with timestamp information.

[0018] S2, model the operation behavior sequence, extract operation intention features, check the compliance of parameter settings, identify and judge the equipment status and experimental results, and integrate multi-source data to form a unified context representation vector.

[0019] S3, based on a unified contextual representation vector, performs rule matching through predefined rules in a rule knowledge base, while using a machine learning model for prediction to identify operational errors, logical errors, safety violations, and potential experimental risks, and outputs the final diagnostic results using a conflict resolution mechanism.

[0020] S4 dynamically generates and outputs guidance information in the form of text, visuals, voice, or animation based on error type, risk level, student's historical performance, and learning preferences.

[0021] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another method for generating virtual experimental environment guidance information is provided, which includes the following steps.

[0022] S101, capture experimental operation data.

[0023] Specifically, it captures students' operational data in the virtual experimental environment in real time, including sequences of operational behaviors, experimental parameter settings, device status data, and interactive events, along with timestamp information.

[0024] An event listener integrated into the virtual experiment platform monitors and records all student interactions with virtual experiment interface elements in real time. The sequence of these interactions is recorded as a time-series data stream, with each event including at least: operation type (e.g., click, drag, select, combine, connect, open, close, etc.), the specific object identifier to which the operation was applied (e.g., specific instrument number, reagent bottle ID, switch button ID), and the timestamp of the operation. For example, the sequence could be: [t1: click (alcohol lamp), t2: drag (measuring cylinder, from position A to position B), t3: select (1M HCl solution),...].

[0025] By reading real-time values ​​from interactive components such as virtual instrument panels, input boxes, sliders, and knobs, the system captures various experimental parameters set by students. These parameters include, but are not limited to, the numerical values ​​of physical quantities (such as temperature, voltage, pressure, volume, and mass), the concentration and dosage of chemical reagents, and the incubation time of biological samples. The units of these parameters are also recorded to ensure the accuracy of subsequent compliance checks. For example, the system captures whether a student sets the heating temperature to 150°C or measures the solution volume as 50.0 mL.

[0026] The simulation engine's application programming interface (API) periodically polls or triggers events to obtain the real-time operating status of various instruments and equipment in the virtual experimental environment. This equipment status data includes: on / off status, operating mode (e.g., heating, cooling, stirring), real-time readings (e.g., current temperature, current value, pH value), and alarm status (e.g., overheating, overpressure). For example, it might capture data such as "the induction cooker is on, current power is 500W" or "the pH meter current reading is 4.5".

[0027] The system captures student interactions with the tutoring system itself, reflecting the student's learning status and intentions. These interactions include: actively clicking the "Help" button, interrupting an experiment using the "Pause" function, triggering the "Review" function to review previous standard operation animations, and using the note-taking function to record key points. These events are also accompanied by precise timestamps.

[0028] The captured raw data undergoes preliminary preprocessing, including data cleaning, format standardization, and timestamp alignment. Finally, the processed data is encapsulated into a structured data packet, which includes at least the Session ID, Student ID, timestamp sequence, and the data content of the aforementioned operations, and then transmitted to the subsequent feature extraction module for further processing.

[0029] S102, Generate a unified context representation vector based on the operational data.

[0030] Specifically, the system models the sequence of operational behaviors, extracts operational intent features, performs compliance checks on parameter settings, identifies and judges equipment status and experimental results, and integrates multi-source data to form a unified contextual representation vector. The multi-source data includes operational intent features extracted from the operational behavior sequence, compliance check results for parameter settings, identification results of equipment status, judgment conclusions of experimental results, and contextual information of the experimental scenario.

[0031] S102.1, The operation behavior sequence is analyzed using a time-series modeling algorithm to extract feature vectors representing the operation intention.

[0032] S102.11 performs data cleaning, denoising, and standardization on the collected raw operation behavior sequences, removes outliers and redundant operations, and aligns operation sequences at different time scales onto a unified time axis.

[0033] S102.12 Extract at least one of the following features from the preprocessed operation behavior sequence: operation type distribution features, operation frequency features, operation duration features, operation sequence pattern features, operation interval time features, and operation object association features.

[0034] S102.13, a pre-trained first LSTM model is used to model the temporal dependency relationship of the operation behavior sequence, capturing the contextual association and long-term dependency relationship between operations.

[0035] S102.14, using the hidden layer state and attention mechanism of the first LSTM model, extract high-level abstract features representing the user's operation intentions. The features include: intention to execute experimental steps, intention to comply with operational procedures, intention to achieve experimental goals, and intention to perform potentially erroneous operations.

[0036] S102.15 concatenates, weights, or reduces the dimensionality of the extracted intent features to generate a feature vector of fixed-dimensional operational intent.

[0037] The pre-trained first LSTM model employs a bidirectional multi-layer architecture designed to extract high-level temporal features from operation sequence and infer operation intent. The input layer receives a pre-processed and feature-engineered feature matrix of the operation sequence. The embedding layer maps high-dimensional sparse classification features to low-dimensional dense vectors. The bidirectional LSTM layer serves as the feature extraction layer, containing LSTM units in both forward and backward directions to fully capture the forward and backward contextual dependencies of the operation sequence, outputting the hidden state sequence for all time steps. An attention mechanism is applied to the hidden state sequence output by the bidirectional LSTM layer, calculating the attention weights for each time step and generating a weighted summed context vector. This vector integrates the most crucial information for intent judgment throughout the sequence. The output layer outputs a high-level feature representation; the context vector, as an abstract representation of the entire input sequence, is passed to subsequent steps for intent classification or as part of the fused features. This model learns normal patterns of operation behavior through unsupervised pre-training or pre-training on a large number of standard operation sequences, thus providing high-quality feature representations for subsequent intent recognition and error detection.

[0038] The above process transforms the original low-level operation sequence into a high-level feature vector representing user intent. First, the original sequence undergoes data cleaning and standardization to eliminate noise and unify the time scale. Then, various statistical features are manually extracted from the preprocessed sequence to characterize the operation patterns, frequency, duration, and sequence. To further capture the complex temporal dependencies in the sequence, a pre-trained first LSTM model is used for deep temporal modeling. This model employs a bidirectional multi-layer structure and incorporates an attention mechanism, enabling it to effectively learn contextual information from long sequences and focus on key operational steps. Through the model's hidden states and attention weights, the system can parse the user's deep operational intent, such as whether they are performing standard steps, attempting to achieve an experimental goal, or engaging in potentially risky operations. Finally, these extracted statistical features are fused (e.g., concatenated or weighted) with the high-level abstract features generated by the LSTM model, and a fixed-dimensional, information-rich feature vector of operational intent is generated through dimensionality reduction techniques for subsequent context fusion and error recognition.

[0039] The training of the first LSTM model enables it to effectively extract high-level temporal features from action sequences and accurately infer the user's action intent. Its training process includes the following steps.

[0040] Step 1, Training data preparation.

[0041] Collect a large amount of historical behavioral sequence data of students performing standard operations and common erroneous operations in a virtual experimental environment. This data should include information such as operation type, operation object, and timestamp, and has been preprocessed and feature-engineered through steps S102.11 and S102.12.

[0042] Each operation sequence sample is labeled with its corresponding true intent tag. The tag is determined based on expert knowledge or post-event analysis, and includes, but is not limited to: step execution correct, step execution error, intent compliance, intent violation, goal achieved, goal not achieved, etc.

[0043] The labeled sequence data is divided into training set, validation set and test set.

[0044] Step 2: Train the model using supervised learning.

[0045] The training objective is to minimize the difference between the predicted intent and the true intent label. The classification cross-entropy loss function is used as the primary training objective, expressed as:

[0046] in, For batch size, The total number of intent categories, It is a sample One-hot encoding of the true intent tag. The sample predicted by the model Belongs to the The probability of class intent.

[0047] Use the Adaptive Moment Estimation (Adam) optimizer to update the model weights.

[0048] The training data is fed into the first LSTM model in batches. Forward propagation calculates the predicted output and loss, backpropagation calculates the gradient, and the optimizer updates the model parameters (including the weights of the LSTM and attention layers) based on the gradient. This process is repeated iteratively until the model's performance on the validation set no longer shows significant improvement or the preset number of iterations is reached.

[0049] Throughout the training process, model performance, such as accuracy and F1 score, is evaluated periodically on the validation set, and the parameters of the best-performing model on the validation set are saved for final deployment.

[0050] S102.2, compare the collected experimental parameter settings with the preset standard parameter range to generate parameter compliance judgment results and deviation measurement values.

[0051] A pre-defined standard parameter knowledge base defines the compliant value range for each configurable parameter in the virtual experimental environment. This range includes the target value, the allowed upper limit, the allowed lower limit, the recommended value interval, and is associated with its physical units.

[0052] The student settings captured by the data acquisition module are compared with the standard range of the corresponding parameters in the knowledge base. The comparison process not only compares the numerical values ​​but also performs unit conversions to ensure dimensional consistency.

[0053] Generates parameter compliance judgment results and deviation metrics. The judgment result is a category label, such as compliant, warning, or error. The deviation metric is a numerical value used to quantify the degree to which the set value deviates from the standard value; its calculation method can be absolute deviation, relative deviation, or standardized deviation based on Z-score. For example, for a parameter with a target value of 100°C and a set value of 150°C, a judgment result of error and a deviation metric of 50°C or 50% can be generated.

[0054] S102.3 Identify the operating status of the instrument based on the equipment status data, and determine success, failure or abnormality based on the experimental results data.

[0055] The system receives real-time device status data streams from the virtual experiment simulation engine, such as: furnace: ON, current temperature: 150°C; centrifuge: OFF, speed: 0 rpm. Using predefined state machines or classification rules, this raw state data is mapped to higher-level, semantically clear state identifiers. For example, [power > 0W, temperature < 50°C] is identified as "heating"; [power > 0W, temperature >= 100°C] is identified as "boiling"; and [on / off state == ON, speed == 0 rpm, duration > 5s] is identified as "device jammed" abnormal state.

[0056] At key experimental points or after the experiment, acquire experimental data such as the pH value of the product, the current value in the circuit, whether the reaction occurred, and the measured weight. Logically compare these results with predefined experimental success criteria. The judgment conclusion is usually a classification result, for example: Success: The results fully met the expected goals; Failure: The results deviated significantly from expectations and could not be corrected by simple adjustments; Anomaly: The result data deviates from expectations or presents an intermediate state that cannot be defined by simple success / failure.

[0057] S102.4, the feature vector of the operation intention, the parameter compliance judgment result, the device status recognition result, the experimental result judgment conclusion, and the experimental scenario context information are fused at the feature level to generate a unified dynamic context representation vector.

[0058] Using the feature vector (e.g., a 128-dimensional vector) of the operational intent generated in S102.15 as a basis, the parameter compliance judgment result and deviation metric value in S102.2 are one-hot encoded and quantified, and converted into feature vectors. Similarly, the equipment status identification result and experimental result judgment conclusion in S102.3 are one-hot encoded or embedded encoded, and converted into feature vectors. Finally, the experimental scenario context information (e.g., experimental subject: chemistry - titration, current step: step 3 - add indicator) is encoded and converted into feature vectors.

[0059] The aforementioned multiple feature vectors are combined using feature fusion algorithms, including direct concatenation, weighted fusion, and neural network-based fusion. Ultimately, a unified, fixed-dimensional dynamic context representation vector is generated. This vector comprehensively represents information such as "who (student) performed what operation (intention) in what scenario (experiment subject, procedure), what parameters were set (compliance), resulting in what device state, and what result was produced."

[0060] S103, generate the final diagnostic result based on the unified context representation vector.

[0061] Specifically, based on a unified contextual representation vector, rule matching is performed using predefined rules in a rule knowledge base. Simultaneously, predictions are made using a machine learning model to identify operational errors, logical errors, safety violations, and potential experimental risks. Finally, a conflict resolution mechanism is used to output the final diagnostic results.

[0062] S103.1 matches the unified context representation vector with the predefined rule conditions in the rule knowledge base. When the context representation vector meets the specific rule conditions, the corresponding rule conclusion is triggered, and the rule matching result is generated, including error type, violation level, rule confidence and associated rule number.

[0063] The predefined rules in the rule knowledge base adopt a "condition-conclusion" structure. The condition part of each rule is a combination of one or more logical judgment clauses that query and judge the information encoded by the unified context representation vector.

[0064] Specifically, by parsing the context representation vector, the rule conditions can perform logical judgments on the following types of information: Operational intent characteristics: Determine whether the operation sequence matches or deviates from a specific intent. For example, operation intent ==="Illegal operation_Directly smelling chemicals" or operation intent !="Standard procedure_Rinsing pipette".

[0065] Parameter compliance: Determine whether parameter settings exceed hard limits or fall within non-recommended ranges. For example, heating temperature >100°C, solution concentration NOT IN [0.1mol / L, 1.0mol / L].

[0066] Equipment Status: Determines whether the equipment is in a dangerous, abnormal, or incorrect state. For example, centrifuge status == "Running" AND centrifuge lid status == "Not Closed", heating plate status == "On" AND container contents == "Empty".

[0067] Experimental Results: Determine whether intermediate or final results indicate experimental failure or abnormality. For example, the final product has a pH value < 6.0 and the circuit current is 0.

[0068] Operation sequence and logic: Determine whether the operation sequence conforms to the logical flow. For example, the current step: "Add reagent B" AND the step that did not occur: "Add reagent A" (i.e., determine whether the preceding step is missing).

[0069] Safety procedures: Determine if basic safe operating procedures have been violated. For example, whether safety goggles are worn (FALSEAND) and the object being handled (corrosive chemicals).

[0070] Experimental scenario context: Determines whether the rules apply to the current experimental scenario. For example, Experiment Subject == "Chemistry" AND Experiment Type == "Titration".

[0071] Multiple clauses in a rule condition are combined using logical operators (such as AND, OR, NOT) to form a complete conditional expression. The rule is triggered only if the current context representation vector makes the expression evaluate to "true".

[0072] Specifically, the system iterates through each rule in the rule knowledge base. For each rule, the logical judgment clauses in its condition section are converted into queries against a unified context representation vector. The rule engine parses these conditions, extracts the corresponding feature values ​​from the context vector, performs logical operations, and calculates the Boolean value of the entire condition expression. When the condition expression of a rule is satisfied (i.e., the calculation result is True), the rule is triggered or activated. Subsequently, the system executes the conclusion section corresponding to that rule, generating a rule matching result. This result is a structured data object, including: error type, violation level, rule confidence, and associated rule number.

[0073] Error type is the error category identifier defined by the rule, such as "Safety Violation - No Protective Equipment," "Operational Error - Parameter Exceeded Limits," or "Logical Error - Missing Step." Violation level is the severity level defined by the rule, such as "High Risk," "Medium Risk," "Low Risk," or "Warning." Rule confidence is a preset confidence value representing the reliability of the rule; for example, for absolutely clear rules, the confidence level is set to 1.0; for rules with edge cases, the confidence level can be set to 0.8. The associated rule number is a unique identifier for the triggered rule in the knowledge base, used for tracing and explaining the decision-making process, enhancing the system's interpretability.

[0074] S103.2 Input the unified context representation vector into the pre-trained second LSTM model to output the prediction results, including the error probability distribution, risk level score, potential error type identifier, and model confidence.

[0075] The second Long Short-Term Memory (LSTM) network model is configured to receive a uniform contextual representation vector and output predictions. This model contains the following hierarchical structure: Input layer: Receives a unified contextual representation vector of dimension d; The first LSTM layer contains multiple LSTM units, used to capture the primary temporal features and dependencies of the input vector, and output the first hidden state sequence. The second LSTM layer is stacked with the first LSTM layer and is used to further extract high-level, abstract temporal patterns from the primary features, outputting the second hidden state sequence. Attention mechanism layer: The second hidden state sequence is weighted and aggregated, the attention weight of each time step is calculated, and a context vector that integrates important information from all time steps is generated; Fully connected output layer: Receives the context vector and generates direct outputs through multiple parallel sub-networks, including error probability distribution vectors, risk level scores, and model confidence.

[0076] It should be noted that the potential error type identifier is not a direct output of the model, but is generated subsequently based on the error probability distribution vector. Specifically, the potential error type identifier is a set of labels corresponding to error types whose probability values ​​exceed a preset threshold.

[0077] The second LSTM model is trained by minimizing a multi-task loss function, which is a weighted sum of multiple sub-losses:

[0078] in, Here is the crossover loss function, and N is the batch size. Let sample i be the true label of the k-th type of error. The model predicts the probability that sample i belongs to the k-th type of error; Let the mean square error loss function be . For sample i, the true risk score, The predicted risk score for sample i; For L2 regularization terms, This is the set of all trainable weight parameters in the second LSTM model; , , This is a hyperparameter.

[0079] Cross-entropy loss is the primary task loss, responsible for supervising the model's classification ability, i.e., accurately predicting the probability distribution of the error type. It calculates the loss by comparing the probability distribution P (predicted) output by the model with the one-hot encoding y of the true label. When the predicted probability... With real labels When they are completely identical, the loss is 0; the greater the difference, the higher the loss value.

[0080] The mean squared error loss function is an auxiliary task loss responsible for supervising the model's regression ability, i.e., accurately predicting a continuous risk level score. It calculates the risk score predicted by the model. Compared with the true risk score The average of the squares of the differences between them. The squared term amplifies the effect of larger errors, making the model particularly sensitive to high-risk operations.

[0081] The purpose of the regularization term is not to fit the data, but to constrain the model itself and prevent overfitting. It calculates the sum of squares (the square of the L2 norm) of all weight parameters W in the model. The larger the weight values, the greater the loss of this term. Adding this term to the total loss "penalizes" larger weight values, encouraging the model to learn smaller, more dispersed weights, thereby reducing model complexity and improving its ability to generalize to unseen data.

[0082] The prediction results generated based on the second LSTM model architecture specifically include: inputting a unified scenario representation vector into the input layer of the second LSTM model; then processing the input vector sequentially through the first and second LSTM layers of the second LSTM model to extract the hidden state sequence; calculating the weights of each time step of the hidden state sequence through an attention mechanism; and finally, weighted summation to obtain a context vector containing relevant information. This context vector is then input into the fully connected output layer to generate an error probability distribution vector, a risk level score, and a model confidence score in parallel. Each probability value in the error probability distribution vector is compared with a preset probability threshold. If the probability value is not less than the preset probability value, the corresponding error type identifier is added to the potential error type identifier set. The potential error type identifier set, together with the error probability distribution vector, risk level score, and model confidence score, constitute the complete model prediction result.

[0083] The training of the second LSTM model enables it to perform both misclassification and risk rating tasks simultaneously, based on a unified context representation vector. Its training process is based on a multi-task learning framework and includes the following steps.

[0084] Step 1, Training data preparation.

[0085] The unified context representation vectors processed in step S102 are used as input features. Each vector corresponds to a snapshot of a historical experimental operation context.

[0086] For each context representation vector sample, two target values ​​are labeled: the true error label (for classification tasks) and the true risk score (for regression tasks).

[0087] The true error label is a one-hot vector that indicates the type of error that actually occurred in the context, such as operation error - parameter over-limit, or safety violation. If there is no error, it is labeled as no error category.

[0088] The true risk score is a continuous value between [0,1], which is assigned by domain experts based on the severity of the consequences of the operation. For example, a minor deviation is assigned 0.2 and a serious safety violation is assigned 1.0.

[0089] The labeled data is divided into training set, validation set and test set.

[0090] Step 2, multi-task learning training.

[0091] The training objective is to simultaneously minimize classification and regression errors and prevent model overfitting.

[0092] Joint optimization is performed using the aforementioned multi-task loss function, as shown in the loss function formula of the second LSTM model above, which will not be repeated here.

[0093] The Adam optimizer is used. Batch context representation vectors are input into a second LSTM model. The model propagates forward, outputting the error probability distribution P and risk score R in parallel. After calculating the total loss L, the gradient is calculated via backpropagation, and all model weight parameters are updated. This process is iterated until the model converges. The overall loss L and metrics for each subtask, such as classification accuracy and regression root mean square error, are monitored on the validation set. The model with the best overall performance on the validation set is selected as the final model.

[0094] S103.3, perform collaborative decision-making on the rule matching result and the model prediction result. When the two results are consistent, directly output the final diagnosis result. When the two results conflict, use a conflict resolution algorithm to handle the inconsistency between the rule and the model and generate the final diagnosis result.

[0095] S103.31, obtain the confidence level of the rule matching result and the confidence level of the model prediction result, multiply the confidence level of the rule matching result by the preset rule engine weight coefficient to obtain the rule support weight vote, and multiply the confidence level of the model prediction result by the preset model weight coefficient to obtain the model support weight vote.

[0096] S103.32 If the number of votes supporting the rule weight is greater than the sum of the number of votes supporting the model weight and the decision threshold, then the rule matching result is adopted.

[0097] S103.33 If the number of votes supporting the model weight is greater than the sum of the number of votes supporting the rule weight and the decision threshold, then the model prediction result shall be adopted.

[0098] S103.34 If the absolute value of the difference between the number of votes supporting the weight of the rule and the number of votes supporting the weight of the model is not greater than the decision threshold, then the meta-rule arbitration is triggered, which includes matching the current conflict context with the meta-rule conditions and executing the arbitration action specified by the successfully matched meta-rule; the conflict context includes the rule matching result and its confidence, the model prediction result and its confidence, and the specific type of experiment simulated in the current virtual experimental environment.

[0099] For example, in a virtual chemical titration experiment, a student performed an operation: directly pouring a 2.5 mol / L NaOH solution into the acid solution to be tested. The system then performed diagnostics using both a rule engine and a machine learning model.

[0100] First, the rule engine detected a safety rule being triggered. Rule number: CHEM-SAFETY-007; Rule condition: IF(Operation == "Add strong alkali") AND (alkali concentration > 2.0 mol / L) THEN; Rule conclusions include: Error type: "Safety violation_High concentration strong alkali operation", Violation level: "High risk", Rule confidence: 1.0, Rule weight vote calculation: Vote rule =conf r *ω r =1.0 * 0.7 = 0.7.

[0101] After analyzing the context of the entire operation sequence, the machine learning model (second LSTM) predicted that the student was likely performing a special "quick neutralization" technique, not an unintentional violation. The primary error type was "no error," the model confidence score was 0.85, and the model weight votes were calculated as follows: Vote. model =conf m *ω m =0.85 * 0.6 = 0.51, Conflict Context: Context = (type r "Safety Violation - High Concentration Strong Alkali Operation", type m "No errors", conf r :1.0,conf m :0.85,ExperimentType:"chemistry_titration").

[0102] Calculating weighted votes: Vote rule =1.0 * 0.7 = 0.7, Vote model =0.85*0.6=0.51.

[0103] The preset decision threshold τ = 0.2. Compare the votes. rule (0.7)>Vote model The calculation result is False. (0.51)+τ(0.2)=>0.7>0.71?

[0104] Next, check the reverse condition: Vote model (0.51)>Vote rule (0.7)+τ(0.2)=>0.51>0.9?, the calculation result is also False.

[0105] Since |0.7-0.51|=0.19<τ(0.2), the difference in votes is within the threshold range, making automatic decision-making impossible. The system then triggers the meta-rule arbitration mechanism.

[0106] The conflict context is matched against the meta-rule knowledge base. One meta-rule is successfully matched: the meta-rule condition is IF(ExperimentType=="chemistry_titration")AND(conflict-related error type=="safety violation")THEN, and the meta-rule action is Action=accept_rule (for safety violations in chemistry experiments, the result from the rule engine is preferred). The action specified by the meta-rule is executed, and the system ultimately adopts the rule matching result.

[0107] The final diagnostic results output includes: the final error type is "Safety Violation_High Concentration Strong Alkali Operation", the final violation level is "High Risk", and the final confidence level is 0.95 = (conf r +conf m Adjustment factor / 2, decision basis is rule "CHEM-SAFETY-007; Arbitration: Meta Rule-Safety-01".

[0108] S103.4 Output the final diagnostic results, including the identified operational error type, logical error description, safety violation, potential experimental risk level, corresponding rule or model basis, and comprehensive confidence score.

[0109] The corresponding rules or models are used to record the direct source of the current diagnostic conclusion, clearly indicating whether the conclusion is derived from the rule engine, predicted by the machine learning model, or an arbitration result after conflict resolution.

[0110] The overall confidence score is a single quantitative value that combines rule confidence and model confidence, used to represent the system's overall confidence in the final diagnostic result of this output.

[0111] S104, dynamically generates and renders guidance information.

[0112] Specifically, guidance information in the form of text, visuals, voice, or animation is dynamically generated and output based on error type, risk level, student's historical performance, and learning preferences.

[0113] S104.1 Query the student profile database based on the current student identifier to obtain the student's historical performance data and learning preference data. The historical performance data includes the frequency of historical errors, operation proficiency score, and recent progress trend; the learning preference data includes the feedback modality of the preference and the level of detail of the preference.

[0114] Historical error frequency indicates how frequently students make mistakes in past experimental operations. It is usually calculated as the average number of errors occurring per unit of time or per unit of operation.

[0115] The proficiency score is a comprehensive assessment value used to measure students' proficiency in operating virtual instruments and the standardization of their process execution. It is calculated by analyzing multiple dimensions of indicators such as students' operating speed, operational fluency (whether there are redundant or hesitant operations), and accuracy in operating complex equipment, through a predefined model (such as a weighted scorecard or regression model), and is usually a normalized value.

[0116] The progress trend characterizes the direction and rate of change in a student's ability over a recent period. It is not a static value, but a trend indicator derived through time series analysis (such as calculating the slope of proficiency scores or error frequencies within a sliding window).

[0117] The feedback modality of preferences indicates the student's preferred or most effective channels for receiving information. It is a categorical variable and typically allows one or more values.

[0118] The level of detail preferred indicates how detailed the feedback information a student expects. It is usually a categorical variable, such as: concise, standard, or detailed.

[0119] S104.2 Match the error type and risk level with the predefined feedback strategy library to obtain the basic feedback content template and urgency level indicator.

[0120] S104.3, dynamically generates guidance information based on the basic feedback template, students' historical performance, and learning preferences, including: 1) Calculate the final content detail coefficient = basic detail + α * (1 - operation proficiency score) + β * historical error frequency, where α and β are adjustment coefficients. Select the concise, standard, or detailed version of the text content based on this detail coefficient.

[0121] 2) If the proficiency score is lower than the score threshold or the historical error frequency is higher than the frequency threshold, then the corresponding principle explanation and standard operating procedure will be obtained based on the specific type of experiment simulated in the current virtual experimental environment and added to the guidance information.

[0122] 3) If the risk level exceeds the safety threshold, the corresponding strong warning tone and safety specification reference will be added to the guidance information content according to the error type.

[0123] S104.4, based on student preferences and urgency, render the generated guidance information into an appropriate output format, including text output, visual output, audio output, and animation output.

[0124] The system reads the student's preference feedback modality and the urgency level indicator obtained from the feedback strategy library. Combining these two points, it generates a list of output instructions to be executed. Based on the generated instruction set, the uniform text content is distributed to different renderers for parallel processing and output.

[0125] The foregoing has described in detail an embodiment of a method for generating virtual experimental environment guidance information. Based on the virtual experimental environment guidance information generation method described in the above embodiment, this invention also provides a virtual experimental environment guidance information generation system corresponding to the method.

[0126] Figure 2 This is a schematic block diagram of a virtual experimental environment guidance information generation system provided in an embodiment of the present invention. In this embodiment, the virtual experimental environment guidance information generation system 200 can be divided into multiple functional modules according to its functions, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory.

[0127] The operation data capture module 210 is used to capture in real time the sequence of students' operation behaviors, experimental parameter settings, equipment status data and interaction events in the virtual experimental environment, and to include timestamp information.

[0128] The scenario representation vector generation module 220 is used to model the operation behavior sequence, extract operation intention features, perform compliance checks on parameter settings, identify and judge the device status and experimental results, and integrate multi-source data to form a unified scenario representation vector.

[0129] The diagnostic result generation module 230 is used to perform rule matching based on a unified context representation vector and predefined rules in a rule knowledge base, while making predictions through a machine learning model to identify operational errors, logical errors, safety violations and potential experimental risks, and output the final diagnostic result using a conflict resolution mechanism.

[0130] The guidance information display module 240 is used to dynamically generate and output guidance information in the form of text, visual, audio or animation based on error type, risk level, student's historical performance and learning preferences.

[0131] The virtual experimental environment guidance information generation system of this embodiment is used to implement the aforementioned virtual experimental environment guidance information generation method. Therefore, the specific implementation of this system can be found in the embodiment section of the virtual experimental environment guidance information generation method above. Thus, its specific implementation can be referred to the description of the corresponding embodiments, and will not be elaborated here.

[0132] Furthermore, since the virtual experimental environment guidance information generation system of this embodiment is used to implement the aforementioned virtual experimental environment guidance information generation method, its function corresponds to the function of the above method, and will not be repeated here.

[0133] Figure 3 This is a schematic diagram of a terminal 300 provided in an embodiment of the present invention, including: a processor 310, a memory 320, and a communication unit 330. The processor 310 is used to implement the process steps of the above-described embodiment of the virtual experimental environment guidance information generation method when implementing the virtual experimental environment guidance information generation program stored in the memory 320.

[0134] The terminal 300 includes a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It can be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0135] The memory 320 can be used to store the execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile memory terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 is able to perform some or all of the steps in the above method embodiments.

[0136] The processor 310 serves as the control center of the storage terminal, connecting various parts of the electronic terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0137] The communication unit 330 is used to establish a communication channel, enabling the storage terminal to communicate with other terminals. It can receive user data sent by other terminals or send user data to other terminals.

[0138] The present invention also provides a computer storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0139] The computer storage medium stores a virtual experimental environment guidance information generation program. When the virtual experimental environment guidance information generation program is executed by the processor, it implements the process steps of the above-described virtual experimental environment guidance information generation method embodiment.

[0140] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or other media capable of storing program code. It includes several instructions to cause a computer terminal (which may be a personal computer, server, or a second terminal, network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0141] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating guidance information for a virtual experimental environment, characterized in that, Includes the following steps: Real-time capture of students' operational behavior sequences, experimental parameter settings, device status data, and interaction events in the virtual experimental environment, along with timestamp information; The operation behavior sequence is modeled, operation intention features are extracted, the parameter settings are checked for compliance, the equipment status and experimental results are identified and judged, and multi-source data are integrated to form a unified context representation vector. Based on a unified contextual representation vector, rule matching is performed using predefined rules in a rule knowledge base. Simultaneously, prediction is performed using a machine learning model to identify operational errors, logical errors, safety violations, and potential experimental risks. Finally, a conflict resolution mechanism is used to output the diagnostic results. Based on error type, risk level, student's historical performance, and learning preferences, guidance information in text, visual, audio, or animation formats is dynamically generated and output.

2. The method according to claim 1, characterized in that, The operation behavior sequence is modeled, operation intention features are extracted, parameter settings are checked for compliance, equipment status and experimental results are identified and judged, and multi-source data are integrated to form a unified contextual representation vector, specifically including: The sequence of operational behaviors is analyzed using a time-series modeling algorithm to extract feature vectors representing the operational intent; The collected experimental parameter settings are compared with the preset standard parameter range to generate parameter compliance judgment results and deviation measurement values; The system identifies the operating status of instruments and equipment based on equipment status data, and determines success, failure, or abnormality based on experimental results data. The feature vector of the operation intention, the parameter compliance judgment result, the device status recognition result, the experimental result judgment conclusion, and the experimental scenario context information are fused at the feature level to generate a unified dynamic context representation vector.

3. The method according to claim 2, characterized in that, The operation behavior sequence is analyzed using a time-series modeling algorithm to extract feature vectors representing the operation intention, specifically including: The collected raw operation sequences are cleaned, denoised, and standardized to remove outliers and redundant operations, and operation sequences at different time scales are aligned to a unified time axis. Extract at least one of the following features from the preprocessed sequence of operation behaviors: operation type distribution features, operation frequency features, operation duration features, operation sequence pattern features, operation interval time features, and operation object association features; A pre-trained first LSTM model is used to model the temporal dependencies of the operation sequence, capturing the contextual associations and long-term dependencies between operations; By using the hidden layer state and attention mechanism of the first LSTM model, high-level abstract features representing the user's operation intentions are extracted. These features include: intention to execute experimental steps, intention to comply with operational procedures, intention to achieve experimental goals, and intention to perform potentially erroneous operations. The extracted intent features are concatenated, weighted, or dimensionality reduced to generate a feature vector of fixed-dimensional operational intent.

4. The method according to claim 1, characterized in that, Based on a unified contextual representation vector, rule matching is performed using predefined rules in a rule knowledge base. Simultaneously, machine learning models are used for prediction to identify operational errors, logical errors, safety violations, and potential experimental risks. A conflict resolution mechanism is employed to output the final diagnostic results, specifically including: The unified context representation vector is matched with predefined rule conditions in the rule knowledge base. When the context representation vector meets a specific rule condition, the corresponding rule conclusion is triggered, and the rule matching result is generated, including error type, violation level, rule confidence and associated rule number. The unified context representation vector is input into the pre-trained second LSTM model to output the prediction results, including the error probability distribution, risk level score, potential error type identification, and model confidence. The rule matching result and the model prediction result are used to make a collaborative decision. When the two results are consistent, the final diagnosis result is directly output. When the two results conflict, the inconsistency between the rule and the model is handled by the conflict resolution algorithm to generate the final diagnosis result. The final diagnostic results are output, including the identified types of operational errors, descriptions of logical errors, safety violations, potential experimental risk levels, corresponding rules or model bases, and a comprehensive confidence score.

5. The method according to claim 4, characterized in that, The unified context representation vector is input into a pre-trained second LSTM model to output prediction results, specifically including: A unified scenario representation vector is input into the input layer of the second LSTM model. The input vector is then processed sequentially through the first and second LSTM layers of the second LSTM model to extract the hidden state sequence. The weights of each time step of the hidden state sequence are calculated using an attention mechanism, and the weighted sum is used to obtain a context vector containing relevant information. This context vector is then input into the fully connected output layer, where an error probability distribution vector, a risk level score, and a model confidence score are generated in parallel. Each probability value in the error probability distribution vector is compared with a preset probability threshold. If the probability value is not less than the preset probability value, the corresponding error type identifier is added to the potential error type identifier set. The potential error type identifier set, together with the error probability distribution vector, risk level score, and model confidence score, constitute the complete model prediction result. The following loss function is used when training the second LSTM model. in, Here is the crossover loss function, and N is the batch size. Let sample i be the true label of the k-th type of error. The model predicts the probability that sample i belongs to the k-th type of error. Let the mean square error loss function be . For sample i, the true risk score, The predicted risk score for sample i; For L2 regularization terms, This is the set of all trainable weight parameters in the second LSTM model; , , This is a hyperparameter.

6. The method according to claim 4, characterized in that, The final diagnostic results are generated by handling inconsistencies between rules and models through conflict resolution algorithms, specifically including: Obtain the confidence scores of the rule matching results and the model prediction results. Multiply the confidence scores of the rule matching results by the preset rule engine weight coefficients to obtain the rule support weight votes. Multiply the confidence scores of the model prediction results by the preset model weight coefficients to obtain the model support weight votes. If the number of votes supporting the rule weight is greater than the sum of the number of votes supporting the model weight and the decision threshold, then the rule matching result is adopted. If the number of votes supported by the model is greater than the sum of the number of votes supported by the rule and the decision threshold, then the model prediction result is adopted. If the absolute value of the difference between the number of votes supporting the weight of the rule and the number of votes supporting the weight of the model is not greater than the decision threshold, then meta-rule arbitration is triggered, which includes matching the current conflict context with the meta-rule conditions and executing the arbitration action specified by the successfully matched meta-rule; the conflict context includes the rule matching result and its confidence, the model prediction result and its confidence, and the specific type of experiment simulated in the current virtual experimental environment.

7. The method according to claim 1, characterized in that, Based on error type, risk level, student's historical performance, and learning preferences, dynamically generate and output guidance information in text, visual, audio, or animated formats, specifically including: The system queries the student profile database based on the current student identifier to obtain the student's historical performance data and learning preference data. The historical performance data includes the frequency of historical errors, operation proficiency scores, and recent progress trends; the learning preference data includes the feedback modality of preferences and the level of detail of preferences. Match error types and risk levels with a predefined feedback strategy library to obtain basic feedback content templates and urgency indicators; Based on the basic feedback template, students' historical performance, and learning preferences, guidance information is dynamically generated, including: The final content detail coefficient is calculated as follows: base detail coefficient + α * (1 - operation proficiency score) + β * historical error frequency, where α and β are adjustment coefficients. Based on this detail coefficient, the concise, standard, or detailed version of the text content is selected. If the proficiency score is lower than the score threshold or the historical error frequency is higher than the frequency threshold, then the corresponding principle explanation and standard operating procedure will be obtained based on the specific type of experiment simulated in the current virtual experimental environment and added to the guidance information content. If the risk level exceeds the safety threshold, the corresponding strong warning tone and safety specification reference will be added to the guidance information content according to the error type. Based on student preferences and urgency, the generated guidance information is rendered into corresponding output formats, including text, visual, audio, and animation.

8. A virtual experimental environment guidance information generation system, characterized in that, include: The operation data capture module is used to capture in real time the sequence of students' operation behaviors, experimental parameter settings, equipment status data and interaction events in the virtual experimental environment, and to include timestamp information. The scenario representation vector generation module is used to model the operation behavior sequence, extract operation intention features, perform compliance checks on parameter settings, identify and judge the device status and experimental results, and integrate multi-source data to form a unified scenario representation vector. The diagnostic result generation module is used to perform rule matching based on a unified context representation vector and predefined rules in a rule knowledge base. At the same time, it uses a machine learning model to make predictions, identify operational errors, logical errors, safety violations and potential experimental risks, and output the final diagnostic results using a conflict resolution mechanism. The guidance information display module is used to dynamically generate and output guidance information in the form of text, visuals, voice, or animation based on error type, risk level, student's historical performance, and learning preferences.

9. A terminal, characterized in that, include: The memory is used to store the program that generates guidance information for the virtual experimental environment; A processor is configured to implement the steps of the virtual experimental environment guidance information generation method as described in any one of claims 1 to 7 when executing the virtual experimental environment guidance information generation program.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a virtual experimental environment guidance information generation program, which, when executed by a processor, implements the steps of the virtual experimental environment guidance information generation method as described in any one of claims 1 to 7.

Images