A Standardized Scoring Method for Experimental Operation Sequence Based on Spatiotemporal Graph Convolutional Networks

Abstract

This paper presents a prescriptive scoring method for experimental operation sequence based on spatiotemporal graph convolutional networks. It relates to the field of data processing technology and includes: constructing a dependency graph of experimental operations by defining sequential dependencies, parallelism, and mutual exclusion relationships between actions; acquiring experimental operation videos and performing action detection to obtain action sets and their corresponding temporal positions; transforming the action sets and their corresponding temporal positions into a spatiotemporal graph; encoding the spatiotemporal graph using a spatiotemporal graph convolutional network; and calculating the dependency satisfaction of sequential dependencies, the parallel rationality of parallelism, and the mutual exclusion violation penalty term based on the encoded node features to comprehensively generate a prescriptive score. This method addresses the problems of traditional relational graphs, such as the single type of edge, lack of decay characteristics in temporal modeling, insufficient decoupling of scoring, and lack of interpretability.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method for scoring the order of experimental operations based on spatiotemporal graph convolutional networks. Background Technology

[0002] Experimental operation skills are a core assessment dimension in science education and vocational skills training, such as physics, chemistry, and biology. The standardized operation sequence is not only related to the accuracy of experimental results, but also directly affects experimental safety. In scenarios such as junior and senior high school physics, chemistry, and biology experimental examinations and vocational qualification certifications, the demand for automated scoring of the standardization of operation sequence is becoming increasingly urgent.

[0003] Currently, the scoring methods for the order of experimental operations are mainly divided into three categories: First, scoring based on manual observation, which suffers from strong subjectivity, low efficiency, and difficulty in large-scale standardized implementation; second, procedural scoring based on sensors or fixed process rules, which requires the deployment of a large number of sensors on experimental equipment, resulting in high equipment costs, poor versatility, and the inability to capture complex relationships such as temporal dependencies and parallel execution between actions; and third, general action recognition methods based on computer vision, which typically focus only on the category recognition of a single action and lack the ability to model the structured logical relationships between actions, performing poorly in judging normative issues such as whether "operation A must precede operation B" or "operations C and D can be performed simultaneously."

[0004] In recent years, graph neural networks have demonstrated advantages in structured relationship modeling, and researchers have attempted to apply them to action sequence analysis. However, existing methods have the following shortcomings: First, the constructed relationship structures are relatively simple, making it difficult to differentiate and model the various differential logical constraints present in actual operations, resulting in insufficient representation capabilities for constraint relationships of different properties; second, the temporal modeling part does not pay enough attention to the differences in the impact between actions over different time spans, making it difficult to effectively capture the dynamic changes in the mutual influence between operational actions; third, the scoring output is mostly a comprehensive single indicator, failing to decompose the scoring process according to different constraint dimensions, resulting in a lack of process transparency and targeted feedback capabilities in the scoring results.

[0005] Therefore, we propose a method that can solve the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide a standardized scoring method for the experimental operation sequence based on spatiotemporal graph convolutional networks, which addresses the problems of traditional methods such as simple relational structure, insufficient attention to the dynamic changes in the influence between actions in temporal modeling, and the inability to decompose the scoring process according to the constraint dimension.

[0007] This invention is achieved through the following technical solution: A prescriptive scoring method for the experimental operation sequence based on spatiotemporal graph convolutional networks specifically includes: By defining the sequential dependencies, parallelism, and mutual exclusion relationships between actions, a dependency knowledge graph of experimental operations is constructed. Acquire experimental operation videos and perform motion detection to obtain a set of motions and their corresponding time positions; The action set and its corresponding time position are transformed into a spatiotemporal graph, where the nodes of the spatiotemporal graph are action instances, the time edges connect adjacent time actions, and the spatial edges are established between the corresponding action sets based on the dependency knowledge graph. Spatiotemporal graph convolutional networks are used to encode the spatiotemporal graph. The spatial graph convolutional layer aggregates spatial neighbor information using a weight matrix corresponding to the edge type based on the edge type of the spatial edge, and the temporal graph convolutional layer introduces a temporal decay attention mechanism along the temporal edge. Based on the encoded node features, the dependency satisfaction of sequential dependencies, the parallel rationality of parallel relationships, and the mutual exclusion violation penalty of mutual exclusion relationships are calculated, and a sequential normality score is generated in combination.

[0008] Furthermore, the dependency knowledge graph is a directed graph, where nodes represent atomic actions and edges represent temporal dependencies. A sequential dependency edge indicates that one action must be completed before another action, a parallel edge indicates that two actions can be executed simultaneously, and a mutual exclusion edge indicates that two actions cannot be executed simultaneously.

[0009] Furthermore, the acquisition of experimental operation videos and the performance of motion detection specifically include: The experimental operation video was analyzed frame by frame, and the spatiotemporal features of the video were extracted using a pre-trained 3D convolutional neural network. The action detection model outputs action category labels and corresponding start and end times to form an action set.

[0010] Furthermore, the action detection model also includes a non-maximum suppression post-processing step, which is used to remove redundant detection results of the same action category within the overlapping time window and retain the action with the highest confidence.

[0011] Furthermore, the decay function of the time decay attention mechanism is an exponential decay function, where the larger the time interval between the two action instances connected by the time edge, the smaller the attention weight.

[0012] Furthermore, the dependency satisfaction degree is calculated as follows: For each pair of actions with sequential dependencies defined in the dependency knowledge graph, compare their order of occurrence in actual operation; If the dependency direction is satisfied, positive points are contributed; otherwise, negative or zero points are contributed.

[0013] Furthermore, the calculation method for the parallel rationality is as follows: For two actions defined as parallelizable in the dependency knowledge graph, calculate their temporal overlap in the experimental operation video. The higher the degree of overlap, the higher the score for parallel rationality.

[0014] Furthermore, the calculation method for the mutual exclusion violation penalty term is as follows: For two actions defined as mutually exclusive in a dependency knowledge graph; If an overlap is detected in time, a penalty score is applied based on the duration of the overlap; the longer the overlap, the greater the penalty.

[0015] Furthermore, this method also includes: A comprehensive order normativity score is generated by fusing the scores of dependency satisfaction of sequential dependencies, parallel rationality of parallel relationships, and mutual exclusion violation penalty of mutual exclusion relationships. The weight coefficients of each score are configured according to the experiment type and scoring requirements.

[0016] Furthermore, this method also includes: The comprehensive sequence standardization score is compared with the preset score level threshold, the corresponding operation standardization rating result is output, and a visual score report is generated. The report marks the specific actions that violate the sequence dependency, parallel relationship or mutual exclusion relationship and the corresponding time position.

[0017] The technical solution of the present invention has at least the following advantages and beneficial effects: This invention discloses a prescriptive scoring method for experimental operation sequence based on spatiotemporal graph convolutional networks. By introducing edge types containing three semantics—sequential dependency, parallelism, and mutual exclusion—during the spatiotemporal graph construction stage, the model can aggregate neighborhood information under different logical relationships in a differentiated manner, thereby achieving fine-grained structured modeling of complex experimental operation procedures and significantly improving the ability to represent multi-temporal relationships between actions.

[0018] In addition, by introducing a temporal decay attention mechanism in the temporal graph convolutional layer, the model can give higher attention weight to actions with closer time intervals, effectively capturing the decay characteristics of action position on operation norm evaluation, thus better adapting to the actual situation of large time span differences in experimental operations, and improving the accuracy and robustness of temporal relationship modeling.

[0019] Furthermore, by decoupling the comprehensive score into three interpretable and independent scoring dimensions—dependency satisfaction, parallel rationality, and mutual exclusion violation penalty—the scoring process is no longer a black box output. Each score clearly reflects the operator's specific performance in terms of temporal dependency, parallel execution, and mutual exclusion constraints, providing a transparent and traceable basis for subsequent teaching feedback and targeted error correction.

[0020] It is worth noting that by extracting video spatiotemporal features through a pre-trained 3D convolutional neural network and combining it with non-maximum suppression post-processing, redundant detection results of the same action category within overlapping time windows are effectively removed, ensuring the accuracy and compactness of the action set and laying a reliable data foundation for subsequent spatiotemporal graph construction and scoring calculation. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Example 1 like Figure 1 The experimental operation sequence normativity scoring method shown is based on spatiotemporal graph convolutional networks and specifically includes: S1. Construct a dependency knowledge graph of experimental operations by defining sequential dependencies, parallelism, and mutual exclusion relationships between actions; Furthermore, the dependency knowledge graph is a directed graph, where nodes represent atomic actions and edges represent temporal dependencies. Sequential dependency edges indicate that one action must be completed before another action, parallel edges indicate that two actions can be executed simultaneously, and mutual exclusion edges indicate that two actions cannot be executed simultaneously. The structured expression for a knowledge graph is defined as follows: , in: For a set of atomically normalized action nodes, a single node within that set... Corresponding to a single basic experimental operation; Let be a set of directed logical edges, where each edge is a single edge. Connect two action nodes; It includes three types of constraint relationships, among which, This is a sequential dependency, meaning that a subsequent action can only proceed after the preceding action has been completed. This is a parallel permission relationship, indicating that there are no mandatory sequential restrictions on the two sets of actions, allowing them to be executed synchronously. To ensure mutual exclusion, the two sets of actions are prohibited from overlapping in time and operating synchronously, specifically including: Sequential dependency : That is, action Execution can only proceed after the entire process is completed. This constitutes a directed acyclic time constraint; Parallel licensing relationship : That is, there is no mandatory sequential constraint between the two actions, and they are allowed to be executed simultaneously and interspersed. and safety mutual exclusion relationship : This means that two actions must not be performed simultaneously or overlap in time, in order to meet experimental safety standards.

[0024] By finely distinguishing the constraints of three different types of logic—sequential, parallel, and mutually exclusive—the modeling granularity closely matches the actual experimental operation specifications. The textual operation procedures are transformed into quantified graph-structured prior knowledge, realizing the digitization and computability of rules, providing authoritative constraint basis for subsequent model reasoning, violation detection, and itemized scoring. Moreover, the graph construction logic is universal and can be quickly transferred and adapted to different chemical, physical, and biological experiments, with strong scalability.

[0025] S2. Acquire experimental operation videos and perform motion detection to obtain a set of motions and their corresponding time positions, specifically including: S21. Perform frame-by-frame analysis on the experimental operation video and extract the spatiotemporal features of the video using a pre-trained 3D convolutional neural network. The spatiotemporal feature extraction process is as follows: Let the input video segment tensor be... ,in The number of frames in a single video segment. The dimensions are: height and width, 3 represents the RGB three channels; 3D-ResNet, I3D and other 3D convolutional feature extractors are used to jointly encode spatial appearance features and short-term motion features to obtain a fixed-dimensional spatiotemporal fusion feature vector. The calculation formula is: , in, This is for 3D convolutional neural network operations; S22. The action detection model outputs action category labels and corresponding start and end times to form an action set. The specific process is as follows: The detection network employs a two-branch structure, with the classification branch outputting the probability distribution of action categories and the optimal predicted category. The calculation formula is ,in Features Corresponding category Predicted confidence probability; time offset output by the time series regression branch. The time boundary of the anchor point is corrected, and the formula for correcting the time parameter is as follows: , in, The actual start time of the action. The actual end time of the action. To preset the start time of the anchor point action segment, To preset the end time of the anchor point action segment, the final original action instance expression is: ,in Detect confidence level for the current action.

[0026] Specifically, the action detection model also includes a non-maximum suppression (T-NMS) post-processing step to remove redundant detection results of the same action category within overlapping time windows, retaining only the action with the highest confidence. The specific method is as follows: Introducing time-series intersection-exchange ratio After eliminating overlapping and redundant detection results, the formula for calculating the temporal intersection-over-union ratio is: , in, and These are the start and end times of the optimal baseline action instance, respectively; the NMS iteration execution logic is as follows: all original detection results under the same category are sorted by confidence level. Sort in descending order; select the instance with the highest current confidence level as the benchmark. Set the suppression threshold If the instance to be filtered and the benchmark instance satisfy... If the result is found to be overlapping or redundant, it is removed; the process is repeated until all results have been filtered.

[0027] The final output is a set of normalized action instances. ,in, For the first The standardized atomic action category label corresponding to each action instance and The first A real-world action example The start and end times of the action.

[0028] As can be seen, 3D convolutional simultaneous modeling of spatial visual features and temporal motion features significantly improves the accuracy of action recognition and temporal boundary localization compared to 2D networks; temporal nonmaximum suppression based on tIoU can accurately eliminate overlapping and redundant detection results, ensuring the uniqueness and validity of action instances in the input spatiotemporal graph; and the output of structured action temporal data enables end-to-end transformation from raw video to standardized action sequences, providing high-quality input for upper-layer rule reasoning.

[0029] S3. Transform the action set and its corresponding time position into a spatiotemporal graph, where the nodes of the spatiotemporal graph are action instances, the time edges connect adjacent time actions, and the spatial edges are established between the corresponding action sets based on the dependency knowledge graph; In other words, the spacetime graph is associated with the actual set of action instances measured. Knowledge graph of operating procedures Construct a spatiotemporal graph that integrates real-time behavior with domain-specific rule constraints. .

[0030] First, the nodes of the spatiotemporal graph are initialized, and each measured action instance is... Mapped to spatiotemporal graph nodes The initial feature vector of a node in a spacetime graph. It is composed of action semantic embedding and time position encoding concatenated, and the calculation formula is: , in, Action categories can learn semantic embedding vectors to encode the semantic information of the action itself; The normalized temporal location encoding function maps the start and end times and duration of an action to a high-dimensional feature space.

[0031] Secondly, construct time edges All action instances will be sorted by start time. Ascending order, for actions with adjacent time sequences Constructing directed time edges The actual sequence of experimental operations is fully preserved.

[0032] Finally, construct spatial logical edges. Match spatiotemporal graph nodes with knowledge graph atomic action nodes, based on Pre-stored relationship types allow you to add corresponding logical edges to node pairs. It embeds hard constraints in operating procedures to achieve a two-way integration of data-driven and knowledge-driven approaches.

[0033] In this dual-branch roadside structure design, the temporal edge depicts the real-time sequence of behavior, while the spatial edge constrains the logic of compliance rules. The two-dimensional information is decoupled and complementary. Action semantics and temporal information are jointly encoded, and node features simultaneously possess semantic and temporal attributes, adapting to subsequent multi-dimensional graph convolutional inference. This achieves deep integration of ruleless prior video data and structured procedural knowledge, solving the defects of pure data-driven models that lack logical constraints and have poor interpretability.

[0034] S4. A spatiotemporal graph convolutional network is used to encode the spatiotemporal graph. By iteratively updating node features layer by layer, neighborhood logical information and temporal correlation information are aggregated. The spatial graph convolutional layer aggregates spatial neighbor information using a weight matrix corresponding to the edge type based on the edge type of the spatial edge. The temporal graph convolutional layer introduces a temporal decay attention mechanism along the temporal edge. Among them, the edge-type-aware spatial graph convolution sets independent trainable weights for different logical relationships, and differentially aggregates sequential, parallel, and mutually exclusive neighbor features. The formula for calculating the spatial aggregation feature is: , in, For nodes In relationship The set of neighboring nodes; A relation-specific weight matrix is ​​used to distinguish the feature propagation logic for different constraints. The neighborhood number normalization coefficient; Assign self-loop weights to nodes, preserving their original features; It is a non-linear activation function to enhance the model's representational ability. For nodes In relationship The neighboring nodes No. Layer feature vectors, For the current node In the The input feature vector of the layer.

[0035] Temporally decaying attention map convolution introduces a time interval decay factor to simulate the objective law that actions with closer time intervals have stronger mutual influence, dynamically calculating the temporal neighborhood attention weights. The formula for calculating the temporal attention weights is: , in, For nodes Time-series neighbors Normalized attention weights For feature splicing operations; A learnable attention parameter vector; This is the temporal feature transformation matrix; As the core time decay term, This is the decay coefficient; the longer the time interval, the smaller the attention weight, and the stronger the weight suppression. The Leaky ReLU activation function is used to perform a non-linear transformation on the result of concatenating the attention vector and node features, enhancing the model's expressive power while preventing gradient saturation. Temporal Neighborhood Nodes No. Layer feature vectors.

[0036] The formula for weighted aggregation of time-series features is: , Finally, by using gating fusion or weighted summation, spatial logical features and temporal dynamic features are integrated to obtain the single-layer final node representation. ;through After multiple convolutional iterations, the output is a global context fusion feature. .

[0037] Thus, by using relational differential convolution to learn independent weights for illegal mutual exclusion, sequential dependence, and legal parallelism, the influence mechanism of different constraints can be accurately modeled; the time decay attention mechanism conforms to the objective law of the influence of experimental operation time sequence and can weaken the interference of distant irrelevant actions; the spatiotemporal dual convolutional collaborative encoding simultaneously completes the understanding of rule constraints and the modeling of temporal behavior, and the feature expression capability is far superior to the single graph structure model.

[0038] It should be noted that there are progressively related technologies between steps S1 to S4 in this embodiment. The dependency knowledge graph constructed in step S1... Three types of constraint relationships are defined, and this set of relationships provides the basis for determining the edge type when establishing spatial logical edges in step S3. The spatiotemporal graph constructed in step S3 contains two types of heterogeneous edges: time edges and spatial edges. The time edges are constructed based on the ascending order of the start times of the action instances, while the spatial edges are constructed based on the three types of relationships in step S1 as constraint rules. The two respectively carry the actual occurrence sequence information of the experimental operation and the logical constraint information of the operation procedure.

[0039] Step S4, spatial graph convolution, takes spatial edges as input and aggregates neighbor information using dedicated weight matrices according to edge type. If there is only one type of relationship, the dedicated weight matrix degenerates into a single shared weight matrix, and the edge type awareness mechanism is equivalent to ordinary graph convolution. Step S4, temporal graph convolution, takes temporal edges as input and introduces an exponential decay factor to weight and aggregate temporal neighbors. The decay coefficient controls the degree of influence of time interval on attention weights.

[0040] The spatial graph convolution output and the temporal graph convolution output are integrated by a fusion function to obtain node features. These features simultaneously contain constraint information in the spatial logical dimension and dynamic correlation information in the temporal dimension, providing feature input for the sub-item scoring in step S5. Between the above four steps, the output of the previous step serves as the input condition or constraint parameter for the next step, and the functional implementation of each step depends on the data structure and relation definition established in the preceding steps.

[0041] S5. Based on the encoded node features, calculate the dependency satisfaction of sequential dependencies, the parallel rationality of parallel relationships, and the mutual exclusion violation penalty of mutual exclusion relationships. These three scoring branches quantify the sequential compliance, parallel rationality, and mutual exclusion violation degree, respectively. Finally, they are weighted and fused to obtain a comprehensive generated sequential normativity score.

[0042] It should be noted that the aforementioned dependency satisfaction degree The calculation method is as follows: For each pair of actions with sequential dependencies defined in the dependency knowledge graph, compare their order of occurrence in actual operation; If the dependency direction is satisfied, positive points are contributed; otherwise, negative or zero points are contributed. The specific calculation process is as follows: First, traverse all sequentially dependent action pairs in the knowledge graph. ; Then, a temporally constrained sign function is introduced, which is combined with the learnable scoring network for computation. The specific formula is as follows: , , in, This is a symbolic function that outputs 1 if the order constraint is satisfied and -1 if the constraint is violated, thus implementing hard rule penalties. It is a lightweight, fully connected scoring head that learns latent features related to action matching. and This is the final feature vector of the corresponding node after L layers of encoding. and These are the spatiotemporal graph instance nodes corresponding to two sets of actions that have sequential dependency constraints.

[0043] The rationality of parallelism The calculation method is as follows: For two actions defined as parallelizable in the dependency knowledge graph, calculate their temporal overlap in the experimental operation video. The higher the degree of overlap, the higher the score for parallel rationality.

[0044] The specific calculation process is as follows: For graph parallel licensing action pairs Using temporal overlap as a quantitative indicator, the specific formula is as follows: , , The higher the score, the more fully parallel actions can be executed synchronously, and the more reasonable and efficient the operation process is.

[0045] The mutual exclusion violation penalty item The calculation method is as follows: For two actions defined as mutually exclusive in a dependency knowledge graph; If an overlap is detected in time, a penalty score is applied based on the duration of the overlap; the longer the overlap, the greater the penalty.

[0046] The specific calculation process is as follows: For safe mutual exclusion action pairs The detection times overlap and the penalty is accumulated. The specific formula is as follows: , , The higher the penalty value, the more serious the safety violation.

[0047] Finally, multi-dimensional indicators are linearly fused using configurable weights, and then normalized using Sigmoid activation. The interval is calculated using the following formula: , in, The weights are configurable by humans to adapt to the stringency of different experimental rules; For bias terms; This is the Sigmoid activation function, used to normalize the scores.

[0048] This method's multi-dimensional decomposition scoring system can accurately locate three types of problems: sequential errors, parallel waste, and safety mutual exclusion violations. The combination of rigid rule symbol constraints and flexible model learning ensures that the mandatory constraints of operating procedures are implemented while adapting to the flexible deviations of real operations. The comprehensive score is normalized and outputs in a way that facilitates quantitative rating, batch assessment, and digital teaching evaluation, making it highly applicable.

[0049] Example 2 As one embodiment, this embodiment further defines the configuration method of the weight coefficients in the comprehensive scoring fusion process based on embodiment 1, so as to adapt to different experimental types and scoring requirements and improve the versatility and scenario adaptability of the solution.

[0050] Because the focus of scoring for experimental operation exams varies significantly across different disciplines, chemical experiments involving hazardous chemicals and high-temperature heating have extremely strict requirements for safety mutual exclusion constraints, and any violations should be severely punished. Meanwhile, some biological experiments emphasize operational efficiency and have high requirements for the rationality of parallel operations, necessitating an increased weighting for the parallel rationality indicator. Routine basic experiments, on the other hand, focus on assessing the standardization of standard operating procedures, requiring priority to be given to ensuring the satisfaction of sequence dependencies.

[0051] To address this, this embodiment provides a configurable weight parameter template mechanism. For different experiment types, multiple standardized weight coefficient configuration schemes are preset, which can adapt to different scoring requirements without modifying the core algorithm logic, simply by adjusting the parameters.

[0052] In this embodiment, the weighting coefficient , , In addition to satisfying the non-negativity constraint described in Example 1, a normalization constraint is further added to ensure the rationality and consistency of the weight allocation. The normalization constraint formula is as follows: , Bias term The default value is The scoring criteria can be fine-tuned according to actual scoring needs to ensure that the scoring results meet the assessment standards.

[0053] The specific weight configuration schemes are divided into the following three categories, each corresponding to different experimental scenarios: (1) Safety-first configuration This configuration is suitable for high-risk experiments involving hazardous chemicals, high-temperature heating, and high-pressure operations. Its core objective is to strengthen the penalties for safety violations and ensure the safety of experimental operations. The weighting coefficients in this configuration meet the following requirements. This ensures that the weight of mutually exclusive violation penalties is the highest, and any security mutually exclusive violation can be fully reflected in the comprehensive score.

[0054] Typical parameter values ​​for this configuration are: , , .

[0055] (2) Process-standard configuration This configuration is suitable for routine, fundamental experiments that focus on assessing standard operating procedures. Its core objective is to evaluate the temporal accuracy of experimental operations. The weighting coefficients in this configuration satisfy the following requirements: Prioritize compliance with the operational sequence and ensure strict adherence to standard operating procedures.

[0056] Typical parameter values ​​for this configuration are: , , .

[0057] (3) Efficiency-balanced configuration This configuration is suitable for experimental scenarios that allow parallel operations and encourage high efficiency. Its core objective is to evaluate the efficiency of experimental operations while ensuring a standardized operational sequence. The weighting coefficients in this configuration satisfy the following requirements: The rationality of parallel execution is given higher scores, guiding experimenters to complete the operation efficiently.

[0058] Typical parameter values ​​for this configuration are: , , .

[0059] In actual deployment, the system can automatically match the corresponding weight configuration scheme according to the exam subject and experiment number, generating a comprehensive score that conforms to the experiment's scoring criteria without manual intervention. For example, chemical experiments are automatically matched with a safety-first configuration, biological high-efficiency experiments are automatically matched with an efficiency-balanced configuration, and routine experiments are automatically matched with a standardized process configuration.

[0060] Through the configurable weight mechanism described above, the scoring framework of this method can flexibly adapt to the scoring standards of different disciplines and experiments. There is no need to retrain the model or modify the core algorithm logic. Only the weight parameters need to be adjusted to complete the scene switching. It has good versatility and convenient deployment, and reduces the cost of algorithm iteration and scene adaptation.

[0061] Example 3 As one embodiment, this embodiment further defines the output and application of the scoring results based on embodiment 1 or embodiment 2, transforming the abstract comprehensive score into a visual report with clear teaching feedback value, realizing the interpretability, traceability, and improvement of the scoring results, and enhancing the educational application value of the solution.

[0062] Specifically, this embodiment obtains a comprehensive order normativity score. and the scores for each item (sequence dependency satisfaction) Parallelism rationality Mutually exclusive violation penalties After that, the following three processing steps are executed in sequence: rating level mapping, violation location and labeling, and generation of a visual rating report, as follows: First, normalize the overall score. The continuous quantitative score is compared with a preset rating threshold to convert the operational standardization rating result into a discrete rating result, facilitating a direct assessment of the standardization of experimental operations. This embodiment adopts a five-level rating system, and the threshold divisions and their meanings are explained in the table below:

[0063] Then, iterating through each action pair involved in the three scoring criteria—dependency satisfaction, parallel rationality, and mutual exclusion violation penalty—and combining rigid time-series rules with quantitative indicators, the specific action pairs with abnormal scores are identified, categorized, and labeled to clarify the violation type, involved actions, and time information, providing a precise basis for subsequent improvements. The specific screening and labeling rules are as follows: (1) Violation of sequence dependency positioning Filter to meet Actions This is marked as a "sequence violation". For the existence of a sequence dependency Two spatiotemporal graph instance nodes, Indicates the preceding action Incomplete That is, to begin executing subsequent actions. Synchronously record the corresponding action category name and the time position of the two actions. and This makes it easier to trace the violation scenarios.

[0064] (2) Violation of mutually exclusive relationship positioning Filtering satisfies Actions This is marked as a "security mutual exclusion violation". For the existence of a safe mutual exclusion relationship Two spatiotemporal graph instance nodes, This indicates that two actions overlap in time. The corresponding action category name and time overlap interval are recorded simultaneously to determine the severity of the violation.

[0065] (3) Parallel relationships are not adequately located Filtering satisfies Actions This is marked as "insufficient parallel utilization". For the existence of a parallel licensing relationship Two spatiotemporal graph instance nodes, As a threshold for parallel rationality, in this embodiment It can be fine-tuned according to the type of experiment; This indicates that two parallel actions were not executed synchronously, resulting in wasted efficiency. Synchronizing the time intervals and non-overlapping periods of parallel actions provides a basis for improving operational efficiency.

[0066] Finally, the rating results, sub-item scoring data, and violation labeling information are integrated to generate a structured and visualized rating report, providing an intuitive display and accurate feedback of the rating results. The report comprises the following four core modules, each with the following functions: (1) Comprehensive score display area This module is used to display the overall evaluation results of the experimental operation, specifically including: normalized comprehensive score. Convert to a percentage score for display; clearly indicate the corresponding rating level (AE level); display the sequence dependency satisfaction in the form of a bar chart. Parallelism rationality Mutually exclusive violation penalties The three sub-scores allow for a direct comparison of the performance of each indicator, enabling quick identification of key issues.

[0067] (2) Time series normalization analysis area This module displays the timeline of all detected actions in the form of a Gantt chart, clearly showing the start time, end time, and duration of each action. At the same time, it marks three types of relationship edges on the Gantt chart with different colors: sequential dependency, parallelism, and mutual exclusion. Violation edges (sequential violation and mutual exclusion violation) are highlighted in red, intuitively showing the temporal relationship of violation actions, which facilitates quick tracing of violation scenarios.

[0068] (3) List of Violations This module lists all detected violations one by one. Each violation includes: violation type (sequential violation, security exclusion violation, insufficient parallel exploitation), the specific action name involved, the time interval in which the action occurred, and a quantified value of the violation severity (e.g., ...). The system displays the time deviation of numerical and sequential violations, along with the corresponding video frame position index. Clicking the index will directly jump to the corresponding video segment, enabling precise tracing of the violation scene.

[0069] (4) Improvement Suggestion Area This module automatically generates customized text suggestions based on the type of violation, combining standard operating procedures to clearly point out the hazards of the violation and the correct operating methods. For example, when a safety-incompatible violation is detected, it prompts, "The operations 'lighting the alcohol lamp' and 'adding alcohol' overlap in time, which seriously violates the safety operating procedures. Please be sure to complete the alcohol addition before lighting the alcohol lamp to avoid fire hazards." When a sequence violation is detected, it prompts, "The operation 'taking out the reagents' should be performed after 'placing the experimental equipment' is completed. Please adjust the operation sequence to ensure that the process is standardized."

[0070] Through the aforementioned visualization reports, the scoring results not only provide a macro-level grade assessment but also pinpoint irregularities in the operational process. This allows candidates to clearly understand the reasons for point deductions and specific violation scenarios, and to make targeted improvements based on the suggestions. Teachers can also use this information to conduct precise teaching and guidance, focusing on explaining frequently violated rules, significantly enhancing the educational application value and user trust of the automated scoring system.

[0071] It should be noted that the technical features described in Embodiments 2 and 3 can be implemented in combination with Embodiment 1, or Embodiments 2 and 3 can be implemented in combination with Embodiment 1 at the same time to obtain a complete scoring solution that has both configurable scoring weights and can output a visual diagnostic report, further expanding the functionality and application scenarios of the solution.

[0072] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An experimental operation sequence normality scoring method based on a spatio-temporal graph convolution network, characterized in that, Specifically, it includes: By defining the sequential dependencies, parallelism, and mutual exclusion relationships between actions, a dependency knowledge graph of experimental operations is constructed. Acquire experimental operation videos and perform motion detection to obtain a set of motions and their corresponding time positions; The action set and its corresponding time position are transformed into a spatiotemporal graph, where the nodes of the spatiotemporal graph are action instances, the time edges connect adjacent time actions, and the spatial edges are established between the corresponding action sets based on the dependency knowledge graph. Spatiotemporal graph convolutional networks are used to encode the spatiotemporal graph. The spatial graph convolutional layer aggregates spatial neighbor information using a weight matrix corresponding to the edge type based on the edge type of the spatial edge, and the temporal graph convolutional layer introduces a temporal decay attention mechanism along the temporal edge. Based on the encoded node features, the dependency satisfaction of sequential dependencies, the parallel rationality of parallel relationships, and the mutual exclusion violation penalty of mutual exclusion relationships are calculated, and a sequential normality score is generated in combination.

2. The spatio-temporal graph convolution network based experimental operation sequence normality scoring method according to claim 1, characterized in that: The dependency knowledge graph is a directed graph. Nodes in the graph represent atomic actions, and edges represent temporal dependencies. A sequential dependency edge indicates that one action must be completed before another action, a parallel edge indicates that two actions can be executed simultaneously, and a mutual exclusion edge indicates that two actions cannot be executed simultaneously.

3. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The acquisition of experimental operation videos and motion detection specifically includes: The experimental operation video was analyzed frame by frame, and the spatiotemporal features of the video were extracted using a pre-trained 3D convolutional neural network. The action detection model outputs action category labels and corresponding start and end times to form an action set.

4. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 3, characterized in that: The action detection model also includes a non-maximum suppression post-processing step, which removes redundant detection results of the same action category within overlapping time windows and retains the action with the highest confidence.

5. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The decay function of the time decay attention mechanism is an exponential decay function, where the larger the time interval between the two action instances connected by the time edge, the smaller the attention weight.

6. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The dependency satisfaction degree is calculated as follows: For each pair of actions with sequential dependencies defined in the dependency knowledge graph, compare their order of occurrence in actual operation; If the dependency direction is satisfied, positive points are contributed; otherwise, negative or zero points are contributed.

7. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The calculation method for the parallel rationality is as follows: For two actions defined as parallelizable in the dependency knowledge graph, calculate their temporal overlap in the experimental operation video. The higher the degree of overlap, the higher the score for parallel rationality.

8. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The calculation method for the mutual exclusion violation penalty is as follows: For two actions defined as mutually exclusive in a dependency knowledge graph; If an overlap is detected in time, a penalty score is applied based on the duration of the overlap; the longer the overlap, the greater the penalty.

9. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: This method also includes: A comprehensive order normativity score is generated by fusing the scores of dependency satisfaction of sequential dependencies, parallel rationality of parallel relationships, and mutual exclusion violation penalty of mutual exclusion relationships. The weight coefficients of each score are configured according to the experiment type and scoring requirements.

10. The experimental operation sequence standardization scoring method based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: This method also includes: The comprehensive sequence standardization score is compared with the preset score level threshold, the corresponding operation standardization rating result is output, and a visual score report is generated. The report marks the specific actions that violate the sequence dependency, parallel relationship or mutual exclusion relationship and the corresponding time position.

Images