A trainee progress tracking system

Abstract

The present application relates to the technical field of supervision and management of training process, and discloses a trainee training progress tracking system, which comprises the following steps: capturing time sequence interaction data of a trainee's operation process, and converting the time sequence interaction data into a capability feature vector representing real operation capability; the system further calculates a trend parameter of capability evolution based on historical vectors, and adjusts a consistency check threshold for identifying the authenticity of current behavior by using the trend parameter, so that an objective progress evaluation is generated only after the check is passed, wherein the evaluation basis is changed from a traditional task completion state to data capable of understanding the quality of the operation process, and the diagnosis of the real mastery degree of the trainee is realized.

Description

Technical Field

[0001] This invention relates to a trainee training progress tracking system, belonging to the technical field of training process supervision and management. Background Technology

[0002] Currently, a common practice is to track the progress of trainees in completing preset task modules through a recording system. This method can provide managers with evidence that trainees have completed all learning steps as required, and it is widely used due to its intuitiveness and ease of deployment in management.

[0003] However, when this management approach is applied to highly skilled positions, inherent technical constraints arise. There is no direct technical correspondence between the task completion status recorded by the system and the goal of whether trainees have mastered practical skills. When managers rely on such systems for decision-making, their pursuit of complete task records can lead to a lack of information on assessing the trainees' true level of competence. For example, a trainee whose progress is recorded as complete may exhibit behavioral characteristics such as hesitation, repeated corrections, or inefficiency in path planning during the operation process, which cannot be assessed by the existing system's data collection and processing mechanisms. This constitutes a potential management risk.

[0004] To address the aforementioned shortcomings, the industry has attempted to introduce operational process records for expert review. However, this approach has proven difficult to implement on a large scale due to the cost and subjectivity of manual review. Therefore, existing technologies generally suffer from the following problems: 1. Insufficient information dimensions for assessment; the system only records the endpoint of the task, losing more process-related information that better reflects actual ability; 2. The assessment process relies on post-training manual review, lacking real-time diagnostic and intervention capabilities during training; 3. The completeness of task records may mask deficiencies in actual operational ability, thus conveying incomplete assessment information to management. Therefore, the technical problem this invention aims to solve is how to design a management and monitoring system that can automatically collect and analyze process interaction data from trainees' operations, and based on this, establish an assessment mechanism that objectively reflects the trainees' true level of mastery, thereby providing managers with a more comprehensive decision-making basis. Summary of the Invention

[0005] This invention provides a trainee training progress tracking system, the main purpose of which is to solve the problem that existing progress tracking systems only record the task completion status and cannot collect and analyze the behavioral characteristics of trainees during the operation process, making it difficult to objectively assess their true level of mastery and potentially leading to incomplete management decision-making basis.

[0006] To achieve the above objectives, the present invention provides a trainee training progress tracking system, the system comprising: The interaction sequence capture module is configured to capture non-preset result-oriented time-series interaction data generated by the interaction between the trainee and the task interface during the trainee's performance of training tasks; The capability vector engine is configured to transform time-series interactive data into capability feature vectors that characterize the learners' operational abilities, and to quantitatively compare the learners' capability feature vectors with preset capability feature vectors to generate capability deviation analysis results. The chronic trend analyzer is configured to calculate trend parameters that characterize the long-term evolution of a learner's abilities based on a set of historical learner ability feature vectors generated from multiple historical training tasks. The consistency verification module has a variable judgment threshold. It is configured to receive trend parameters and adjust the judgment threshold according to the trend parameters. Then, it uses the adjusted judgment threshold to verify the consistency between the current task's trainee ability feature vector and its historical trainee ability feature vector set, so as to output a verification conclusion on the authenticity of the ability deviation analysis results. The progress assessment module is configured to generate an assessment of the trainees' actual training progress based on the capability deviation analysis results if and only if the verification conclusion is true.

[0007] Preferably, the time-series interaction data includes at least one of: mouse movement trajectory data, keyboard input timing data, or interface element interaction sequence data.

[0008] Preferably, the capability feature vector is a baseline vector generated by aggregating time-series interaction data of one or more expert users performing the same training task through a preset vectorization model.

[0009] Preferably, it also includes: an adaptive recommendation module, which is configured to determine the corresponding reinforcement training content based on the specific dimensions of the bias in the ability bias analysis results, in a preset knowledge graph that stores the correspondence between ability weaknesses and reinforcement training content, and push the reinforcement training content to the trainees.

[0010] Preferably, the trend parameter is a state stability index that quantitatively represents the growth gradient of the trainee's ability over time or the stability of the trainee's operating mode.

[0011] Preferably, the adjustment of the judgment threshold in the consistency verification module follows a preset adjustment rule, which is such that: when the trend parameter indicates that the student's ability shows a long-term positive growth trend, the judgment threshold is raised; when the trend parameter indicates that the student's ability shows a long-term stagnation or negative trend, the judgment threshold is lowered.

[0012] Preferably, the process by which the capability vector engine transforms time-series interaction data into learner capability feature vectors includes: performing a preset statistical transformation on the time-series interaction data to extract process quality feature values ​​that characterize operational redundancy, decision hesitation frequency, or behavioral smoothness, and combining the process quality feature values ​​into a multi-dimensional vector.

[0013] Preferably, the consistency verification module performs consistency verification by comparing the current task’s student ability feature vector with the set of historical student ability feature vectors that serve as the baseline of the student’s own behavior, in order to identify statistically significant atypical fluctuations and thus identify abnormal behavior patterns such as spoofing or gaming.

[0014] Preferably, the system further includes a processor and a memory, the memory storing a computer program, and the processor is configured to execute the computer program to implement the functions of the interactive sequence capture module, the capability vector engine, the chronic trend analyzer, the consistency verification module, and the progress assessment module.

[0015] Preferably, the system also includes a user terminal, and the adaptive recommendation module is configured to push reinforcement training content to the user terminal for presentation, thereby forming a closed loop from capability diagnosis to training intervention.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. By setting up an implicit capability probe module to capture the interaction sequence data of trainees during task execution, and using a capability feature vector generation and comparison engine to transform this unstructured sequence data into a trainee capability feature vector that can be quantitatively compared, the evaluation basis for trainee training progress changes from the traditional endpoint state data that only reflects whether the task has been executed to process quality data that can directly characterize their operational proficiency decision-making mode and knowledge mastery. The evaluation gap between task completion and trainees' actual ability in the original management method is bridged under the operation mechanism of this system, and the progress evaluation results obtained by managers are directly related to the trainees' actual work level.

[0017] 2. After obtaining the results of the competency deviation analysis, the diagnostic path adaptive recommendation module within the system will automatically match and push corresponding reinforcement training content from the knowledge graph based on the specific competency gaps indicated by the analysis results. In this way, a feedback loop is formed between the diagnostic information output by the comparison engine and the intervention actions executed by the recommendation module. The operation of this loop makes the allocation of training resources no longer dependent on preset fixed procedures or manual judgment, but driven by the real competency needs exposed by trainees in operation, enabling the entire training process to have the ability to self-optimize for individual differences.

[0018] 3. This system further incorporates a behavioral pattern consistency verification module. This module does not analyze the interaction behavior of a single task, but rather analyzes multiple ability feature vectors generated by the same learner in multiple different types of tasks over a longitudinal period. By examining the consistency and fluctuation patterns of these vectors over time, the system can identify occasional atypical behavioral patterns that do not conform to the learner's own ability baseline. The existence of this mechanism provides an internal verification layer for the authenticity of the ability deviation analysis results, avoiding deviations in the system's assessment of true ability due to the learner's occasional extraordinary or abnormal performance, thus enhancing the stability and credibility of the entire progress tracking system.

[0019] 4. To further enhance the adaptability of the supervision logic, the system also links the behavior pattern consistency verification module with a chronic ability trend analyzer. This analyzer continuously integrates historical ability feature vectors to generate a trend parameter characterizing the long-term ability evolution of trainees, such as the rate of ability growth or the stability of the state. Subsequently, the adaptive risk response threshold adjustment logic in the consistency verification module receives this trend parameter and dynamically adjusts its judgment threshold for identifying abnormal behavior patterns based on this parameter. In this way, the system's tolerance for errors in supervision judgment will be appropriately relaxed for a trainee whose abilities are continuously improving; while for a trainee whose abilities have stagnated for a long time, supervision will become more sensitive. The scale of supervision thus changes from a fixed standard to a dynamically changing personalized standard that matches the individual's development trajectory. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the overall system architecture and closed-loop evaluation process of the present invention. Figure 2 This is a typical pattern diagram illustrating the evolution trend of student operation redundancy in this invention; Figure 3 This is a timing diagram of the module interaction for dynamic threshold adjustment in this invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] In the specific implementation of this invention, the system design strictly follows the principles of data processing minimization, purpose limitation, and user informed consent to ensure that trainees' privacy is fully protected. The time-series interaction data collected by the system is limited to the operational behavior during the execution of specific training tasks and does not involve any personal information unrelated to the task. Before data collection, the system will fully inform trainees of the type, purpose, and processing method of the collected data through clear agreements or prompts and obtain their authorization and consent. All collected raw data and derived capability feature vectors are encrypted, stored, and processed in the background and are used only for capability assessment, trend analysis, and personalized training recommendations described in this invention, and are not used for any other purpose.

[0023] This invention provides a trainee training progress tracking system. In application scenarios, such as the management process for training and certification of financial risk auditors, its system architecture and data processing flow are clearly demonstrated. Logically, the system includes an interaction sequence capture module, a capability vector engine, a chronic trend analyzer, a consistency verification module, and a progress assessment module. In optional implementations, an adaptive recommendation module may also be included. During system operation, the interaction sequence capture module collects the process data stream generated by trainees operating on a simulated business system. The capability vector engine receives this data stream and transforms it into structured capability feature vectors. The chronic trend analyzer calculates trend parameters representing the trainee's long-term performance based on historical capability feature vector sets. The consistency verification module dynamically adjusts its internal judgment threshold based on these trend parameters and verifies the validity of the current vector. After successful verification, the progress assessment module generates an assessment report related to the trainee's operational capabilities, thus providing an objective basis for training managers' intervention and certification decisions. A common challenge in training management practice is that traditional progress recording systems can only reflect the trainee's... While it's possible to assess whether the prescribed training tasks have been completed, the quality of the operational process during task completion cannot be quantified. For example, a trainee might spend several times longer than the standard procedure and undergo numerous trial-and-error operations to finally complete a simulated risk audit task. Traditional systems would still mark this as completed, resulting in information gaps at the management level. To address this challenge, the interaction sequence capture module in this invention is configured to utilize existing event listening mechanisms in the operating system or application programming interface to capture non-preset result-oriented time-series interaction data generated by the trainee's interaction with the task interface during training tasks. This data is structurally organized as a timestamp sequence containing multiple event records, each of which includes at least the event occurrence time, event type (e.g., mouse movement, keyboard press, interface element click), and related coordinates or key values. In this way, the module transforms all previously overlooked hesitation, repetition, or correction behaviors during the trainee's operation into a complete, objective, and analytically applicable raw data record, providing a data foundation to compensate for the aforementioned lack of management information.

[0024] The acquired time-series interaction data is raw and unstructured. Directly using it for evaluation would face management obstacles due to its high data dimensionality and lack of comparability. Therefore, the system's capability vector engine is configured to execute a process that transforms raw data into structured indicators. This engine receives the time-series interaction data output by the interaction sequence capture module and performs preset statistical transformations on it to extract process quality features representing operational redundancy, decision hesitation frequency, or behavioral smoothness. These process quality features are then combined into a multi-dimensional learner capability feature vector. For example, operational redundancy characteristic value The decision hesitation frequency characteristic value can be obtained by calculating the ratio of the actual length of the student's mouse movement trajectory to the optimal trajectory length for completing the same task. The system can count the number of times a user hovers their cursor over a key decision button for more than a preset time threshold (e.g., ...). The number of unclicked events (in milliseconds) is obtained, while the behavior smoothness feature value is... This can be obtained by calculating the standard deviation of the rate of change of curvature of the mouse movement trajectory. Assuming a specific task, the calculation yields... , , Then, the student's ability feature vector for this task can be represented as: Subsequently, the capability vector engine generates the student's capability feature vector. With a preset capability feature vector For quantitative comparison, this capability feature vector is a baseline vector generated by aggregating time-series interaction data of one or more expert users performing the same training task through a pre-defined vectorization model. For example, if the capability feature vector is... The comparison will then generate a capability deviation analysis result, such as a deviation vector. This deterministic data processing flow transforms process behavior into a standardized capability assessment criterion that can be used for quantitative comparison.

[0025] The results of a single operational assessment may fluctuate, while long-term tracking of trainee ability changes provides a more stable basis for management judgment, which is necessary for identifying individuals who are continuously improving or stagnating in the long term. To this end, the system incorporates a chronic trend analyzer, configured to calculate a trend parameter characterizing the long-term evolution of a trainee's ability based on a set of historical trainee ability feature vectors generated from multiple historical training tasks. In one specific implementation, this trend parameter can be a quantitative representation of the growth gradient of a trainee's ability over time. The analyzer extracts the trainee's most recent... For example The set of student ability feature vectors for the task A key dimension (e.g., representing operational redundancy) The numerical sequence of a dimension is used, and the least squares method is applied to this sequence for linear regression analysis. The slope of the resulting line is defined as the capability growth gradient of that dimension. For example, if a trainee's operational redundancy is as follows in 5 consecutive tasks: Calculations show that the ability growth gradient is negative, indicating that operational efficiency is improving. To ensure the timeliness of this trend parameter, the chronic trend analyzer uses a sliding time window mechanism during operation. That is, after each new task is completed, the latest trainee ability feature vector is included in the calculation set, while the oldest vector is removed, thus ensuring that the trend parameter can dynamically reflect the trainee's recent development status. By introducing the chronic trend analyzer, the system connects a series of discrete evaluation points into an ability development curve, providing managers with decision-making information to judge trainee growth potential.

[0026] In the management system, the reliability of assessment data is the foundation for all subsequent decisions, especially in critical stages such as certification and assessment. Mechanisms need to be established to identify atypical data generated by cheating or random fluctuations, preventing them from impacting management decisions. To this end, the system further includes a consistency verification module with variable thresholds to internally verify the validity of the trainee ability feature vectors obtained from the current task. Specifically, this module verifies the validity of the trainee ability feature vectors from the current task. The system performs a statistical comparison with the set of historical student ability feature vectors, which serves as the baseline for the student's behavior, to identify statistically significant atypical fluctuations. The judgment threshold is not a fixed value but is dynamically adjusted by an adaptive risk response threshold adjustment logic. This logic receives trend parameters output by a chronic trend analyzer and adjusts the judgment threshold accordingly. The adjustment rule is such that when the trend parameter indicates a long-term positive growth trend in the student's ability, the judgment threshold is appropriately increased, giving the student greater tolerance for performance fluctuations. Conversely, when the trend parameter indicates a long-term stagnation or negative trend in the student's ability, the judgment threshold is correspondingly decreased, making the system more sensitive to any operations that deviate from its inherent behavioral pattern. For example, the judgment threshold... The calculation can be set according to the following formula: ,in, This represents the final judgment threshold after dynamic adjustment; This represents a preset baseline threshold, which can be calibrated through statistical analysis of a large number of student samples; This represents a sensitivity coefficient used to adjust the weight of the trend parameter on the threshold. This represents the aforementioned growth gradient, which characterizes the changes in the trainees' abilities in a specific dimension.

[0027] Using the adjusted judgment threshold After verifying the consistency between the current trainee's ability feature vector and their historical behavioral patterns, this module outputs a verification conclusion regarding the authenticity of the ability deviation analysis results. This mechanism enhances the stability and credibility of the evaluation results. The progress evaluation module is configured as the system's final data output stage, aiming to provide managers with a verified basis for decision-making. Its operation follows a precondition: only when the verification conclusion output by the consistency verification module is true will this module generate an evaluation of the trainees' actual training progress based on the ability deviation analysis results generated by the ability vector engine. If the verification conclusion is false, the system will mark this task as data anomaly and prompt the manager for manual review. This design ensures that every evaluation report submitted to management has undergone internal verification of data authenticity, thereby... The system ensures the quality of information upon which management decisions are based from a process perspective. Furthermore, to connect the diagnosis and intervention stages, the system can include an adaptive recommendation module. This module is configured to determine corresponding reinforcement training content from a pre-defined knowledge graph based on specific dimensions of deviation found in the capability deviation analysis results. This knowledge graph stores the correspondence between capability vector dimensions, specific capability weaknesses, and corresponding reinforcement training modules in the form of a data structure. For example, when the system detects that a trainee's deviation value in the dimension of decision hesitation is consistently too high, the adaptive recommendation module will automatically query the knowledge graph and push a reinforcement training module specifically targeting decision decisiveness to the trainee's user terminal. In this way, the system forms a feedback path from diagnosis to intervention, enabling the system to automatically recommend training content based on individual needs.

[0028] Example 1: In a certification assessment scenario for compliance specialists combating financial violations in financial institutions, the system of this invention is deployed on a transaction analysis simulation platform. The core task of this assessment is to require trainees to identify potential illegal networks from a dataset containing tens of thousands of transaction records within a limited time and submit corresponding suspicious transaction reports. In this scenario, the assessment record of trainee A shows that he / she has completed all training subjects and submitted reports that meet the quantity requirements and whose conclusions are correct. According to the traditional management system based on task completion status, the trainee's progress assessment result is qualified. At the same time, this system runs synchronously as a background supervision and management tool. The system's interaction sequence capture module records all time-series interaction data between trainee A and the simulation platform interface during the execution of the assessment task. Subsequently, the capability vector engine receives this data and transforms it into a trainee capability feature vector representing his / her operational capabilities. ,when Compared with the preset capability feature vector After comparison, the generated capability deviation analysis results showed that trainee A's deviation values ​​in the two dimensions of operational redundancy and number of decision hesitations were higher than the preset normal fluctuation range. This indicates that although the trainee found the correct answer, his analysis process involved a large number of ineffective path explorations and repeated hesitations. The difference between this management approach that only focuses on the end result and the evaluation approach that focuses on process quality is a long-standing problem in the management of this type of job training. That is, it is difficult to ensure management efficiency while taking into account the in-depth assessment of the individual's mastery of capabilities.

[0029] This system addresses this issue through the coordinated operation of its internal mechanisms. First, the system's chronic trend analyzer retrieves the historical set of student ability feature vectors generated from all past training tasks for student A and calculates the trend parameter of their ability growth gradient. The calculation shows that the student's ability growth gradient in the operational redundancy dimension is close to zero, indicating that their inefficient analysis pattern is not a coincidence in this assessment but a long-standing behavioral pattern. This objective trend parameter is then sent to the consistency verification module and received by its internal adaptive risk response threshold adjustment logic, used to dynamically lower the threshold for identifying abnormal behavioral patterns. This process demonstrates the synergistic effect between the two modules: the long-term trend data provided by the chronic trend analyzer provides a dynamic and personalized verification benchmark for the consistency verification module's single-behavior identification, making the monitoring scale no longer fixed but matched with the student's own development trajectory. Furthermore, the system does not directly address the problem of how to determine whether a student's correct answer stems from chance, but rather transforms the evaluation basis from endpoint state data into ability feature vectors that characterize process quality, thereby improving the original evaluation problem. The system was redefined; it no longer focuses on whether the answer is correct, but rather on whether the process of arriving at the answer possesses the necessary operational characteristics. Since there is a stronger correlation between the quality of the operational process and the level of ability, this shift in evaluation criteria, from a management logic perspective, avoids the risk that correct results might mask insufficient ability, as is common in traditional methods. Ultimately, because Trainee A's ability characteristic vector in this assessment matches his long-standing inefficient behavioral patterns, the consistency verification module outputs a true result. The progress assessment module then generates an assessment report based on the aforementioned ability deviation analysis results, which showed significant discrepancies. This report's conclusion was not "qualified," but rather that there were risks in operational efficiency and decision-making patterns, recommending specialized intensive training. After receiving this assessment report, the assessment manager, combined with the intensive training recommendations for analysis path planning automatically pushed by the adaptive recommendation module, decided to postpone granting Trainee A the job qualification. Trainee A was redirected to a training path focused on improving his analytical methodology, thus preventing a potentially unqualified person from being deployed to a critical compliance position. The management risks that might have arisen due to insufficient information dimensions in the original management process were effectively controlled under the operation of this system.

[0030] Example 2: To verify the effectiveness of the system of the present invention in distinguishing between the task completion status and operational ability of trainees, a comparative experiment was conducted. The purpose of the experiment was to objectively evaluate the performance difference between the present system and the traditional progress tracking system in identifying trainees whose task results are correct but whose operational processes have potential risks. To this end, a supply chain management simulation training platform based on enterprise resource planning software was built. This platform can simulate and generate purchase order processing tasks that include problems such as data inconsistency, demand fluctuations, and interruption risks. The experiment recruited 20 trainees who had completed basic theoretical training but had no practical experience and randomly divided them into two groups of 10 each. One group used the traditional progress tracking system, and the other group used the present invention system. The system in the control group only recorded whether the trainees correctly completed the approval process of the purchase order within the specified time, that is, the end result of the task was used as the evaluation basis. The experimental group deployed the present invention system, which, without interfering with the trainees' operation, used its interaction sequence capture module to record all time-series interaction data of the trainees during the order processing process. To increase the discrimination of the experiment, 4 specific trainees were pre-selected among the 20 trainees. They were known to be accustomed to using trial and error methods, which could achieve the task objectives but had low process efficiency.

[0031] Two groups of trainees were required to independently complete 20 pre-set purchase order processing tasks within 4 hours. After the experiment, the task completion status of all trainees was statistically analyzed. The results showed that the accuracy rate of the task endpoint results for all trainees in both the control and experimental groups, including 4 specific trainees, was greater than or equal to 90%. Based solely on this indicator, all trainees should be rated as qualified. However, for the trainees in the experimental group, the capability vector engine in this invention transformed their time-series interaction data during the task process into trainee capability feature vectors containing dimensions such as operational redundancy and number of decision hesitations. For the aforementioned 4 specific trainees, the capability deviation analysis results generated after comparing their trainee capability feature vectors with the capability feature vectors showed that the deviation value of the operational redundancy dimension exceeded 3.5, and the deviation value of the number of decision hesitations exceeded 10. Both of these values ​​were higher than the average level of other trainees in the experimental group. To intuitively present the differences in the evaluation capabilities of the two systems, the key evaluation indicators were compiled, as shown in Table 1.

[0032] Table 1: Comparison of key evaluation indicators between the experimental group and the control group.

[0033]

[0034] The data in Table 1 shows that for trainees with a conclusion accuracy rate below 90%, such as trainees 04 and 14, both systems can identify their unqualified status. However, for specific trainees with a conclusion accuracy rate within the acceptable range but inefficient operation processes, such as trainees 03, 05, 13, and 15, the control group system, due to the limitations of its evaluation dimensions, failed to identify the potential risks in their operation process. In contrast, the experimental group system, through analysis of process quality data, marked these trainees as having risks. This difference stems from the capability vector engine within the system of this invention, which transforms the trainees' inefficient operational behaviors into structured and quantifiable risk indicators, namely, a higher capability feature vector deviation value. The experimental results confirm that, compared to management methods that rely solely on the task endpoint status for evaluation, the system of this invention, by introducing analysis of operational process quality, can provide managers with supplementary information about trainees' operational capabilities, especially in identifying high-risk trainees whose results are acceptable but whose processes are questionable, demonstrating its application value.

[0035] Example 3: This example combines Figures 1 to 3 A description of a student training progress tracking system, such as... Figure 1 As shown, the interaction sequence capture module is responsible for capturing time-series interaction data that is not result-oriented and generated during student operations. This data is then transmitted to the capability vector engine, which converts the interaction data into student capability feature vectors and compares them with expert benchmarks to generate capability deviation analysis results. These results are archived in the historical student capability feature vector set and used by the adaptive recommendation module to match and push relevant reinforcement training content based on the knowledge graph. Meanwhile, the historical student capability feature vector set is used by the chronic trend analyzer to calculate trend parameters representing the long-term evolution of student capabilities. These parameters are sent to the consistency verification module to dynamically adjust its internal judgment threshold. After verifying the consistency between the current vector and the historical pattern using the adjusted threshold, if the verification conclusion is true, the progress evaluation module generates an objective progress evaluation based on the capability deviation analysis results as a basis for the manager's decision-making. If the verification conclusion is false, the system marks this task as data anomaly and prompts for manual review.

[0036] like Figure 2 As shown in the figure, the task sequence is used as the horizontal axis and the operational redundancy is used as the vertical axis to depict the performance trajectory of three trainees in 20 consecutive tasks. Trainee A's curve shows an overall downward trend, indicating that his operational redundancy decreases as the task progresses and he is in a state of continuous improvement. Trainee B's curve fluctuates steadily within a fixed value range, indicating that his ability is stagnant. Trainee C's curve shows irregular and large fluctuations, indicating that his operation mode is unstable and fluctuates greatly.

[0037] like Figure 3 As shown, after a trainee completes a task, the capability vector engine generates a current capability feature vector and triggers trend analysis. The chronic trend analyzer extracts the most recent 20 historical vector sets from the historical database, performs linear regression analysis on the key dimension numerical sequences, and calculates the capability growth gradient. Based on the positive, negative, or zero value of this gradient, trend parameters are set for continuous capability improvement, capability decline, or capability stagnation, respectively. Subsequently, these trend parameters are sent to the consistency verification module. After receiving the parameters, the module executes the corresponding threshold adjustment logic. For positive growth trends, the judgment threshold is increased to provide greater tolerance for fluctuations, while for stagnant or negative trends, the judgment threshold is decreased to increase the sensitivity of supervision. After completing the update of the internal threshold parameters, it prepares for the next consistency verification.

[0038] Example 4: In the initial deployment phase of the system of this invention, when applied to a simulated collision avoidance operation training scenario for large ship operators, a key engineering problem that needs to be solved first is how to establish a set of standardized benchmark parameters for this specific scenario. Specifically, this means how to determine the capability feature vectors used for comparison and how to calibrate the judgment thresholds used by the consistency verification module to identify abnormal behavior patterns. Without a standardized calibration procedure, the setting of these core parameters may lead to a lack of a unified and verifiable basis for subsequent evaluation and management. To solve this problem, before the system is officially put into use, an offline standardized parameter calibration procedure is executed. First, a group of operators who have passed the highest level of professional qualification certification in this field and have no bad operation records are selected as an expert group, and they are invited to repeatedly perform a series of standard tasks covering various typical collision avoidance scenarios on the simulation platform. During this process, the interaction sequence capture module records the complete time-series interaction data of each expert for each task. Subsequently, the capability vector engine processes these raw data in batches, generating a corresponding multi-dimensional capability feature vector for each operation of each expert, thus forming an original set of capability feature vectors. Next, in order to establish a benchmark capability feature vector that can represent the high-level operation characteristics in this field, The system performs statistical processing on the aforementioned original set. This processing includes two steps: the first step is to remove outlier data, which involves calculating the Euclidean distance between all vectors in the set and their geometric centers, and removing vectors with a distance greater than three standard deviations as atypical operational data caused by random factors. The second step is to calculate the mean of the components of all remaining vectors in each dimension on the purified vector set. The vector formed by these means is ultimately determined as the capability feature vector for this training scenario. This approach, which is based on group statistics rather than individual-based procedures, yields an evaluation benchmark that is statistically significant and representative of the field.

[0039] After determining the capability feature vector Next, the procedure proceeds to determine the threshold in the consistency verification module. The calibration phase of relevant parameters; specifically, to determine the basic threshold. The system uses the aforementioned purified set of capability feature vectors to calculate the mean of each vector and its set, i.e. The deviations, and based on the distribution of all these deviation values, the deviation values ​​located at the upper limit of the 95% confidence interval are set as the base threshold. This value represents the upper limit of normal behavioral fluctuations for a high-level operator; it is used to determine the sensitivity coefficient. The system will further introduce a set of historical operation data of novice trainees and their corresponding ability progress records. By traversing different [0.1, 2.0] intervals with a step size of 0.1, the system will... Value, and use the formula Repeated simulation calculations in different The system's detection rate for known cheating behaviors and its false alarm rate for high performance from normally improving students are compared. Ultimately, a value is selected that maximizes the overall metric: detection rate minus false alarm rate. The threshold parameter setting is based on a data-driven optimization process with clear optimization goals. By executing the above complete offline calibration procedure, the key parameters of the core evaluation model and supervision logic of the system are determined through this procedure when it is applied to new training fields. The existence of this procedure enables the system of the present invention to have reproducibility for cross-domain deployment and implementation, and provides an engineering foundation for its application consistency in different management scenarios.

[0040] Example 5: In an annual qualification review scenario for air traffic controllers using the system of this invention, which has been running for a long time, the system faces two management problems. First, after the software interface or operating procedures are updated, the original capability feature vector may not reflect the current operating mode. Second, in critical certification assessments, it is necessary to effectively identify occasional atypical behavioral patterns that are inconsistent with the trainee's long-term capability baseline, such as situations where someone else operates on their behalf. To address these issues, the system is configured to automatically execute an annual baseline model reconstruction procedure before the start of each annual review cycle. This procedure first utilizes all valid interaction data generated by the top 5% of controllers in performance records during daily simulation training over the past year, and recalculates and updates the system's built-in capability feature vector according to the statistical processing method described in Example 3. This allows the system to reflect new operational techniques and procedural changes. At the same time, the system will also analyze the correlation data between all the marked capability gaps and their corresponding reinforcement training content in the past year, and optimize the knowledge graph on which the adaptive recommendation module is based, removing weakly correlated recommendation paths and enhancing the weights of frequently occurring paths with significant training effects.

[0041] In a specific review assessment within this scenario, student B, whose historical student ability feature vector set showed that they had been in the critical ability zone for a long time and whose ability growth gradient was close to zero, submitted a set of operational process data. After processing this data, the system's ability vector engine generated an ability feature vector for the current task student. Its relationship with the updated capability feature vector The deviation value is lower than a preset monitoring lower limit; however, in the consistency verification module, since it receives the trend parameter output by the chronic trend analyzer, which characterizes the long-term stagnation of the trainee's ability, its internal adaptive risk response threshold adjustment logic has already adjusted the judgment threshold used for this verification. Set at a low numerical level; therefore, when this When compared with its own accumulated set of historical learner ability feature vectors, which are statistically represented by another characteristic, the calculated behavioral deviation value far exceeds the dynamically lowered judgment threshold. Ultimately, the consistency verification module output a false verification conclusion for the assessment data. As a result, the progress assessment module prevented the generation of a qualified assessment conclusion and marked the assessment as an atypical operational event with high statistical bias, automatically triggering the management review process. The initiation of this process enabled management personnel to intervene in the investigation and ultimately confirm the violations in the assessment, thereby avoiding potential errors in personnel qualification certification due to inaccurate data.

[0042] Example 6: In an annual qualification review scenario for air traffic controllers using the system of this invention that has been running for a long time, the system faces two management problems. First, after the software interface or operating procedures are updated, the original capability feature vector may not reflect the current operating mode. Second, in critical certification assessments, it is necessary to effectively identify occasional atypical behavioral patterns that are inconsistent with the trainee's long-term capability baseline, such as situations where someone else operates on their behalf. To address these issues, the system is configured to automatically execute an annual baseline model reconstruction procedure before the start of each annual review cycle. This procedure first utilizes all valid interaction data generated by the top 5% of controllers in performance records during daily simulation training over the past year, and recalculates and updates the system's built-in capability feature vector according to statistical processing methods. This allows the system to reflect new operational techniques and procedural changes. At the same time, the system will also analyze the correlation data between all the marked capability gaps and their corresponding reinforcement training content in the past year, and optimize the knowledge graph on which the adaptive recommendation module is based, removing weakly correlated recommendation paths and enhancing the weights of frequently occurring paths with significant training effects.

[0043] In a specific review assessment within this scenario, student B, whose historical student ability feature vector set showed that they had been in the critical ability zone for a long time and whose ability growth gradient was close to zero, submitted a set of operational process data. After processing this data, the system's ability vector engine generated an ability feature vector for the current task student. Its relationship with the updated capability feature vector The deviation value is lower than a preset monitoring lower limit; however, in the consistency verification module, since it receives the trend parameters representing the long-term stagnation of the trainee's ability output by the chronic trend analyzer, its internal adaptive risk response threshold adjustment logic has already adjusted the judgment threshold used for this verification in advance. Set at a low numerical level; therefore, when this When compared with its own accumulated set of historical learner ability feature vectors, which statistically represent another characteristic, the calculated behavioral deviation value far exceeds the dynamically lowered judgment threshold. Ultimately, the consistency verification module output a false verification conclusion for the assessment data. As a result, the progress assessment module prevented the generation of a qualified assessment conclusion and marked the assessment as an atypical operational event with high statistical bias, automatically triggering the management review process. The initiation of this process enabled management personnel to intervene in the investigation and ultimately confirm the violations in the assessment, thereby avoiding potential errors in personnel qualification certification due to inaccurate data.

[0044] To further verify the technical solution of dynamically adjusting the judgment threshold based on trend parameters in this invention, and to compare its technical advantages with conventional technical solutions that use fixed thresholds, the following comparative experiment was conducted.

[0045] Comparative Example 1: This comparative example aims to illustrate the inherent limitations of a conventional technical solution that relies solely on historical data statistical distribution to set a fixed judgment threshold in an annual qualification review scenario for air traffic controllers, identical to Example 6, when identifying specific atypical behavioral patterns. Except for the following key differences, the system, test tasks, data acquisition and processing methods, and trainee B (a trainee whose historical trainee ability feature vector set shows long-term stagnation and a growth gradient close to zero) used in this comparative example are completely consistent with the conditions described in Example 6. The key difference lies in the fact that the trainee training progress tracking system in this comparative example does not integrate the trend parameters output by the chronic trend analyzer in its consistency verification module. Instead, it uses a conventional method recognized in the art to set its judgment threshold: based on the historical trainee ability feature vector set of trainee B's past 20 tasks, it calculates the mean and standard deviation of the key dimension of operational redundancy, and uses the upper limit of the confidence interval covering 95% of the data points (approximately the mean plus 1.96 times the standard deviation) as a fixed judgment threshold. .

[0046] The experiment proceeded as follows: Trainee B also submitted operational data from the annual review assessment. After processing this data, the system's capability vector engine generated the trainee's capability feature vector for the current task. After analysis, the In the data analysis, the operational redundancy metric was 1.21, significantly better than its historical average. Subsequently, the consistency verification module initiated the verification process. First, the module calculated the statistical baseline for student B in this dimension based on the stored historical data: the average operational redundancy of the past 20 tasks. =1.83; Standard deviation of redundancy in operations over the past 20 tasks =0.08; Based on the aforementioned conventional technical solution, a fixed judgment threshold is set. Set as: This threshold represents the upper limit of fluctuation in student B's normal operational performance with a 95% probability; next, the module will... Compared with the fixed threshold, since the current task's operation redundancy value of 1.21 is lower than its historical average of 1.83 and does not exceed the upper limit threshold of 1.987, the system determines that the feature vector of this operation is within the normal statistical fluctuation range of its historical behavior. The key evaluation indicators and final conclusions of the experiment are summarized in Table 2.

[0047] Table 2: Verification results of the assessment data of trainee B using the conventional technical solution with fixed thresholds.

[0048]

[0049] The experimental results show that, although the conventional technical solution of setting a fixed judgment threshold based on the statistical distribution of historical data conforms to the prevailing standards in terms of methodology, its static monitoring scale makes the fixed threshold too lenient for trainee B, whose abilities have stagnated for a long time. It fails to recognize that the current assessment data (1.21) is a statistically very low probability event compared to its highly stable baseline (1.83±0.08). The design principle of this solution lacks consideration of the trainee's ability evolution and cannot automatically tighten the monitoring scale after recognizing the trainee's long-term stagnation, as this invention does. This leads to the missed judgment of the cheating behavior of having someone else perform the operation, which conveys false assessment information to the manager and constitutes a management risk.

[0050] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A student training progress tracking system, characterized in that, The system includes: The interaction sequence capture module is configured to capture non-preset result-oriented time-series interaction data generated by the interaction between the trainee and the task interface during the trainee's performance of training tasks; The capability vector engine is configured to transform time-series interactive data into capability feature vectors that characterize the learners' operational abilities, and to quantitatively compare the learners' capability feature vectors with preset capability feature vectors to generate capability deviation analysis results. The chronic trend analyzer is configured to calculate trend parameters that characterize the long-term evolution of a learner's abilities based on a set of historical learner ability feature vectors generated from multiple historical training tasks. The consistency verification module has a variable judgment threshold. It is configured to receive trend parameters and adjust the judgment threshold according to the trend parameters. Then, it uses the adjusted judgment threshold to verify the consistency between the current task's trainee ability feature vector and its historical trainee ability feature vector set, so as to output a verification conclusion on the authenticity of the ability deviation analysis results. The progress assessment module is configured to generate an assessment of the trainees' actual training progress based on the capability deviation analysis results if and only if the verification conclusion is true.

2. The trainee training progress tracking system according to claim 1, characterized in that, Time-series interaction data includes at least one of the following: mouse movement trajectory data, keyboard input timing data, or interface element interaction sequence data.

3. The trainee training progress tracking system according to claim 1, characterized in that, The capability feature vector is a baseline vector generated by aggregating time-series interaction data of one or more expert users performing the same training task through a pre-defined vectorization model.

4. The trainee training progress tracking system according to claim 1, characterized in that, Also includes: The adaptive recommendation module is configured to determine the corresponding reinforcement training content based on the dimensions of bias in the ability bias analysis results, in a preset knowledge graph that stores the correspondence between ability weaknesses and reinforcement training content, and push the reinforcement training content to the learner.

5. The trainee training progress tracking system according to claim 1, characterized in that, The trend parameter is a state stability index that quantitatively represents the growth gradient of trainees' abilities over time or the stability of trainees' operating patterns.

6. The trainee training progress tracking system according to claim 1, characterized in that, The adjustment of the judgment threshold in the consistency verification module follows a preset adjustment rule. The adjustment rule is that when the trend parameter represents a long-term positive growth trend in the trainee's ability, the judgment threshold is raised. When the trend parameter indicates that the trainee's ability has stagnated or is showing a negative trend for a long period of time, the judgment threshold is lowered.

7. The trainee training progress tracking system according to claim 1, characterized in that, The process by which the capability vector engine transforms time-series interaction data into learner capability feature vectors includes: performing preset statistical transformations on the time-series interaction data to extract process quality feature values ​​that characterize operational redundancy, number of decision hesitations, or behavioral smoothness, and combining the process quality feature values ​​into a multi-dimensional vector.

8. A student training progress tracking system according to claim 1, characterized in that, The consistency verification module performs consistency verification by comparing the current task’s student ability feature vector with the set of historical student ability feature vectors that serve as the baseline of the student’s own behavior.

9. A student training progress tracking system according to claim 1, characterized in that, The system also includes a processor and a memory containing computer programs. The processor is configured to execute the computer programs to implement the functions of the interactive sequence capture module, the capability vector engine, the chronic trend analyzer, the consistency check module, and the progress assessment module.

10. A student training progress tracking system according to claim 4, characterized in that, The system also includes a user terminal, and the adaptive recommendation module is configured to push reinforcement training content to the user terminal for presentation.

Images