Quality evaluation method and system for skill training operation behavior, medium and product

Abstract

The invention provides a skill training operation behavior quality evaluation method and system, a medium and a product, and relates to the technical field of industrial skill training quality evaluation, and the method comprises the steps: obtaining a time sequence operation data flow of a training operator in a skill training task execution process, carrying out the fragmentation processing of the time sequence operation data flow according to a preset time window, and obtaining a time sequence operation data flow; generating a behavior snapshot sequence; extracting operation frequency data, fixation focus position data and reference resource access record data of the practical training operator from the behavior snapshot sequence; performing activeness analysis on the current operation frequency data to obtain an operation activeness feature, performing distribution analysis on the fixation focus position data to obtain a focus stability feature, and performing deep analysis on the reference resource access record data to obtain a resource participation depth feature; and generating an operation quality evaluation result of the practical training operator according to the operation activeness feature, the focus stability feature and the resource participation depth feature.

Description

Technical Field

[0001] This application relates to the field of industrial skills training quality assessment technology, and in particular to a method, system, medium and product for assessing the quality of skills training operational behavior. Background Technology

[0002] With the rapid development of industrial digital transformation and intelligent manufacturing technologies, immersive skills training platforms are playing an increasingly important role in areas such as complex industrial operation training and equipment maintenance skills training. Especially in high-risk industrial scenarios and precision assembly operation training, the operational quality and skill mastery of trainees directly affect the safety and standardization of operations in actual production.

[0003] In related technologies, to assess the quality of trainees' performance in skills training tasks, a single-duration statistical method is typically used for quality evaluation. Specifically, the skills training platform records the task start timestamp when the training task begins and the task end timestamp when the trainee submits the task. The task completion time is calculated by determining the time difference between the task end timestamp and the task start timestamp. The system compares the task completion time with a preset standard completion time range. If the task completion time is less than the lower limit of the standard completion time range, it is determined that the trainee has engaged in hasty and perfunctory completion; if the task completion time is greater than the upper limit of the standard completion time range, it is determined that the trainee is not proficient in the operation or has been idle. Simultaneously, some skills training platforms also record the number of mouse clicks and keyboard keystrokes by the trainee, calculating the operation frequency per unit time. When the operation frequency falls below a preset threshold, a low activity warning is triggered. Furthermore, the system provides feedback to training managers in a binary judgment of "qualified" or "unqualified" for subsequent training adjustments and skills certification.

[0004] However, using the above-mentioned single-duration statistical method, the evaluation system judges based on only two single dimensions: task completion time and operation frequency. However, the actual operational behavior of trainees in the process of performing skills training tasks has complex and diverse characteristics. These single-dimensional statistical data are difficult to fully reflect the trainees' true operational status and task participation level, which leads to poor accuracy in the evaluation of the quality of skills training operation behavior in related technologies. Summary of the Invention

[0005] This application provides a method, system, medium, and product for quality assessment of skills training operations, which is used to improve the accuracy of skills training operations quality assessment.

[0006] Firstly, this application provides a quality assessment method for skills training operational behavior, applied to the aforementioned quality assessment system. The method includes: acquiring the time-series operational data stream of the trainee during the execution of a skills training task; segmenting the time-series operational data stream according to a preset time window to generate a sequence of behavioral snapshots; extracting the trainee's current operational frequency data from each behavioral snapshot in the sequence, and performing activity analysis on the current operational frequency data to obtain operational activity characteristics; extracting the trainee's gaze focus position data on the operation interface from each behavioral snapshot in the sequence, and performing distribution analysis on the gaze focus position data to obtain focus stability characteristics; extracting the trainee's reference resource access record data from each behavioral snapshot in the sequence, and performing in-depth analysis on the reference resource access record data to obtain resource participation depth characteristics; and generating an operational quality assessment result for the trainee based on the operational activity characteristics, focus stability characteristics, and resource participation depth characteristics.

[0007] By adopting the above technical solution, the time-series operation data stream is segmented into a sequence of behavioral snapshots after passing through a preset time window. This discretizes continuous operational behaviors into analyzable time units, making each behavioral snapshot an independent analytical object. Activity analysis of current operation frequency data quantifies the operator's activity level; distribution analysis of gaze focus position data reflects the operator's attention allocation pattern; and in-depth analysis of reference resource access record data reflects the operator's learning engagement. These three types of features characterize operational behaviors from different dimensions: operational activity reveals performance at the action execution level, focus stability reflects the state at the cognitive attention level, and resource engagement depth demonstrates the degree of knowledge acquisition. These three features complement each other to form a multi-dimensional evaluation system. By comprehensively generating operational quality evaluation results from these three types of features, the one-sidedness of single-dimensional evaluation can be avoided, improving the accuracy of judging complex operational behaviors. This solves the technical problem of poor accuracy in evaluating the quality of skills training operational behaviors in related technologies, achieving the technical effect of improving the accuracy of skills training operational behavior evaluation.

[0008] Optionally, an activity analysis is performed on the current operation frequency data to obtain operation activity characteristics. Specifically, this includes: acquiring historical operation data with the same task type as the current skills training task, and determining the mean and standard deviation of the operation frequency of the trainees in the historical operation data, using the mean operation frequency as the baseline operation frequency; determining the average operation frequency of the current operation frequency data within a preset time window, and performing a deviation analysis between the average operation frequency and the baseline operation frequency to obtain the operation frequency deviation value; dividing the operation frequency deviation value by the standard deviation to obtain an activity score; extracting records of text editing operations performed by the trainees within the preset time window from each behavior snapshot, and determining the number of editing operations performed by the trainees based on the records; extracting the number of newly added characters and the number of deleted characters from the records, adding the number of newly added characters and the number of deleted characters to obtain the total character change, and dividing the number of editing operations by the total character change to obtain the editing intensity value; and using the activity score and the editing intensity value as operation activity characteristics.

[0009] By employing the above technical solutions, the mean and standard deviation of operation frequency in historical operation data provide a reference benchmark for the current operation frequency, making the deviation analysis statistically significant. Dividing the operation frequency deviation by the standard deviation standardizes frequency fluctuations, eliminating dimensional differences between different task types, and allowing the resulting activity score to be compared across tasks. The editing intensity value, formed by dividing the number of editing operations in the text editing operation record by the sum of the number of added and deleted characters, reflects the level of operational precision required for a unit character change. The activity score measures the frequency of operations, while the editing intensity value measures the precision of operations; the combined operation activity feature can distinguish between high-frequency ineffective operations and low-frequency effective operations. This dual-dimensional feature combination avoids misjudging quick, perfunctory operations as high-quality behavior and also prevents misjudging low-frequency operations involving deep thought as negative behavior, improving the accuracy of operation quality identification.

[0010] Optionally, a distribution analysis of the gaze focus position data is performed to obtain focus stability characteristics. Specifically, this includes: dividing the interface into a task execution area, a reference resource area, and an auxiliary tool area, and assigning a unique area identifier to each area; determining the target area where the trainee's gaze focus is located within a preset time window based on the gaze focus position data; matching the target area with the area identifier to determine the first dwell time of the gaze focus in the task execution area, the second dwell time of the gaze focus in the reference resource area, and the third dwell time of the gaze focus in the auxiliary tool area; determining the first time percentage of the gaze focus in the task execution area based on the first dwell time and the total duration of the preset time window; and determining the first time percentage of the gaze focus in the task execution area based on the second dwell time and the total duration of the preset time window. The second time proportion of the examination resource area is determined; the third time proportion of the gaze focus in the auxiliary tool area is determined based on the third dwell time and the total time; the information entropy value of the gaze focus is determined based on the first, second, and third time proportions; threshold analysis is performed on the information entropy value to determine that when the information entropy value is less than the first preset entropy value, the trainee's focus is concentrated in a single area, and when the information entropy value is greater than the second preset entropy value, the trainee's focus is dispersed in multiple areas, and the first preset entropy value is less than the second preset entropy value; the preset time window is divided into multiple continuous time segments, the number of area switching of the gaze focus between two adjacent continuous time segments is determined, and the focus switching frequency of the trainee on the operation interface is determined based on the number of area switching and the total time; the information entropy value and focus switching frequency are used as focus stability characteristics.

[0011] By adopting the above technical solution, the user interface is divided into three functional areas and assigned unique identifiers, providing a spatial reference framework for precise positioning of the focus. After converting the duration of focus in each area into a time percentage, the influence of time window length on the analysis is eliminated, making behavioral snapshots of different durations comparable. The three time percentages are used to calculate the dispersion of focus distribution through information entropy. Comparing the information entropy value with a preset threshold value can qualitatively determine whether the focus is concentrated or dispersed. The ratio of the number of area switching times between consecutive time segments to the total duration forms the focus switching frequency, which quantifies the dynamic change characteristics of the focus. The information entropy value reflects the spatial distribution characteristics of the focus, and the focus switching frequency reflects the temporal change characteristics of the focus. The combined focus stability characteristics can comprehensively characterize attention patterns. This spatiotemporal dual-dimensional analysis method can identify seemingly stable but actually detached pseudo-focus behavior, improving the accuracy of judging the true state of focus.

[0012] Optionally, a deep analysis of the reference resource access records is performed to obtain resource participation depth characteristics. Specifically, this includes: extracting the resource identifier, access timestamp, and browsing depth value for each accessed resource from the reference resource access record data; determining the access duration of each accessed resource based on the access timestamp, and determining the content browsing ratio of each accessed resource based on the browsing depth value; multiplying the browsing depth value of each accessed resource by the access duration to obtain a deep participation score for each accessed resource; determining the target resource participation level based on the deep participation scores of all accessed resources and the total duration of a preset time window; determining the total number of accesses to different resources and the number of times the same resource was accessed repeatedly in the reference resource access record data; determining the resource coverage breadth of the training operator based on the total number of accesses and the number of times the same resource was accessed repeatedly; and using resource participation level and resource coverage breadth as resource participation depth characteristics.

[0013] By adopting the above technical solution, the access status of each resource is fully recorded by three elements: resource identifier, access timestamp, and browsing depth value, providing a comprehensive data foundation for in-depth analysis. Access duration measures time investment, while content browsing ratio measures content coverage. These two are multiplied together to form a deep engagement score, which comprehensively reflects the learning quality of a single resource. The relationship between the deep engagement score of all resources and the total duration determines a normalized target resource engagement level, making the engagement levels of tasks with different durations comparable. The total number of accesses reflects the frequency of resource utilization, while the number of repeated accesses reflects the degree of repetitive learning. The resource coverage breadth determined by both reflects the comprehensiveness of learning. Target resource engagement focuses on the depth dimension, while resource coverage breadth focuses on the breadth dimension. The combined resource engagement depth feature can distinguish between superficial browsing behavior and in-depth study behavior, improving the accuracy of judging the quality of resource utilization.

[0014] Optionally, based on operational activity characteristics, focus stability characteristics, and resource participation depth characteristics, the operational quality assessment results of the trainee are generated. Specifically, this includes: integrating operational activity characteristics, focus stability characteristics, and resource participation depth characteristics to construct a multi-dimensional feature vector; extracting activity values ​​from the operational activity characteristics and comparing them with a first judgment threshold to obtain an operational activity comparison result; extracting stability values ​​from the focus stability characteristics and comparing them with a second judgment threshold to obtain a focus stability comparison result; extracting participation values ​​from the resource participation depth characteristics and comparing them with a third judgment threshold to obtain a resource participation depth comparison result; determining the trainee's current operational state as a deep analysis state when the operational activity characteristics are less than the first judgment threshold and the focus stability characteristics are greater than the second judgment threshold based on the operational activity comparison results; determining the trainee's current operational state as an idle state when the operational activity characteristics remain continuously at zero for a preset duration based on the focus stability comparison results; and further determining the trainee's current operational state as an idle state based on the resource participation depth characteristics. When the participation depth comparison results determine that the resource participation depth feature is less than the third judgment threshold and the operation activity feature is greater than the fourth judgment threshold, the current operation status of the trainee is determined to be a perfunctory operation status. The first occurrence frequency and first duration of the deep analysis status in the behavior snapshot sequence are determined; the second occurrence frequency and second duration of the idle status in the behavior snapshot sequence are determined; and the third occurrence frequency and third duration of the perfunctory operation status in the behavior snapshot sequence are determined. A first descending order ranking analysis is performed on the first, second, and third occurrence frequencies to obtain a frequency ranking sequence; and a second descending order ranking analysis is performed on the first, second, and third durations to obtain a duration ranking sequence. Based on the frequency ranking sequence and duration ranking sequence, the current dominant operation mode of the trainee is determined from the deep analysis status, idle status, and perfunctory operation status. Based on the dominant operation mode and multi-dimensional feature vector, the current comprehensive quality score and current operation quality level of the trainee are determined. The current operation quality level, current comprehensive quality score, and current dominant operation mode are used as the operation quality evaluation results.

[0015] By adopting the above technical solution, the three types of features are integrated into a multi-dimensional feature vector, unifying discrete features into the same mathematical space, facilitating subsequent comprehensive analysis. The comparison of activity, stability, and participation values ​​with their corresponding judgment thresholds produces three comparison results; the combination of these results determines the operational state type. Deep analysis of the state requires both low activity and high stability; the idle state requires zero activity for a preset duration; and the perfunctory operation state requires both low participation and high activity. This multi-condition combination judgment improves the accuracy of state identification. The frequency and duration of the three states are sorted in descending order to form a frequency sequence and a duration sequence. Comprehensive analysis of these two sequences determines the dominant operational mode, avoiding misjudgments of occasional states. The dominant operational mode and the multi-dimensional feature vector together determine the comprehensive quality score and quality level, achieving a step-by-step progression from features to states to evaluation, forming a complete quality assessment logic chain.

[0016] Optionally, after using the current operational quality level, current comprehensive quality score, and current dominant operational mode as the operational quality assessment results, the method further includes: comparing the current operational quality level with a preset quality level classification standard to obtain a comparison result; generating a first-level intervention signal when the current operational quality level is determined to be in the low-quality level category based on the comparison result; generating a second-level intervention signal when the current operational quality level is determined to be in the medium-quality level category based on the comparison result and the current dominant operational mode is a perfunctory operational mode; selecting a corresponding intervention strategy based on the first-level or second-level intervention signal, wherein: when a first-level intervention signal exists, guidance is generated. The system provides guidance and incentive reminders; generates motivational reminders when a second-level intervention signal is present; acquires the trainee's historical operational quality level, historical comprehensive quality score, and historical dominant operational mode; performs trend analysis on historical operational quality level, historical comprehensive quality score, and historical dominant operational mode to determine the operational quality change trend; adjusts the difficulty coefficient of subsequent training tasks when the operational quality change trend shows a downward trend in operational quality level or comprehensive quality score; and records the guidance or incentive reminders and operational quality change trends in the trainee's personal skill file.

[0017] By adopting the above technical solution, the comparison between the current operational quality level and the preset classification standard discretizes the continuous quality score into actionable level categories, providing a basis for differentiated intervention decisions. Low quality triggers a first-level intervention signal, while medium quality accompanied by a perfunctory approach triggers a second-level intervention signal. This tiered triggering mechanism ensures targeted intervention. Guidance prompts provide specific improvement directions for low quality, while motivational reminders offer positive incentives for medium quality. Differentiated intervention strategies enhance intervention effectiveness. Comparative analysis of historical and current data reveals quality change trends, which predict future development directions. A downward trend triggers a difficulty adjustment mechanism, reducing task difficulty to alleviate operator learning pressure and creating an adaptive task configuration. Intervention information and trends are recorded in individual skill profiles, accumulating a data foundation for subsequent personalized training program development and achieving a closed-loop optimization of assessment-intervention-improvement.

[0018] Optionally, before acquiring the time-series operation data stream of the trainee during the execution of the skills training task, the method further includes: acquiring the trainee's identity information and extracting the trainee's historical skill level data from the skill archive based on the identity information; determining the trainee's skill proficiency level based on the historical skill level data and performing a personalized threshold configuration operation based on the skill proficiency level to obtain a preset frequency threshold; acquiring the task attribute information of the current skills training task, including the task difficulty coefficient, task type identifier, and expected completion time; performing a time window configuration operation based on the matching relationship between the task difficulty coefficient and the skill proficiency level to obtain a preset time window; determining the data type range to be collected based on the task type identifier and configuring the data collection range of the time-series operation data stream; setting the total collection time of the behavior snapshot sequence based on the expected completion time; establishing an association session between the trainee and the current skills training task, configuring the preset time window, preset frequency threshold, data collection range, and total collection time in the association session to complete the initialization of the data collection environment before acquiring the time-series operation data stream.

[0019] By adopting the above technical solutions, historical skill level data associated with identity information provides a basis for personalized configuration, and the determination of skill proficiency levels allows for individualized assessment standards. The preset frequency threshold determined by proficiency levels can adapt to the operating habits of operators at different skill levels, avoiding misjudgments caused by uniform standards. The matching relationship between task difficulty coefficients and proficiency levels ensures the rationality of time window configuration; higher-difficulty tasks are configured with longer time windows to provide sufficient operation time, while lower-difficulty tasks are configured with shorter time windows to improve assessment efficiency. The data collection scope determined by task type identifiers ensures the relevance of collected data, with different types of tasks focusing on different operational characteristics. The total collection time set by the expected completion time avoids indefinite data collection, ensuring the timeliness of the assessment. The associated session centrally configures various parameters, forming a unified assessment environment and ensuring consistency in subsequent data collection and analysis.

[0020] In a second aspect, embodiments of this application provide a quality assessment system, which includes: one or more processors and a memory; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors invoke the computer instructions to cause the quality assessment system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a quality assessment system, cause the quality assessment system to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a quality assessment system, cause the quality assessment system to perform the method described in the first aspect and any possible implementation thereof. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a quality assessment method for skills training operations in this application. Figure 2 This is a schematic diagram of the physical device structure of a quality assessment system in the embodiments of this application. Detailed Implementation

[0024] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0025] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0026] This application provides a method for quality assessment of skills training operational behaviors, see reference. Figure 1 , Figure 1 This is a flowchart illustrating a quality assessment method for skills training operations in this application, comprising the following steps: Step S101: Obtain the time-series operation data stream of the trainee during the execution of the skills training task, and perform segmentation processing on the time-series operation data stream according to the preset time window to generate a behavior snapshot sequence; Step S102: Extract the current operation frequency data of the trainee from each behavior snapshot in the behavior snapshot sequence, and perform activity analysis on the current operation frequency data to obtain operation activity characteristics; Step S103: Extract the gaze focus position data of the trainee on the operation interface from each behavior snapshot in the behavior snapshot sequence, and perform distribution analysis on the gaze focus position data to obtain focus stability characteristics; Step S104: Extract reference resource access record data of the training operator from each behavior snapshot in the behavior snapshot sequence, and perform in-depth analysis on the reference resource access record data to obtain resource participation depth features; Step S105: Based on the operational activity characteristics, focus stability characteristics, and resource participation depth characteristics, generate the operational quality assessment results of the trainees.

[0027] In the above embodiment, taking automotive engine assembly skills training as an example, it is assumed that the trainee needs to complete the assembly task of a piston connecting rod assembly on a virtual simulation platform. This task includes multiple stages such as part identification, assembly sequence planning, tool selection and use, and assembly precision control, and the entire operation process is expected to take 30 minutes. Various interactive data of the operator are collected in real time through front-end data tracking technology to form a continuous time-series operation data stream. The time-series operation data stream includes multi-dimensional information such as mouse click events, keyboard input records, part dragging trajectories, tool switching operations, and view adjustment actions. Each operation event has precise timestamps, operation type identifiers, operation object numbers, operation parameter values, and other attributes. The data stream is sliced ​​according to a preset 30-second time window to generate a sequence of behavioral snapshots. For example, a complete assembly task will generate approximately 60 behavioral snapshots, each snapshot encapsulating all operation events and their associated information within that time period. In the process of extracting operation activity features, the total number of operation events within each behavioral snapshot is first counted. In a 30-second snapshot, 45 mouse clicks, 12 keyboard inputs, 8 part drags, and 3 tool switches were detected, totaling 68 operation events, resulting in a current operation frequency of 2.27 times / second. Activity analysis was performed on this frequency by retrieving operation records from 200 skilled workers in the historical database for the same task type. The historical average operation frequency was calculated to be 2.0 times / second, with a standard deviation of 0.5 times / second. The standardized deviation of the current operation frequency of 2.27 times / second compared to the historical mean was calculated to be 0.54. Combined with the effectiveness analysis of the operation content, it was found that 85% of the assembly actions were effective, while 15% were ineffective repetitive actions. The overall operation activity characteristic value was calculated to be 0.75, indicating that the operator maintained a moderate operating pace and the operation efficiency was at a normal level.

[0028] In the above embodiment, when extracting focus stability features, the virtual assembly interface is divided into three gaze areas: the assembly work area, the process drawing area, and the toolbar area. An integrated eye-tracking module acquires the operator's gaze focus coordinate sequence, collecting 30 gaze point data points per second. In a 30-second snapshot, the system recorded a dwell time of 18 seconds in the assembly work area, 8 seconds in the process drawing area, and 4 seconds in the toolbar area. Distribution analysis of the gaze focus position data shows that the time proportions for each area are 60%, 27%, and 13%, respectively, resulting in an information entropy value of 0.92 for the gaze distribution. Simultaneously, the number of area switching events was recorded as 5, averaging one gaze area switch every 6 seconds. Based on the concentration of focus distribution and switching frequency, a comprehensive analysis yielded a focus stability feature value of 0.82, indicating that the operator's attention is focused, and their gaze remains stable in the key operation area without frequent wandering or distraction. The analysis of resource engagement depth features involves the operator's access behavior to various reference resources. The data recorded that the operator accessed the 3D assembly animation twice during the snapshot period, with a total viewing time of 8 seconds and a video viewing progress of 85%; consulted the technical document once, with a dwell time of 5 seconds and a document scroll depth of 60% of the page; and accessed the assembly diagram three times, with an average dwell time of 3 seconds each time. In-depth analysis of the reference resource access record data revealed that, based on the importance weight of the resource type and the access depth, the participation score for the 3D assembly animation was 0.85 × 8 = 6.8, the participation score for the technical document was 0.60 × 5 = 3.0, and the participation score for the assembly diagram was 1.0 × 9 = 9.0, for a total participation score of 18.8. Dividing this by the 30-second time window yielded a resource participation depth feature value of 0.63. This value reflects the operator's reasonable utilization of auxiliary resources, demonstrating sufficient reference to technical materials without excessive reliance on external assistance.

[0029] In the above embodiment, the operation quality assessment result is generated using a multi-dimensional feature fusion algorithm, which establishes an assessment model based on 5000 historical training data sets. The operation activity feature (0.75), focus stability feature (0.82), and resource participation depth feature (0.63) are input into the assessment model. After feature weighted fusion and nonlinear mapping, a comprehensive quality score of 78 is output. According to the scoring interval mapping rule, below 60 points indicates improvement is needed, 60 to 80 points is good, and above 80 points is excellent. The operator's current operation quality level is judged to be good. A detailed assessment report is also generated, indicating that the operator performed well in assembly sequence planning and had good tool proficiency, but there is still room for improvement in precision control. It is recommended that the operator strengthen their understanding of assembly tolerances in subsequent practice, refer to more technical videos on precision assembly, and improve operational precision. This multi-dimensional fusion assessment method can accurately identify the operator's true skill level, avoid the one-sidedness of single-indicator evaluation, and provide a scientific quality assessment basis for skills training.

[0030] Through the above steps, the time-series operational data stream is segmented into a sequence of behavioral snapshots after passing through a preset time window. This discretizes continuous operational behaviors into analyzable time units, making each behavioral snapshot an independent analytical object. Comparison analysis of current operational frequency data with a preset frequency threshold quantifies the operator's activity level; distribution analysis of gaze focus position data reflects the operator's attention allocation pattern; and in-depth analysis of reference resource access record data reflects the operator's learning engagement. These three types of features characterize operational behavior from different dimensions: operational activity reveals performance at the action execution level, focus stability reflects the state at the cognitive attention level, and resource engagement depth demonstrates the degree of knowledge acquisition. These three features complement each other to form a multi-dimensional evaluation system. By comprehensively generating operational quality evaluation results from these three types of features, the one-sidedness of single-dimensional evaluation can be avoided, improving the accuracy of judging complex operational behaviors. This solves the technical problem of poor accuracy in evaluating the quality of skills training operational behaviors in related technologies, achieving the technical effect of improving the accuracy of skills training operational behavior evaluation.

[0031] The entity performing the above steps can be a system or a device, such as an oxygen supply mask, or a controller or processor in the device or system, or a standalone controller or processor, or other processing devices or processing units with similar processing functions, but is not limited to these.

[0032] In an optional embodiment, an activity analysis is performed on the current operation frequency data to obtain operation activity characteristics. Specifically, this includes: acquiring historical operation data with the same task type as the current skills training task, and determining the mean and standard deviation of the operation frequency of the trainee in the historical operation data, using the mean operation frequency as the baseline operation frequency; determining the average operation frequency of the current operation frequency data within a preset time window, and performing a deviation analysis between the average operation frequency and the baseline operation frequency to obtain an operation frequency deviation value; performing a first calculation on the operation frequency deviation value and standard deviation to obtain an activity score; extracting records of text editing operations performed by the trainee within the preset time window from each behavior snapshot, and determining the number of editing operations performed by the trainee based on the records; extracting the number of newly added characters and the number of deleted characters from the records, and performing a second calculation on the number of editing operations, the number of newly added characters, and the number of deleted characters to obtain an editing intensity value; and using the activity score and the editing intensity value as operation activity characteristics.

[0033] In the above embodiment, taking the piston connecting rod assembly task as an example, historical operation data with the same type identifier as the current task is first retrieved from the skill file database. The database stores detailed operation records of 1000 operators completing this task in the past 12 months. Each record contains multi-dimensional information such as operator skill level, task completion time, operation frequency statistics, and final score. 280 high-quality sample data points with a skill level marked as proficient and a task score exceeding 85 are selected. These samples represent the standard operation level for this task type. The baseline operation frequency is determined by statistically processing the selected sample data. The arithmetic mean of the operation frequencies of the 280 samples is calculated, yielding a baseline operation frequency of 1.5 operation events per second. Simultaneously, the standard deviation of these samples is calculated to be 0.3 events per second, reflecting the normal operation frequency fluctuation range within the proficient operator group. These two statistical parameters together constitute the operation frequency benchmark for this task type, providing a quantitative reference standard for subsequent deviation analysis. Analysis of the current trainees' operation frequency revealed 24 valid operation events within a 30-second time window, resulting in an average operation frequency of 0.8 times / second. Comparing this to the baseline operation frequency of 1.5 times / second, a deviation of -0.7 times / second was observed. This negative deviation indicates that the current trainees' operation speed is significantly lower than the average level of skilled workers, suggesting they may be operating cautiously or in a state of deep thought.

[0034] In the above embodiment, the activity score is calculated using a standardized method. The operation frequency deviation of -0.7 is divided by the historical standard deviation of 0.3, resulting in an activity score of -2.33. This score, after standardization by division, is statistically significant. -2.33 indicates that the current operation frequency is approximately 2.33 standard deviations below the baseline, representing a significantly low level of operational activity. This score is rounded to two decimal places and used as the core quantitative indicator of operational activity. The calculation of the editing intensity value focuses on the operator's fine-grained operational behaviors. Text editing operation records are extracted from snapshots of the same behavior. These records mainly come from text interaction behaviors such as inputting assembly parameters, filling out check forms, and recording operation instructions. Within a 30-second window, five independent text editing operations were detected, including three parameter value inputs and two description text modifications. Through character-level difference analysis, the number of newly added characters was 30, mainly input assembly tolerance values ​​and operation points; the number of deleted characters was 5, reflecting the operator's correction behavior of the input content. The calculation of the editing intensity value is achieved through explicit division. First, add the 30 newly added characters to the 5 deleted characters, resulting in a total of 35 character changes. Then, divide the 5 editing operations by the total of 35 character changes to obtain an editing intensity value of 0.14. This value reflects the degree of fine-tuning performed by the operator. A lower editing intensity value indicates that the operator involves more characters in each edit, demonstrating a more definite operational intention; while a higher editing intensity value may suggest frequent minor modifications, reflecting uncertainty or iterative adjustment behavior.

[0035] In the above embodiment, the final operational activity feature is composed of an activity score and an editing intensity value. Combining an activity score of -2.33 and an editing intensity value of 0.14 creates a two-dimensional feature vector, which comprehensively reflects the operator's operational behavior pattern. A negative activity score coupled with a low editing intensity value suggests that the operator may be in a state of careful consideration, deliberating carefully before each operation. This pattern is common among beginners or is a normal reaction when facing complex assembly steps.

[0036] In an optional embodiment, a distribution analysis of the gaze focus position data is performed to obtain focus stability characteristics. Specifically, this includes: dividing the user interface into a task execution area, a reference resource area, and an auxiliary tool area, and assigning a unique area identifier to each area; determining the target area where the trainee's gaze focus is located within a preset time window based on the gaze focus position data; matching the target area with the area identifier to determine the first dwell time of the gaze focus in the task execution area, the second dwell time of the gaze focus in the reference resource area, and the third dwell time of the gaze focus in the auxiliary tool area; determining the first time percentage of the gaze focus in the task execution area based on the first dwell time and the total duration of the preset time window; and determining the gaze focus based on the second dwell time and the total duration. The second time percentage of the focus is in the reference resource area; the third time percentage of the focus is in the auxiliary tool area based on the third dwell time and the total time; the information entropy value of the focus is determined based on the first time percentage, the second time percentage, and the third time percentage; threshold analysis is performed on the information entropy value to determine that when the information entropy value is less than the first preset entropy value, the focus of the trainee is concentrated in a single area, and when the information entropy value is greater than the second preset entropy value, the focus of the trainee is dispersed in multiple areas, and the first preset entropy value is less than the second preset entropy value; the preset time window is divided into multiple continuous time segments, the number of times the focus switches between two adjacent continuous time segments is determined, and the focus switching frequency of the trainee on the operation interface is determined based on the number of area switching and the total time; the information entropy value and the focus switching frequency are used as focus stability features.

[0037] In the above embodiment, the virtual assembly operation interface is first divided into functional areas. The assembly operation area occupies approximately 60% of the central display area, including the 3D assembly model, parts library, and assembly operation space. This area is assigned area identifier 0 and defined as the task execution area. The right 20% of the interface displays process drawings, technical parameters, and assembly instruction documents, and is assigned area identifier 1, defined as the reference resource area. The top and left sidebar together occupy 20% of the area, including various assembly tool icons, measuring instrument options, and system function buttons, and are assigned area identifier 2, defined as the auxiliary tool area. This division is based on the logical relationship of the interface's functional layout, ensuring that each pixel uniquely belongs to a specific area. Gaze focus position data is collected in real time using an integrated eye-tracking device, with a sampling frequency of 30 data points per second. Within a 30-second time window, a total of 900 gaze point coordinates are collected. Each coordinate data includes horizontal and vertical coordinate values ​​and a timestamp information. A coordinate mapping algorithm is used to match each gaze point to its corresponding interface area. For example, the coordinate point (850, 450) is located within the assembly work area after mapping and is marked as region 0; the coordinate point (1420, 380) is located in the process drawing display area and is marked as region 1. The dwell time statistics are performed using continuous gaze point clustering. A gaze point located in the same region within 50 consecutive milliseconds is considered a valid dwell time, and the total dwell time for each region is accumulated. In a 30-second snapshot, the statistics show that the gaze focus stayed for 18 seconds in the task execution area, containing 12 independent gaze events; 9 seconds in the reference resource area, containing 8 gaze events; and 3 seconds in the auxiliary tool area, containing 4 gaze events. This duration distribution reflects that the operator mainly focuses their attention on the core assembly operation while moderately referring to technical documents.

[0038] In the above embodiments, the calculation of time percentages directly reflects the attention allocation pattern. The first time percentage is 18 seconds divided by 30 seconds, resulting in 0.6, indicating that 60% of the time is spent looking at the assembly work area. The second time percentage is 9 seconds divided by 30 seconds, resulting in 0.3, indicating that 30% of the time is spent looking at reference resources. The third time percentage is 3 seconds divided by 30 seconds, resulting in 0.1, indicating that 10% of the time is spent focusing on tool selection. This 6:3:1 attention allocation ratio conforms to the typical pattern of skilled operators, who mainly focus on operational execution, appropriately refer to technical guidance, and occasionally adjust tool settings. The information entropy value is calculated using the Shannon entropy formula to quantify the dispersion of the gaze distribution. Substituting the three time percentage values ​​into the formula, the calculation process is: negative 0.6 multiplied by the logarithm of 0.6 plus the logarithm of 0.3 multiplied by 0.3 plus the logarithm of 0.1 multiplied by 0.1, resulting in an information entropy value of approximately 1.295. The preset threshold for the first entropy value is 0.8, and the preset threshold for the second entropy value is 1.5. The current entropy value of 1.295 falls between two thresholds, indicating a relatively balanced focus distribution, neither highly concentrated in a single area nor excessively dispersed. This moderate entropy value typically corresponds to normal operating conditions. Calculating the focus switching frequency requires further analysis of temporal characteristics. The 30-second window was divided into six consecutive 5-second segments, and the change in gaze focus area between adjacent segments was tracked. The first 5-second segment primarily focused on area 0, the second segment switched to area 1, the third segment returned to area 0, the fourth segment remained in area 0, the fifth segment switched to area 2, and the sixth segment switched back to area 1. A total of four area switches occurred between adjacent segments, translating to a switching frequency of eight times per minute. This frequency reflects the operator's activity level in shifting their gaze between different functional areas.

[0039] In the above embodiment, the final focus stability characteristic is composed of an information entropy value of 1.295 and a focus switching frequency of 8 times / minute. A low information entropy value combined with a moderate switching frequency indicates that the operator maintains good concentration, with their gaze primarily focused on the core operational area, while also being able to consult reference materials and adjust tool settings as needed. This focus distribution pattern aligns with the cognitive patterns of skill operation, avoiding both blind operation and preventing excessive reference to information from affecting operational fluency.

[0040] In an optional embodiment, a deep analysis is performed on the reference resource access records to obtain resource participation depth features. Specifically, this includes: extracting the resource identifier, access timestamp, and browsing depth value of each accessed resource from the reference resource access record data; determining the access duration of each accessed resource based on the access timestamp, and determining the content browsing ratio of each accessed resource based on the browsing depth value; multiplying the browsing depth value of each accessed resource by the access duration to obtain a deep participation score for each accessed resource; determining the target resource participation degree based on the deep participation scores of all accessed resources and the total duration of a preset time window; determining the total number of accesses to different resources and the number of times the same resource is accessed repeatedly in the reference resource access record data; determining the resource coverage breadth of the training operator based on the total number of accesses and the number of times the same resource is accessed repeatedly; and using the resource participation degree and resource coverage breadth as resource participation depth features.

[0041] In the above embodiment, a complete sequence of access behaviors was extracted from the reference resource access record data. Within a 30-second time window, the operator accessed three key resources: a 3D assembly model of the piston connecting rod, an assembly process specification document, and a standard torque parameter table. Each resource access record contained a unique resource identifier, timestamps for the start and end of the access, and a browsing depth value. The resource identifier for the 3D assembly model of the piston connecting rod was R001, and the access began at the 5th second and ended at the 17th second. During this period, the operator performed 15 model rotation operations, viewing various angles and assembly details of the part. The resource identifier for the assembly process specification document was R002, and the access began at the 18th second and ended at the 23rd second, with the document scrolling to 85% of its total length. The resource identifier for the standard torque parameter table was R003, and the access began at the 24th second and ended at the 26th second, during which the operator viewed three key parameter values. The duration of the access was calculated using the timestamp difference. The access duration for the 3D assembly model of the piston connecting rod was 12 seconds, a relatively long access time reflecting the operator's in-depth study of the core assembly object. Access to the assembly process specification document lasted 5 seconds, indicating that the operator quickly browsed the key process points. Access to the standard torque parameter table lasted 2 seconds, reflecting a behavior pattern of precisely searching for specific parameters. The content browsing ratio was determined based on the characteristic indicators of different resource types. The browsing depth value of the 3D assembly model was calculated by the ratio of the number of operation interactions to the number of standard operations, with 15 rotation operations corresponding to a browsing depth value of 1.0; the browsing depth value of the document was directly calculated as a scroll position percentage of 0.85; the browsing depth value of the parameter table was calculated by the ratio of the number of parameters viewed to the total number of parameters, with 3 parameters corresponding to a browsing depth value of 0.3.

[0042] In the above embodiment, the deep engagement score is calculated by multiplying the browsing depth value by the access duration. The deep engagement score for the piston connecting rod 3D assembly model is 1.0 multiplied by 12 seconds, resulting in 12 points; the deep engagement score for the assembly process specification document is 0.85 multiplied by 5 seconds, resulting in 4.25 points; and the deep engagement score for the standard torque parameter table is 0.3 multiplied by 2 seconds, resulting in 0.6 points. This multiplication operation rewards in-depth browsing behavior while considering the value of time investment, avoiding high scores from superficial access patterns. The total deep engagement score for the three resources is 16.85 points. The target resource engagement level is determined based on the deep engagement scores of all resources and the total duration of the time window. The system normalizes the total deep engagement score of 16.85 points with the total duration of the 30-second time window, calculating a target resource engagement level of 0.56. This figure indicates that the operator spent 56% of the effective time during this period deeply engaging with various reference resources, maintaining reasonable utilization of auxiliary resources without over-relying on external assistance and neglecting actual operation. The system sets a reasonable range for participation based on different task types; for assembly tasks, the reasonable participation level is typically between 0.4 and 0.7. The analysis of resource coverage breadth focuses on the operator's breadth of utilization of different resources. Statistics show a total of 4 accesses to different resources, including the initial access to three different resources and a repeated access to the piston connecting rod 3D assembly model at the 25th second; therefore, the number of repeated accesses to the same resource is 1. Based on the total of 4 accesses and 1 repeated access, the resource coverage breadth is calculated, yielding 0.75 through a specific proportional relationship. This high breadth value indicates that the operator can fully utilize multiple types of reference resources while repeatedly reviewing important resources, reflecting a combination of comprehensive and focused knowledge acquisition.

[0043] In the above embodiments, the resource engagement depth feature is ultimately composed of a target resource engagement degree of 0.56 and a resource coverage breadth of 0.75. This combination of moderate engagement degree and high coverage breadth reflects the operator's adoption of a learning strategy that is broad in scope but moderately in-depth. The operator not only examines the three-dimensional structure of the assembly object but also refers to process specifications and technical parameters, forming a three-dimensional knowledge acquisition model. This resource utilization model typically corresponds to operators who study diligently and think actively, enabling them to understand and master assembly skills from multiple perspectives.

[0044] In an optional embodiment, the operational quality assessment result of the trainee is generated based on operational activity features, focus stability features, and resource participation depth features. Specifically, this includes: integrating the operational activity features, focus stability features, and resource participation depth features to construct a multi-dimensional feature vector; extracting activity values ​​from the operational activity features and comparing them with a first judgment threshold to obtain an operational activity comparison result; extracting stability values ​​from the focus stability features and comparing them with a second judgment threshold to obtain a focus stability comparison result; extracting participation values ​​from the resource participation depth features and comparing them with a third judgment threshold to obtain a resource participation depth comparison result; determining the trainee's current operational state as a deep analysis state when the operational activity feature is less than the first judgment threshold and the focus stability feature is greater than the second judgment threshold based on the operational activity comparison result; determining the trainee's current operational state as an idle state when the operational activity feature remains continuously at zero for a preset duration based on the focus stability comparison result; and further determining the trainee's current operational state as an idle state based on the root... When the resource participation depth characteristic is less than the third judgment threshold and the operation activity characteristic is greater than the fourth judgment threshold, the current operation status of the trainee is determined to be a perfunctory operation status. The first occurrence frequency and first duration of the deep analysis status in the behavior snapshot sequence are determined; the second occurrence frequency and second duration of the idle status in the behavior snapshot sequence are determined; and the third occurrence frequency and third duration of the perfunctory operation status in the behavior snapshot sequence are determined. A first descending order ranking analysis is performed on the first, second, and third occurrence frequencies to obtain a frequency ranking sequence; and a second descending order ranking analysis is performed on the first, second, and third durations to obtain a duration ranking sequence. Based on the frequency ranking sequence and duration ranking sequence, the trainee's current dominant operation mode is determined from the deep analysis status, idle status, and perfunctory operation status. Based on the dominant operation mode and multi-dimensional feature vector, the trainee's current comprehensive quality score and current operation quality level are determined. The current operation quality level, current comprehensive quality score, and current dominant operation mode are used as the operation quality evaluation results.

[0045] In the above embodiment, the features extracted from the aforementioned dimensions are integrated to construct a five-dimensional feature vector. This vector includes five components: an activity score of -2.33, an editing intensity value of 0.14, an information entropy value of 1.295, a focus switching frequency of 6 times per minute, and a resource participation degree of 0.56. This multi-dimensional vector comprehensively depicts the operator's behavioral pattern within the time window, with each dimension reflecting a different aspect of the operation quality from a different perspective. This feature vector serves as the basic input for subsequent state determination and quality scoring. The operation state determination employs a multi-condition combination judgment logic. The activity score of -2.33 is extracted from the activity feature and compared with the first judgment threshold of -1.0, concluding that the activity level is significantly low. The information entropy value of 1.295 is extracted from the focus stability feature and compared with the second judgment threshold of 2.0, concluding that the focus is relatively concentrated. When both the activity score and the information entropy value are less than -1.0 and less than 2.0, the current operation state is determined to be a deep analysis state. This state indicates that although the operator's operation frequency is low, their attention is highly concentrated, and they are engaged in in-depth thinking and analysis. Identifying an idle state requires stricter criteria. It checks whether the user activity level remains zero for three consecutive time windows, meaning there are absolutely no user actions within 90 seconds. It also verifies that the focus entropy value is less than 0.3, indicating minimal eye movement. In this embodiment, although the user's activity level is low, some user actions still occur, and the focus shifts moderately between different areas; therefore, the criteria for an idle state are not met.

[0046] In the above embodiments, the determination of the perfunctory operation state focuses on the mismatch between resource utilization and operation frequency. The participation value of 0.56 is extracted from the resource participation depth feature and compared with the third judgment threshold of 0.5; the result is slightly higher than the threshold. Simultaneously, it is checked whether the activity score is greater than the fourth judgment threshold of 1.5, indicating abnormally high-frequency operation. When the resource participation is less than 0.5 and the activity score is greater than 1.5, it is judged as a perfunctory operation state. This state is usually manifested as rapid clicking without in-depth resource viewing. In this embodiment, the operator's resource participation is moderate, and the activity is low, which does not meet the characteristics of a perfunctory operation. A comprehensive analysis was performed on 60 behavioral snapshots throughout the complete task execution cycle. The deep analysis state appeared 35 times in these snapshots, with a cumulative duration of 1050 seconds, accounting for 58.3% of the total duration. The idle state appeared twice, each lasting 90 seconds, totaling 180 seconds, accounting for 10% of the total duration. The perfunctory operation state appeared 8 times, with a cumulative duration of 240 seconds, accounting for 13.3% of the total duration. The operator was in a normal operation state for the remaining time. This statistical analysis provides the temporal distribution characteristics of operator behavior patterns. The dominant operating mode is determined through a dual ranking mechanism. The frequency of occurrence of the three states is sorted in descending order, resulting in a frequency ranking sequence of: deep analysis (35 times), perfunctory operation (8 times), and idle (2 times). The duration is then sorted in descending order, resulting in a duration ranking sequence of: deep analysis (1050 seconds), perfunctory operation (240 seconds), and idle (180 seconds). Since the first state in both ranking sequences is deep analysis, deep analysis is determined to be the current dominant operating mode. This dual verification mechanism avoids the bias that might arise from a single indicator.

[0047] In the above embodiment, the comprehensive quality score is calculated using a weighted fusion algorithm. First, each feature value is normalized, mapping the activity score to the 0-1 range to obtain 0.2, the complementary value of the information entropy value to 0.35, and the resource participation value to 0.56. Using weight coefficients of 0.2, 0.3, and 0.5 to correspond to the three dimensions of operation frequency, focus concentration, and resource utilization, respectively, the comprehensive score is calculated as 0.2 multiplied by 0.2 plus 0.3 multiplied by 0.35 plus 0.5 multiplied by 0.56, resulting in 0.425. Mapping this normalized score to a percentage scale yields a final quality score of 72.5. The operation quality level is divided based on a preset scoring range. 0-60 points are defined as the level requiring improvement, 60-80 points as the good level, and 80-100 points as the excellent level. The current score of 72.5 falls within the good level range, indicating that the operator's overall operation quality has reached a satisfactory level, but there is still room for improvement. Based on the assessment that the dominant operational mode is deep analysis, the learner is considered a cautious learner. Although their operation speed is slow, their learning attitude is serious, and they are able to deeply understand the task content. The final operation quality assessment result includes three core elements. The current operation quality level is good, the overall quality score is 72.5 points, and the dominant operational mode is deep analysis. This triad assessment result not only provides a quantitative quality score but also reveals the learner's behavioral characteristics and learning style. This assessment result can also be fed back into the teaching management system to provide a basis for personalized teaching guidance. For learners with a deep analysis mode, it is recommended to give them more time to think and avoid inappropriate negative evaluations due to slow operation speed.

[0048] In an optional embodiment, after using the current operational quality level, current comprehensive quality score, and current dominant operational mode as the operational quality assessment results, the method further includes: comparing the current operational quality level with a preset quality level classification standard to obtain a level comparison result; generating a first-level intervention signal when the current operational quality level is determined to belong to the low-quality level category based on the level comparison result; generating a second-level intervention signal when the current operational quality level is determined to belong to the medium-quality level category and the current dominant operational mode is the perfunctory operational mode based on the level comparison result; and selecting a corresponding intervention strategy based on the first-level intervention signal or the second-level intervention signal, wherein: when a first-level intervention signal exists, a first-level intervention signal is generated. The system generates guiding prompts; when a second-level intervention signal is present, it generates incentive reminders; it acquires the trainee's historical operational quality level, historical comprehensive quality score, and historical dominant operational mode; it performs trend analysis on the historical operational quality level, historical comprehensive quality score, and historical dominant operational mode to determine the operational quality change trend; when the operational quality change trend shows a downward trend in the operational quality level or comprehensive quality score, it adjusts the difficulty coefficient of subsequent training tasks; and it records the guiding prompts or incentive reminders and the operational quality change trend in the trainee's personal skill file.

[0049] In the above embodiment, the obtained operational quality level of "Good" is carefully compared with the preset quality level classification standard. The preset standard divides the quality level into three categories: low quality (including unqualified and needing improvement), medium quality (including passable and good), and high quality (including excellent and outstanding). The current operational quality level of "Good" belongs to the medium quality category, and further checks are made to see if the dominant operational mode is a perfunctory operation mode. Since the current dominant operational mode is in-depth analysis rather than perfunctory operation, it is determined that no intervention signal needs to be generated, and the operator's learning status is basically normal. In another scenario, if the operational quality level is "Needing Improvement" and belongs to the low quality category, a first-level intervention signal will be generated immediately. This is the strongest level of intervention, indicating that the operator's skill mastery is significantly insufficient. The generation mechanism of guiding prompts is triggered based on the first-level intervention signal. The guiding prompts include specific operational suggestions and learning resource recommendations. For example, a dialog box may pop up indicating that the piston installation sequence is incorrect, suggesting rewatching the standard assembly process video, and providing a link button to directly jump to relevant teaching resources. This guiding intervention directly targets the core of the problem, helping the operator quickly locate and correct the error.

[0050] In the above embodiments, if the operation quality level is good but the dominant operation mode is perfunctory operation, a second-level intervention signal will be generated. This is a medium-intensity intervention level, indicating that although the operator has met the basic requirements, their learning attitude needs improvement. The second-level intervention signal triggers the generation of incentive reminders. These reminders appear in a non-intrusive manner, such as a floating prompt at the edge of the screen, stating that there is room for improvement in your operation efficiency and that trying to optimize the assembly sequence by referring to the process drawings might yield better results. This gentle reminder neither interrupts the operation process nor hinders the operator's learning methods. Historical data comparison and analysis provide temporal information for judging quality trends. The operator's most recent five performances of similar tasks are extracted from the skill profile database. The historical operation quality level sequence is shown as Pass, Pass, Good, Good, Good, showing a steady upward trend. The historical comprehensive quality score sequence is 65, 68, 70, 71, and 72.5, with an average increase of 1.875 points each time. The historical dominant operation mode gradually shifts from initial normal operation to in-depth analysis, indicating that the operator's learning strategy is continuously being optimized. The trend of operational quality changes was calculated using linear regression analysis. Least squares fitting of the historical rating sequence yielded a trend line with a slope of 1.9, indicating a stable upward trend in ratings. Changes in quality levels were analyzed using a transition probability matrix; the probability of transitioning from passable to good was 0.6, and the probability of maintaining good was 0.8, indicating a steady improvement in the operator's skill level. The evolution of dominant operational patterns was analyzed using the percentage of pattern duration; the percentage of in-depth analysis patterns increased from 20% to 58%, showing that operators are increasingly inclined towards in-depth thinking and analysis.

[0051] In the above embodiment, the difficulty coefficient of subsequent training tasks is intelligently adjusted based on the positive development of the quality change trend. The current task has a difficulty coefficient of 1.0; considering the steady improvement of the operator's skills, the difficulty coefficient of the next task is increased to 1.2. The increased difficulty is reflected in the increase in the number of assembly parts from 8 to 10, the increase in assembly accuracy requirements from ±0.5 mm to ±0.3 mm, and the introduction of assembly steps requiring special tools. This gradual adjustment of difficulty maintains the challenge of learning while avoiding the frustration caused by sudden increases in difficulty. Updating the individual skills profile ensures the complete recording and long-term tracking of assessment information. Complete information about this assessment is written into the profile, including task execution time, good quality level, comprehensive score of 72.5 points, in-depth analysis of the dominant mode, and no intervention triggers. The results of the quality change trend analysis are also recorded, including a score increase slope of 1.9, a level improvement probability of 0.6, and a 58% proportion of in-depth analysis modes. The profile also saves potential intervention content generated by the system but not triggered, providing a reference for the development of subsequent personalized teaching plans. This accumulated archival data will be used to identify more precise individual capabilities, supporting long-term skills development planning and training path optimization.

[0052] In an optional embodiment, before acquiring the time-series operation data stream of the trainee during the execution of the skills training task, the method further includes: acquiring the trainee's identity information and extracting the trainee's historical skill level data from the skill archive based on the identity information; determining the trainee's skill proficiency level based on the historical skill level data and configuring benchmark parameters for activity analysis based on the skill proficiency; acquiring the task attribute information of the current skills training task, including the task difficulty coefficient, task type identifier, and expected completion time; performing a time window configuration operation based on the matching relationship between the task difficulty coefficient and the skill proficiency level to obtain a preset time window; determining the data type range to be collected based on the task type identifier and configuring the data collection range of the time-series operation data stream; setting the total collection time of the behavior snapshot sequence based on the expected completion time; establishing an association session between the trainee and the current skills training task, configuring the preset time window, benchmark parameters, data collection range, and total collection time in the association session to complete the initialization of the data collection environment before acquiring the time-series operation data stream.

[0053] In the above embodiment, before the skills training task begins, operator identification and historical data extraction are performed first. When the operator logs into the system by swiping their employee card, the card reader obtains the identification information E11110015 from the chip embedded in the employee card. Based on this unique identifier, the operator's complete historical record is retrieved from the skills archive. The archive data shows that the operator has completed 15 assembly tasks of various types, including 12 engine assembly tasks, with an average completion time of 25 minutes, an average operation accuracy rate of 88%, a highest historical score of 92 points, a lowest score of 75 points, and an average score of 82 points over the past three months. Based on the statistical analysis of this historical data, the operator's skill proficiency level is determined to be intermediate, in the transitional stage between novice and expert. The baseline parameter configuration for activity analysis is dynamically adjusted according to the operator's skill proficiency level. For operators with intermediate proficiency, the configured operation frequency statistical baseline is 1.3 operations per second, which is about 8% higher than the novice baseline of 1.2 operations / second, but about 13% lower than the expert baseline of 1.5 operations / second. This differentiated setting reflects the typical operating rhythm of operators at different skill levels. Simultaneously, the allowable deviation range for activity analysis is configured, with the standard deviation parameter for intermediate operators set at 0.35 times / second, stricter than the 0.4 for beginners but more lenient than the 0.25 for experts. This benchmark parameter configuration considers both the relatively high stability already possessed by intermediate operators and allows for reasonable fluctuations during their learning and improvement process, providing a personalized evaluation benchmark for subsequent activity analysis.

[0054] In the above embodiment, the current skill training task to be executed is the precision assembly of a turbocharger. Detailed task attribute information is obtained from the task management module. The task difficulty coefficient is set at 1.5, significantly higher than the basic piston assembly task difficulty coefficient of 1.0. This is because turbochargers contain more precision parts, have stricter assembly tolerance requirements, and require the use of specialized tools and measuring instruments. The task type is identified as PM_ASSEMBLY, indicating a precision assembly category; this type of task requires the collection of more comprehensive operational data. The expected completion time is set at 600 seconds, or 10 minutes, a standard value determined based on the average time taken by an expert group to complete the task. The time window configuration fully considers the matching relationship between task difficulty and operator ability. Since the turbocharger assembly difficulty coefficient of 1.5 is higher than that of regular tasks, and the operator is at an intermediate level, the preset time window is extended from the standard 30 seconds to 45 seconds. This adjustment allows each behavioral snapshot to include a longer period of operational behavior, helping to capture detailed operations and pauses in thought during the precision assembly process, and avoiding missing key behavioral characteristics due to a too-short window. A longer time window can better smooth out instantaneous fluctuations in operation frequency, resulting in more stable and reliable statistical characteristics. The data acquisition range is configured specifically for the characteristics of precision assembly tasks. Full-level operation log collection is enabled, recording detailed information for every mouse click, keyboard input, part selection, and tool switch. The eye-tracking data acquisition module is set to high-precision mode, with a sampling rate increased to 60 times per second, focusing on changes in the operator's gaze focus during critical assembly steps. Resource access records are configured in a technical document-first mode, prioritizing the collection of operator access to assembly drawings, tolerance tables, and process specifications, while a simplified mode is used for accessing general help documents. Furthermore, an operation playback recording function is enabled, fully saving the operator's screen operation video for detailed post-operation analysis.

[0055] In the above embodiment, the total collection time is appropriately extended based on the expected completion time. Although the expected completion time for the task is 600 seconds, considering that intermediate operators may need additional checking and adjustment time, the total collection time for the behavior snapshot sequence is set to 720 seconds, or 12 minutes. This 20% time redundancy ensures that even if the operator completes the task slightly slowly, the complete operation process can still be collected. If the operator completes the task ahead of schedule, subsequent behavior will continue to be collected for 30 seconds after the task is submitted, for analyzing the operator's self-checking and review behavior. The establishment of the associated session integrates all configuration parameters into a unified operating environment. A session identifier, Session_11110315_E11110015_PM, is created, which includes a date and timestamp, operator identifier, and task type. The session data packet encapsulates all initialization parameters, including a preset time window of 45 seconds, activity analysis benchmark parameters (operation frequency benchmark 1.3 times / second, standard deviation parameter 0.35), data collection range configuration table, and a total collection time of 720 seconds. This data packet is transmitted encrypted simultaneously to both the front-end data acquisition unit and the back-end analysis module, ensuring that all components of the entire quality assessment system use consistent configuration parameters. Upon receiving the configuration, the front-end acquisition unit automatically adjusts sensor parameters and cache size, while the back-end analysis module pre-allocates corresponding computing resources and storage space. Once all components have completed initialization and returned to a ready state, the user interface displays a message indicating that the data acquisition environment initialization is complete and the task can begin. Only then can the operator see the task details and start operating the system.

[0056] In this embodiment, the time-series operation data stream is segmented into a sequence of behavioral snapshots after passing through a preset time window. This discretizes continuous operational behaviors into analyzable time units, making each behavioral snapshot an independent analytical object. Comparison analysis of current operation frequency data with a preset frequency threshold quantifies the operator's activity level; distribution analysis of gaze focus position data reflects the operator's attention allocation pattern; and in-depth analysis of reference resource access record data reflects the operator's learning engagement. These three types of features characterize operational behavior from different dimensions: operational activity reveals performance at the action execution level, focus stability reflects the state at the cognitive attention level, and resource engagement depth demonstrates the degree of knowledge acquisition. These three features complement each other to form a multi-dimensional evaluation system. By comprehensively generating operational quality evaluation results from these three types of features, the one-sidedness of single-dimensional evaluation can be avoided, improving the accuracy of judging complex operational behaviors.

[0057] The quality assessment system in the embodiments of this invention is described below from the perspective of hardware processing. (See attached document.) Figure 2 , Figure 2 This is a schematic diagram of the physical device structure of a quality assessment system in the embodiments of this application.

[0058] It should be noted that, Figure 2 The structure of the quality assessment system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0059] like Figure 2 As shown, the quality assessment system includes a Central Processing Unit (CPU) 201, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 202 or a program loaded from storage portion 208 into Random Access Memory (RAM) 203, such as performing the methods described in the above embodiments. The RAM 203 also stores... It contains various programs and data required for system operation. CPU 201, ROM 202, and RAM 203 are interconnected via bus 204. Input / output (I / O) interface 205 is also connected to bus 204.

[0060] The following components are connected to I / O interface 205: input section 206 including audio input devices, push-button switches, etc.; output section 207 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 208 including a hard disk, etc.; and communication section 209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 209 performs communication processing via a network such as the Internet. Drive 210 is also connected to I / O interface 205 as needed. Removable media 211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 210 as needed so that computer programs read from them can be installed into storage section 208 as needed.

[0061] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the computer program is executed by central processing unit (CPU) 201, it performs the various functions defined in the present invention.

[0062] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0063] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0064] Specifically, the quality assessment system of this embodiment includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements the quality assessment method for skills training operation behavior provided in the above embodiment.

[0065] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the quality assessment system described in the above embodiments; or it may exist independently and not incorporated into the quality assessment system. The storage medium carries one or more computer programs that, when executed by a processor of the quality assessment system, cause the quality assessment system to implement the quality assessment method for skills training operations provided in the above embodiments.

[0066] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0067] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for quality assessment of skills training operational behavior, characterized in that, include: Acquire the time-series operation data stream of the trainee during the execution of the skills training task, and segment the time-series operation data stream according to a preset time window to generate a behavior snapshot sequence; Extract the current operation frequency data of the trainee from each behavior snapshot in the behavior snapshot sequence, and perform activity analysis on the current operation frequency data to obtain operation activity characteristics; Extract the gaze focus position data of the trainee on the operation interface from each behavioral snapshot in the behavioral snapshot sequence, and perform distribution analysis on the gaze focus position data to obtain focus stability characteristics; Extract reference resource access record data of the training operator from each behavior snapshot in the behavior snapshot sequence, and perform in-depth analysis on the reference resource access record data to obtain resource participation depth features; Based on the operational activity characteristics, the focus stability characteristics, and the resource participation depth characteristics, the operational quality assessment results of the training operators are generated.

2. The method according to claim 1, characterized in that, The step of performing activity analysis on the current operation frequency data to obtain operation activity characteristics specifically includes: Obtain historical operation data with the same task type as the current skills training task, and determine the mean and standard deviation of the operation frequency of the trainee in the historical operation data, and use the mean operation frequency as the baseline operation frequency. Determine the average operating frequency of the current operating frequency data within the preset time window, and perform a deviation analysis between the average operating frequency and the baseline operating frequency to obtain the operating frequency deviation value; Divide the operation frequency deviation by the standard deviation to obtain the activity score; Extract records of text editing operations performed by the trainee within the preset time window from each behavior snapshot, and determine the number of editing operations performed by the trainee based on the records; The number of newly added characters and the number of deleted characters are extracted from the record. The number of newly added characters and the number of deleted characters are added together to obtain the total number of character changes. The number of editing operations is divided by the total number of character changes to obtain the editing intensity value. The activity score and the editing intensity value are used as the operational activity features.

3. The method according to claim 1, characterized in that, The distribution analysis of the gaze focus position data to obtain focus stability characteristics specifically includes: The user interface is divided into a task execution area, a reference resource area, and an auxiliary tool area, and a unique area identifier is assigned to each of the task execution area, the reference resource area, and the auxiliary tool area. Based on the gaze focus position data, determine the target area where the training operator's gaze focus is located within the preset time window; The target region is matched with the region identifier to determine the first duration of the gaze focus in the task execution region, the second duration of the gaze focus in the reference resource region, and the third duration of the gaze focus in the assistive tool region; The first time percentage of the gaze focus in the task execution area is determined based on the first dwell time and the total duration of the preset time window. The second time percentage of the gaze focus in the reference resource area is determined based on the second dwell time and the total time. The third time percentage of the gaze focus in the assistive tool area is determined based on the third dwell time and the total duration. The information entropy value of the gaze focus is determined based on the first time proportion, the second time proportion, and the third time proportion. A threshold analysis is performed on the information entropy value to determine that when the information entropy value is less than a first preset entropy value, the focus of the trainee is concentrated in a single area; when the information entropy value is greater than a second preset entropy value, the focus of the trainee is dispersed in multiple areas, and the first preset entropy value is less than the second preset entropy value. The preset time window is divided into multiple continuous time segments, the number of times the gaze focus switches between two adjacent continuous time segments is determined, and the focus switching frequency of the training operator on the operation interface is determined based on the number of region switching and the total duration. The information entropy value and the focus switching frequency are used as the focus stability features.

4. The method according to claim 1, characterized in that, The in-depth analysis of the reference resource access records to obtain resource participation depth characteristics specifically includes: Extract the resource identifier, access timestamp, and browsing depth value for each accessed resource from the reference resource access record data; The access duration of each accessed resource is determined based on the access timestamp, and the content browsing ratio of each accessed resource is determined based on the browsing depth value. Multiply the browsing depth value of each accessed resource by the access duration to obtain the depth participation score of each accessed resource; The target resource engagement is determined based on the deep engagement scores of all accessed resources and the total duration of the preset time window. Determine the total number of accesses to different resources and the number of times the same resource is accessed repeatedly in the reference resource access record data; The resource coverage breadth of the training operator is determined based on the total number of visits and the number of repeated visits. The resource participation degree and the resource coverage breadth are used as the resource participation depth features.

5. The method according to claim 1, characterized in that, The step of generating the operational quality assessment result for the trainee based on the operational activity feature, the focus stability feature, and the resource participation depth feature specifically includes: The operational activity feature, the focus stability feature, and the resource participation depth feature are integrated to construct a multi-dimensional feature vector; The activity value is extracted from the operation activity feature, and the activity value is compared with the first judgment threshold to obtain the operation activity comparison result; The stability value is extracted from the focus stability feature, and the stability value is compared with the second determination threshold to obtain the focus stability comparison result. The participation value is extracted from the resource participation depth feature, and the participation value is compared with the third judgment threshold to obtain the resource participation depth comparison result. When the operation activity feature is determined to be less than the first judgment threshold and the focus stability feature is greater than the second judgment threshold based on the operation activity comparison result, the current operation state of the training operator is determined to be the deep analysis state. When the operation activity feature remains at zero for a preset duration based on the focus stability comparison result, the current operation state of the training operator is determined to be an idle state. When the resource participation depth feature is determined to be less than the third judgment threshold and the operation activity feature is greater than the fourth judgment threshold based on the resource participation depth comparison result, the current operation state of the training operator is determined to be a perfunctory operation state. The first occurrence frequency and first duration of the deep analysis state in the behavior snapshot sequence are determined; the second occurrence frequency and second duration of the idle state in the behavior snapshot sequence are determined; and the third occurrence frequency and third duration of the perfunctory operation state in the behavior snapshot sequence are determined. A first descending order sorting analysis is performed on the first occurrence frequency, the second occurrence frequency, and the third occurrence frequency to obtain a frequency sorting sequence; and a second descending order sorting analysis is performed on the first duration, the second duration, and the third duration to obtain a duration sorting sequence. The current dominant operation mode of the trainee is determined from the deep analysis state, the idle state, and the perfunctory operation state based on the frequency sorting sequence and the duration sorting sequence. The current comprehensive quality score and current operation quality level of the trainee are determined based on the dominant operation mode and the multidimensional feature vector. The current operational quality level, the current comprehensive quality score, and the current dominant operational mode are used as the operational quality assessment results.

6. The method according to claim 5, characterized in that, After using the current operational quality level, the current comprehensive quality score, and the current dominant operational mode as the operational quality assessment result, the method further includes: The current operation quality level is compared with the preset quality level classification standard to obtain the level comparison result; When the current operation quality level is determined to be low quality level based on the level comparison results, a first-level intervention signal is generated. When the current operation quality level is determined to be medium quality level based on the level comparison results and the current dominant operation mode is perfunctory operation mode, a second-level intervention signal is generated. The corresponding intervention strategy is selected based on either the first-level intervention signal or the second-level intervention signal, wherein: When the first-level intervention signal is present, a guiding prompt message is generated; When a second-level intervention signal is present, an incentive reminder message is generated; Obtain the historical operation quality level, historical comprehensive quality score, and historical dominant operation mode of the training operator; By performing trend analysis on the historical operation quality level and the current operation quality level, the historical comprehensive quality score and the current comprehensive quality score, and the historical dominant operation mode and the current dominant operation mode, the operational quality change trend of the trainee can be determined. When the trend of operational quality changes shows that the operational quality level or the comprehensive quality score is declining, the difficulty coefficient of subsequent training tasks will be adjusted. The guidance or incentive information, along with the trend of changes in operational quality, will be recorded in the trainee's personal skills file.

7. The method according to claim 1, characterized in that, Before acquiring the time-series operation data stream of the trainee during the execution of the skills training task, the method further includes: Obtain the identity information of the trainee, and extract the trainee's historical skill level data from the skill archive based on the identity information; The skill proficiency level of the trainee is determined based on the historical skill level data, and the baseline parameters for the activity analysis are configured based on the skill proficiency level. Obtain the task attribute information of the current skills training task, including the task difficulty coefficient, task type identifier, and expected completion time; The time window configuration operation is performed based on the matching relationship between the task difficulty coefficient and the skill proficiency level to obtain the preset time window; The data type range to be collected is determined based on the task type identifier, and the data collection range of the time-series operation data stream is configured. Set the total collection duration of the behavior snapshot sequence according to the expected completion time; Establish an association session between the training operator and the current skill training task, and configure the preset time window, the benchmark parameters, the data acquisition range and the total acquisition duration in the association session to complete the initialization of the data acquisition environment before acquiring the time-series operation data stream.

8. A quality assessment system, characterized in that, The quality assessment system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the quality assessment system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the quality assessment system, the quality assessment system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on a quality assessment system, the quality assessment system performs the method as described in any one of claims 1-7.

Images