Intelligent interaction and automatic scoring management method and system for vocational skill competition

Abstract

The application discloses a kind of intelligent interaction and automatic scoring management method and system for professional skill competition, it is related to digital management and intelligent scoring technical field.The application is limited to be applied to the scene of new a gas internal staff professional skill competition, and the object of competition is the staff authenticated by new a gas internal identity system, and the application realizes overall process design around the whole process data reliable collection of competition, intelligent objective scoring and achievement safety control.The session configuration is generated by the event management server and the task package is issued;After the session is started, the controlled competition environment is established in the competition end, and interface interaction, simulation operation, sensing, result file and / or video abstract data are collected and eventized reporting;Data acquisition and alignment service corrects timestamp based on time synchronization metadata and quality inspection aggregation, generates evidence package containing event index, key abstract, root hash, rule version and double signature;Scoring engine loads and compiles rules to automatically calculate score points / penalty points and output scoring report containing evidence reference, combined with stability and benchmark score and grading review audit, lock achievement after generation training list and issue achievement.

Description

Technical Field

[0001] This invention relates to the field of digital management and intelligent scoring technology, and in particular to intelligent interaction and automatic scoring management methods and systems. Background Technology

[0002] Vocational skills competitions typically cover multiple events and scenarios, generating multi-source data such as operation logs, task completion status, quality inspection results, work duration, and interaction behaviors. However, existing competition organization and scoring methods have the following shortcomings: competition data is scattered across multiple business systems or terminal devices, making it difficult to form a unified view of participants, events, work steps / tasks, and time; scoring relies on manual aggregation, and judges' manual scoring is highly subjective, making it difficult to provide traceable scoring basis for each work step; and there is a lack of verifiable data support when disputes occur. Existing automatic scoring mechanisms mostly rely on simple weighting or single-dimensional statistics, failing to effectively suppress occasional errors, exhibiting poor comparability across sessions, and struggling to identify abnormal fluctuations. When scores are abnormal, there is a lack of tiered review mechanisms, resulting in low efficiency and high cost in dispute resolution. Existing automatic scoring systems typically only score the final result and lack evidence solidification and rule version binding, making reproduction difficult. Furthermore, scoring results are difficult to automatically transform into deficiency diagnosis and personalized training tasks, failing to form a closed-loop improvement path of scoring, diagnosis, improvement, and re-competition.

[0003] Therefore, there is an urgent need for a competition scoring management method and system that can achieve automatic collection of multi-source data, reproducible scoring models, detectable anomalies, traceable verification, and closed-loop application of results. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] On the one hand, the present invention provides an intelligent interactive and automatic scoring management method for vocational skills competitions, the method comprising the following steps:

[0006] S1. Conduct pre-competition configuration and competition arrangement, and establish the relationship between competition information, round information, workstation information, employee information, job qualification information, affiliated organization information, referee information, data collection strategy information, and scoring rule version information;

[0007] S2. Establish a controlled competition environment on the participating end, load the task package and perform employee identity authentication, job qualification verification, authorized terminal verification and intranet environment verification, and collect multi-source data such as interface interaction, gas business simulation platform operation, equipment sensor data, result files and video summaries during the competition process in an event-driven manner.

[0008] S3. Perform time alignment, quality verification, and session aggregation on the collected multi-source events to generate evidence packages corresponding to the sessions, and bind the rule version number to the evidence package;

[0009] S4. Call the scoring engine to read the rule version and evidence package, automatically execute the scoring points, deduction points, step status and aggregation rules, and output the sub-scores, total score, deduction points and evidence references;

[0010] S5. Based on the scoring results, perform stability scoring, benchmarking scoring, level determination, anomaly judgment and graded review, trigger review and final audit for abnormal samples, and audit and record the modification chain.

[0011] S6. After the results are locked, the results are released, displayed on the dashboard, pushed to the system, aggregated with ability tags, and generated training tasks, so that the competition results are transformed into training lists and subsequent improvement plans.

[0012] On the other hand, the present invention also provides an intelligent interactive and automatic scoring management system for vocational skills competitions, the system comprising:

[0013] The event management server is used for event configuration, round arrangement, workstation binding, participant and referee account management, rule version management, results release and appeal archiving;

[0014] The competition client is used to establish a controlled competition environment, display tasks and interactive interfaces, collect process data, and report events.

[0015] The judge's client is used to view automatic scoring results, retrieve evidence, conduct review, and confirm the final judgment.

[0016] Data acquisition and alignment services are used to perform event-based, time-aligned, quality-verified, and evidence package generation on multi-source data.

[0017] The rule compilation and scoring engine service is used to convert scoring criteria into machine-executable rules and perform automatic scoring.

[0018] Evidence storage and auditing services are used to store and audit evidence packages, rule versions, scoring reports, and review and modification chains.

[0019] The dashboard and message push service is used to display the competition status, rankings and abnormal information in real time, and to push alarms and handling prompts to relevant personnel.

[0020] Furthermore, the present invention preferably adopts a unified data object model, including at least a Session, an Event, an EvidenceBundle, and a ScoreReport, to ensure data interoperability and semantic consistency between modules.

[0021] Furthermore, the present invention preferably converts the scoring criteria into a machine-executable rule model, wherein the rule model includes at least a step state machine, scoring point rules, deduction point rules, aggregation rules, rule version number, and signature information.

[0022] Furthermore, the present invention preferably embeds time synchronization metadata in the evidence package, so that data from different sources can be replayed and reviewed based on the same timeline.

[0023] Furthermore, the present invention preferably establishes a mapping library of scoring items, ability labels, and training tasks, so that the scoring results can be automatically converted into a list of ability deficiency diagnoses and training tasks.

[0024] Compared to existing solutions, the beneficial effects achieved by this invention are:

[0025] To improve the objectivity and real-time nature of scoring, multi-source data is automatically collected and calculated, reducing errors from manual statistical analysis and enabling a shift from experience-based judgment to data-driven approaches. Because the collected data is limited to interface interactions, gas job simulations, tool sensing, result files, and video summary data from internal employee job competitions at ENN Gas, the scoring criteria are more consistent across different scenarios.

[0026] To improve the stability and fairness of scoring, stability scores and benchmarking scores are introduced to reduce the impact of occasional anomalies on the overall score and improve comparability within the same and different sessions. In a preferred embodiment, benchmarking calculation is limited to an internal employee sample within the same position, competition item, round, and / or qualification level, thereby improving the fairness of the results of ENN Gas's internal job competitions.

[0027] Improve the efficiency and reliability of anomaly handling; abnormal fluctuations can be automatically identified and trigger tiered reviews, reducing dispute costs; at the same time, audit logs ensure the traceability of the review process.

[0028] To promote a closed-loop skill enhancement system, the system automatically generates gap-filling indicators and training tasks, enabling competition results to be directly converted into training plans and re-competition feedback. The system optimizes the establishment of a mapping library of scoring items, capability tags, and training tasks: each scoring / deduction point is bound to a capability tag and a training resource ID; after scoring, capability tags are aggregated and sorted according to the percentage of deduction points or their frequency of occurrence, and the Top N tags are selected to generate a training list, recording the training list version number and the basis for its generation.

[0029] This invention is limited to internal employee training, skills competitions, and assessments within ENN Gas, and is not intended for public events. By limiting the applicable objects, data sources, work environments, rule sources, and result applications to internal enterprise technical scenarios, the boundaries of the technical solution are made clearer. Furthermore, by embedding time synchronization metadata in the evidence package and incorporating this metadata into the scoring report generation process, this invention enables different data sources to reconstruct events based on the same timeline during playback and review, avoiding misjudgments of event sequence caused by terminal clock drift or network jitter, thereby reducing review disputes and misjudgment rates. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the overall process of this invention.

[0031] Figure 2 This is a flowchart illustrating the calculation process for individual, organizational, and session scores in this invention.

[0032] Figure 3 This is a schematic diagram illustrating the capability layering, grouping control, and three-level audit execution of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This invention provides an intelligent interactive and automatic scoring management method and system for vocational skills competitions. Its core lies in the collaboration between the cloud or central server and multiple workstations of participants and judges to achieve automatic multi-source data collection, automated rule execution, automatic scoring, tiered review, and audit evidence storage during the competition process. This ultimately forms a closed-loop improvement cycle of competition, scoring, diagnosis, improvement, and re-competition. The following will elaborate on the entire process through multiple embodiments, from system deployment, pre-competition configuration, data collection, evidence package generation, automatic scoring, tiered review, to results publication.

[0035] To avoid ambiguity in mathematical symbols, English abbreviations, and function definitions used in this manual, the following unified explanations are provided:

[0036] Score raw : This represents the raw composite score output by the rule engine based on the scoring points, deduction points, and aggregation rules;

[0037] K: Represents the number of segments after dividing the competition process by time window or by step segment; K is an integer greater than or equal to 1.

[0038] s i : represents the score of the i-th sub-segment, where i = 1, 2, ..., K, and the scores of all sub-segments form the sequence S = {s1, s2, ..., s}. K};

[0039] disp(S): Represents the dispersion function, used to measure the degree of fluctuation of the sub-segment score sequence S. It is preferred to use the standard deviation sd(S) or the median absolute deviation MAD(S).

[0040] clip(x,a,b): This means clipping the variable x to the closed interval [a,b].

[0041] γ min : Indicates the lower bound of the stability penalty factor;

[0042] γ: Represents the stability correction factor, satisfying γ=clip(1-disp(S) / disp) ref ,γ min ,1);

[0043] Score stable : Represents the stability score, satisfying the Score stable =Score raw ×γ;

[0044] rank(): When there are tied scores, rank() uses a preset tie-ranking rule (e.g., taking the average or lowest rank of the tied scores) and writes the rule identifier into the scoring report metadata; when N=1, pctl is 1; when the sample set is empty, the benchmark score degenerates to be calculated based on the sample set of the same session or processed according to the preset default value, and the reason for the degeneration is recorded in the scoring report metadata.

[0045] N: Represents the number of samples participating in the sorting;

[0046] pctl: Represents the quantile value, satisfying pctl = rank(Score) stable ) / N;

[0047] Δ0: Represents the initial time deviation obtained after time synchronization at the start of the session;

[0048] ρ: Represents the clock drift rate during session execution;

[0049] t corr : Represents a unified timestamp corrected based on time synchronization metadata, satisfying t corr =t local -Δ0-ρ×(t local -t start );

[0050] disp ref : Represents the reference dispersion, used to normalize disp(S); disp ref It can be the median / mean of disp(S) for this competition item in the same sample set or a specified baseline version sample set, or a preset constant for the competition item, and its value source and version number are fixed in the scoring report metadata; in the preferred embodiment, disp ref The preferred sample is drawn from internal employees of ENN Energy who are in the same position, compete in the same event, in the same round, and / or have the same qualification level.

[0051] w raw w stable w bench : Indicates the use of Score fusion raw Score stable Score bench The weighting coefficients satisfy w raw +w stable +w bench =1;

[0052] Example 1: System Composition and Deployment

[0053] The intelligent interactive and automatic scoring management system for vocational skills competitions of the present invention can be deployed as a distributed architecture consisting of a cloud or central server, multiple workstations for participants, and judges; in a specific deployment embodiment, the system includes the following modules:

[0054] The event management server is used for event configuration, round / workstation arrangement, participant and referee account management, scoring rule management, results release and appeal archiving; and interacts with ENN Gas's internal human resources system, training system and identity authentication system to verify employee participation permissions.

[0055] Competition client: Deployed on PCs / industrial control computers / tablets at each workstation, used to establish a controlled competition environment, display tasks and interactive interfaces, collect process data and report it; the workstations are preferably internal competition workstations, training workstations and / or controlled industrial control workstations of ENN Energy.

[0056] The referee's client application is used to view automatic scoring results, replay evidence, conduct reviews, and confirm final judgments.

[0057] Data acquisition and alignment services: used to access multi-source data and perform event processing, time alignment, quality verification, and evidence package generation;

[0058] Rule compilation and scoring engine service: used to convert scoring criteria into executable rules and perform automatic scoring, outputting sub-scores and evidence of deduction points;

[0059] Evidence storage and auditing services: used to bind, store, and audit evidence, rule versions, scoring results, and review and modification chains;

[0060] Kanban and push notification services: Used to display competition progress, rankings and anomaly alerts in real time, and push notification information to organizers / referees (e.g., mobile push, IM push, etc.).

[0061] This embodiment modularizes competition management, data collection, rule enforcement, review and auditing, and result display, ensuring clear boundaries of responsibility for different functional modules while maintaining mutual collaboration. This architecture is beneficial for adapting to multi-workstation concurrent competition scenarios and for scaling server resources module by module as the number of competitions or participants increases, thereby improving the system's concurrent processing capabilities, deployment flexibility, and operational controllability. Furthermore, the clear data links between the participants, referees, and central services reduce layers of forwarding and redundant verification in information transmission, improving the efficiency of competition operations.

[0062] Example 2: Key Data Objects and Field Conventions

[0063] To achieve data interoperability and semantic consistency among the various modules of the system, this invention preferably adopts the following unified data object definition:

[0064] A session is a complete process of a participant engaging in a competition at a specific workstation. It includes at least the following: competition ID, round ID, workstation ID, participant ID, start time, end time, and rule version number. Preferably, it also includes the employee ID, company, job category, job qualification level, workstation equipment identifier, and job standard version number.

[0065] Event: The smallest recording unit in the competition process, uniformly adopting an event-based structure, which includes at least: Event ID, Session ID, Timestamp, Source Type (UI Interaction / Simulation Platform / Sensor / File / Video Summary), Event Type, Event Parameters, and Serial Number; among them, the Source Type preferably further includes identity verification events, environmental integrity verification events, and job simulation events;

[0066] EvidenceBundle: A collection of evidence aggregated by session, which includes at least: event sequence fragment index, key file digest, key window data digest, root hash, rule version number, and signature information;

[0067] ScoreReport: The output of the scoring engine, which includes at least: sub-scores, total score, list of deductions, evidence references for each deduction, rule version number, generation time, and signature information.

[0068] Furthermore, in this embodiment, the objects are associated with each other through session ID, rule version number, event ID and root hash, so that the competition behavior, scoring results, review records and training tasks of the same participant in the same session can be indexed and tracked in the same data link.

[0069] This embodiment, by adopting a unified object model, avoids data ambiguity caused by different modules using different field names and semantic definitions, thus establishing stable relationships between personnel, workstations, competition items, steps, time, evidence, and scores during the competition. This not only facilitates the scoring engine's automatic parsing and calculation of multi-source data but also allows judges to quickly locate the corresponding operation process by event ID, session ID, or evidence reference, thereby improving data organization efficiency, scoring accuracy, and post-competition review efficiency.

[0070] Example 3: Pre-competition configuration and event arrangement

[0071] This embodiment describes in detail the system preparation and configuration work before the start of the competition.

[0072] Competition configuration: The competition management server enters the competition information, including: competition topic / task package, competition duration, list of allowed devices, data collection strategy (whether video summarization is enabled, whether sensors are connected), and referee roles and permissions; the competition information is limited to competition information for internal employees of ENN Gas, and preferably corresponds to positions such as inspection, safety inspection, pressure regulation, emergency repair, customer service, and transmission and distribution operation;

[0073] Workstation resource orchestration: Generates the binding relationships between participants, workstations, equipment, and judges for each round, and generates a task distribution plan;

[0074] Data integration: If the existing system already contains registration information, identity information, equipment ledger, simulation platform account, etc., the event management server can automatically retrieve these data through interface integration, reducing errors caused by manual data entry.

[0075] Scoring rule version generation: Generate a rule version number (RuleVersion) for each competition item and calculate the rule summary (summary hash value), which serves as the basis for binding the subsequent evidence package and scoring report.

[0076] This embodiment pre-binds the competition items, workstations, personnel, equipment, rules, and data collection strategies before the competition begins. This allows each session to automatically inherit the corresponding configuration upon startup, reducing the workload of on-site configuration, manual verbal confirmation, and manual rule switching. In particular, by pre-fixing the rule versions and rule summaries, it effectively avoids the problem of mistakenly loading different scoring rules for the same competition item at different workstations and in different rounds, reducing scoring disputes caused by rule inconsistencies, and improving the standardization of pre-competition preparation, the smoothness of the competition process, and the consistency of execution during the competition.

[0077] Example 4: Mechanized Representation, Verification, and Compilation of Scoring Rules

[0078] To avoid merely remaining at the level of competition scoring sheets / rule descriptions, this invention converts the scoring criteria into a machine-executable rule model, preferably including:

[0079] Rule model structure: Step state machine: defines the start condition, end condition, and allowed duration of the step; Scoring points: trigger conditions, scores, and upper limits; Deduction points: violation conditions (number of violations / duration / threshold exceeding / order error, etc.) plus deduction value and upper limit; Aggregation rules: sub-item weights, total score cap, and tie-breaking auxiliary fields; wherein, the trigger conditions and violation conditions preferably correspond to the internal job operation specifications, safety operation procedures, service standards, and competition step requirements of ENN Gas;

[0080] Static validation: Validate the rules before publishing, including at least: field validity, reference validity (each condition can be mapped to collectable events / window statistics), dependency acyclicity, and the reasonableness of capping / weighting;

[0081] Compilation and Release: The rule model is compiled into an executable graph / bytecode that the scoring engine can execute, and the compiled product is signed to form a combination of rule version number, rule summary, compilation summary and signature, which is used by the participants and the scoring engine for verification.

[0082] This embodiment transforms the scoring criteria from natural language rules into programmable rules, enabling the same rule to be repeatedly executed according to a unified logic in different sessions, work positions, and with different referees.

[0083] Example 5: Controlled Environment and Intelligent Interactive Data Acquisition at the Participant's End

[0084] This embodiment describes in detail the process of environmental preparation and data collection at the start of the competition for the participating end.

[0085] 1. Controlled Environment Establishment: At the start of the session, the participating client creates a controlled environment (e.g., one or more of the following: container, virtual machine, whitelisted process, restricted file directory). The controlled environment must at least: allow only the running of the competition client and programs required for the competition; restrict external network access, allowing only access to the whitelisted addresses of the competition management server; monitor and perform summary calculations on output files in key directories; preferably, further allow: only the running of job simulation programs, training programs, and result acquisition programs corresponding to the internal job competitions of ENN Gas; only allow access to the competition management server, identity authentication server, training server, and competition resource server within ENN Gas's internal private network; detect USB peripheral access, remote control processes, screen sharing processes, non-whitelisted browsers, and unauthorized communication programs.

[0086] 2. Task Package Loading: The participating client retrieves the task package (containing competition instructions, rule version number, rule summary, and data collection strategy) from the competition management server. If the summary verification fails, the competition will be refused and an alarm will be triggered.

[0087] 3. Intelligent Interactive Data Collection: The participating platform provides an interactive interface that supports at least the following interactive actions: start / submit / request prompt / confirmation steps; each interaction generates a UI event and records the timestamp, sequence number, and parameters (e.g., button ID, input text length, number of prompts).

[0088] 4. Process Data Acquisition: Participants will generate events from one or more of the following sources according to the acquisition strategy: simulation platform events (step completion, parameter setting, fault location, etc.); workstation equipment / tool ​​sensor data (torque, pressure, position, etc., converted into threshold events or window statistics events); result file events (submitted files, generated reports, output product parameters); video evidence summary events; in the internal employee competition scenario of ENN Gas, the simulation platform events preferably include one or more of the following: gas facility inspection operation events, safety inspection judgment events, pressure regulation parameter setting events, emergency repair process events, and customer service handling events.

[0089] This embodiment establishes a controlled environment at the competition site. On the one hand, it restricts the execution of unauthorized programs and access to non-whitelisted networks, reducing environmental uncertainties caused by external interference, irrelevant software, or unauthorized access during the competition. On the other hand, by collecting interface interactions, simulation operations, sensor signals, and result files with fine granularity, the competition process, which would otherwise be difficult to record completely, can be structured, time-based, and computationally preserved. This is beneficial for the scoring engine to automatically determine scoring and deduction conditions based on the actual competition process, and also facilitates the review of specific interactive actions, operation sequences, and output results in the event of disputes, improving the completeness of process records and the accuracy of evidence acquisition.

[0090] Example 6: Multi-source data alignment, quality verification, and evidence package generation

[0091] This embodiment describes in detail how the data acquisition service processes multi-source data and generates tamper-proof evidence packages.

[0092] 1. Event Reporting and Buffering: The participating end uses a local queue with breakpoint resume to report the event sequence; each event is accompanied by a sequence number, and the server confirms the event based on the sequence number.

[0093] 2. Time alignment: Time synchronization is performed at the start of the session to obtain the deviation Δ0; during operation, the drift rate ρ is estimated based on the heartbeat and corrected; time synchronization metadata is written into the evidence package;

[0094] In one embodiment, the time synchronization metadata includes at least: initial deviation Δ0, clock drift rate ρ, number of synchronization samples M, and minimum round-trip time (RTT). min The variance of the bias sample, Var(Δ), and the confidence level, conf; where conf can be based on RTT. min The time alignment is calculated using a preset monotonic function with Var(Δ) and used to determine the reliability of time alignment during quality verification. When conf is below the threshold, the system marks the session as having a time synchronization anomaly and triggers a review or lowers the credibility level of the evidence.

[0095] 3. Quality verification: Check for gaps in the event sequence, missing key events, whether the source is on the whitelist, and whether the submitted file is bound to the session;

[0096] Evidence gap rate r gap Calculate as follows: Let the minimum sequence number of the same session be seq. min The maximum sequence number is seq max The actual number of events received is n. recv Then r gap =1 − n recv / (seq max -seq min +1). When r gap If the threshold is exceeded or there are missing events in the set of critical steps, the session will be transferred to an isolation queue and the generation of a final review report will be prohibited until the missing events are completed through resuming interrupted transmission or the reason for the missing events is confirmed and the reason code is recorded by the judge.

[0097] 4. Evidence Bundle Generation: Calculate the bundle root (hash chain or Merkle root) for the event sequence and key file / video digests, and form a SIG. c lient / sig s double signature on the server; the scoring report uses the bundle root and rule version number ID to bind evidence to the rule;

[0098] In one embodiment, to ensure that different implementations calculate consistent payload hashes for the same event payload, the event payload is normalized and serialized before hash calculation: JSON object keys are sorted lexicographically; UTF-8 encoding is uniformly used; irrelevant whitespace characters between object keys and values, and between array elements, are removed; numerical values ​​are represented in decimal notation (scientific notation and trailing zeros are prohibited); and time fields are uniformly formatted as ISO-8601. The normalized byte sequence is then input into a preset hash algorithm to obtain the payload hash.

[0099] In one embodiment, the evidence package includes at least: session ID, workstation ID, participant ID, rule version number, time synchronization metadata (Δ0, ρ, confidence level), event list, artifact hash, root hash, client signature, and server signature. The root hash is generated using a hash chain or Merkle tree: h is calculated for each event. i =H(event i ), iteratively calculate in pairs until the root is obtained; or calculate the chained summary h sequentially. i =H(h {i-1} ||event i The participating client generates a client signature by signing the root hash of the packet, and the server verifies the signature and then signs it to generate a server signature, thus forming a double signature.

[0100] In one embodiment, the initial time deviation Δ0 is defined as the difference between the server time and the participant's local time, and the clock drift rate ρ represents the linear drift coefficient of the participant's local clock relative to the server clock (in milliseconds per millisecond). start This indicates the local clock start timestamp recorded by the participating terminal when the session started; in another implementation, the event object only carries t. local The data collection and alignment service calculates t based on Δ0 and ρ. corr And it is written into the evidence package and scoring report, and the t reported by the participant. corr (If it exists) It is only used for consistency verification and recording deviation statistics to avoid timestamp conflicts.

[0101] In one embodiment, the event list adopts an append-only sequential log structure. Each event includes at least: event ID, session ID, sequence number, local timestamp, source type, event type, and payload hash; the payload is calculated using JSON serialization. When receiving events, the server deduplicates them using the sequence number and session ID as idempotent keys. Upon detecting a sequence number gap, the event is placed in an isolation queue, a gap alarm is generated, and the generation of a final review report is prohibited until the gap is filled through breakpoint resuming or the reason for the missing information is confirmed by the judge.

[0102] Example 7: Automatic Scoring Execution and Interpretive Output

[0103] This embodiment describes in detail how the scoring engine loads rules, executes scoring, and outputs an interpretable scoring report.

[0104] 1. Loading rules: Load and execute graphs / bytecode according to rule version numbers and verify signatures;

[0105] 2. Execute scoring: The scoring engine reads the evidence package and advances the step state according to the step state machine; it judges the trigger conditions for each scoring point / deduction point: for frequency type / sequence type conditions, it is based on event sequence statistics; for duration / threshold over-limit conditions, it is based on window statistical events or window aggregation of the original samples; for submitted file / result parameter conditions, it is judged based on file summary and parsing results.

[0106] 3. Explanatory Output: For each deduction point, output evidence references (list of event IDs, window start and end, file digest hash value);

[0107] 4. Stability Score and Benchmark Score (Implementation Algorithm): Divide the process events into K segments by window or step segment to obtain the sub-segment score sequence S; calculate the dispersion disp, and take the reference dispersion disp. ref Get Score stable =Score raw ×clip(1-disp / disp ref ,γ min ,1); Calculate the quantile pctl based on the distribution of the same session or baseline version to obtain the score. bench =100×pctl; This will change the value of disp and disp. ref γ min The quantile algorithm and the version number of the benchmark distribution are written into the metadata of the scoring report to ensure that the review can be reproduced.

[0108] In one embodiment, the scoring report outputs at least: the trigger condition ID, trigger evidence reference (event ID set / window start and end / file hash), trigger time range, and rule version number for each deduction point; and for subjective item scoring, it outputs the person who entered the data, the time, the reason, and the evidence reference.

[0109] Furthermore, the session scores or team scores can be generated based on individual comprehensive scores according to preset aggregation rules. The aggregation rules may include average, weighted average, extreme value removal average, or weighted aggregation method with different weights set according to job roles.

[0110] This embodiment employs targeted calculation methods for different types of competition data, enabling scoring to move beyond a single data source or dimension and simultaneously consider the operational process, result quality, completion timeliness, and behavioral stability. In particular, by outputting a corresponding event ID, window range, or file hash for each deduction point, judges can directly pinpoint the specific basis for the deduction, reducing the workload of repeatedly reviewing original materials. Furthermore, by introducing stability scores and benchmarking scores, the continuity and relative position of the participants' performance can be further reflected without altering the original scores, helping to suppress the impact of occasional errors on the overall score and improving comparability and fairness between the same and different competitions.

[0111] In one embodiment, the participating end uses an append-only sequential log structure for the event sequence. After receiving the data using the session ID and sequence number as idempotent keys, the data collection and alignment service verifies the continuity of the digest and the sequence number. If gaps or discontinuities exist, the data is placed in an isolation queue, and the generation of a final review report is prohibited until the data is resumed and the missing data is filled in, or the judge confirms the cause of the missing data and records the cause code. After the session ends, the participating end signs the end digest or the set of segmented digests to generate a client signature. After the server verifies the client signature, it then signs the evidence packet root hash, rule version number, and time synchronization metadata digest to generate a server signature, thus forming a verifiable double-signature closed loop.

[0112] To ensure that the evidence cited in the scoring report can be independently verified by a third party, the system includes, or can generate on demand, a corresponding Merkle path proof for each deduction point when generating the scoring report. Specifically, starting from the leaf hash of the cited event, it provides a set of sibling node hashes at each level of the Merkle tree and indicates the left and right positions of the leaves at each level. The third party recalculates the leaf hash based on the session ID, sequence number, unified timestamp, source type, event type, and payload hash, and then iterates through the sibling node hashes to obtain the root hash, which is then compared with the root hash of the evidence package. Simultaneously, the client signature and server signature are verified separately. The signature input for the server signature includes at least the root hash of the evidence package, the scoring rule version number, and the time synchronization metadata digest, thereby completing the closed-loop verification of "evidence citation → evidence package → rule version → signature".

[0113] Example 8: Tiered Review and Audit Evidence Preservation

[0114] This embodiment describes in detail the hierarchical review mechanism and anomaly triggering strategy of automatic initial review, secondary review and final review.

[0115] 1. Automatic Initial Assessment: The scoring engine generates an initial assessment report and enters the queue for review;

[0116] 2. Judge's review: Judges can view the evidence cited for deductions and replay relevant evidence; if a score needs to be changed, they must fill in the reason for the change and select the corresponding evidence to be cited, thus creating a review record;

[0117] 3. Final review by the chief referee: After confirming the review results, the chief referee locks the score and generates a final review report.

[0118] 4. Modification Chain Audit: The evidence storage and auditing service links and stores the initial review report, re-review record, and final review report in chronological order to ensure post-competition traceability and verifiability. The tiered review mechanism is used to trigger re-reviews and final reviews for abnormal samples after automatic initial review, and to generate traceable audit records for each change. However, in this invention, it is implemented through the technical binding of competition evidence and rule versions.

[0119] To improve the efficiency of handling abnormal disputes, the system introduces a computable exception triggering strategy: if the participant's score quantile (pctl) is lower than p... low or higher than p high And at the same time, the evidence gap rate is greater than r. gap If any of the following conditions are met—missing critical steps, abnormal environmental integrity, or sudden changes across rounds—a re-evaluation will be automatically triggered; if the re-evaluation changes are significant, the score will be affected. review -Score init If the score exceeds the threshold ΔT or k×MAD (MAD is the median absolute deviation of the score difference), a final review is automatically triggered. The system generates a review work order for each trigger, which includes at least: root hash, rule version number, trigger reason code, evidence citation index, and threshold parameter version number. In a preferred embodiment, the environmental integrity anomaly also includes any one of the following: employee authentication anomaly, device fingerprint mismatch, unauthorized peripheral access, non-whitelist process startup, remote control detection anomaly, and screen sharing detection anomaly; the quantile PCTL is preferably calculated from an internal employee sample set of the same position, competition item, round, and / or qualification level.

[0120] In one embodiment, the exception triggering parameter set has an independent version number, and is written into the scoring report metadata along with the rule version number; wherein p low p high r gap The values ​​of ΔT and k are configurable, with default values ​​of 0.05, 0.95, 0.01, 5, and 3, respectively. The system embeds the parameter version number and the statistical sample range identifier used during triggering when generating a review work order, ensuring that the PCTL and MAD calculation results can be reproduced on the same sample set during review and post-event auditing.

[0121] Example 9: Results Release, Kanban Display, and Message Push

[0122] Once the final results are locked, the event management server will change the results status from "Pending Release" to "Released." Participants can log in to the query portal to view their scores, rankings, and scoring reports. Results can be released in batches or at scheduled times to meet the organizational needs of different events.

[0123] 1. Results Release: After the final review is completed, the competition management server will release the results, rankings, and report links, and participants can access these information according to their permissions.

[0124] 2. Real-time dashboard: The dashboard display should include at least: workstation progress (start / in progress / completed / abnormal), automatic scoring completion rate, review queue length, and current ranking (which can indicate the initial review / final review status).

[0125] 3. Push Strategy: Push notifications will be triggered when any of the following events occur: session disconnection or no valid events for an extended period; evidence gap (discontinuous event sequence numbers); abnormal environment integrity (detection of non-whitelisted processes / external network access); the difference between the review and initial evaluation exceeds the threshold.

[0126] The system establishes a scoring item, capability tag, and training task mapping library. Each scoring point / deduction point is bound to a capability tag and a training resource ID. After scoring, the capability tags are aggregated and sorted according to the percentage of deduction points or the frequency of occurrence. The Top N tags are selected to generate a training list, and the version number and generation basis of the training list are recorded, realizing the automatic conversion of competition results into training plans.

[0127] This embodiment automatically publishes, visualizes, and pushes abnormal messages after the scores are locked, enabling competition administrators to grasp the overall competition progress and risk status more promptly, reducing reliance on slow methods such as manual phone notifications and manual statistical table updates. Furthermore, by directly mapping deductions to ability tags and training resources, the system automatically transforms scores into problem identification and training arrangements. This allows participants to receive more than just a final score; they clearly understand the corresponding ability points for their weaknesses and subsequent improvement paths. This improves the efficiency of post-competition review and the relevance of training plans, truly achieving closed-loop management driven by competition results for ability improvement.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for intelligent interaction and automatic scoring management in vocational skills competitions, characterized in that: include: The competition management server generates a competition session configuration based on the competition information, round information, workstation information, participant information, referee information, data collection strategy information, and scoring rule version information, and sends the task package bound to the competition session configuration to the participant terminal. Before starting the competition session, the participating end verifies the version number and rule summary of the scoring rules in the task package. After the verification is successful, a controlled competition environment is established, and the interface interaction data, simulation platform operation data, device or tool sensor data, result file data and / or video summary data during the competition are collected according to the collection strategy. The collected data is converted into events and reported to form an event sequence, wherein each event includes at least a local timestamp, a corrected unified timestamp, and an event payload / event parameter. The data acquisition and alignment service receives the event sequence, corrects the event timestamps based on time synchronization metadata to unify the timestamps, and performs quality verification and session aggregation on the event sequence to generate an evidence package corresponding to the competition session. The evidence package includes at least an event index, a key data digest, a time synchronization metadata digest, an evidence package root hash, and client and server signatures. The evidence package root hash is calculated iteratively using a Merkle tree or hash chain based on the session identifier, sequence number, corrected unified timestamp, source type, and event type.

2. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 1, characterized in that: The participant signs the evidence root hash to generate a client signature; the event management server or data collection and alignment service verifies the client signature. After the verification is successful, the participant signs the evidence root hash, the scoring rule version number, and the time synchronization metadata digest to generate a server signature. The scoring engine reads the evidence package and the scoring rule version bound to it, applies the executable rules compiled from the scoring rules to the evidence package, and automatically calculates the step status, scoring points, deduction points, and aggregation rules to generate a scoring report. The scoring report includes at least the sub-item score, the original comprehensive score, the deduction point list, the evidence reference for each deduction point, the root hash of the evidence package, the scoring rule version number, the time-synchronized metadata digest, and the signature verification result, so as to realize the binding of the scoring result with the evidence package and rule version. The system calculates stability score and benchmarking score based on the scoring report to obtain individual comprehensive score, and performs anomaly judgment, re-evaluation, final review and audit evidence storage based on the individual comprehensive score, evidence status and environmental integrity status; After the scores are locked, the system generates a training list and publishes the scores based on the mapping relationship between the scoring items, ability tags, and training tasks.

3. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 1, characterized in that, The competition session configuration, events, evidence packages, and scoring reports are all represented using a unified data object. The competition session object must include at least the competition item ID, round ID, workstation ID, participant ID, start time, end time, and scoring rule version number; An event object must include at least the event ID, session ID, timestamp, source type, event type, event parameters, and sequence number; The evidence package object includes at least the event sequence fragment index, key file digest, key window data digest, time synchronization metadata digest, root hash, scoring rule version number, and signature information; The scoring report should include at least the sub-item score, total score, list of deductions, evidence cited for each deduction, scoring rule version number, generation time, and signature verification result.

4. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 1, characterized in that, The scoring rule version is generated in the following manner: The scoring criteria are converted into a machine-executable rule model, which includes a step state machine, scoring point rules, deduction point rules, and aggregation rules. Before release, the rule model is statically validated. The static validation includes at least field validity validation, reference validity validation, dependency acyclicity validation, and capping value and weight rationality validation. After the static verification is passed, the rule model is compiled into an executable graph or bytecode that the scoring engine can execute, and the compilation product is signed to form a rule release result containing the scoring rule version number, rule summary, compilation summary and signature information.

5. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 1, characterized in that, The establishment of a controlled competition environment by the participating end includes: By using one or more methods such as containers, virtual machines, whitelisted process control, and restricted file directories, the participating end can be restricted to only running the competition client and programs required for the competition. Restrict external network access for participating devices, allowing them to access only the whitelisted addresses of the event management server; Monitor and calculate summaries for output files in key directories; Furthermore, when loading the task package, the participating client verifies the scoring rule version number, rule summary, and collection strategy in the task package. If the verification fails, the competition session will be refused to start and an alarm will be sent to the competition management server.

6. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 1, characterized in that, The generation and timestamp correction of the time synchronization metadata include: At the start of the competition session, the participating terminal and the competition management server perform at least M round-trip time synchronizations, where M is an integer greater than or equal to 3. The time deviation corresponding to the time synchronization sample with the smallest round-trip delay is selected as the initial time deviation Δ0. The time synchronization process is preferably carried out in the internal private network environment of ENN Energy. During the competition session, the participating terminals send heartbeat time synchronization packets at preset heartbeat intervals. The system estimates the clock drift rate ρ based on multiple deviation samples through linear regression.

7. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 1, characterized in that, The root hash and signature information of the evidence package are generated in the following manner: The event payload is normalized and serialized before the payload hash is calculated. The normalization and serialization includes at least sorting of object key names, uniform encoding, removal of irrelevant whitespace characters, and normalization of numerical format. The leaf hash is calculated by using the session identifier, sequence number, corrected unified timestamp, source type, event type, and payload hash as the basic fields of the leaf node; The leaf hashes are arranged in ascending order of sequence number, and the root hash of the evidence packet is obtained by iterative calculation using a Merkle tree or hash chain method. The participating client signs the root hash of the evidence packet to form a client signature; After the server verifies the client's signature, it signs the evidence root hash, scoring rule version number, and time synchronization metadata digest to form a server signature. The scoring report uses the root hash of the evidence package, the version number of the scoring rule, the time synchronization metadata digest, and the signature verification result to bind the scoring result to the evidence package.

8. The intelligent interactive and automatic scoring management method for vocational skills competitions according to claim 2, characterized in that, The anomaly detection, review, final audit, and audit evidence preservation include: When a participant's score percentile is lower than a preset lower threshold or higher than a preset upper threshold, and simultaneously meets any of the following conditions: evidence gap rate exceeding a preset gap threshold, missing key steps, abnormal environmental integrity or sudden changes across rounds, unauthorized terminal access, external network access attempts, remote control process detection, cross-workstation login, or the number of retries for failed identity verification exceeding a threshold, the system automatically triggers a re-evaluation and generates a review work order. The review work order includes at least the evidence root hash, scoring rule version number, trigger reason code, evidence citation index, and threshold parameter version number. When the difference between the re-evaluation score and the initial evaluation score exceeds a preset score threshold, or exceeds a preset multiple of the median absolute deviation of the score difference, the system automatically triggers a final review. The evidence storage and auditing service links the initial evaluation report, re-evaluation record, and final review report in chronological order for evidence storage, forming a traceable modification chain.

9. An intelligent interactive and automatic scoring management system for vocational skills competitions, characterized in that, include: The event management server is used for event configuration, round and workstation arrangement, participant and referee account management, scoring rule version management, task package distribution, result release and appeal archiving. The competition client is used to establish a controlled competition environment, display competition tasks and interactive interfaces, collect competition process data, and generate event sequences. The judge's client is used to view automatic scoring results, retrieve and cite evidence, replay relevant evidence, execute review and final confirmation. The data acquisition and alignment service is used to receive the event sequence, perform timestamp correction, quality verification, and session aggregation on multi-source events, and generate evidence packages. The rule compilation and scoring engine service is used to convert scoring criteria into executable rules, perform automatic scoring based on the scoring rule version and evidence package, and output a scoring report containing evidence references to deduction points; Evidence storage and auditing services are used to bind and store evidence packages, scoring rule versions, scoring reports, review records, and final audit reports, and generate traceable audit records. The event management server, participant client, referee client, data collection and alignment service, rule compilation and scoring engine service, and evidence storage and auditing service establish data associations through session ID, event ID, scoring rule version number, and evidence packet root hash.

10. The system according to claim 9, characterized in that, It also includes dashboard and message push services, as well as a results application module; The dashboard and message push service are used to display workstation progress, automatic scoring completion rate, review queue length, ranking status and abnormal alarms, and push prompt information to the organizer or referee when a session disconnection occurs, there are no valid events for a long time, there is a gap in the event sequence, the environment integrity is abnormal, or the review modification exceeds the threshold. The results application module is used to maintain the mapping relationship between scoring items, ability tags and training tasks. After the score is locked, the ability tags are aggregated and sorted according to the score ratio or frequency of occurrence of the deduction points, the target ability tags are selected to generate a training list, and the version number and generation basis of the training list are recorded.

Images