A simulation machine maintenance quality evaluation method and system, an electronic device, and a storage medium

Abstract

The application discloses a kind of simulation machine maintenance quality evaluation method, system, electronic equipment and storage medium. Including establishing the standardized task library configured with evaluation definition parameter and calibration parameter;According to evaluation configuration and historical operation quality file extraction evaluation task set, generate evaluation task sequence;Collect evaluation process key data and generate evidence chain bag, carry out edit distance comparison to operation sequence and standard operation sequence, obtain behavior trajectory consistency score;Calculate task weight based on calibration parameter, combine scoring factor and abnormal behavior correction factor to calculate each evaluation task score and total score;According to task score and total score, generate multidimensional operation quality portrait vector, and fuse and update with historical operation quality file, output maintenance quality evaluation report and maintenance operation correction instruction or maintenance strategy adjustment parameter.The application can improve the standardization degree of simulation machine maintenance quality evaluation, improve maintenance quality and efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method, system, electronic device, and storage medium for assessing the maintenance quality of a simulator. Background Technology

[0002] Flight simulators play a core role in civil aviation training, and simulator maintenance personnel are responsible for ensuring stable equipment operation, training continuity, quality, and compliance. Currently, the evaluation of simulator maintenance personnel's operational quality relies heavily on internal training and experience-based judgment within each organization, lacking a market-oriented, cross-organizational, quantifiable, and auditable maintenance operation evaluation system.

[0003] Different simulator manufacturers exhibit variations in system architecture, interfaces, maintenance tools, log formats, and failure modes. Furthermore, differences in the corresponding civil aircraft models and configurations also affect the difficulty of the tasks, making direct comparisons of maintenance quality evaluations from different organizations impossible. The existing evaluation system also suffers from the following shortcomings:

[0004] The static benchmark has a drawback: once the existing evaluation benchmark is set, it becomes fixed for a long time and cannot be adaptively adjusted with the iteration of equipment technology, the upgrading of maintenance processes and the improvement of the overall level of operators. As a result, the scoring standards gradually become detached from reality and the discrimination decreases.

[0005] The data silo problem stems from the fact that each organization establishes its own assessment database, which is not interconnected. This makes it impossible to effectively calibrate benchmark parameters using industry-wide data and to identify industry-level operational quality distribution patterns.

[0006] The existing assessment has a timeliness deficiency. It only records the maintenance operation results at a certain point in time and does not track the natural decline of the maintenance personnel's operation quality over time. This results in a lack of scientific basis for setting the validity period of the assessment results, and there is a safety hazard that the assessment is valid but the actual operation ability has deteriorated significantly.

[0007] The assessment tasks are rigid and lack inherent defects. Existing assessment tasks are mainly designed manually, which makes it difficult to cover long-tail failure scenarios in real operation. Furthermore, they cannot adaptively adjust the difficulty based on the historical performance of the assessed individuals, resulting in insufficient assessment efficiency and discrimination.

[0008] Therefore, this application provides a method for assessing the maintenance quality of a simulator to solve the above-mentioned technical problems. Summary of the Invention

[0009] The purpose of this invention is to provide a method, system, electronic device, and storage medium for assessing the quality of simulator maintenance operations, in order to solve the technical problems in the prior art where the assessment results of simulator maintenance operation quality are incomparable due to differences in manufacturers, models, and configurations; the assessment benchmark is static and cannot be updated adaptively; data silos between institutions cannot be calibrated across institutions; the assessment results lack timely tracking of operational quality; and the assessment tasks are fixed and cannot be adjusted adaptively.

[0010] To address the aforementioned technical problems, this invention provides a method for assessing the maintenance quality of a simulator, comprising:

[0011] Establish a standardized task library to store multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters. The evaluation definition parameters include at least a standard operation sequence.

[0012] Determine the assessment configuration, obtain the historical operation quality file of the person being assessed, and extract the assessment task set from the standardized task library based on the assessment configuration and the historical operation quality file to generate an assessment task sequence;

[0013] Execute and collect key data during the evaluation process, generate an evidence chain package, compare the edit distance of the operation sequence in the evidence chain package with the standard operation sequence, and calculate the behavioral trajectory consistency score.

[0014] Equivalence calibration and quantitative scoring are performed. The task weight is calculated based on the calibration parameters. Multiple scoring factors are extracted based on the evidence chain package. The abnormal behavior correction factor is updated in combination with the behavior trajectory consistency score. The score of each evaluation task is calculated based on the task weight, the scoring factors and the updated abnormal behavior correction factor. The total score of this evaluation is obtained by summing them up.

[0015] Generate and update the operation quality profile. Based on the score of each evaluation task and the total score of this evaluation, generate a multi-dimensional operation quality profile vector. Then, merge the multi-dimensional operation quality profile vector with the historical operation quality file using an exponentially weighted moving average method to update the historical operation quality file. At the same time, output a maintenance quality evaluation report and generate maintenance operation correction instructions or maintenance strategy adjustment parameters based on the evaluation report.

[0016] In some specific embodiments, a standardized task library is established to store multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters. The evaluation definition parameters at least include a standard operating sequence, and further include:

[0017] For different manufacturers, models, positions and special domains, a set of predefined assessment tasks is set up, and assessment definition parameters are configured for each assessment task. The assessment definition parameters include at least the task objective, quantitative indicator threshold, evidence chain requirements, reference time, basic weight, and standard operation sequence.

[0018] For each assessment task, calibration parameters are configured, including equivalent calibration parameters and project response theory parameters. The equivalent calibration parameters include system complexity coefficient, task impact coefficient, and environmental factor coefficient. The project response theory parameters include at least task discrimination parameter, difficulty parameter, and guessing parameter.

[0019] In some specific embodiments, determining the evaluation configuration, obtaining the historical operational quality profile of the person being evaluated, and extracting an evaluation task set from the standardized task library based on the evaluation configuration and the historical operational quality profile to generate an evaluation task sequence, further includes:

[0020] Obtain the assessment level, specific domain, job type, manufacturer, and model parameters as the assessment configuration for this assessment;

[0021] Obtain the historical operational quality file of the person being evaluated. If the historical operational quality file exists, extract the currently determined operational proficiency level value of the person being evaluated from it. Then, using a computerized adaptive testing algorithm, based on the project response theory parameters, dynamically select the combination of evaluation tasks with the largest project information content under the currently determined operational proficiency level value from the standardized task library, and satisfy the special domain coverage constraint to form an evaluation task set, and generate an evaluation task sequence and scoring configuration.

[0022] If the historical operation quality file does not exist, evaluation tasks are randomly selected from the standardized task library according to the evaluation configuration to form an evaluation task set, and an evaluation task sequence and scoring configuration are generated.

[0023] In some specific embodiments, key data during the evaluation process is executed and collected to generate an evidence chain package. The operation sequence in the evidence chain package is compared with the standard operation sequence by edit distance, and a behavioral trajectory consistency score is calculated. Further steps include:

[0024] During the evaluation process, data is collected from simulators, maintenance terminals, and measuring instruments. The data includes at least operation logs, alarm records, configuration change records, key parameter snapshots, operation sequences, command input timing, and environmental context information.

[0025] The collected key data is bound with timestamps, device identifiers, and workstation identifiers, and then hashed and digitally signed to generate an evidence chain package.

[0026] Extract a snapshot of the behavior trajectory of the person being evaluated from the evidence chain package, use the operation sequence in the behavior trajectory snapshot as the operation sequence in the evidence chain package, compare the edit distance with the standard operation sequence of the corresponding evaluation task, calculate the normalized similarity score, and use it as the behavior trajectory consistency score.

[0027] The data in the evidence chain package is subjected to consistency verification. If the verification fails, the basic anomaly correction factor is set to zero or a predetermined correction value.

[0028] In some specific embodiments, equivalence calibration and quantitative scoring are performed. Task weights are calculated based on the calibration parameters, and multiple dimensions of scoring factors are extracted based on the evidence chain package. Abnormal behavior correction factors are updated in conjunction with the behavioral trajectory consistency score. The score for each evaluation task is calculated based on the task weights, the scoring factors, and the updated abnormal behavior correction factors. These scores are then accumulated to obtain the total score for this evaluation. Further steps include:

[0029] Obtain the basic weight, system complexity coefficient, task impact coefficient, and environmental factor coefficient of the evaluation task. Multiply the basic weight by the system complexity coefficient, task impact coefficient, and environmental factor coefficient to obtain the calibrated task weight.

[0030] The quality score, process score, timeliness score, and evidence completeness score of each evaluation task are extracted and calculated from the evidence chain package as the scoring factor, wherein the timeliness score is determined based on the ratio of the actual time spent on the task to the reference time.

[0031] The basic anomaly correction factor is corrected based on the behavioral trajectory consistency score to obtain the final abnormal behavior correction factor. The correction method is to multiply the basic anomaly correction factor by a coefficient determined by the behavioral trajectory consistency score.

[0032] The task weights are multiplied by the weighted sum of the quality score, process score, timeliness score, and evidence completeness score, and then multiplied by the final abnormal behavior correction factor to obtain the score for each assessment task. The scores of all tasks in this assessment are added together, and the reliability contribution bonus score is added, while the risk event deduction items are subtracted to obtain the total score for this assessment.

[0033] In some specific embodiments, an operational quality profile is generated and updated. Based on the score of each evaluation task and the total score of this evaluation, a multi-dimensional operational quality profile vector is generated. This multi-dimensional operational quality profile vector is then fused with the historical operational quality archive using an exponentially weighted moving average method to update the historical operational quality archive. Simultaneously, a maintenance quality evaluation report is output, and maintenance operation correction instructions or maintenance strategy adjustment parameters are generated based on the evaluation report. Further steps include:

[0034] Based on the score of each assessment task, it is mapped to multiple predefined quality dimensions to generate a multi-dimensional operational quality profile vector for this assessment. The quality dimensions include at least safety and compliance, fault diagnosis, tools and measurement, and system understanding and integration.

[0035] Obtain the historical operational quality file of the person being evaluated, and use an exponentially weighted moving average method to fuse the multi-dimensional operational quality profile vector of this evaluation with the values ​​of the corresponding dimensions in the historical operational quality file to obtain the updated historical operational quality file, and record it on a rolling basis to form a quality time series file.

[0036] Based on the total score of this assessment and the updated historical operation quality file, the operation quality level of the assessed person is determined, and according to the specific domain, manufacturer and model information covered by the assessment task, the corresponding specific compatibility evaluation and platform model endorsement are generated to form a maintenance quality assessment report.

[0037] Output the total score and multi-dimensional operation quality profile vector of this evaluation, and input the evaluation results into the operation quality degradation prediction model. At the same time, based on the behavior trajectory consistency score and abnormal behavior correction factor in the evaluation report, generate maintenance operation correction instructions or maintenance strategy adjustment parameters.

[0038] In some specific embodiments, a dynamic benchmark adaptation step is also included:

[0039] Periodically collect all evaluation data within a preset period;

[0040] Using a statistical analysis algorithm, a weighted quantile value is calculated based on the completion time distribution of each assessment task in the full assessment data, which serves as a new reference time.

[0041] Using a smoothing coefficient, the new reference time is weighted and averaged with the historical reference time to obtain the revised reference time, and the quantitative indicator thresholds and weight coefficients are automatically revised periodically in sync.

[0042] The revised reference time, quantitative indicator thresholds, and weight coefficients will be reviewed and updated to the standardized task library, and the version history will be recorded.

[0043] Based on the same concept, the present invention also provides a simulator maintenance quality assessment system, comprising:

[0044] The standardized task library creation module is configured to create a standardized task library that stores multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters, and the evaluation definition parameters include at least a standard operation sequence.

[0045] The assessment configuration determination module is configured to determine the assessment configuration, obtain the historical operation quality file of the person being assessed, and extract an assessment task set from the standardized task library based on the assessment configuration and the historical operation quality file to generate an assessment task sequence.

[0046] The behavioral trajectory consistency score calculation module is configured to execute and collect key data during the evaluation process, generate an evidence chain package, compare the edit distance of the operation sequence in the evidence chain package with the standard operation sequence, and calculate the behavioral trajectory consistency score.

[0047] The evaluation score calculation module is configured to perform equivalence calibration and quantitative scoring. It calculates task weights based on the calibration parameters, extracts scoring factors from multiple dimensions based on the evidence chain package, updates abnormal behavior correction factors in conjunction with the behavior trajectory consistency score, calculates the score of each evaluation task based on the task weights, the scoring factors, and the updated abnormal behavior correction factors, and accumulates them to obtain the total score of this evaluation.

[0048] The historical operation quality file update and report output module is configured to generate and update operation quality profiles. Based on the score of each evaluation task and the total score of this evaluation, a multi-dimensional operation quality profile vector is generated. The multi-dimensional operation quality profile vector is then merged with the historical operation quality file using an exponentially weighted moving average method to update the historical operation quality file. Simultaneously, a maintenance quality evaluation report is output, and maintenance operation correction instructions or maintenance strategy adjustment parameters are generated based on the evaluation report.

[0049] Based on the same concept, the present invention also provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of a simulator maintenance quality assessment method.

[0050] Based on the same concept, the present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a simulator maintenance quality assessment method.

[0051] Compared with existing technologies, its advantages are as follows:

[0052] This invention discloses a method, system, electronic device, and storage medium for assessing the maintenance quality of simulators. By using a task equivalence calibration model, the basic weights are multiplied by the system complexity coefficient, task impact coefficient, and environmental factor coefficient to normalize and calibrate the task weights on devices from different manufacturers and with different configurations, so that the scoring results can be compared horizontally.

[0053] Reduce human subjectivity: By automatically collecting logs and measurement data, an evidence chain package containing timestamps, hash check values ​​and digital signatures is generated. Combined with behavior trajectory consistency comparison and abnormal behavior detection, the proportion of subjective judgment is reduced and auditable traceability is achieved.

[0054] Improve industry efficiency: The assessment results are output in the form of a multi-dimensional operational quality profile vector, and the historical operational quality files are updated on a rolling basis using an exponentially weighted moving average method, which can be directly used for job matching and operational quality migration.

[0055] Benchmark Adaptability: Through a dynamic benchmark adaptation mechanism, full evaluation data is collected periodically, and a statistical analysis algorithm is used to calculate the weighted quantile value as a new reference time. After being weighted by a smoothing coefficient, the value is updated to the task library, ensuring that the scoring system remains timely as the industry's technical level evolves.

[0056] Cross-agency calibration with data privacy protection: Through a federalized cross-agency calibration protocol, each participating agency uploads only statistical summaries instead of raw data locally, achieving globally consistent benchmark calibration.

[0057] Operational quality timeliness tracking and safety early warning: Through the operational quality decay prediction model, the operational quality status of the assessed personnel is dynamically estimated based on the exponential decay function and the activity correction factor. Within the assessment validity period, the risk of operational quality decay is proactively identified and a retraining early warning is triggered.

[0058] Improved efficiency of adaptive assessment: By using a computerized adaptive testing algorithm based on the theoretical parameters of project response and the information content function of project, the combination of assessment tasks with the largest information content is dynamically selected, thereby improving assessment efficiency and the accuracy of distinguishing the quality of boundary operations while ensuring assessment coverage.

[0059] Multi-layered anomaly detection and anti-fraud: Integrating task-level data consistency detection, behavior trajectory snapshot comparison based on edit distance, cross-session individual anomaly detection and batch-level group anomaly detection, a multi-layered fairness assurance system is constructed. Attached Figure Description

[0060] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0061] Figure 1This is a flowchart illustrating some specific embodiments of the simulator maintenance quality assessment method of the present invention;

[0062] Figure 2 This is a schematic diagram of the structure of a simulator maintenance quality assessment system according to some specific embodiments of the present invention;

[0063] Figure 3 This is a schematic diagram of the structure of an electronic device according to some specific embodiments of the present invention;

[0064] In the diagram, 710 is the processor; 720 is the memory; 730 is the input device; and 740 is the output device. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0067] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0068] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0069] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0071] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0072] Reference Figure 1 A method for assessing the maintenance quality of a simulator, comprising:

[0073] S101, Establish a standardized task library to store multiple evaluation tasks, wherein each evaluation task is configured with at least evaluation definition parameters and calibration parameters, and the evaluation definition parameters include at least a standard operation sequence;

[0074] S102, determine the evaluation configuration, obtain the historical operation quality file of the person being evaluated, and extract the evaluation task set from the standardized task library according to the evaluation configuration and the historical operation quality file to generate the evaluation task sequence;

[0075] S103, execute and collect key data during the evaluation process, generate an evidence chain package, compare the edit distance of the operation sequence in the evidence chain package with the standard operation sequence, and calculate the behavior trajectory consistency score.

[0076] S104, perform equivalence calibration and quantitative scoring, calculate task weights based on the calibration parameters, extract scoring factors of multiple dimensions based on the evidence chain package, update abnormal behavior correction factors in combination with the behavior trajectory consistency score, calculate the score of each evaluation task based on the task weights, the scoring factors and the updated abnormal behavior correction factors, and sum them up to obtain the total score of this evaluation.

[0077] S105, Generate and update the operation quality profile. Based on the score of each evaluation task and the total score of this evaluation, generate a multi-dimensional operation quality profile vector. Then, merge the multi-dimensional operation quality profile vector with the historical operation quality file using an exponentially weighted moving average method to update the historical operation quality file. At the same time, output a maintenance quality evaluation report and generate maintenance operation correction instructions or maintenance strategy adjustment parameters based on the evaluation report.

[0078] Specifically, in this embodiment of the invention, a standardized task library is established. This task library stores multiple assessment tasks in database form. Each assessment task is configured with at least assessment definition parameters and calibration parameters. The assessment definition parameters include task objectives, quantitative indicator thresholds, evidence chain requirements, reference time, basic weights, and standard operation sequences. The calibration parameters include equivalent calibration parameters and project response theory parameters. The equivalent calibration parameters specifically include system complexity coefficients, task impact coefficients, and environmental factor coefficients. The project response theory parameters specifically include task discrimination parameters, difficulty parameters, and guessing parameters. When determining the assessment configuration, the assessment level, special domain, job type, manufacturer, and model parameters for this assessment are obtained, and the historical operation quality file of the assessed person is obtained. If the historical operation quality file exists, the currently determined operation proficiency level value of the assessed person is extracted from it; otherwise, tasks are directly extracted according to the assessment configuration. During the task extraction phase, if the person being evaluated has a currently defined operational proficiency level, a computerized adaptive testing algorithm is used. Based on the project response theory parameters, it dynamically selects the combination of assessment tasks with the largest project information content at the current operational proficiency level from the standardized task library, while satisfying the special domain coverage constraint, to form an assessment task set, and then generates an assessment task sequence and scoring configuration. If the person being evaluated does not have a historical operational quality file, assessment tasks are randomly extracted from the standardized task library according to the assessment configuration, forming an assessment task set, and then generating an assessment task sequence and scoring configuration. During the assessment, the person being evaluated operates item by item according to the assessment task sequence in the assessment environment. Key data from the simulator, maintenance terminal, and measuring instruments are automatically collected through the data acquisition interface, including at least the operation log, alarm record, configuration change record, key parameter snapshot, operation sequence, command input timing, and environmental context information. The collected key data is bound with timestamps, equipment identifiers, and workstation identifiers, and hash calculations and digital signatures are performed to package and generate an evidence chain package. Extract a snapshot of the behavior trajectory of the person being evaluated from the evidence chain package. Compare the operation sequence in the behavior trajectory snapshot with the standard operation sequence of the corresponding evaluation task by editing distance. Calculate the normalized similarity score as the behavior trajectory consistency score. At the same time, perform consistency verification on the data in the evidence chain package. If the verification fails, set the basic anomaly correction factor to zero or a predetermined correction value.In the equivalence calibration and quantitative scoring stage, the basic weight, system complexity coefficient, task impact coefficient, and environmental factor coefficient of the assessment task are obtained. The basic weight is multiplied by the system complexity coefficient, task impact coefficient, and environmental factor coefficient to obtain the calibrated task weight. The quality score, process score, timeliness score, and evidence completeness score of each assessment task are extracted and calculated from the evidence chain package as scoring factors for multiple dimensions. The timeliness score is determined based on the ratio of the actual time consumed by the task to the reference time. Specifically, a limiting function is used to restrict the complement of the ratio of the actual time consumed to the reference time to the range of zero to one. The basic anomaly correction factor is corrected based on the behavioral trajectory consistency score. The basic anomaly correction factor is multiplied by the coefficient determined by the behavioral trajectory consistency score to obtain the final anomaly behavior correction factor. The calibrated task weight is multiplied by the weighted sum of the quality score, process score, timeliness score, and evidence completeness score, and then multiplied by the final anomaly behavior correction factor to obtain the score of each assessment task. The scores of all assessment tasks in this assessment are added together, and the reliability contribution reward score is added. The risk event deduction item is subtracted to obtain the total score of this assessment. In the operation quality profile generation and updating phase, the score for each assessment task is mapped to multiple predefined quality dimensions, including at least safety and compliance, fault diagnosis, tools and measurement, and system understanding and integration, generating a multi-dimensional operation quality profile vector for this assessment. The historical operation quality files of the assessed entity are obtained, and an exponentially weighted moving average method is used to fuse the multi-dimensional operation quality profile vector of this assessment with the corresponding dimension values ​​from the historical operation quality files, resulting in an updated historical operation quality file. This updated file is then continuously updated to form a quality time-series archive. Based on the total score of this assessment and the updated historical operation quality files, the operation quality level of the assessed entity is determined. Furthermore, based on the specific domains, manufacturers, and models covered by the assessment tasks, corresponding specific compatibility evaluations and platform / model endorsements are generated, forming a maintenance quality assessment report. The total score and multi-dimensional operation quality profile vector of this assessment are output, and the assessment results are input into an operation quality degradation prediction model. Simultaneously, based on the behavioral trajectory consistency score and abnormal behavior correction factor in the assessment report, maintenance operation correction instructions or maintenance strategy adjustment parameters are generated to guide subsequent maintenance operations or equipment maintenance plan optimization.

[0079] In some applications, a standardized task library is established to store multiple assessment tasks. Each assessment task is configured with at least assessment definition parameters and calibration parameters. The assessment definition parameters include at least a standard operating sequence, including a set of predefined assessment tasks for different manufacturers, models, positions, and specialized domains. Assessment definition parameters are configured for each assessment task, including at least a task objective, quantitative indicator threshold, evidence chain requirements, reference time, basic weight, and standard operating sequence. Calibration parameters are configured for each assessment task, including equivalent calibration parameters and project response theory parameters. The equivalent calibration parameters include system complexity coefficient, task impact coefficient, and environmental factor coefficient. The project response theory parameters include at least a task discrimination parameter, a difficulty parameter, and a guessing parameter.

[0080] Understandably, when establishing a standardized task library, a set of predefined assessment tasks is created for different simulator manufacturers, models, job types, and specialized domains. For each assessment task, assessment definition parameters are configured, including task objectives, quantitative indicator thresholds, evidence chain requirements, reference time, basic weights, and standard operating procedure sequences. Specifically, the task objectives clarify the concrete work results to be achieved, the quantitative indicator thresholds determine the acceptable boundaries for task completion, the evidence chain requirements specify the set of evidence items that must be collected and solidified during the assessment process, the reference time measures task completion efficiency, the basic weights characterize the task's fundamental importance within its specialized domain, and the standard operating procedure sequences serve as a benchmark for subsequent behavioral trajectory comparisons. Simultaneously, calibration parameters are configured for each assessment task, including equivalent calibration parameters and project response theory parameters. The equivalent calibration parameters further include system complexity coefficients, task impact coefficients, and environmental factor coefficients. The system complexity coefficient reflects the number of interfaces, subsystems, coupling relationships, and diagnostic path complexity involved in the task. The task impact coefficient reflects the degree of influence of task failure on training downtime risk, training degradation risk, compliance risk, or quality risk. The environmental factor coefficient reflects the impact of differences in the operational support environment on task difficulty. The project response theory parameters include at least task discrimination parameters, difficulty parameters, and guessing parameters, used in subsequent computerized adaptive testing to calculate the task's information content and task completion probability.

[0081] In some applications, the assessment configuration is determined, the historical operational quality profile of the person being assessed is obtained, and based on the assessment configuration and the historical operational quality profile, an assessment task set is extracted from the standardized task library to generate an assessment task sequence. This includes obtaining the assessment level, specialized domain, job type, manufacturer, and model parameters as the assessment configuration for this assessment; obtaining the historical operational quality profile of the person being assessed; if the historical operational quality profile exists, extracting the currently determined operational proficiency level value of the person being assessed from it, and using a computerized adaptive testing algorithm, based on the project response theory parameters, dynamically selecting the assessment task combination with the largest project information content under the currently determined operational proficiency level value from the standardized task library, while satisfying the specialized domain coverage constraint, to form an assessment task set, and generating an assessment task sequence and scoring configuration; if the historical operational quality profile does not exist, assessment tasks are randomly extracted from the standardized task library according to the assessment configuration to form an assessment task set, and generating an assessment task sequence and scoring configuration.

[0082] Understandably, when determining the assessment configuration, the assessment level, specific domain, job type, manufacturer, and model parameters are obtained as the assessment configuration for this assessment. The historical operational quality file of the assessed individual is retrieved. This file records the multi-dimensional operational quality vector values ​​from each assessment and a time-series quality file updated using an exponentially weighted moving average. If this historical operational quality file exists, the assessed individual's current determined operational proficiency level is extracted from it. This operational proficiency level is an objective value calculated using a model based on historical assessment data. After obtaining the current determined operational proficiency level, a computerized adaptive testing algorithm is used. Based on the pre-configured item response theory parameters for each assessment task in the standardized task library, the item information content of each candidate assessment task at the current operational proficiency level is calculated. The calculation of item information content is based on task discrimination parameters, difficulty parameters, guessing parameters, and the current operational proficiency level value, derived through an item information content function. This function reflects the task's discrimination accuracy for a specific operational proficiency level. The algorithm prioritizes assessment tasks with the largest amount of project information. Simultaneously, it considers the specialized domain parameters in the assessment configuration to ensure that the selected task combination covers the required specialized domain range, avoiding assessment content concentration in a single area. This selection process is repeated until the preset maximum number of tasks is reached, forming an assessment task set. This then generates an assessment task sequence containing task order, scoring configuration, and weighting information. If the assessed person's historical performance record is missing (i.e., this is their first assessment or historical data is incomplete), assessment tasks are randomly selected from the standardized task library according to preset coverage rules based on the assessment level, specialized domain, job type, manufacturer, and model parameters in the assessment configuration. This forms an assessment task set, which in turn generates the assessment task sequence and corresponding scoring configuration.

[0083] In some applications, key data during the evaluation process is executed and collected to generate an evidence chain package. The operation sequences in the evidence chain package are compared with the standard operation sequences by edit distance to calculate the behavioral trajectory consistency score. This includes collecting data from simulators, maintenance terminals, and measuring instruments during the evaluation process. The data includes at least operation logs, alarm records, configuration change records, key parameter snapshots, operation sequences, command input timing, and environmental context information. The collected key data is bound with timestamps, device identifiers, and workstation identifiers, and hashed and digitally signed to generate the evidence chain package. A behavioral trajectory snapshot of the person being evaluated is extracted from the evidence chain package. The operation sequences in the behavioral trajectory snapshot are used as the operation sequences in the evidence chain package and compared with the standard operation sequences of the corresponding evaluation task by edit distance to calculate a normalized similarity score as the behavioral trajectory consistency score. The data in the evidence chain package is subjected to consistency verification. If the verification fails, the basic anomaly correction factor is set to zero or a predetermined correction value.

[0084] Understandably, during the evaluation process, multi-source data from simulators, maintenance terminals, and measuring instruments is automatically collected via a data acquisition interface. The collected data includes at least simulator operation logs, alarm event records, configuration change records, key parameter snapshots, operation sequences, command input timing, and environmental context information. The environmental context information includes the equipment operating status baseline, support resource configuration, spare parts availability, network connectivity, and external support conditions. The collected key data is bound to timestamps, equipment identifiers, and workstation identifiers to form traceable data units. These bound data units are then hashed to generate hash verification values, digital signatures are added, and the data is packaged into an evidence chain. This evidence chain is used to ensure the evaluation process is verifiable, auditable, and tamper-proof. A snapshot of the evaluated person's behavior trajectory is extracted from the generated evidence chain. This behavioral trajectory snapshot is a subset of evidence formed by a structured record of the maintenance terminal's operation sequence, command input timing, and interface interaction trajectory during the evaluation process. The operation sequences in the behavioral trajectory snapshot are used as operation sequences in the evidence chain package. These sequences are compared with the pre-configured standard operation sequences for the corresponding evaluation task using edit distance. Edit distance refers to the minimum number of edit operations required to transform one sequence into another. By normalizing the edit distance, a similarity score between zero and one is calculated and used as the behavioral trajectory consistency score. Simultaneously, consistency checks are performed on the data in the evidence chain package, including verifying whether there are contradictions between system log times and manual recording times, whether there is a causal relationship between changes in key parameters and result improvements, and whether the operation sequences match the system log time sequence. If the check fails, the basic anomaly correction factor is set to zero. If the check passes but some doubts exist, the basic anomaly correction factor is set to a predetermined correction value based on the severity of the doubts.

[0085] In some applications, equivalence calibration and quantitative scoring are performed. Task weights are calculated based on the calibration parameters, and multiple scoring factors are extracted from the evidence chain. Abnormal behavior correction factors are updated by combining the behavioral trajectory consistency score. The score for each evaluation task is calculated based on the task weights, the scoring factors, and the updated abnormal behavior correction factors. These scores are then accumulated to obtain the total evaluation score. This includes obtaining the basic weights, system complexity coefficients, task impact coefficients, and environmental factor coefficients of the evaluation tasks. The basic weights are multiplied by the system complexity coefficients, task impact coefficients, and environmental factor coefficients to obtain the calibrated task weights. Each evaluation task is extracted from and calculated from the evidence chain. The quality score, process score, timeliness score, and evidence completeness score are used as the scoring factors. The timeliness score is determined based on the ratio of the actual time spent on the task to the reference time. The basic anomaly correction factor is corrected based on the behavioral trajectory consistency score to obtain the final anomaly behavior correction factor. The correction method is to multiply the basic anomaly correction factor by a coefficient determined by the behavioral trajectory consistency score. The task weight is multiplied by the weighted sum of the quality score, process score, timeliness score, and evidence completeness score, and then multiplied by the final anomaly behavior correction factor to obtain the score for each evaluation task. The scores of all tasks in this evaluation are added together, and the reliability contribution bonus score is added. The risk event deduction item is subtracted to obtain the total score of this evaluation.

[0086] Understandably, when performing equivalence calibration and quantitative scoring, the basic weight, system complexity coefficient, task impact coefficient, and environmental factor coefficient of the evaluation task are obtained. The basic weight is multiplied by the system complexity coefficient, task impact coefficient, and environmental factor coefficient to obtain the calibrated task weight. The system complexity coefficient reflects the impact of the number of interfaces, subsystems, and diagnostic path complexity involved in the task on the task difficulty. The task impact coefficient reflects the degree of impact of task failure on the risks of training stoppage, training degradation, and compliance risks. The environmental factor coefficient reflects the impact of differences in the operational support environment on the task difficulty. The quality score, process score, timeliness score, and evidence completeness score of each evaluation task are extracted and calculated from the evidence chain package as scoring factors across multiple dimensions. The quality score evaluates the degree of indicator recovery, stability, and error margin after task completion. The process score evaluates the achievement of key safety and compliance points, procedural standardization, and risk control point execution during task execution. The timeliness score is determined based on the ratio of the actual task time to the reference time, specifically using a limiting function to restrict the complement of the ratio between zero and one. The evidence completeness score evaluates the completeness and consistency of the task's corresponding logs, measurement records, before-and-after comparisons, and retest reports. The basic anomaly correction factor is adjusted based on the behavioral trajectory consistency score by multiplying it by a coefficient determined by the behavioral trajectory consistency score. This coefficient is calculated based on the trajectory consistency score and a preset trajectory penalty weight, ranging from zero to one. The higher the trajectory consistency score, the closer the correction coefficient is to one, resulting in the final anomaly behavior correction factor. The calibrated task weight is multiplied by the weighted sum of the quality, process, timeliness, and evidence completeness scores, and then multiplied again by the final anomaly behavior correction factor to obtain the score for each evaluation task. The scores for all assessment tasks in this evaluation are added together, and the reliability contribution bonus points are added. The risk event deduction points are subtracted to obtain the total score for this evaluation. The reliability contribution bonus points are used to add extra points for positive behaviors such as high reliability operation, redundancy risk avoidance, and compliance closed-loop management demonstrated in the evaluation. The risk event deduction points are used to deduct points specifically for behaviors such as illegal operation, violation of safety red lines, and equipment operation risk events that occur in the evaluation.

[0087] In some of these applications, an operation quality profile is generated and updated. Based on the score of each evaluation task and the total score of this evaluation, a multi-dimensional operation quality profile vector is generated. The multi-dimensional operation quality profile vector is then fused with the historical operation quality archive using an exponentially weighted moving average method to update the historical operation quality archive. Simultaneously, a maintenance quality evaluation report is output, and maintenance operation correction instructions or maintenance strategy adjustment parameters are generated based on the evaluation report.

[0088] Understandably, based on the score of each assessment task, it is mapped to multiple predefined quality dimensions to generate a multi-dimensional operational quality profile vector for this assessment. These quality dimensions include at least safety and compliance, fault diagnosis, tools and measurement, system understanding and integration, configuration and change control, documentation and evidence chain, timeliness and resources, and risk identification and emergency response. The mapping method involves decomposing the score of each assessment task according to the quality dimension to which the task belongs. The scores of multiple tasks within the same dimension are aggregated using a weighted sum or average method to form the quality value for each dimension. The historical operational quality archive of the assessed entity is obtained, recording the updated multi-dimensional operational quality vector values ​​after each assessment and the corresponding update time. An exponentially weighted moving average method is used to fuse the multi-dimensional operational quality profile vector generated in this assessment with the corresponding dimension values ​​in the historical operational quality archive. The fusion formula is: the updated dimension value equals the update weight multiplied by the dimension value of this assessment plus one minus the update weight multiplied by the historical dimension value. The update weight ranges from zero to one and can be configured differently according to the assessment level and assessment type. After merging, an updated historical operational quality file is obtained, and a quality time-series file is formed by rolling records in chronological order. This file reflects the evolution of the assessed person's operational quality over time. Based on the total score of this assessment and the updated historical operational quality file, the assessed person's operational quality level is determined. The operational quality level is comprehensively judged by the ratio of the total score to the full score, combined with whether the quality values ​​of each dimension reach the preset threshold. At the same time, based on the specialized domains, manufacturers, and models covered by this assessment task, corresponding specialized compatibility evaluations and platform model endorsements are generated. The specialized compatibility evaluation is used to identify the assessed person's operational compatibility in a specific specialized field, and the platform model endorsement is used to indicate that the assessed person is qualified to operate equipment from a specific manufacturer or a specific model simulator. All of the above information together form the maintenance quality assessment report. Output the total score and multi-dimensional operational quality profile vector of this assessment, and input the assessment results into the operational quality decay prediction model. This model dynamically estimates the decay of the assessed person's operational quality over time based on the exponential decay function and activity correction factor, which is used for subsequent retraining warnings and assessment effectiveness management. At the same time, based on the behavioral trajectory consistency score and abnormal behavior correction factor in the assessment report, generate maintenance operation correction instructions or maintenance strategy adjustment parameters to guide subsequent maintenance operations or equipment maintenance plan optimization.

[0089] In some applications, a dynamic benchmark adaptation step is also included: periodically collecting full evaluation data within a preset period; using a statistical analysis algorithm, calculating a weighted quantile value based on the completion time distribution of each evaluation task in the full evaluation data, as a new reference time; using a smoothing coefficient, weighting the new reference time with the historical reference time to obtain a revised reference time, and simultaneously periodically and automatically revising the quantitative indicator thresholds and weight coefficients; updating the revised reference time, quantitative indicator thresholds, and weight coefficients to the standardized task library after review, and recording the version history.

[0090] Understandably, this also includes a dynamic benchmark adaptation step to keep the scoring system relevant as industry technology evolves. Full assessment data is collected periodically within a preset period, which can be set according to the actual application scenario, such as quarterly or semi-annually. The collected full assessment data includes at least all completion times, quantitative indicator achievement values, and corresponding weighting coefficient application records for each assessment task within the period. A statistical analysis algorithm is used to calculate weighted quantile values ​​as new reference times based on the distribution of completion times for each assessment task in the full assessment data. The calculation of weighted quantiles is based on quantile regression methods, allowing the selection of specific quantile points according to the distribution characteristics of the assessment data to reflect the completion time corresponding to the upper-middle level in the industry, avoiding interference from extreme values ​​on the benchmark. A smoothing coefficient is used to weight and average the calculated new reference time with historical reference times to obtain the revised reference time. The smoothing coefficient ranges from zero to one, used to control the impact of single-period data fluctuations on benchmark revision, preventing drastic changes in the benchmark due to accidental fluctuations. While revising the reference time, the quantitative indicator thresholds and weighting coefficients are simultaneously and automatically revised periodically. The quantitative indicator thresholds can be calibrated based on the statistical distribution of the evaluation data within the period, and the weighting coefficients can be adjusted according to changes in task discrimination. After the revision is completed, the revised reference time, quantitative indicator thresholds, and weighting coefficients are reviewed and confirmed by the expert committee, then updated to the corresponding evaluation task field in the standardized task library, and the version history is recorded, including the revision time, revision content, reviewer, and effective date, to ensure the traceability of the benchmark revision process.

[0091] The following describes another embodiment of the simulator maintenance quality assessment method of the present invention:

[0092] Definitions of abbreviations and key terms in this embodiment:

[0093] Standardized Task Library: A pre-established set of reusable assessment tasks; each task includes at least the task objective, quantitative indicators / thresholds, evidence chain requirements, reference time, and weight / calibration parameters, used to standardize the assessment criteria of different institutions.

[0094] Evidence Package (EP): A collection of evidence that packages and solidifies key data from the evaluation process. It includes at least system logs, configuration changes, and measurement data, and is accompanied by timestamps, hash verification, and digital signatures to ensure verifiability, auditability, and tamper-proofness.

[0095] Equivalence Calibration Model: A model that normalizes the difficulty of tasks under different manufacturers, models, configurations, and environments.

[0096] Quantitative Scoring Model: Based on the evidence chain package, it calculates the task completion quality (Q), process compliance (S), timeliness (T), and evidence completeness (E) by weighting, and can combine anomaly correction factors to obtain the task / total score.

[0097] Ability Vector: Outputs operational quality results in the form of a multi-dimensional vector, used for transferable representations across organizations (e.g., dimensions such as safety compliance, diagnostic capabilities, tool measurement, configuration management, etc.).

[0098] Dynamic Baseline Adaptation (DBA): A mechanism that automatically revises task baseline parameters (reference time, threshold, weight coefficient) periodically based on historical evaluation big data, so that the scoring system remains timely as industry technology evolves.

[0099] The Federated Cross-Institutional Calibration Protocol (FCICP) is a protocol mechanism that, while protecting the data privacy of each institution, uses a federated learning framework to jointly model cross-institutional assessment data, thereby achieving globally consistent calibration without the need for centralized sharing of raw data.

[0100] The Competency Decay Prediction Model (CDPM) is a predictive model that dynamically estimates the current operational quality status of maintenance personnel based on their historical assessment records and on-duty operation frequency, predicts the risk of operational quality decay, and triggers retraining or review recommendations.

[0101] The evaluation system implemented here adopts a combination of "core certificate + endorsement + platform / model endorsement" to balance market-based mobility with personnel risk control.

[0102] The assessment is recommended to be divided into five levels, from L1 to L5.

[0103] The evaluation adopts a combined structure of theory, sandbox / simulation, on-site practice, and working sample review, with the core being the evidence chain package and quantitative scoring of on-site practice.

[0104] The system architecture of this embodiment includes:

[0105] Task Library Module: Stores standardized task definitions, indicator thresholds, evidence chain requirements, task parameters, and equivalence coefficients.

[0106] Assessment scheduling module: Extracts tasks based on parameters such as special domain, job / role, and assessment level, generates assessment plans and randomized task combinations, and distributes them to the assessment environment.

[0107] Data acquisition module: Automatically collects data from sources such as simulators, maintenance terminals, measuring instruments, and networks, including logs, alarms, configuration differences, snapshots of key parameters, and timestamps.

[0108] Evidence chain management module: Packages, anonymizes (optional), performs hash verification, digital signature and storage of collected data; supports audit query and verification.

[0109] Scoring and Calibration Module: Performs task equivalence calibration model, reference time calculation, quantitative scoring model, abnormal behavior detection and scoring correction.

[0110] Operational Quality Profile and Report Module: Generates multi-dimensional operational quality profile vectors, grade determinations, special endorsements / certifications, and report verification information.

[0111] Audit and Reporting Module: Provides cross-organizational statistics, discrimination analysis, question bank calibration, and consistency analysis.

[0112] Dynamic benchmark adaptive module: Periodically collects full evaluation data, uses statistical analysis methods (including but not limited to quantile regression and IRT item response theory) to automatically revise the task reference time, threshold and weight coefficients, and updates them to the task library module after review by the expert committee.

[0113] The federated calibration protocol module deploys lightweight federated nodes locally in each participating institution, uploading only gradient / statistical summaries rather than raw data. It updates global calibration parameters through federated aggregation algorithms (such as FedAvg or FedProx), achieving cross-institutional benchmark consistency without exposing the original evaluation data of each institution.

[0114] Operation quality decay prediction module: Based on the maintenance personnel's historical evaluation records, report issuance date, on-duty operation frequency and operation type logs, the module uses a time-series decay model (such as exponential decay function combined with forgetting curve theory) to dynamically estimate the current operation quality score. When the predicted operation quality score is lower than the set threshold, a retraining / review warning is pushed to the report holder and their affiliated organization.

[0115] The adaptive assessment task generation module reads the existing operational quality profile of the person being assessed and uses the Computerized Adaptive Testing (CAT) algorithm to dynamically select the task combination with the most information from the standardized task library. This reduces redundant tasks while ensuring assessment coverage, improves assessment efficiency, and enhances the accuracy of distinguishing boundary operational quality levels.

[0116] The typical process corresponding to this embodiment includes:

[0117] Establish / maintain a standardized task library and configure indicator thresholds, evidence chain requirements, basic weights, and equivalence parameters for each task.

[0118] Assessment Configuration: Select (or automatically identify) the assessment level and special domain, and optionally input parameters such as job type, manufacturer, and machine model; if the assessed person has a historical operation quality profile, it will be synchronously input into the adaptive assessment task generation module.

[0119] Task extraction: The evaluation and scheduling module randomly extracts a set of tasks from the task library based on parameters (or the adaptive evaluation task generation module dynamically selects tasks based on the CAT algorithm) and generates task sequences and scoring configurations.

[0120] Execution assessment: Candidates complete tasks in a specified environment; the data acquisition module automatically collects process data.

[0121] Generate evidence chain package: The evidence chain management module packages logs, configurations, measurements, timestamps, etc., and performs hashing and digital signature; it also performs de-identification and access control when necessary.

[0122] Equivalent calibration and scoring: The scoring and calibration module calculates task weights, reference time, and scores for each dimension, and performs anomaly detection and correction.

[0123] Operational quality profile and report output: Generate a multi-dimensional operational quality profile, level determination, special endorsement / certification, and report verification information; simultaneously input the results of this assessment into the operational quality degradation prediction module to update the quality time series file of the personnel.

[0124] Audit and review: In abnormal or sampling scenarios, review and arbitration are carried out through evidence chain packages.

[0125] Dynamic benchmark update: The dynamic benchmark adaptive module periodically (e.g., quarterly) aggregates all evaluation data and executes the benchmark parameter revision process; if the federated calibration protocol is enabled, each institution node only uploads a statistical summary, and the federated aggregation algorithm outputs the updated global calibration parameters, which are then pushed to each task library module after review.

[0126] Operational quality timeliness monitoring: The operational quality decay prediction module continuously estimates the operational quality of the assessed personnel periodically. When the predicted operational quality score is lower than the preset threshold, a retraining / review warning notification is automatically triggered.

[0127] The evidence chain package is used to solidify key evidence in the evaluation process in a verifiable and auditable manner, reducing the risk of subjective scoring and tampering, including:

[0128] Automatically collects data including: simulator operation logs, alarm / event records, configuration change records, key parameter snapshots, network / clock synchronization information, etc.

[0129] Measurement and verification data: such as visual geometric error measurement, color difference ΔE measurement, platform positioning error / jitter spectrum, frame synchronization / frame loss statistics, etc.

[0130] Manually supplemented data: measurement forms, screenshots / photos, risk assessments, retest records, change orders / work order records (can be anonymized);

[0131] Consistency verification information: timestamp, hash value, digital signature (examinee signature + invigilator signature + system signature), device identifier and workstation identifier;

[0132] Access Controls and Data Masking: Fields involving commercially sensitive configurations or customer information are masked, and masking rules are recorded for auditing purposes;

[0133] Behavioral Trajectory Snapshot: During the evaluation process, the operation sequence, command input timing, and interface interaction trajectory of the maintenance terminal are recorded in a structured manner to form a Behavioral Trajectory Snapshot (BTS), which is cross-compared with the system log to support abnormal behavior detection and anti-fraud verification.

[0134] Environmental context information: Record environmental parameters during the assessment, including but not limited to: equipment operating status baseline, current support resource configuration, spare parts availability status, network connectivity, external support conditions, etc., to support the objective assignment of the Ke coefficient and post-audit.

[0135] In this model, the task equivalence calibration defines the calibrated weights for task j (referring to the j-th evaluation task in the standardized task library). (The final weight of task j after equalization calibration), the calculation formula is as follows:

[0136] ;

[0137] in, :Task The basic weight is a pre-defined basic weight value based on the core level and industry-standard importance of the task, and is not affected by external factors such as machine type, manufacturer, or environment;

[0138] System complexity coefficient, used to characterize the number of interfaces, subsystems, coupling relationships, and diagnostic path complexity involved in the task.

[0139] : Task impact coefficient, used to characterize the impact of task failure or non-resumption on training interruption risk, training degradation risk, compliance risk / quality risk.

[0140] : Environmental factor coefficient, used to characterize differences in the operational support environment, such as night flight support, spare parts availability, remote support limitations, and on-call resources.

[0141] Note: Differences in manufacturers, models, and equipment configurations can affect... / / It incorporates parameters such as reference time to achieve comparable scoring regardless of manufacturer or model.

[0142] The reference time and baseline threshold include:

[0143] To avoid unfairness caused by differences in "equipment speed" or "tool differences" between different units, a reference time can be set for the task. And update it through historical data or standard bench calibration.

[0144] Key metric thresholds are expressed using a "margin from threshold" normalization, prioritizing relative thresholds (e.g., relative tolerance percentage) over absolute values ​​to improve consistency across devices.

[0145] Dynamic benchmark revision: reference time The threshold is no longer fixed, but is periodically revised by the dynamic benchmark adaptive module in the following manner:

[0146] ;

[0147] in, Task number, referring to the first task in the standardized task library. Evaluation task;

[0148] :Task The revised new reference time;

[0149] :Task The historical baseline reference time before revision;

[0150] Tasks in the full evaluation data during this period Weighted quantile estimates of completion time (e.g., P60 quantile).

[0151] : Smoothing coefficient, with a value range of (0,1), to prevent drastic changes in the benchmark caused by fluctuations in single-period data;

[0152] The revisions must be reviewed and confirmed by the expert committee before they can take effect and be recorded in the version history.

[0153] The task score and total score model includes:

[0154] Each task (Referring to the first in the standardized task library) Score of the assessment task (Task The final assessment score can be calculated in the following form:

[0155] ;

[0156] in, :Task The final weights after equivalence calibration;

[0157] :Task The quality score, with a value range of [0,1], is used to evaluate the degree of indicator recovery, stability, error margin, etc. after the task is completed;

[0158] :Task The process score, with a value range of [0,1], is used to evaluate the reach of key safety / compliance points, the standardization of steps, and the execution of risk control points during task execution.

[0159] :Task The evidence chain integrity score, with a value range of [0,1], is used to evaluate the completeness and consistency of task-related logs, measurement records, comparisons before and after changes, retest reports, etc.

[0160] :Task An abnormal behavior correction factor (anti-cheating), with a value range of [0,1], is used when abnormal behavior or contradictory evidence is detected. This may trigger task invalidation and review.

[0161] : Task 's timeliness score, with a value range of [0, 1], and the typical form is:

[0162] ;

[0163] Among them:

[0164] (x, min, max): Clamping function, used to limit the calculation result x within the interval [min, max]; when x < min, output min, when x > max, output max, and in other cases output x;

[0165] : The actual time taken by the evaluated person to complete the task ; [[ID=二十二]]

[0166] : Task 's benchmark reference time;

[0167] , , , : Respectively, the weight coefficients corresponding to the quality score, process score, timeliness score, and evidence chain integrity score, with a value range of (0, 1), and satisfy = 1, which can be configured according to the special domain / level;

[0168] Total score Can be:

[0169] ;

[0170] Among them, : The final total score of the evaluated person in this evaluation;

[0171] : Summation operator, referring to the sum of the scores of each task in the task set extracted in this evaluation; for accumulation;

[0172] : The set of valid tasks extracted in this evaluation;

[0173] : Task 's final evaluation score;

[0174] : Reliability contribution reward score, used to give extra points for positive behaviors such as high reliability operations, redundant risk avoidance, and compliance closed-loop management reflected in the evaluation;

[0175] Risk event deduction items are used to deduct points specifically for behaviors such as violations of safety red lines, equipment operation risk events, etc. that occur during the assessment.

[0176] Behavioral trajectory consistency correction: In In the calculation, an additional behavioral trajectory consistency score is introduced. The calculation method involves comparing the operation sequence in the behavior trajectory snapshot with the standard operation sequence of the task using edit distance, and then normalizing to obtain a similarity score. The final correction factor is updated as follows:

[0177] ;

[0178] in, Task number, referring to the first task in the standardized task library. Evaluation task;

[0179] :Task The final abnormal behavior correction factor;

[0180] :Task The basic anomaly correction factor, with a value range of [0,1], is calculated from basic anti-fraud rules such as data consistency, tool permissions, and result anomalies.

[0181] μ: Trajectory consistency penalty weight, with a value range of [0,1], which can be configured according to the specific domain;

[0182] :Task The behavioral trajectory consistency score, with a value range of [0,1], is calculated by normalizing the edit distance comparison between the operation sequence in the behavioral trajectory snapshot and the standard operation sequence of the task.

[0183] The operational quality degradation prediction model includes:

[0184] To track the timeliness of maintenance personnel's operational quality, an operational quality degradation prediction model is introduced. For maintenance personnel... (Referring to the target maintenance personnel being evaluated) in the quality dimension (Referring to the corresponding quality dimension in the multidimensional operational quality profile) The current estimated operational quality value at time t (the target time for operational quality assessment). Defined as:

[0185] ;

[0186] in, This person is in the dimension The score from the most recent assessment;

[0187] The base of the natural logarithm, used to construct the exponentially decaying function;

[0188] Quality Dimension The corresponding rate of decay of inherent abilities was obtained by fitting historical re-examination data (the decay rate is different for different dimensions, for example, the decay rate of operational dimensions is faster than that of knowledge-based dimensions).

[0189] : The time interval between the target time t and the person's most recent assessment;

[0190] Activity correction factor, based on the individual's activity level. The actual frequency and type of on-the-job operations during this period will be adjusted (the higher the operation frequency, the slower the attenuation):

[0191] ;

[0192] in, Maintenance personnel In terms of quality The activity correction factor is set to [0,1];

[0193] These are the maintenance personnel's unique ID and the quality dimension ID, respectively.

[0194] Quality Dimension The corresponding attenuation influence coefficient, with a value range of [0,1], is a configurable parameter used to control the maximum influence of the operation frequency on the activity correction.

[0195] : Operation frequency influence coefficient, a configurable positive parameter used to control the convergence speed of the operation frequency on the activity correction;

[0196] Maintenance personnel exist Quality dimension within the time interval Statistical values ​​of the frequency of on-the-job operations for the corresponding positions;

[0197] The base of the natural logarithm.

[0198] when (Quality Dimension) When the corresponding retraining warning threshold is reached, the system triggers a retraining warning; when (Quality Dimension) When the corresponding report failure / degradation threshold is reached, the system triggers a report degradation or failure prompt.

[0199] The adaptive evaluation task selects the following models:

[0200] For assessees with a historical operational quality profile, the adaptive assessment task generation module dynamically selects tasks using a CAT algorithm based on Item Information Function (IIF).

[0201] ;

[0202] in, :Task The level of operational proficiency of the person being assessed The amount of project information is used to measure the accuracy of the task in distinguishing the level of operational proficiency of the person being evaluated; the larger the value, the higher the accuracy.

[0203] : Task number, referring to the j-th evaluation task in the standardized task library;

[0204] The current estimated operational proficiency level of the person being evaluated is initialized from their historical operational quality profile.

[0205] Complete the task at your level of operational proficiency. The probability is obtained by fitting a three-parameter logistic model (3PL);

[0206] : right The first derivative of represents the rate at which the probability of task completion changes with the level of operational proficiency of the person being evaluated.

[0207] The task selection strategy is as follows: under the premise of satisfying the special domain coverage constraint and the upper limit of the total number of tasks, priority is given to selecting the task with the largest IIF under the current estimated operational proficiency level, so as to minimize the number of evaluation tasks while maximizing the discrimination accuracy of the operational proficiency boundary of the evaluated person.

[0208] The abnormal behavior detection and anti-cheating rules include:

[0209] Anomaly detection can be triggered based on at least one of the following rules:

[0210] Data consistency: discrepancies exist between system log times and manual recording times; changes in key parameters lack a causal relationship with improved results;

[0211] Behavioral trajectory: The operation sequence jumps abnormally, is unusually short in duration but lacks a chain of evidence; the behavioral trajectory snapshot does not match the system log timeline;

[0212] Tools and Permissions: Using unauthorized tools or performing unauthorized operations;

[0213] Abnormal results: The indicator meets the standard but lacks retest records or the retest fluctuations are abnormal;

[0214] Cross-session consistency anomaly: The operational quality profile vector of the same person being evaluated in multiple evaluations is compared longitudinally. If the deviation of a single score from the historical mean exceeds the set standard deviation threshold (e.g., ±2σ), it is automatically marked as a suspected anomaly and enters the manual review queue.

[0215] Group anomaly detection: Statistical analysis is performed on the batch evaluation results under the same examination room / same invigilator. If the batch score distribution is significantly different from the historical benchmark distribution (e.g., the p-value of the KS test is lower than the set threshold), a batch-level audit is triggered.

[0216] After being triggered, it can Set the correction value to 0-1, or proceed with the review process.

[0217] Here is an example of the pseudocode for the scoring algorithm:

[0218] Input: task_set J, parameters (alpha, beta, gamma, delta, mu),

[0219] reference_times tref, base_weights, calibration_coeffs Kc, Ki,Ke,

[0220] ability_profile AP (optional, for adaptive mode),

[0221] historical_scores HS (optional, for decay prediction)

[0222] Output: TotalScore, AbilityVector, DecayWarnings (optional)

[0223] TotalScore = 0

[0224] AbilityVector = init_vector(dim=8) # S, D, M, I, C, E, T, R

[0225] DecayWarnings = []

[0226] # ---- Step 0: Adaptive task selection (if AP available) ----

[0227] if AP is not None:

[0228] theta = estimate_ability(AP)

[0229] J = adaptive_task_selection(J_pool, theta,

[0230] domain_coverage_constraints)

[0231] # ---- Step 1: Main scoring loop ----

[0232] for each task j in J:

[0233] # 1a) Evidence extraction & validation

[0234] evidence = load_evidence_package(j)

[0235] valid = verify_hash_and_signature(evidence)

[0236] if not valid:

[0237] Fc_base = 0

[0238] else:

[0239] Fc_base = anomaly_factor(evidence) # 0..1

[0240] # 1b) Behavioral trajectory consistency correction

[0241] bts = load_behavioral_trajectory(j)

[0242] bts_score = edit_distance_similarity(bts, reference_sequence(j))

[0243] Fc = Fc_base * (1 - mu * (1 - bts_score))

[0244] # 2) Equivalence calibration

[0245] W = base_weight(j) * Kc(j) * Ki(j) * Ke(j)

[0246] # 3) Scoring components

[0247] Q = quality_score(evidence) # 0..1

[0248] P = process_score(evidence) # 0..1

[0249] T = clamp(1 - actual_time(evidence) / tref(j), 0, 1)

[0250] E = evidence_integrity(evidence) # 0..1

[0251] Score = W * (alpha*Q + beta*P + gamma*T + delta*E) * Fc

[0252] TotalScore += Score

[0253] # 4) Update ability vector

[0254] AbilityVector += map_task_to_ability(j, Q, P, T, E, Fc)

[0255] # 5) Reliability bonus / risk penalty

[0256] TotalScore += reliability_bonus(AbilityVector) \

[0257] - risk_penalty(AbilityVector)

[0258] # ---- Step 2: Cross-session anomaly detection ----

[0259] if HS is not None:

[0260] flag = cross_session_anomaly_check(TotalScore,

[0261] HS.mean, HS.std, sigma_thresh=2.0)

[0262] if flag:

[0263] mark_for_manual_review(candidate_id)

[0264] # ---- Step 3: Competency decay prediction (for certified personnel)----

[0265] if HS is not None:

[0266] for each dimension d in AbilityVector.dimensions:

[0267] CV = competency_decay(HS.last_score[d],

[0268] decay_rate[d],

[0269] delta_t,

[0270] activity_factor(candidate_id, d))

[0271] if CV < threshold_warning[d]:

[0272] DecayWarnings.append((d, CV, "retrain_recommended"))

[0273] if CV < threshold_invalid[d]:

[0274] DecayWarnings.append((d, CV, "certificate_downgrade"))

[0275] return TotalScore, normalize(AbilityVector), DecayWarnings

[0276] The multi-dimensional operational quality profile includes:

[0277] The evaluation results not only output "pass / fail" or a grade, but also output a multi-dimensional operational quality profile (AbilityVector) for cross-institutional transferable representation.

[0278] Recommended quality dimensions (can be adjusted according to implementation method):

[0279] Dimension Code Dimension Name S Safety and Compliance D Fault Diagnosis M Tools and Measurement I System understanding and integration C Configuration / Change Control E Documents and Chain of Evidence T Timeliness and Resources R Risk Identification and Emergency Response

[0280] Operational quality profiles can be matched with job requirement matrices, serving as an objective basis for recruitment, promotion, training, and inter-unit mobility.

[0281] Operational quality profile dynamic update mechanism: After each assessment is completed, the system integrates the operational quality vector results of this assessment into the personnel's historical operational quality profile file in the form of exponentially weighted moving average (EWMA) to form a rolling updated capability profile curve, which is used to support the operational quality degradation prediction model and cross-site anomaly detection.

[0282] ;

[0283] in, Quality dimension number, referring to the corresponding quality dimension in the multi-dimensional operational quality profile;

[0284] After the update, the quality dimension The corresponding historical archive operation quality profile vector value;

[0285] The quality dimensions obtained in this assessment The corresponding real-time operational quality vector result;

[0286] Before the update, the quality dimension The corresponding historical archive operation quality profile vector value;

[0287] Update weights (smoothing coefficients), with a value range of (0,1), which can be configured differently according to the evaluation level and evaluation type.

[0288] Here are some examples of the evaluation components and weights:

[0289] The following combination can be used: theoretical examination 20%, simulation / sandbox operation 20%, on-site practical operation 50%, and work sample review 10%.

[0290] The on-site practical training focuses on evidence chain packages and quantitative scoring, while the review of working samples can adopt anonymous blind review and consistency test.

[0291] Adaptive weight allocation: For reassessment scenarios (regular review for certified personnel), the weight of each assessment stage can be dynamically adjusted based on the output of the operation quality decay prediction module. The proportion of on-site practical operation can be appropriately increased for dimensions with a higher risk of predicted operation quality decay, so as to improve the reassessment's relevance and efficiency.

[0292] The standardized task library includes:

[0293] The mission library is built according to specialized domains, covering, but not limited to: visuals / displays, motion platforms, avionics / interfaces, software / configuration, quality management, aircraft material management, network and information security, etc.

[0294] Each task includes: task objective, pass threshold, risk point list, quantifiable output items, allowed tool range, evidence chain requirements, equivalence parameters, and reference time.

[0295] The task library field template is as follows:

[0296] field name illustrate Task ID Unique Identifier Specialized Domain Such as visual / motion / avionics, etc. Job type (Role, optional) Such as routine maintenance / technical support, etc. Applicable manufacturers (Vendor, optional) Such as CAE / L3 / General Applicable aircraft types (AircraftType, optional) Such as A320 / B737 / General Applicable Platform / Configuration (PlatformConfig, optional) Platform-level configuration description Task Name (Title) Brief Name Task Objective Description of the purpose of the assessment Preconditions / Initial State Assess the initial state of the environment Key Steps for the Task Key operating steps Key Safety / Compliance Control Points The safe point that must be reached Quantitative Metrics Measurable output items Acceptable thresholds / tolerances The boundary value for determining compliance Reference time tref (ReferenceTime) Standard completion time Evidence Requirements Minimum set of evidence requirements Allowed Tools and Constraints List of licensed tools Calibration of parameters BaseWeight, Kc, Ki, Ke Anomaly triggering rules (optional) Anomaly detection rule configuration Retest Criteria Retest Triggering Conditions and Requirements Standard operation sequence (ReferenceSequence, newly added) Structured description of reference operation sequences used for behavior trajectory comparison IRT parameters (IRTParams, newly added) The three-parameter logistic model parameters (a, b, c) are used for adaptive task selection. Versioning and Maintenance Log Field version history

[0297] Example task:

[0298] TaskID Domain Role (optional) Vendor (optional) Type (optional) Metric Threshold Evidence Kc / Ki / Ke BaseW VIS-01 Visual Routine maintenance General General Geometric error ≤ Threshold (e.g., ≤ X pixels / corner) Calibration logs + measurement reports + before and after screenshots + behavior trajectory snapshots 1.2 / 1.5 / 1.0 10 VIS-02 Visual technical support General General Color difference ΔE ≤ Threshold (e.g., ≤ X) Color measurement data + correction parameter differences + retest records + behavior trajectory snapshots 1.3 / 1.4 / 1.0 10 MOT-01 sports Routine maintenance General General Positioning error ≤Tolerance (e.g., ≤X mm / deg) Controller logs + measurement curves + safety loop test records + environmental context information 1.4 / 1.6 / 1.1 12 AVIO-01 Avionics / interfaces technical support General General ARINC data consistency Error rate = 0 or ≤ threshold Bus packet capture / interface logs + configuration comparison + behavior trajectory snapshot 1.5 / 1.7 / 1.0 12 SWC-01 Software / Configuration technical support General General Version consistency 100% consistent Version list + diff report + rollback verification log 1.2 / 1.6 / 0.9 10 QA-01 Quality Management Quality Management General General Audit non-compliance closed loop The process is closed-loop and the evidence is complete. Rectification record + Re-inspection record + Signature and timestamp 1.1 / 1.8 / 1.0 9 MAT-01 Aviation Material Management Aviation Material Management General General Spare parts availability / alternative review The strategy is satisfied and the records are complete. Inventory records + Replacement parts review form + Traceability number 1.0 / 1.5 / 1.2 8

[0299] The list of required evidence chains includes:

[0300] To ensure objectivity and auditability, a minimum evidence set can be defined for different specialized domains, as shown in the following example:

[0301] General categories: Task start / end timestamps, login / permission records, critical alarm list, operation command / parameter change records, retest results, behavior trajectory snapshots, and environmental context information;

[0302] Visual data: geometric / fusion / color measurement data files, calibration tool output, before and after comparison screenshots, synchronization / frame loss statistics;

[0303] Motion-related data: Positioning / tracking error curves, servo / drive / pressure / current curves, alarm recovery logs, and safety loop test records;

[0304] Avionics / interfaces: bus packet capture, interface logs, clock synchronization status, input / output consistency reports;

[0305] Software / Configuration: Version list, difference comparison report, rollback verification log, key service status and performance statistics;

[0306] Quality / Aerospace Materials: Audit records, closed-loop evidence, traceability numbers, alternative parts review and risk assessment.

[0307] The equivalent parameter table is as follows:

[0308] To facilitate implementation, Kc / Ki / Ke can be designed as tiered or continuous values. An example of tiered values ​​is given below:

[0309] parameter meaning Range of values ​​(example) Basis for determination (example) Kc System complexity coefficient 1.0–1.6 Number of interfaces, number of subsystems, coupling relationships, and diagnostic path length Ki Task Influence Coefficient 1.0–2.0 Risks of training suspension, risk of training downgrade, and compliance / quality risk levels. Ke Environmental factor coefficient 0.8–1.2 Night flight support, limited spare parts, remote support restrictions, resource constraints BaseWeight Basic weights 1–20 (or tiers) Task type and basic difficulty, historical discrimination, expert calibration

[0310] The Federal Calibration Protocol Parameter Table includes:

[0311] parameter meaning Range of values ​​(example) illustrate λ (smoothing coefficient) Benchmark Dynamic Revision Smoothing Weights 0.1–0.3 To prevent drastic changes in the benchmark due to fluctuations in single-period data φ (EWMA updated weights) Operational quality profile with rolling updates of weights 0.2–0.5 Differentiated configuration based on assessment level rd (decay rate) Decay rate of each quality dimension Dimensional related Fitting data from historical re-examinations μ (trajectory penalty weight) Consistency of Behavioral Trajectory Penalty Weight 0.0–0.3 Configure by special domain σ (Inter-session anomaly threshold) Cross-game scoring deviation threshold 2.0 (multiple of standard deviation) Adjustable by level

[0312] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0313] like Figure 2 As shown, the present invention also provides a simulator maintenance quality assessment system, comprising:

[0314] The standardized task library establishment module 201 is configured to establish a standardized task library to store multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters, and the evaluation definition parameters include at least a standard operation sequence.

[0315] The assessment configuration determination module 202 is configured to determine the assessment configuration, obtain the historical operation quality file of the person being assessed, and extract an assessment task set from the standardized task library based on the assessment configuration and the historical operation quality file to generate an assessment task sequence.

[0316] The behavioral trajectory consistency score calculation module 203 is configured to execute and collect key data during the evaluation process, generate an evidence chain package, compare the edit distance of the operation sequence in the evidence chain package with the standard operation sequence, and calculate the behavioral trajectory consistency score.

[0317] The evaluation score calculation module 204 is configured to perform equivalence calibration and quantitative scoring. It calculates the task weight based on the calibration parameters, extracts multiple dimensions of scoring factors based on the evidence chain package, updates the abnormal behavior correction factor in combination with the behavior trajectory consistency score, calculates the score of each evaluation task based on the task weight, the scoring factors and the updated abnormal behavior correction factor, and accumulates them to obtain the total score of this evaluation.

[0318] The historical operation quality file update and report output module 205 is configured to generate and update operation quality profiles. Based on the score of each evaluation task and the total score of this evaluation, it generates a multi-dimensional operation quality profile vector and merges the multi-dimensional operation quality profile vector with the historical operation quality file through an exponentially weighted moving average method to update the historical operation quality file. At the same time, it outputs a maintenance quality evaluation report and generates maintenance operation correction instructions or maintenance strategy adjustment parameters based on the evaluation report.

[0319] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of the claims of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0320] like Figure 3 As shown, the present invention also provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of a simulator maintenance quality assessment method.

[0321] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 3 The structure shown in this embodiment of the invention includes an electronic device comprising one or more processors 710 and a memory 720; the processors 710 in this electronic device may be one or more. Figure 3Taking a processor 710 as an example; a memory 720 is used to store one or more programs; the one or more programs are executed by the one or more processors 710, so that the one or more processors 710 implement a simulator maintenance quality assessment method as described in any one of the embodiments of the present invention.

[0322] The electronic device may also include an input device 730 and an output device 740.

[0323] The processor 710, memory 720, input device 730, and output device 740 in this electronic device can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0324] The memory 720 in this electronic device serves as a computer-readable storage medium, capable of storing one or more programs. These programs can be software programs, computer-executable programs, or modules, such as the program instructions / modules corresponding to the simulator maintenance quality assessment method provided in this embodiment of the invention. The processor 710 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 720, thereby implementing the simulator maintenance quality assessment method described in the above embodiment.

[0325] The memory 720 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 720 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 720 may further include memory remotely located relative to the processor 710, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0326] Input device 730 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 740 may include display devices such as a display screen.

[0327] The present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a simulator maintenance quality assessment method.

[0328] Specifically, the computer storage medium in this embodiment of the invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0329] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing the maintenance quality of a simulator, characterized in that, include: Establish a standardized task library to store multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters. The evaluation definition parameters include at least a standard operation sequence. Determine the assessment configuration, obtain the historical operation quality file of the person being assessed, and extract the assessment task set from the standardized task library based on the assessment configuration and the historical operation quality file to generate an assessment task sequence; Execute and collect key data during the evaluation process, generate an evidence chain package, compare the edit distance of the operation sequence in the evidence chain package with the standard operation sequence, and calculate the behavioral trajectory consistency score. Equivalence calibration and quantitative scoring are performed. The task weight is calculated based on the calibration parameters. Multiple scoring factors are extracted based on the evidence chain package. The abnormal behavior correction factor is updated in combination with the behavior trajectory consistency score. The score of each evaluation task is calculated based on the task weight, the scoring factors and the updated abnormal behavior correction factor to obtain the total score of this evaluation. Generate and update the operation quality profile. Based on the score of each evaluation task and the total score of this evaluation, generate a multi-dimensional operation quality profile vector. Then, merge the multi-dimensional operation quality profile vector with the historical operation quality file through an exponentially weighted moving average method to update the historical operation quality file. At the same time, output a maintenance quality evaluation report and generate maintenance operation correction instructions or maintenance strategy adjustment parameters based on the evaluation report. During the evaluation process, data is collected from simulators, maintenance terminals, and measuring instruments. The data includes at least operation logs, alarm records, configuration change records, key parameter snapshots, operation sequences, command input timing, and environmental context information. The collected key data is bound with timestamps, device identifiers, and workstation identifiers, and then hashed and digitally signed to generate an evidence chain package. Extract a snapshot of the behavior trajectory of the person being evaluated from the evidence chain package, use the operation sequence in the behavior trajectory snapshot as the operation sequence in the evidence chain package, compare the edit distance with the standard operation sequence of the corresponding evaluation task, calculate the normalized similarity score, and use it as the behavior trajectory consistency score. The data in the evidence chain package is subjected to consistency verification. If the verification fails, the basic anomaly correction factor is set to zero or a predetermined correction value. This also includes a dynamic benchmark adaptation step: Periodically collect all evaluation data within a preset period; Using a statistical analysis algorithm, a weighted quantile value is calculated based on the completion time distribution of each assessment task in the full assessment data, which serves as a new reference time. Using a smoothing coefficient, the new reference time is weighted and averaged with the historical reference time to obtain the revised reference time, and the quantitative indicator thresholds and weight coefficients are automatically revised periodically in sync. The revised reference time, quantitative indicator thresholds, and weight coefficients will be reviewed and updated to the standardized task library, and the version history will be recorded.

2. The simulator maintenance quality assessment method according to claim 1, characterized in that, Establish a standardized task library to store multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters. The evaluation definition parameters include at least a standard operating sequence and further include: For different manufacturers, models, positions and special domains, a set of predefined assessment tasks is set up, and assessment definition parameters are configured for each assessment task. The assessment definition parameters include at least the task objective, quantitative indicator threshold, evidence chain requirements, reference time, basic weight, and standard operation sequence. For each assessment task, calibration parameters are configured, including equivalent calibration parameters and project response theory parameters. The equivalent calibration parameters include system complexity coefficient, task impact coefficient, and environmental factor coefficient. The project response theory parameters include at least task discrimination parameter, difficulty parameter, and guessing parameter.

3. The simulator maintenance quality assessment method according to claim 2, characterized in that, The evaluation configuration is determined, the historical operational quality profile of the evaluated individual is obtained, and based on the evaluation configuration and the historical operational quality profile, an evaluation task set is extracted from the standardized task library to generate an evaluation task sequence, further including: Obtain the assessment level, specific domain, job type, manufacturer, and model parameters as the assessment configuration for this assessment; Obtain the historical operational quality file of the person being evaluated. If the historical operational quality file exists, extract the currently determined operational proficiency level value of the person being evaluated from it. Then, using a computerized adaptive testing algorithm, based on the project response theory parameters, dynamically select the combination of evaluation tasks with the largest project information content under the currently determined operational proficiency level value from the standardized task library, and satisfy the special domain coverage constraint to form an evaluation task set, and generate an evaluation task sequence and scoring configuration. If the historical operation quality file does not exist, evaluation tasks are randomly selected from the standardized task library according to the evaluation configuration to form an evaluation task set, and an evaluation task sequence and scoring configuration are generated.

4. The simulator maintenance quality assessment method according to claim 1, characterized in that, The process involves performing equivalence calibration and quantification scoring, calculating task weights based on the calibration parameters, extracting multiple dimensions of scoring factors based on the evidence chain package, updating the abnormal behavior correction factor in conjunction with the behavioral trajectory consistency score, and calculating the score for each evaluation task based on the task weights, the scoring factors, and the updated abnormal behavior correction factor to obtain the total score for this evaluation. This further includes: Obtain the basic weight, system complexity coefficient, task impact coefficient, and environmental factor coefficient of the evaluation task. Multiply the basic weight by the system complexity coefficient, task impact coefficient, and environmental factor coefficient to obtain the calibrated task weight. The quality score, process score, timeliness score, and evidence completeness score of each evaluation task are extracted and calculated from the evidence chain package as the scoring factor, wherein the timeliness score is determined based on the ratio of the actual time spent on the task to the reference time. The basic anomaly correction factor is corrected based on the behavioral trajectory consistency score to obtain the final abnormal behavior correction factor. The correction method is to multiply the basic anomaly correction factor by a coefficient determined by the behavioral trajectory consistency score. The task weights are multiplied by the weighted sum of the quality score, process score, timeliness score, and evidence completeness score, and then multiplied by the final abnormal behavior correction factor to obtain the score for each assessment task. The scores of all tasks in this assessment are added together, and the reliability contribution bonus score is added, while the risk event deduction items are subtracted to obtain the total score for this assessment.

5. The simulator maintenance quality assessment method according to claim 1, characterized in that, Generate and update an operational quality profile. Based on the score of each assessment task and the total score of this assessment, generate a multi-dimensional operational quality profile vector. Then, merge the multi-dimensional operational quality profile vector with the historical operational quality archive using an exponentially weighted moving average method to update the historical operational quality archive. Simultaneously, output a maintenance quality assessment report, and generate maintenance operation correction instructions or maintenance strategy adjustment parameters based on the assessment report. Further includes: Based on the score of each assessment task, it is mapped to multiple predefined quality dimensions to generate a multi-dimensional operational quality profile vector for this assessment. The quality dimensions include at least safety and compliance, fault diagnosis, tools and measurement, and system understanding and integration. Obtain the historical operational quality file of the person being evaluated, and use an exponentially weighted moving average method to fuse the multi-dimensional operational quality profile vector of this evaluation with the values ​​of the corresponding dimensions in the historical operational quality file to obtain the updated historical operational quality file, and record it on a rolling basis to form a quality time series file. Based on the total score of this assessment and the updated historical operation quality file, the operation quality level of the assessed person is determined, and according to the specific domain, manufacturer and model information covered by the assessment task, the corresponding specific compatibility evaluation and platform model endorsement are generated to form a maintenance quality assessment report. Output the total score and multi-dimensional operation quality profile vector of this evaluation, and input the evaluation results into the operation quality degradation prediction model. At the same time, based on the behavior trajectory consistency score and abnormal behavior correction factor in the evaluation report, generate maintenance operation correction instructions or maintenance strategy adjustment parameters.

6. A simulator maintenance quality assessment system, characterized in that, include: The standardized task library creation module is configured to create a standardized task library that stores multiple evaluation tasks. Each evaluation task is configured with at least evaluation definition parameters and calibration parameters, and the evaluation definition parameters include at least a standard operation sequence. The assessment configuration determination module is configured to determine the assessment configuration, obtain the historical operation quality file of the person being assessed, and extract an assessment task set from the standardized task library based on the assessment configuration and the historical operation quality file to generate an assessment task sequence. The behavioral trajectory consistency score calculation module is configured to execute and collect key data during the evaluation process, generate an evidence chain package, compare the edit distance of the operation sequence in the evidence chain package with the standard operation sequence, and calculate the behavioral trajectory consistency score. The evaluation score calculation module is configured to perform equivalence calibration and quantitative scoring. It calculates task weights based on the calibration parameters, extracts scoring factors from multiple dimensions based on the evidence chain package, updates abnormal behavior correction factors in conjunction with the behavior trajectory consistency score, and calculates the score of each evaluation task based on the task weights, the scoring factors, and the updated abnormal behavior correction factors to obtain the total score of this evaluation. The historical operation quality file update and report output module is configured to generate and update operation quality profiles. Based on the score of each evaluation task and the total score of this evaluation, a multi-dimensional operation quality profile vector is generated. The multi-dimensional operation quality profile vector is then fused with the historical operation quality file using an exponentially weighted moving average method to update the historical operation quality file. Simultaneously, a maintenance quality evaluation report is output, and maintenance operation correction instructions or maintenance strategy adjustment parameters are generated based on the evaluation report. During the evaluation process, data is collected from simulators, maintenance terminals, and measuring instruments. The data includes at least operation logs, alarm records, configuration change records, key parameter snapshots, operation sequences, command input timing, and environmental context information. The collected key data is bound with timestamps, device identifiers, and workstation identifiers, and then hashed and digitally signed to generate an evidence chain package. Extract a snapshot of the behavior trajectory of the person being evaluated from the evidence chain package, use the operation sequence in the behavior trajectory snapshot as the operation sequence in the evidence chain package, compare the edit distance with the standard operation sequence of the corresponding evaluation task, calculate the normalized similarity score, and use it as the behavior trajectory consistency score. The data in the evidence chain package is subjected to consistency verification. If the verification fails, the basic anomaly correction factor is set to zero or a predetermined correction value. This also includes a dynamic benchmark adaptation step: Periodically collect all evaluation data within a preset period; Using a statistical analysis algorithm, a weighted quantile value is calculated based on the completion time distribution of each assessment task in the full assessment data, which serves as a new reference time. Using a smoothing coefficient, the new reference time is weighted and averaged with the historical reference time to obtain the revised reference time, and the quantitative indicator thresholds and weight coefficients are automatically revised periodically in sync. The revised reference time, quantitative indicator thresholds, and weight coefficients will be reviewed and updated to the standardized task library, and the version history will be recorded.

7. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method according to any one of claims 1 to 5.

Images