A method and system for evaluating a service technician training program

Abstract

This invention relates to the field of computer-aided skills training program analysis technology, and more particularly to a method and system for evaluating training programs for maintenance technicians. It includes the following steps: S1: Based on the leap characteristics of trainees' cognitive structures during the formation of maintenance skills, the training program is divided into N consecutive stages with capability qualitative change points as boundaries; S2: At the capability qualitative change boundaries of adjacent consecutive stages, based on the degree of influence of the baseline cognitive unit corresponding to the boundary on subsequent consecutive stages, data variation acting on a single observable parameter is constructed; S3: Based on the matching results of the constraint relationship between the single observable parameter involved in the data variation and the associated unvaried observable parameter, logical conflicts acting on trainees are generated. This invention quantifies cognitive stability through staged training and data variation, and transforms diagnostic strategies into behavioral biases by combining conflict experiments, significantly improving the accuracy of higher-order thinking assessment and attribution reliability in maintenance skills training.

Description

Technical Field

[0001] This invention relates to the field of computer-aided skills training program analysis technology, and in particular to a method and system for evaluating training programs for maintenance technicians. Background Technology

[0002] Maintenance technicians refer to skilled personnel who perform fault diagnosis, condition restoration, and performance maintenance on vehicles already in use. Training programs refer to formal planning documents that specify the training objectives, curriculum system, teaching resources, and assessment standards for maintenance technicians. The evaluation methods for training programs in existing vehicle repair institutions mainly include three categories: summative evaluation based on the average score of theoretical examinations and the pass rate of practical assessments; process evaluation based on student satisfaction questionnaires and trainers' subjective summaries; and follow-up evaluation based on post-graduation return rate statistics. In standardized training scenarios, the above methods provide basic support for the compliance review and phased adjustment of training programs through quantitative indicator screening and empirical judgment.

[0003] However, existing assessment methods have inherent flaws in addressing the quantitative representation of the dynamic evolution of cognitive structures during the formation of vehicle maintenance skills. Specifically, assessment data is mostly derived from cross-sectional sampling at the training endpoint, lacking the ability to collect fine-grained behavioral data and conduct longitudinal correlation analysis on the continuous leap from basic cognition to autonomous strategy generation. Assessment indicators focus on the accuracy of knowledge memorization or the standardization of operational actions, failing to quantitatively measure the extent to which the training program shapes trainees' diagnostic logic construction ability and constraint relationship reasoning ability under conditions of incomplete information. Assessment conclusions rely on horizontal comparisons of group average scores, making it difficult to isolate the interference of differences in trainees' baseline abilities and vehicle technical conditions on the effectiveness of the program.

[0004] Therefore, there is an urgent need to study an evaluation method for training programs that are oriented towards the cognitive leap pattern of vehicle maintenance skills, in order to reveal the true shaping effect of training programs on trainees' ability to autonomously generate diagnostic strategies. Summary of the Invention

[0005] To overcome the drawback of diagnostic thinking being unquantifiable, this invention provides a method and system for evaluating training programs for maintenance technicians.

[0006] The technical implementation of this invention is a method for evaluating a training program for maintenance technicians, comprising the following steps: S1: Based on the leap characteristics of trainees' cognitive structure during the formation of maintenance skills, the training program is divided into N continuous stages with the qualitative change point of ability as the boundary; S2: At the boundary of qualitative change in capability between adjacent consecutive stages, based on the degree of influence of the benchmark cognitive unit corresponding to the boundary on subsequent consecutive stages, construct the data variation acting on a single observable parameter. S3: Based on the constraint relationship matching results between the single observable parameter involved in data variation and the associated unvaried observable parameter, generate logical conflicts that act on the trainees. S4: Based on the differences between trainees' operational behavior under information-masked conditions and their operational behavior under conditions without information-masked conditions after the effect of logical conflict, evaluate the training program's effect on shaping trainees' ability to autonomously generate diagnostic strategies.

[0007] Preferably, the step of constructing data variation acting on a single observable parameter at the boundary of qualitative change in capabilities between adjacent consecutive stages, based on the degree of influence of the baseline cognitive unit corresponding to the boundary on subsequent consecutive stages, includes: Obtain the frequency of invocation of the baseline cognitive unit corresponding to the boundary in subsequent consecutive stages; Based on the causal correspondence between the technical principles described by the benchmark cognitive unit and the observable parameters at the physical level, the correlation between the benchmark cognitive unit and a single observable parameter is established.

[0008] Preferably, establishing the correlation between the baseline cognitive unit and a single observable parameter includes: The benchmark cognitive unit consists of a single cognitive element that is at the boundary of qualitative change in ability in a continuous stage and provides basic support for the learning effect in subsequent continuous stages. Based on the nonlinear cumulative law of the influence of the frequency of calls to the baseline cognitive unit in subsequent continuous stages on the deviation of a single observable parameter, a nonlinear correspondence between the call frequency and the deviation of a single observable parameter is established. Extract the baseline cognitive unit corresponding to the boundary, determine the single observable parameter associated with the baseline cognitive unit based on the correlation relationship, determine the deviation of the single observable parameter based on the nonlinear correspondence relationship, modify the value of the single observable parameter to deviate from the standard range, and maintain the physical possibility of the modified single observable parameter value.

[0009] Preferably, the generation of logical conflicts acting on trainees based on the constraint relationship matching results between the single observable parameter involved in data variation and the associated unvaried observable parameter includes: Obtain K unmutated observable parameter values ​​that have physical or system design constraints with the mutated single observable parameter value, where K is an integer greater than or equal to one; Combine the mutated single observable parameter value with K unmutated observable parameter values ​​to form a parameter value combination; The parameter value combination is matched with a preset standard constraint set, which includes the range of allowable numerical combinations specified by the physical constraint relationship between observable parameters and the system design constraint relationship; If the combination of parameter values ​​does not fall within the range of allowed value combinations specified by the preset standard constraint set, then the combination of parameter values ​​is determined to violate physical constraints or system design constraints. If the combination of parameter values ​​falls within the range of allowed combinations of values ​​specified by the preset standard constraint set, then the combination of parameter values ​​is determined to not violate physical constraints and not violate system design constraints. Based on the determination result of whether the combination of parameter values ​​violates physical constraints or system design constraints, a first type of logical conflict is generated; Based on the result of the parameter constraint mismatch determination, a second type of logical conflict is generated.

[0010] Preferably, generating a second type of logical conflict based on the determination result of parameter constraint mismatch includes: Obtain alternative training vehicles that have the same vehicle model identification but different technical status as the training vehicles used in the first type of logical conflict assessment task; Perform the same data mutation operation on the alternative training platform; If the numerical combination between the mutated single observable parameter value and the K unmutated observable parameter values ​​does not violate physical constraints and system design constraints, then time series data of the mutated single observable parameter and the K unmutated observable parameters under the same operating condition are obtained. Calculate the cross-correlation function between the mutated single observable parameter time series data and the K unmutated observable parameter time series data; If the cross-correlation function value deviates from the preset normal range, it is determined that there is a parameter constraint mismatch between the mutated single observable parameter and the K unmutated observable parameters.

[0011] Preferably, generating a second type of logical conflict based on the determination result of parameter constraint mismatch includes: Map the result of parameter constraint mismatch to a second type of logical conflict event; The second type of logical conflict and the generated first type of logical conflict are applied to the trainees who have completed the data variation verification. The first type of logical conflict is used to verify the trainees' mastery of physical constraints and system design constraints, while the second type of logical conflict is used to verify the trainees' ability to judge the consistency of constraints among multiple observable parameters when the technical state of the training carrier deviates from the preset standard constraint set.

[0012] Preferably, the evaluation of the training program's effect on shaping trainees' ability to autonomously generate diagnostic strategies, based on the difference between trainees' operational behavior under information-masking conditions and without information-masking conditions after the effect of logical conflict, includes: After the generated first and second type of logical conflicts are applied to the trainees who have completed the data mutation verification, an information masking operation is applied to the trainees. Information masking operations include information masking during the period when the first type of logical conflict and the second type of logical conflict are acting simultaneously, and information masking during the period when the first type of logical conflict and the second type of logical conflict are acting at intervals. Simultaneous information masking during the same period refers to hiding the real-time value of at least one observable parameter or hiding at least one physical constraint relationship description during the process of students processing the first type of logical conflict and the second type of logical conflict in parallel. Information masking during the interval refers to hiding the historical time series data of the observable parameters involved in the second type of logical conflict within a preset time period after the trainee completes the first type of logical conflict verification and before the second type of logical conflict verification begins. Acquire trainees’ operational behavior data on handling first-type and second-type logical conflicts under conditions without information masking, and use it as baseline behavior data; Acquire behavioral data of trainees handling first-type and second-type logical conflicts under conditions of information masking, as comparative behavioral data; The behavioral data for comparison were compared with the baseline behavioral data. The differences included differences in diagnostic path length, diagnostic time, and diagnostic accuracy.

[0013] Preferably, the differences compared include differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy, including: The difference in diagnostic path length is obtained by subtracting the total number of diagnostic steps in the baseline behavioral data from the total number of diagnostic steps in the comparative behavioral data. The difference in diagnostic time was obtained by subtracting the total diagnostic time from the baseline behavioral data from the total diagnostic time in the comparative behavioral data. The difference in diagnostic accuracy was obtained by subtracting the percentage of correct diagnostic conclusions from the percentage of correct diagnostic conclusions in the baseline behavioral data from the percentage of correct diagnostic conclusions in the comparative behavioral data. The ability of trainees to autonomously generate diagnostic strategies under conditions of incomplete information is determined by comparing the differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy with preset difference thresholds.

[0014] Preferably, the step of determining the trainee's ability to autonomously generate diagnostic strategies under conditions of incomplete information by comparing the differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy with a preset difference threshold includes: If the difference in diagnostic path length is less than or equal to the first preset threshold, the difference in diagnostic time is less than or equal to the second preset threshold, and the difference in diagnostic conclusion accuracy is greater than or equal to the third preset threshold, then the trainee is deemed to have the ability to generate diagnostic strategies autonomously. If the difference in diagnostic path length is greater than the first preset threshold, the difference in diagnostic time is greater than the second preset threshold, or the difference in diagnostic conclusion accuracy is less than the third preset threshold, then the trainee is deemed not to have the ability to generate diagnostic strategies independently.

[0015] Preferably, a training program evaluation system for maintenance technicians includes: The training program feature extraction module is used to extract the data variation of continuous stage boundaries and baseline cognitive units and construct a single observable parameter; The logic conflict generation module is used to match the constraint relationship between mutated parameters and unmutated parameters to generate logic conflicts; The information masking and behavior collection module is used to apply information masking and collect student operation behavior data; The competency development assessment module is used to compare the differences in operational behavior under conditions of information masking and without information masking in order to evaluate the effectiveness of the training program.

[0016] Beneficial Effects: This invention achieves observable quantification of the stability of trainees' basic cognition by breaking down the training program into continuous stages that match the rhythm of cognitive leaps and constructing data variations associated with baseline cognitive units at the stage boundaries. Based on this, logical conflicts are generated and an information masking control experiment is introduced, using differences in diagnostic path length, time consumption, and accuracy as metrics to transform the implicit ability to autonomously generate diagnostic strategies into explicit behavioral deviation data. This invention effectively overcomes the inherent deficiency of difficulty in quantifying diagnostic thinking in the field of maintenance skills training, significantly improving the accuracy of the evaluation and attribution reliability of the training program's effect on shaping trainees' higher-order thinking abilities. Attached Figure Description

[0017] Figure 1 This is a flowchart of an evaluation method for a training program for maintenance technicians according to the present invention; Figure 2 This is a structural diagram of an evaluation system for a training program for maintenance technicians according to the present invention. Detailed Implementation

[0018] The embodiments of the present invention will be described below with reference to the accompanying drawings.

[0019] Example 1: An evaluation method for a training program for maintenance technicians, such as... Figure 1 As shown, it includes the following steps: S1: Based on the leap characteristics of trainees' cognitive structure during the formation of maintenance skills, the training program is divided into N continuous stages with the qualitative change point of ability as the boundary; It should be noted that the trainee's cognitive structure refers to the internal mental representation framework by which an individual organizes, stores, and retrieves maintenance knowledge. The leap characteristics are used to describe the discontinuous nature of this structure's migration from the basic level to higher levels. Specifically, leap characteristics refer to the discontinuous, qualitative leap in trainees' maintenance skills learning, from being able to mechanically memorize single knowledge points (such as normal sensor values) to being able to independently perform multi-factor causal reasoning. For example, a trainee may go from only being able to say that the normal water temperature is 95℃ to being able to analyze that a water temperature of -20℃ will cause abnormal fuel injection pulse width correction, which in turn will lead to an overly rich air-fuel mixture. The training program is divided into N consecutive stages, aiming to construct an analytical unit system that precisely corresponds to the cognitive development sequence, thereby decomposing the implicit evolution of thinking ability into discrete analytical units that can be measured and observed.

[0020] The qualitative leap point refers to the critical position where a trainee's cognitive structure undergoes a fundamental change in the transition zone between adjacent stages. Specifically, it is defined as the moment when a trainee can, for the first time, independently connect baseline cognitive units belonging to two or more isolated knowledge modules within the causal reasoning chain of fault diagnosis without relying on external prompts. The continuous stage is defined as the relatively stable period of knowledge internalization and skill accumulation between two adjacent cognitive qualitative leaps. Taking engine electronic control system repair as an example, a trainee evolves from an isolated memory state where they can only identify the standard voltage values ​​of a single sensor to a systematic reasoning state where they can understand the causal transmission chain between intake pressure fluctuations and fuel injection pulse width. The two states constitute a typical continuous stage. Through this structured decomposition, this method transforms the originally vague and general diagnostic thinking growth curve into a quantitative analysis object with clear node identifiers. The specific identification method is as follows: compare the completion of the same fault diagnosis task in two cognitive tests. If the student can independently complete the diagnostic reasoning steps that could not be completed in the previous test in the later test, and no longer relies on prompts, it is determined to be a qualitative change point. When the individual qualitative change point determination results of more than half of the students at the same teaching node meet the above conditions, the node is determined to be a common ability qualitative change point, which is used for the stage decomposition of the training program.

[0021] S2: At the boundary of qualitative change in capabilities between adjacent consecutive stages, based on the degree of influence of the baseline cognitive unit corresponding to the boundary on subsequent consecutive stages, construct the data variation acting on a single observable parameter, including: Obtain the frequency of invocation of the baseline cognitive unit corresponding to the boundary in subsequent consecutive stages; Based on the causal correspondence between the technical principles described by the benchmark cognitive unit and the observable parameters at the physical level, establish the correlation between the benchmark cognitive unit and a single observable parameter. The benchmark cognitive unit consists of a single cognitive element that is at the boundary of qualitative change in ability in a continuous stage and provides basic support for the learning effect in subsequent continuous stages. Based on the nonlinear cumulative law of the influence of the frequency of calls to the baseline cognitive unit in subsequent continuous stages on the deviation of a single observable parameter, a nonlinear correspondence between the call frequency and the deviation of a single observable parameter is established. Extract the baseline cognitive unit corresponding to the boundary, determine the single observable parameter associated with the baseline cognitive unit based on the correlation relationship, determine the deviation of the single observable parameter based on the nonlinear correspondence relationship, modify the value of the single observable parameter to deviate from the standard range, and maintain the physical possibility of the modified single observable parameter value.

[0022] It should be noted that data variation refers to a numerical modification of a single observable parameter that deviates from its standard range while maintaining physical probability. Data variation quantifies the learner's mastery of the baseline cognitive unit as the deviation of the observable parameter. This deviation serves as the quantitative input for subsequent generation of logical conflicts and assessment and diagnostic strategies. Specifically, based on the number of times the baseline cognitive unit is relied upon in subsequent successive stages (i.e., the frequency of invocation), a pre-calibrated nonlinear function (such as a power function) is used. ,in For call frequency, The deviation magnitude, where a and b are constants obtained through experimental calibration (a>0, b>1), calculates the modification amount of a single observable parameter. Subtracting or adding this deviation magnitude to the original standard value yields a new value that maintains physical possibility. This value serves as input for subsequent logical conflict generation, thus transforming the student's mastery of the baseline cognitive unit into a measurable parameter anomaly. The call frequency refers to the number of times the baseline cognitive unit is relied upon in subsequent consecutive teaching tasks. Obtaining the call frequency aims to quantify the actual impact weight of the baseline cognitive unit corresponding to the boundary on subsequent learning. The causal correspondence refers to the input-output correspondence between the technical principle described by the baseline cognitive unit and the single observable parameter at the physical level, such as the water temperature signal affecting the fuel injection pulse width. Establishing correlations aims to... Cognitive elements are fixed to specific measurable vehicle operating data; a benchmark cognitive unit refers to a single cognitive element at the boundary of qualitative change in capability and that provides basic support for subsequent learning stages. Its determination logic involves selecting knowledge nodes with prerequisite support at the boundaries of consecutive stages. Specifically, a benchmark cognitive unit refers to a core knowledge point at the boundary between two training stages that provides basic support for subsequent learning. For example, in the transition stage from sensor recognition to actuator control, the correspondence between the water temperature signal and the fuel injection quantity correction is a benchmark cognitive unit. The nonlinear cumulative law refers to the accelerated amplification characteristic of the influence of the frequency of reference to a benchmark cognitive unit on the deviation of a single observable parameter. For example, when the frequency of reference increases exponentially, the deviation... The increase follows an exponential pattern; the purpose of establishing a nonlinear correspondence is to ensure that the deviation of data variation accurately reflects the true amplification effect of cognitive deficits in subsequent learning. The specific calibration method is as follows: collect parameter deviation data after students complete subsequent learning at different call frequencies, use exponential regression or piecewise interpolation to fit the functional relationship, and store this function in the cognitive feature database. When collecting data, ensure that the sample size meets the statistical validity requirements, and use reliability tests (such as Cronbach's Alpha coefficient) to verify the internal consistency of the data; the purpose of determining a single observable parameter based on the correlation is to pinpoint the specific measurement value that has a physical causal connection with the baseline cognitive unit; the purpose of determining the deviation magnitude based on the nonlinear correspondence is to... This ensures that the intensity of variation precisely corresponds to the degree of cognitive impact. Modifying a single observable parameter from its standard range means adjusting the parameter value away from the standard range while maintaining its physical rationality. Maintaining physical possibility means that the modified parameter value is feasible in the real physical world. For example, with an engine coolant temperature sensor, if the baseline cognitive unit is the correspondence between the coolant temperature signal and the fuel injection quantity correction, data variation can change the coolant temperature signal value from the standard 95 degrees Celsius to minus 20 degrees Celsius, a value that is physically possible in extremely cold environments. Through the above constructive tests, this step transforms the trainee's mastery of the baseline cognitive unit into abnormal response data of observable parameters, providing quantitative input for subsequent logical conflict generation.

[0023] S3: Based on the constraint matching results between the single observable parameter involved in data variation and the associated unvaried observable parameter, generate logical conflicts that act on the trainees, including: Obtain K unmutated observable parameter values ​​that have physical or system design constraints with the mutated single observable parameter value, where K is an integer greater than or equal to one; Combine the mutated single observable parameter value with K unmutated observable parameter values ​​to form a parameter value combination; The parameter value combination is matched with a preset standard constraint set, which includes the range of allowable numerical combinations specified by the physical constraint relationship between observable parameters and the system design constraint relationship; If the combination of parameter values ​​does not fall within the range of allowed value combinations specified by the preset standard constraint set, then the combination of parameter values ​​is determined to violate physical constraints or system design constraints. If the combination of parameter values ​​falls within the range of allowed combinations of values ​​specified by the preset standard constraint set, then the combination of parameter values ​​is determined to not violate physical constraints and not violate system design constraints. Based on the determination result of whether the combination of parameter values ​​violates physical constraints or system design constraints, a first type of logical conflict is generated; Based on the result of the parameter constraint mismatch determination, a second type of logical conflict is generated; Obtain alternative training vehicles that have the same vehicle model identification but different technical status as the training vehicles used in the first type of logical conflict assessment task; It should be noted that before performing the data mutation operation, time series data of the alternative training carrier before mutation should be collected under the same operating conditions, the cross-correlation function value before mutation should be calculated, and it should be verified that it is within the preset normal range. If it has deviated from the normal range before mutation, the alternative training carrier should be discarded, and another alternative carrier with small technical differences or normal cross-correlation should be selected. Only when the cross-correlation function value before mutation is normal can the deviation after mutation be attributed to the data mutation operation.

[0024] Perform the same data mutation operation on the alternative training platform; If the numerical combination between the mutated single observable parameter value and the K unmutated observable parameter values ​​does not violate physical constraints and system design constraints, then time series data of the mutated single observable parameter and the K unmutated observable parameters under the same operating condition are obtained. Calculate the cross-correlation function between the mutated single observable parameter time series data and the K unmutated observable parameter time series data; If the cross-correlation function value deviates from the preset normal range, it is determined that there is a parameter constraint mismatch between the mutated single observable parameter and the K unmutated observable parameters; Map the result of parameter constraint mismatch to a second type of logical conflict event; The second type of logical conflict and the generated first type of logical conflict are applied to the trainees who have completed the data variation verification. The first type of logical conflict is used to verify the trainees' mastery of physical constraints and system design constraints, while the second type of logical conflict is used to verify the trainees' ability to judge the consistency of constraints among multiple observable parameters when the technical state of the training carrier deviates from the preset standard constraint set.

[0025] It should be noted that physical constraints refer to the necessary numerical relationships between parameters determined by physical laws, such as the constant voltage-to-current ratio stipulated by Ohm's law; system design constraints refer to the correspondence between parameters determined by the design logic of the vehicle control system, such as the following mapping between the accelerator pedal position signal and the throttle opening signal; the purpose of obtaining K unmutated observable parameter values ​​that have the above-mentioned constraint relationships with the mutated single observable parameter value is to construct a test set to verify whether the trainee can identify the contradictions in the self-consistency between parameters. The K unmutated observable parameter values ​​represent the group of normal parameters affected by the mutated parameter, and the value of K is: the total number of all unmutated parameters that have a constraint relationship with the mutated parameter in the preset standard constraint set; the parameter value combination is a joint judgment unit formed by packaging the mutated parameter and the associated unmutated parameter. The preset standard constraint set is a pre-built database containing all allowed value combinations. The specific method for setting it involves iterating through the standard operating ranges and interlock logic tables of each sensor and actuator in the vehicle maintenance manual, performing a Cartesian product operation on the standard value intervals of any two parameters with physical or design constraints to form allowed combination ranges, and storing these ranges in a lookup table. It should be noted that this method requires that the physical constraints and system design constraints involved be clearly recorded in the vehicle maintenance manual or technical specifications. For undocumented or ambiguous constraints, domain experts supplement them based on experience and include them in the preset standard constraint set. If a parameter combination does not have a known constraint relationship, it will not participate in the matching determination of the first type of logical conflict. Matching refers to comparing the current parameter value combination with each allowed combination range in the preset standard constraint set.

[0026] A violation judgment indicates that the parameter value combination does not belong to the allowed combination range, representing a physical or logical contradiction in the current vehicle data, which the trainee should identify; a non-violation judgment indicates that the parameter value combination belongs to the allowed combination range, representing that the current data is self-consistent under standard constraints; the generation logic of the first type of logical conflict is to directly map the violation judgment result to a test event, which serves to verify the trainee's understanding of physical constraints and system design constraints; the method for obtaining alternative training vehicles is to search the vehicle repair shop's database of vehicles under repair for vehicles with the same model identification as the current training vehicle but with significantly different mileage or repair history; The purpose of performing the same data variation operation is to reproduce the test conditions of the same cognitive weakness on a carrier with a deviated technical state. If the numerical combination does not violate the constraints, time series data is obtained because the parameters appear self-consistent at this time, but the temporal correlation has undergone implicit distortion. Time series data refers to the continuous sampled values ​​of each parameter changing over time under the same operating condition. The cross-correlation function value is used to determine the degree to which the coordinated change pattern of the two parameters deviates from the factory state on the time axis. The higher the cross-correlation function value, the more synchronized the waveforms of the two parameters are; the lower the value, the weaker the synchronicity. The peak delay time reflects the change of one parameter relative to the other parameter. The method for setting the lag degree and the preset normal range is to collect a sufficient number of cross-correlation function values ​​of the same type of normal vehicles under the same working conditions, calculate the mean and standard deviation, and use the standard deviation plus or minus a preset multiple as the boundary of the normal range. The sufficient number is determined according to the confidence level and allowable error in statistics, or by referring to the common experience data in the vehicle repair industry. The logic for judging parameter constraint mismatch is that the cross-correlation function value exceeds the normal range. After judging the mismatch, a second type of logical conflict is generated to verify the trainee's ability to judge the consistency of the constraint relationship between multiple observable parameters when the technical state of the training vehicle deviates from the preset standard constraint set. The first type of logic... The first type of conflict test examines the rule memorization and retrieval in standard scenarios, while the second type of logical conflict test examines the adaptability of constraint relationships in non-standard scenarios. Taking the variation of water temperature sensor data as an example, after modifying the water temperature signal to 130 degrees Celsius, the first type of logical conflict is manifested in the fact that the water temperature value (130℃) and the intake air temperature value (30℃) violate the allowable temperature difference range specified in the vehicle maintenance manual under warm-up conditions. The second type of logical conflict is manifested in the fact that the time-series cross-correlation function value of water temperature and fuel injection pulse width in older vehicles deviates from the normal range due to component aging. Thus, the trainee's abstract reasoning ability of constraint relationships is externalized as data on the identification and attribution behavior of logical conflicts.

[0027] It should be noted that logical conflict refers to the contradiction between the mutated parameters and the relevant normal parameters that violates physical or design rules. It is divided into two categories: static numerical conflict and dynamic timing mismatch. For example, the temperature difference between the water temperature of -20℃ and the intake air temperature of 5℃ is too large (static conflict); the timing relationship between the rise of water temperature and the change of fuel injection pulse width is abnormal in older vehicles (dynamic mismatch).

[0028] S4: Based on the differences in trainees' operational behavior under information-masking conditions and without information-masking conditions after the effect of logical conflict, evaluate the training program's effect on shaping trainees' ability to autonomously generate diagnostic strategies, including: After the generated first and second type of logical conflicts are applied to the trainees who have completed the data mutation verification, an information masking operation is applied to the trainees. Information masking operations include information masking during the period when the first type of logical conflict and the second type of logical conflict are acting simultaneously, and information masking during the period when the first type of logical conflict and the second type of logical conflict are acting at intervals. Simultaneous information masking during the same period refers to hiding the real-time value of at least one observable parameter or hiding at least one physical constraint relationship description during the process of students processing the first type of logical conflict and the second type of logical conflict in parallel. Information masking during the interval refers to hiding the historical time series data of the observable parameters involved in the second type of logical conflict within a preset time period after the trainee completes the first type of logical conflict verification and before the second type of logical conflict verification begins. It should be noted that the information masking during the interval is only used to test the possibility that trainees can infer the timing anomaly from the static parameter combination at the current moment without relying on historical data. In order to eliminate the contradiction with the original design intention of the second type of conflict, in the actual implementation of this method, the information masking operation during the interval is limited to the verification stage of the second type of logical conflict, while the generation stage of the second type of logical conflict (i.e., the calculation of the cross-correlation function value) still uses complete historical time series data; that is, the data is complete when the logical conflict is generated, and the historical data is masked only when testing trainees.

[0029] Acquire trainees’ operational behavior data on handling first-type and second-type logical conflicts under conditions without information masking, and use it as baseline behavior data; Acquire behavioral data of trainees handling first-type and second-type logical conflicts under conditions of information masking, as comparative behavioral data; The behavioral data to be compared with the baseline behavioral data were compared. The differences included differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy. The difference in diagnostic path length is obtained by subtracting the total number of diagnostic steps in the baseline behavioral data from the total number of diagnostic steps in the comparative behavioral data. The difference in diagnostic time was obtained by subtracting the total diagnostic time from the baseline behavioral data from the total diagnostic time in the comparative behavioral data. The difference in diagnostic accuracy was obtained by subtracting the percentage of correct diagnostic conclusions from the percentage of correct diagnostic conclusions in the baseline behavioral data from the percentage of correct diagnostic conclusions in the comparative behavioral data. The ability of trainees to autonomously generate diagnostic strategies under conditions of incomplete information is determined by comparing the differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy with preset difference thresholds. If the difference in diagnostic path length is less than or equal to the first preset threshold, the difference in diagnostic time is less than or equal to the second preset threshold, and the difference in diagnostic conclusion accuracy is greater than or equal to the third preset threshold, then the trainee is deemed to have the ability to generate diagnostic strategies autonomously. If the difference in diagnostic path length is greater than the first preset threshold, the difference in diagnostic time is greater than the second preset threshold, or the difference in diagnostic conclusion accuracy is less than the third preset threshold, then the trainee is deemed not to have the ability to generate diagnostic strategies independently.

[0030] It should be noted that information masking refers to deliberately removing some key information elements during the process of handling logical conflicts, thereby artificially creating a diagnostic task environment with incomplete information. Specifically, information masking means intentionally hiding some real-time parameters, constraint descriptions, or historical waveform data during the trainee's diagnostic process to test their reasoning ability under incomplete information; for example, not displaying the fuel injection pulse width value or hiding the water temperature change curve of the past 5 minutes. The reason for distinguishing between the two time periods of simultaneous action and interval action is that the former is intended to reproduce the complex situation of information channel congestion and signal interference in the scenario of multiple faults occurring simultaneously, while the latter aims to simulate the actual working condition of intermittent concealment of fault phenomena or loss of historical operating data fragments.

[0031] Baseline behavioral data, obtained by recording trainees' complete operational sequences in response to two types of logical conflicts under a standard situation with complete and unobstructed information presentation, primarily serves to establish a reference level of trainees' abilities when information is abundant. Comparative behavioral data, collected after information censorship is applied, is used to capture operational strategy adjustments and performance fluctuations caused by information deprivation. The difference in diagnostic path length measures the net impact of information deficiency on the number of investigation steps; the difference in diagnostic time reflects the hindering effect of the interfering environment on handling efficiency; and the difference in diagnostic conclusion accuracy characterizes the impact of information censorship on the final outcome. To determine the degree of reliability degradation, the calibration scheme for the above three thresholds involves first collecting data on the performance differences of multiple senior maintenance technicians under the same occlusion conditions, statistically calculating the mean and standard deviation, and taking the mean as the critical value. The technicians collected should cover different professional fields and years of experience to ensure the thresholds have broad representativeness. If the trainee's performance on all three indicators is not significantly worse than the critical value, it indicates that the information deficiency has not caused a substantial degradation in behavioral performance. At this point, it is determined that the trainee has already acquired the ability to autonomously generate diagnostic strategies, thus confirming the effectiveness of the training program in cultivating higher-order thinking abilities. Conversely, if any one of the three indicators is significantly worse than the critical value, it indicates that the information deficiency has not caused a substantial degradation in behavioral performance. If the indicator exceeds the critical value, it indicates that the trainees are not yet able to maintain a stable diagnostic level in an information-limited environment, and the teaching effectiveness of the training program in the strategy self-construction stage needs to be strengthened. Through this comparative attribution analysis, step S4 transforms the previously unobservable autonomous diagnostic thinking into a quantifiable measure of behavioral deviation, providing clear data support for the iterative optimization of the training program. It should be noted that the evaluation of the training program is achieved in the following way: all trainees under the same training program are assessed one by one according to the above method, and the proportion of trainees with the ability to generate diagnostic strategies autonomously is counted. If this proportion is significantly higher than the preset pass rate threshold (e.g., 90%), the training program is deemed effective. If it is necessary to compare the advantages and disadvantages of two different training programs, the pass rate of trainees under the two programs is counted separately, and the statistical significance of the difference is judged by the proportional hypothesis test (e.g., chi-square test). The preset pass rate threshold is set according to the training objectives, such as national vocational skills assessment standards or internal enterprise assessment requirements. The length of the preset time period is the same as the time window length used when calculating the cross-correlation function. The specific duration is preset by the testers according to the actual working conditions, so as to cover the process from the beginning of parameter variation to the tendency to stabilize.

[0032] Example 2: Based on Example 1, a training program evaluation system for maintenance technicians, such as... Figure 2 As shown, it includes: The training program feature extraction module is used to extract the data variation of continuous stage boundaries and baseline cognitive units and construct a single observable parameter; The logic conflict generation module is used to match the constraint relationship between mutated parameters and unmutated parameters to generate logic conflicts; The information masking and behavior collection module is used to apply information masking and collect student operation behavior data; The competency development assessment module is used to compare the differences in operational behavior under conditions of information masking and without information masking in order to evaluate the effectiveness of the training program.

[0033] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for evaluating a training program for maintenance technicians, characterized in that, Includes the following steps: S1: Based on the leap characteristics of trainees' cognitive structure during the formation of maintenance skills, the training program is divided into N continuous stages with the qualitative change point of ability as the boundary; S2: At the boundary of qualitative change in ability between adjacent consecutive stages, based on the degree of influence of the benchmark cognitive unit corresponding to the boundary on subsequent consecutive stages, data variation acting on a single observable parameter is constructed; the benchmark cognitive unit refers to a single cognitive element that is at the boundary of qualitative change in ability and provides basic support for learning in subsequent stages. S3: Based on the constraint relationship matching results between the single observable parameter involved in data variation and the associated unvaried observable parameter, generate logical conflicts that act on the trainees. S4: Based on the differences between trainees' operational behavior under information-masked conditions and their operational behavior under conditions without information-masked conditions after the effect of logical conflict, evaluate the training program's effect on shaping trainees' ability to autonomously generate diagnostic strategies.

2. The method for evaluating a training program for maintenance technicians according to claim 1, characterized in that, At the boundary of qualitative change in capabilities between adjacent consecutive stages, based on the degree of influence of the baseline cognitive unit corresponding to the boundary on subsequent consecutive stages, the data variation acting on a single observable parameter is constructed, including: Obtain the frequency of invocation of the baseline cognitive unit corresponding to the boundary in subsequent consecutive stages; Based on the causal correspondence between the technical principles described by the benchmark cognitive unit and the observable parameters at the physical level, the correlation between the benchmark cognitive unit and a single observable parameter is established.

3. The method for evaluating a training program for maintenance technicians according to claim 2, characterized in that, The establishment of the correlation between the baseline cognitive unit and a single observable parameter includes: The benchmark cognitive unit consists of a single cognitive element that is at the boundary of qualitative change in ability in a continuous stage and provides basic support for the learning effect in subsequent continuous stages. Based on the nonlinear cumulative law of the influence of the frequency of calls to the baseline cognitive unit in subsequent continuous stages on the deviation of a single observable parameter, a nonlinear correspondence between the call frequency and the deviation of a single observable parameter is established. Extract the baseline cognitive unit corresponding to the boundary, determine the single observable parameter associated with the baseline cognitive unit based on the correlation relationship, determine the deviation of the single observable parameter based on the nonlinear correspondence relationship, modify the value of the single observable parameter to deviate from the standard range, and maintain the physical possibility of the modified single observable parameter value.

4. The method for evaluating a training program for maintenance technicians according to claim 1, characterized in that, The matching results of the constraint relationships between the single observable parameter involved in data variation and the associated unvaried observable parameter generate logical conflicts that act on the trainees, including: Obtain K unmutated observable parameter values ​​that have physical or system design constraints with the mutated single observable parameter value, where K is an integer greater than or equal to one; Combine the mutated single observable parameter value with K unmutated observable parameter values ​​to form a parameter value combination; The parameter value combination is matched with a preset standard constraint set, which includes the range of allowable numerical combinations specified by the physical constraint relationship between observable parameters and the system design constraint relationship; If the combination of parameter values ​​does not fall within the range of allowed value combinations specified by the preset standard constraint set, then the combination of parameter values ​​is determined to violate physical constraints or system design constraints. If the combination of parameter values ​​falls within the range of allowed combinations of values ​​specified by the preset standard constraint set, then the combination of parameter values ​​is determined to not violate physical constraints and not violate system design constraints. Based on the determination result of whether the combination of parameter values ​​violates physical constraints or system design constraints, a first type of logical conflict is generated; Based on the result of the parameter constraint mismatch determination, a second type of logical conflict is generated.

5. The method for evaluating a training program for maintenance technicians according to claim 4, characterized in that, The generation of a second type of logical conflict based on the parameter constraint mismatch determination result includes: Obtain alternative training vehicles that have the same vehicle model identification but different technical status as the training vehicles used in the first type of logical conflict assessment task; Perform the same data mutation operation on the alternative training platform; If the numerical combination between the mutated single observable parameter value and the K unmutated observable parameter values ​​does not violate physical constraints and system design constraints, then time series data of the mutated single observable parameter and the K unmutated observable parameters under the same operating condition are obtained. Calculate the cross-correlation function between the mutated single observable parameter time series data and the K unmutated observable parameter time series data; If the cross-correlation function value deviates from the preset normal range, it is determined that there is a parameter constraint mismatch between the mutated single observable parameter and the K unmutated observable parameters.

6. The method for evaluating a training program for maintenance technicians according to claim 4, characterized in that, The generation of a second type of logical conflict based on the parameter constraint mismatch determination result includes: Map the result of parameter constraint mismatch to a second type of logical conflict event; The second type of logical conflict and the generated first type of logical conflict are applied to the trainees who have completed the data variation verification. The first type of logical conflict is used to verify the trainees' mastery of physical constraints and system design constraints, while the second type of logical conflict is used to verify the trainees' ability to judge the consistency of constraints among multiple observable parameters when the technical state of the training carrier deviates from the preset standard constraint set.

7. The method for evaluating a training program for maintenance technicians according to claim 4, characterized in that, The evaluation of the training program's effect on shaping trainees' ability to autonomously generate diagnostic strategies is based on the differences between trainees' operational behaviors under information-masking conditions and those under conditions without information-masking, following the application of logical conflict. After the generated first and second type of logical conflicts are applied to the trainees who have completed the data mutation verification, an information masking operation is applied to the trainees. Information masking operations include information masking during the period when the first type of logical conflict and the second type of logical conflict are acting simultaneously, and information masking during the period when the first type of logical conflict and the second type of logical conflict are acting at intervals. Simultaneous information masking during the same period refers to hiding the real-time value of at least one observable parameter or hiding at least one physical constraint relationship description during the process of students processing the first type of logical conflict and the second type of logical conflict in parallel. Information masking during the interval refers to hiding the historical time series data of the observable parameters involved in the second type of logical conflict within a preset time period after the trainee completes the first type of logical conflict verification and before the second type of logical conflict verification begins. Acquire trainees’ operational behavior data on handling first-type and second-type logical conflicts under conditions without information masking, and use it as baseline behavior data; Acquire behavioral data of trainees handling first-type and second-type logical conflicts under conditions of information masking, as comparative behavioral data; The behavioral data for comparison were compared with the baseline behavioral data. The differences included differences in diagnostic path length, diagnostic time, and diagnostic accuracy.

8. The method for evaluating a training program for maintenance technicians according to claim 7, characterized in that, The differences compared include differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy, including: The difference in diagnostic path length is obtained by subtracting the total number of diagnostic steps in the baseline behavioral data from the total number of diagnostic steps in the comparative behavioral data. The difference in diagnostic time was obtained by subtracting the total diagnostic time from the baseline behavioral data from the total diagnostic time in the comparative behavioral data. The difference in diagnostic accuracy was obtained by subtracting the percentage of correct diagnostic conclusions from the percentage of correct diagnostic conclusions in the baseline behavioral data from the percentage of correct diagnostic conclusions in the comparative behavioral data. The ability of trainees to autonomously generate diagnostic strategies under conditions of incomplete information is determined by comparing the differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy with preset difference thresholds.

9. The method for evaluating a training program for maintenance technicians according to claim 8, characterized in that, The method of determining the trainee's ability to autonomously generate diagnostic strategies under conditions of incomplete information, based on comparisons of differences in diagnostic path length, diagnostic time, and diagnostic conclusion accuracy with preset difference thresholds, includes: If the difference in diagnostic path length is less than or equal to the first preset threshold, the difference in diagnostic time is less than or equal to the second preset threshold, and the difference in diagnostic conclusion accuracy is greater than or equal to the third preset threshold, then the trainee is deemed to have the ability to generate diagnostic strategies autonomously. If the difference in diagnostic path length is greater than the first preset threshold, the difference in diagnostic time is greater than the second preset threshold, or the difference in diagnostic conclusion accuracy is less than the third preset threshold, then the trainee is deemed not to have the ability to generate diagnostic strategies autonomously.

10. A system for evaluating training programs for maintenance technicians, used to implement the method for evaluating training programs for maintenance technicians as described in any one of claims 1-9, characterized in that, include: The training program feature extraction module is used to extract the data variation of continuous stage boundaries and baseline cognitive units and construct a single observable parameter; The logic conflict generation module is used to match the constraint relationship between mutated parameters and unmutated parameters to generate logic conflicts; The information masking and behavior collection module is used to apply information masking and collect student operation behavior data; The competency development assessment module is used to compare the differences in operational behavior under conditions of information masking and without information masking in order to evaluate the effectiveness of the training program.

Images