Error-proof design effect evaluation experiment method and system

Abstract

The invention discloses an error-proof design effect evaluation experiment method and system, and the method comprises the steps: generating an operation scene with an error-proof mechanism and without an error-proof mechanism through a simulation system, obtaining operation behavior data and video records, and generating a multi-dimensional data set and a video data set; extracting behavior features from the multi-dimensional data set, and generating a classified behavior feature set; determining a behavior mode set of different experience levels according to the classified behavior feature set; generating a synchronous behavior video set through the video data set and the behavior mode set; counting a difference feature set under an error prevention mechanism and a difference feature set under a non-error prevention mechanism according to the synchronization behavior video set; and determining an influence evaluation result of the error prevention mechanism on the operation behavior from the difference feature set. Therefore, the error-proof design effect evaluation experiment method effectively evaluates the applicability of an error-proof mechanism in different pilot groups, and provides important support for improving the flight safety.

Description

Technical Field

[0001] The present invention relates to the technical field of error-proofing effect evaluation, and in particular to an error-proofing design effect evaluation experimental method and system. Background Art

[0002] As a key research field, aviation human-computer interaction design is directly related to the improvement of flight safety and operational efficiency, and its importance is self-evident. In the modern aviation industry, the interaction design between pilots and cockpit systems not only affects the accuracy of task execution, but also determines the safety of life and property in extreme cases. However, many current evaluation methods are still at the level of theoretical analysis or single experiments, which makes it difficult to fully reflect the complex scenarios in actual operations. Especially in the verification of the effect of error-proofing design, there are generally problems such as insufficient sample coverage, single experimental tasks, and insufficient in-depth data analysis. These limitations lead to insufficient basis for design optimization, which in turn affects the reliability and practicality of the cockpit system.

[0003] Behind the shortcomings of existing methods, the core challenges faced by aviation cockpit error-proofing design evaluation have gradually become prominent. Among them, the diversity of flight crew operating behaviors, the effectiveness verification of error-proofing mechanisms, and the identification and quantification of unconscious operations have become the three major technical focuses. Due to the differences in experience and reaction patterns among different flight crews, error-proofing designs that have not been fully verified may fail in actual applications; at the same time, the lack of systematic capture and analysis of unconscious operations makes it difficult for design improvements to reduce human errors in a targeted manner. These technical factors have not been properly addressed, making it difficult for evaluation methods to accurately measure the actual effects of error-proofing designs, which in turn leads to the unique problem of how to scientifically verify designs in dynamic and complex environments.

[0004] Therefore, how to accurately identify unconscious operations and quantify the actual effect of error-proofing design by comparing the operating performance of different flight crew groups with and without error-proofing mechanisms in a simulated aircraft cockpit system, combined with multi-dimensional experimental data and video analysis, has become a key issue that needs to be overcome in this study. Solving this problem will directly promote cockpit interaction design to a higher level of safety and adaptability. Summary of the invention

[0005] The objective of the present invention is to provide an experimental method and system for evaluating the effect of error-proof design. By simulating a comparative experimental scenario with an error-proof mechanism and a non-error-proof mechanism, combined with multi-dimensional behavioral data (such as operation interval, error rate, panel switching frequency) and video frame difference analysis technology, it realizes the accurate capture and quantitative characterization of pilots' unconscious operations. At the same time, it establishes a parametric relationship model between the error-proof mechanism and task accuracy. The present invention can not only dynamically evaluate the differences in operation modes of pilots with different experience levels, but also quantify the actual intervention effect of error-proof design through time series analysis and clustering algorithms, providing an optimization basis based on objective experimental data for aviation cockpit interaction design, and ultimately promoting the evolution of the man-machine system towards higher safety redundancy and operation adaptability.

[0006] To achieve the above objective, in the first aspect, the present invention provides an experimental method for evaluating the effect of error-proof design, including: generating operation scenarios with an error-proof mechanism and a non-error-proof mechanism through a simulation system, obtaining operation behavior data and video records, and generating a multi-dimensional data set and a video data set. Extracting behavioral characteristics from the multi-dimensional data set to generate a classified set of behavioral characteristics. Determining a set of behavior patterns for different experience levels according to the classified set of behavioral characteristics. Generating a synchronized behavior video set through the video data set and the set of behavior patterns. Counting a set of difference characteristics under the error-proof mechanism and the non-error-proof mechanism according to the synchronized behavior video set. Determining the evaluation result of the influence of the error-proof mechanism on operation behavior from the set of difference characteristics. Extracting operation behavior difference characteristics and time series data from the multi-dimensional data set to determine a parametric representation of the diversity influence. Detecting unconscious operation segments through the video data set to obtain a quantitative feature set of unconscious operations. Extracting behavior response data related to the error-proof mechanism from the quantitative feature set to determine intervention effect parameters. Fitting a relationship model between the error-proof mechanism and task accuracy through the operation behavior data and task accuracy indicators to generate a quantitative result of the design effect.

[0007] In a second aspect, the present invention provides an anti-error design effect evaluation experimental system, including: a data set generation module, a behavior feature set generation module, a pattern set determination module, a behavior video set generation module, a statistics module, an impact on evaluation result determination module, a parameterized representation determination module, a quantization feature set acquisition module, an intervention effect parameter determination module, and a quantization result generation module. The data set generation module is used to generate operation scenarios with and without an anti-error mechanism through a simulation system, obtain operation behavior data and video records, and generate a multi-dimensional data set and a video data set. The behavior feature set generation module is used to extract behavior features from the multi-dimensional data set and generate a classified behavior feature set. The pattern set determination module is used to determine a set of behavior patterns at different experience levels according to the classified behavior feature set. The behavior video set generation module is used to generate a synchronized behavior video set through the video data set and the behavior pattern set. The statistics module is used to statistically analyze a difference feature set under the anti-error mechanism and without the anti-error mechanism according to the synchronized behavior video set. The impact on evaluation result determination module is used to determine the evaluation result of the impact of the anti-error mechanism on the operation behavior from the difference feature set. The parameterized representation determination module is used to extract operation behavior difference features and time series data from the multi-dimensional data set and determine a parameterized representation of the diversity impact. The quantization feature set acquisition module is used to detect unconscious operation segments according to the video data set and obtain a quantization feature set of unconscious operations. The intervention effect parameter determination module is used to extract behavior response data related to the anti-error mechanism from the quantization feature set and determine an intervention effect parameter. The quantization result generation module is used to fit a relationship model between the anti-error mechanism and the task accuracy through the operation behavior data and the task accuracy index, and generate a quantization result of the design effect.

[0008] Compared with the prior art, the anti-error design effect evaluation experimental method and system according to the present invention have the following beneficial effects:

[0009] 1. By generating operation scenarios with and without an anti-error mechanism through a simulation system, it is possible to systematically compare and analyze the differences in operation behaviors under the two mechanisms; through the generation and analysis of the multi-dimensional data set and the video data set, comprehensive data support is provided, making the evaluation results more accurate and reliable.

[0010] 2. Extracting behavior features from the multi-dimensional data set, generating a classified behavior feature set, and determining a set of behavior patterns at different experience levels according to these feature sets, this refined analysis helps to deeply understand the differences in operation behaviors of different pilots; generating a synchronized behavior video set through the video data set and the behavior pattern set can visually observe and analyze the operation behaviors, further improving the accuracy of the evaluation.

[0011] 3. It is capable of detecting unconscious operation segments and obtaining a quantitative feature set of unconscious operations, which is crucial for evaluating the effectiveness of the error-proofing mechanism in reducing unconscious operations.

[0012] 4. It takes into account the diversity of operation behaviors. By extracting the differential features of operation behaviors and time series data, it determines a parametric representation of the diversity impact, which helps to design an error-proofing mechanism that better adapts to the needs of different pilots.

[0013] 5. By fitting the relationship model between the error-proofing mechanism and task accuracy, it generates a quantitative result of the design effect. This scientific method provides an objective evaluation basis and reduces the influence of subjective judgment.

[0014] 6. The present invention is not only applicable to the aviation field, but also can be extended to other fields that require error-proofing design, such as medical treatment, industrial control, etc. Its wide applicability makes the research results have higher practical value.

[0015] 7. It solves the problems existing in the current evaluation methods, such as insufficient sample coverage, single experimental task, and insufficient in-depth data analysis, and provides new ideas and methods for evaluating the effectiveness of error-proofing design. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flowchart of an experimental method for evaluating the error-proofing design effect according to an embodiment of the present invention;

[0017] Figure 2 is a schematic wireframe diagram of the structure of an experimental system for evaluating the error-proofing design effect according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] The following will describe in detail the specific embodiments of the present invention with reference to the accompanying drawings. It should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0019] Unless otherwise clearly stated, in the whole specification and claims, the term "comprising" or its variations such as "comprises" or "including" etc. will be understood to include the stated elements or components, without excluding other elements or other components.

[0020] For the convenience of understanding, the main implementation concepts of the embodiments of the present invention will be briefly described first.

[0021] Taking aviation human-computer interaction design as a key research area directly affects flight safety and operation efficiency. However, current evaluation methods have problems such as insufficient sample coverage, single experimental tasks, and insufficient in-depth data analysis. These limitations lead to insufficient basis for design optimization, affecting the reliability and practicality of cockpit systems. Specifically, the diversity of flight crew operation behaviors, the effectiveness verification of error prevention mechanisms, and the identification and quantification of unconscious operations have become three major technical focuses. Due to differences in experience and reaction patterns among different flight crew, unvalidated error prevention designs may fail in actual applications. At the same time, the lack of systematic capture and analysis of unconscious operations makes it difficult to target design improvements to reduce human errors. These technical factors have not been properly addressed, resulting in the evaluation method being unable to accurately measure the actual effects of error prevention designs, and thus giving rise to the unique problem of how to scientifically verify designs in a dynamic and complex environment.

[0022] In modern aviation cockpit systems, the technical challenges of error prevention design evaluation are particularly prominent. For example, in a typical flight simulation training scenario, the system needs to process the operation data of multiple pilots simultaneously. Among them, pilot A may have rich flight experience, while pilot B is a novice. When the two operate the same cockpit system separately, the system needs to capture and analyze the differences in their operation behaviors. Specifically, the system needs to record multi-dimensional data such as operation interval time, error trigger times, and panel switching frequencies. However, due to the lack of effective data synchronization and analysis methods, the system is difficult to accurately identify the changes in operation patterns caused by experience differences. In addition, during simulated emergency situations, pilots may perform unconscious operations, such as repeatedly pressing a certain button or ignoring key warning messages. These unconscious operations are often difficult to capture through conventional data analysis methods, resulting in a lack of pertinence in the design of error prevention mechanisms. At the same time, the system also needs to consider the task characteristics of different flight phases, such as operation differences during takeoff, cruise, and landing, which further increases the complexity of data analysis.

[0023] If these technical problems cannot be effectively solved, they will have a serious impact on aviation safety and flight efficiency. First, due to the inability to accurately evaluate the actual effect of error-proofing design, some potential operational risks may be ignored. For example, a seemingly reasonable error-proofing design may increase the complexity of operation and the cognitive load of pilots under certain conditions. Secondly, the lack of accurate identification and quantification of unconscious operations makes it difficult for the system to optimize the interface design and operation process in a targeted manner, which may lead to the continued existence of human errors. Furthermore, if the operational differences of pilots with different experience levels cannot be effectively captured and analyzed, it will be difficult to design an adaptable error-proofing mechanism, which may cause the system to perform inconsistently when facing different users. Finally, due to the lack of comprehensive multidimensional data analysis capabilities, it is difficult for the system to establish an accurate relationship model between the error-proofing mechanism and task accuracy, which will directly affect the overall performance evaluation of the cockpit system and the determination of the optimization direction. Therefore, there is an urgent need for an error-proofing design effect evaluation method that can comprehensively consider factors such as multidimensional data, video analysis, and unconscious operation recognition to improve the safety and adaptability of the cockpit system.

[0024] Therefore, the present invention proposes a new error-proofing design effect evaluation experimental method and system, which aims to comprehensively and accurately evaluate the error-proofing design effect of aircraft cockpit control devices in a scientific and objective manner, and provide strong support for the optimization of cockpit interaction design.

[0025] Embodiment 1

[0026] Figure 1 is a flow chart of an error-proofing design effect evaluation experimental method according to an embodiment of the present invention, such as Figure 1As shown in the figure, Embodiment 1 provides a method for evaluating the effect of error-proof design, including: Step S100, generating operation scenarios with and without error-proof mechanisms through a simulation system, obtaining operation behavior data and video records, and generating a multi-dimensional data set and a video data set; Step S200, extracting behavior features from the multi-dimensional data set to generate a classified behavior feature set; Step S300, determining a set of behavior patterns for different experience levels according to the classified behavior feature set; Step S400, generating a synchronized behavior video set through the video data set and the set of behavior patterns; Step S500, statistically analyzing the difference feature set under the error-proof mechanism and without the error-proof mechanism according to the synchronized behavior video set; Step S600, determining the evaluation result of the influence of the error-proof mechanism on operation behavior from the difference feature set; Step S700, extracting operation behavior difference features and time series data from the multi-dimensional data set to determine a parametric representation of the diversity influence; Step S800, detecting unconscious operation segments through the video data set to obtain a quantitative feature set of unconscious operations; Step S900, extracting behavior response data related to the error-proof mechanism from the quantitative feature set to determine the intervention effect parameter; Step S1000, fitting a relationship model between the error-proof mechanism and task accuracy through the operation behavior data and task accuracy indicators to generate a quantitative result of the design effect.

[0027] Specifically, when solving the problem of evaluating the error-proof design of an aircraft cockpit, the present invention first considers how to comprehensively capture and analyze the operation behavior of pilots. Specifically, operation scenarios with and without error-proof mechanisms are generated through a simulation system, operation behavior data and video records are obtained, and a multi-dimensional data set and a video data set are generated. This method can collect both quantitative and qualitative data, providing a comprehensive basis for subsequent analysis.

[0028] Next, the present invention focuses on solving how to extract meaningful behavior features from a large amount of data. By extracting behavior features from the multi-dimensional data set, a classified behavior feature set is generated. This step can effectively convert the original data into analyzable features, laying a foundation for subsequent behavior pattern recognition.

[0029] After determining the behavior features, the present invention considers how to distinguish the behavior of pilots with different experience levels. A set of behavior patterns for different experience levels is determined according to the classified behavior feature set. This step can help us understand the influence of experience on operation behavior, so as to design a more adaptable error-proof mechanism.

[0030] To analyze the operation behavior more intuitively, the present invention proposes to generate a synchronized behavior video set through the video data set and the set of behavior patterns. This method synchronizes quantitative data with video data, enabling analysts to more comprehensively understand the context of operation behavior.

[0031] After obtaining the synchronized behavior video set, the present invention proceeds to solve the problem of how to quantify the effect of the anti-error mechanism. By statistically analyzing the difference feature sets with and without the anti-error mechanism and determining the evaluation result of the influence of the anti-error mechanism on the operation behavior from them. This step can directly quantify the effect of the anti-error design and provide a basis for subsequent optimization.

[0032] Considering the diversity of operation behaviors, the present invention proposes to extract the operation behavior difference features and time series data through a multi-dimensional data set to determine the parametric representation of the diversity influence. This method can capture the operation differences between different pilots and help design a more flexible anti-error mechanism.

[0033] Aiming at the difficulty of unconscious operations, the present invention proposes to detect unconscious operation segments according to the video data set and obtain the quantified feature set of unconscious operations. Extract the behavior response data related to the anti-error mechanism from the quantified feature set to determine the intervention effect parameters. This step can effectively identify and quantify unconscious operations and provide a targeted optimization direction for the anti-error design.

[0034] Finally, in order to comprehensively evaluate the effect of the anti-error design, the present invention proposes to fit the relationship model between the anti-error mechanism and the task accuracy through the operation behavior data and the task accuracy index to generate the quantified result of the design effect. This method can establish a direct connection between the anti-error mechanism and the task execution effect and provide a scientific basis for design optimization.

[0035] Among them, the simulation system refers to the software and hardware system used to simulate the aircraft cockpit environment, which can be specifically implemented by virtual reality technology or physical simulation cabins. Among them, the multi-dimensional dataset refers to a data set containing multiple dimensions such as operation time, operation type, operation frequency, etc., which can be specifically stored and managed using a structured database. Among them, the video dataset refers to a set of video files recording the operation process, which can be specifically recorded using high-definition camera equipment. Among them, the behavior characteristics refer to the key indicators that can characterize the operation behavior, which can be specifically represented by the operation interval time, the number of error triggers, the panel switching frequency, etc. Among them, the behavior pattern set refers to the typical operation patterns of operators with different experience levels, which can be specifically obtained by classifying the behavior characteristics using a clustering algorithm. Among them, the synchronized behavior video set refers to a data set in which the operation behavior data is synchronized with the video record, which can be specifically realized using timestamp matching technology. Among them, the difference feature set refers to a set of key indicators reflecting the effect of the error prevention mechanism, which can be specifically extracted from the synchronized behavior video set using statistical analysis methods. Among them, the parametric representation of the diversity impact refers to a mathematical model used to describe the differences in operation behavior, which can be specifically realized using dimensionality reduction techniques such as principal component analysis. Among them, unconscious operations refer to unexpected operations performed by operators in an unconscious state, which can be specifically identified from video data using video analysis technology. Among them, the intervention effect parameter refers to an indicator used to quantify the intervention effect of the error prevention mechanism, which can be specifically represented by response time, error rate, etc. Among them, the task accuracy refers to the accuracy of completing the operation task, which can be specifically measured using indicators such as error rate and completion time.

[0036] The present invention generates operation scenarios with and without an error prevention mechanism through a simulation system, and at the same time obtains a multi-dimensional dataset and a video dataset, realizing a comprehensive capture of operation behaviors. By extracting behavior characteristics, determining behavior patterns, and generating a synchronized behavior video set, the impact of the error prevention mechanism is systematically analyzed. In particular, by detecting unconscious operation segments and quantifying their characteristics, a targeted optimization direction is provided for error prevention design. Finally, by establishing a relationship model between the error prevention mechanism and task accuracy, a quantitative evaluation of the design effect is realized.

[0037] The working principle of the present invention is first to generate an operation scenario through a simulation system. The simulation system can be a virtual reality environment or a physical simulation cabin, which can accurately simulate the operation environment of an aircraft cockpit. On this basis, the system records operation behavior data and video data simultaneously. The operation behavior data includes multiple dimensions such as operation time, type, frequency, etc., forming a multi-dimensional data set. The video data records the entire operation process through high-definition camera devices. Next, the system extracts behavior features from the multi-dimensional data set. This step uses data mining techniques to extract key indicators such as operation interval time, error trigger times, panel switching frequency, etc. After these features are standardized, clustering algorithms are used for classification to form a set of behavior patterns with different experience levels. Then, the system synchronizes the set of behavior patterns with the video data set. This step uses timestamp matching technology to ensure the precise correspondence between the behavior data and the video images. The synchronized data set provides a rich information source for subsequent analysis. Based on the synchronized behavior video set, the system statistically analyzes the difference features between the error prevention mechanism and the non-error prevention mechanism. This step quantifies the actual effect of the error prevention mechanism through comparative analysis. At the same time, the system also extracts the difference features and time series data of the operation behavior, and uses techniques such as principal component analysis to determine the parametric representation of the diversity impact. In particular, the system detects unconscious operation segments through video analysis techniques. This step can capture operation behaviors that are difficult to discover through conventional data analysis. The system further quantifies the features of these unconscious operations, providing a targeted optimization direction for error prevention design. Finally, the system conducts correlation analysis between the operation behavior data and the task accuracy indicators. By establishing a regression model, the system can quantify the relationship between the error prevention mechanism and the task accuracy, thereby generating a quantitative result of the design effect. The design of this working principle fully considers the comprehensiveness of the data and the depth of the analysis, and can effectively evaluate the actual effect of the error prevention design.

[0038] As a preferred implementation, the experimental method for evaluating the anti-error design effect of the present invention can be applied to the evaluation of the cockpit system of a certain model of civil airliner. Specifically, first, an operation scenario is constructed using a cockpit simulator based on virtual reality technology. The simulator accurately replicates the layout and functions of the real cockpit, including the main display screen, control panel, warning system, etc. During the experiment, 20 pilots with different experience levels are recruited to participate in the test. Among them, 10 are experienced senior pilots with flight hours exceeding 5000 hours; the other 10 are novice pilots with flight hours less than 500 hours. Each pilot completes the same flight tasks in two scenarios with and without an anti-error mechanism respectively. The tasks include three stages: takeoff, cruise, and landing, and each stage lasts for 15 minutes. During the experiment, the system records the operation behavior data in real time through multiple sensors, including button pressing time, switch switching frequency, screen touch position, etc. At the same time, a 4K high-definition camera records the operation process of the pilot from multiple angles. These data are synchronously stored in a high-performance server to form a multi-dimensional data set and a video data set. After the data collection is completed, machine learning algorithms are used to extract behavioral features from the multi-dimensional data set. Specifically, calculate the interval time of each operation, count the number of error triggers, analyze the panel switching mode, etc. After these features are standardized, the K-means clustering algorithm is used to divide the pilots into two groups with high and low experience levels. Next, the system uses timestamp matching technology to accurately align the behavior data with the video images to generate a synchronized behavior video set. On this basis, statistical analysis is carried out on the impact of the anti-error mechanism on various operation indicators, such as the percentage reduction in error rate, the degree of improvement in operation efficiency, etc. In particular, the system uses computer vision technology to analyze the video data to identify unconscious operation segments. For example, detect behaviors such as repeated ineffective button presses and ignoring key warning information. These unconscious operations are quantified as features such as frequency and duration. Finally, the system uses a multiple linear regression model to establish the relationship between the anti-error mechanism parameters and the task completion accuracy. The model inputs include the trigger frequency, response time, etc. of the anti-error mechanism, and the outputs are the error rate and time of task completion. Through this model, the actual effect of the anti-error design can be quantitatively evaluated and a basis for further optimization can be provided.

[0039] In this embodiment, the step S200 includes: step S201, obtaining the operation interval time, the number of error triggers, and the panel switching frequency through the multi-dimensional data set; step S202, using a standardization tool to process the operation interval time, the number of error triggers, and the panel switching frequency to generate standardized feature data; step S203, grouping according to the operation frequency and error rate through a clustering algorithm based on the standardized feature data to generate a classified set of behavioral features.

[0040] In some of the above embodiments, during the implementation of the present invention, there is still a problem of how to effectively extract and classify behavioral characteristics from a multi-dimensional dataset. In this regard, the present invention further proposes to obtain the operation interval time, the number of error triggers, and the panel switching frequency from the multi-dimensional dataset; use a standardization tool to process the operation interval time, the number of error triggers, and the panel switching frequency to generate standardized feature data; group according to the operation frequency and error rate through a clustering algorithm based on the standardized feature data to generate a classified behavioral feature set. This technical solution can effectively extract and classify behavioral characteristics through systematic processing and analysis of the multi-dimensional dataset, providing a more accurate and reliable data basis for the subsequent evaluation of the anti-error design effect.

[0041] Specifically, first, obtain the operation interval time, the number of error triggers, and the panel switching frequency from the multi-dimensional dataset. These metrics can comprehensively reflect the time characteristics, accuracy, and complexity of the operation behavior. Among them, the operation interval time reflects the coherence and proficiency of the operation, the number of error triggers directly reflects the accuracy of the operation, and the panel switching frequency reflects the complexity of the operation and the operator's attention allocation. Next, use a standardization tool to process these raw data. Standardization is an important data preprocessing step that can eliminate the dimensional differences between different metrics and make the data comparable. For example, the Z-score standardization method can be used to convert each metric into a standard normal distribution with a mean of 0 and a standard deviation of 1. The processed data is more suitable for subsequent clustering analysis. Finally, based on the standardized feature data, group according to the operation frequency and error rate through a clustering algorithm to generate a classified behavioral feature set. The clustering algorithm can choose methods such as K-means or hierarchical clustering according to the distribution characteristics of the data. Through clustering, data points with similar operation characteristics can be grouped into one category, thus forming different classifications of behavioral patterns.

[0042] The advantage of this method is that it can extract the most representative and distinguishable behavioral characteristics from a large amount of multi-dimensional data, and through mathematical and standardized processing, these characteristics have good comparability and analyzability. Through the clustering algorithm, these characteristics can be further classified into different behavioral patterns, which provides a scientific basis for the subsequent design and evaluation of the anti-error mechanism.

[0043] As a preferred implementation method, the following specific steps can be considered:

[0044] 1. Data acquisition: Extract the operation interval time series {t1, t2, …, tn}, the number of error triggers series {e1, e2, …, en}, and the panel switching frequency series {f1, f2, …, fn} from the multi-dimensional dataset.

[0045] 2. Standardization: Apply Z-score standardization to each sequence. Taking the operation interval time as an example, calculate the mean μt and standard deviation σt, and then standardize each time point ti: ti’ = (ti - μt) / σt. Similarly, process the number of error triggers and the panel switching frequency.

[0046] 3. Feature vector construction: For each operation sample, construct a three-dimensional feature vector (ti’, ei’, fi’).

[0047] 4. Cluster analysis: Use the K-means algorithm, set the number of clusters k (for example, k = 5), and cluster all feature vectors. The algorithm will return k cluster centers and the class labels of each sample.

[0048] 5. Generation of behavior feature set: According to the clustering results, take each cluster center as a typical behavior feature to generate a behavior feature set {C1, C2, …, Ck}, where each Ci represents a specific operation behavior pattern.

[0049] Through this method, complex multi-dimensional operation data can be transformed into a limited number of typical behavior patterns, greatly simplifying the subsequent analysis process. For example, it may be found that a behavior pattern shows "short operation interval time, few error triggers, and high panel switching frequency", which may represent a skilled but perhaps overly fast operation method. Another pattern may show "long operation interval time, many error triggers, and low panel switching frequency", which may represent an unskilled or fatigued operation state. This classified behavior feature set provides an important basis for the design and evaluation of the error-proof mechanism. For example, corresponding error-proof strategies can be designed for different behavior patterns, or the effectiveness of existing error-proof mechanisms for different behavior patterns can be evaluated. In addition, this method can also be used to monitor the change trend of operation behaviors and timely detect potential risk behaviors.

[0050] Based on the above analysis, it can be seen that the present invention is different from traditional methods that may only focus on a single indicator. This method comprehensively considers multiple dimensions such as operation interval time, number of error triggers, and panel switching frequency, and can more comprehensively characterize operation behaviors. Through standardization, the problem of inconsistent dimensions between different indicators is solved, making the data have better comparability and analyzability. Using the clustering algorithm to automatically identify different behavior patterns avoids the subjectivity and inconsistency that may be brought by manual division. This method has good scalability and can increase or adjust feature indicators according to needs to adapt to different application scenarios. By generating a classified behavior feature set, it provides a solid data foundation for subsequent quantitative analysis and evaluation.

[0051] In this embodiment, step S400 includes: step S401, obtaining the correspondence between the behavior pattern set and the video data set; step S402, aligning the operation behavior timestamps in the behavior pattern set with the video records in the video data set through the frame matching function to generate an initial synchronized behavior video set; step S403, if there is data missing in the initial synchronized behavior video set, using the interpolation method to fill in the missing operation logs and video gaps to generate the synchronized behavior video set.

[0052] In some of the above embodiments, during the implementation of the present invention, there is also a problem of how to accurately synchronize the behavior pattern and the video data to generate a high-quality synchronized behavior video set. In response to this, the present invention further proposes a technical solution for generating a synchronized behavior video set from the video data set and the behavior pattern set. This technical solution first obtains the correspondence between the behavior pattern set and the video data set, which lays the foundation for subsequent synchronization operations. By establishing a mapping between the behavior pattern and the video data, it can be ensured that the operation behavior can be accurately corresponded to the corresponding video segment in subsequent processing. Next, the frame matching function is used to align the operation behavior timestamps in the behavior pattern set with the video records in the video data set to generate an initial synchronized behavior video set. This step is the key to achieving accurate synchronization. The frame matching function may involve various algorithms, such as direct matching based on timestamps, image matching based on feature points, etc. Through these technologies, the time points of the operation behavior can be accurately corresponded to the video frames, thereby generating a preliminary synchronized behavior video set. After generating the initial synchronized behavior video set, the present invention also takes into account the possibility of data missing. If there is data missing in the initial synchronized behavior video set, the interpolation method is used to fill in the missing operation logs and video gaps, and finally a complete synchronized behavior video set is generated. The interpolation method may include techniques such as linear interpolation and spline interpolation, and the specific selection depends on the characteristics of the data and the missing situation. The advantage of this method is that it can handle the problem of incomplete data that may occur in actual operations, improving the integrity and continuity of the synchronized behavior video set. By filling in the missing data, it can be ensured that subsequent analysis will not be biased or incorrect due to data missing.

[0053] In practical applications, the method can be implemented as follows:

[0054] First, the corresponding relationship between the set of behavior patterns and the video dataset can be obtained by establishing a mapping table or database. For example, timestamps or unique identifiers can be used to associate each operation behavior with the corresponding video clip. Then, during the frame matching process, a timestamp-based matching algorithm can be adopted. Specifically, the timestamps of the operation behaviors can be compared with the timestamps of the video frames to find the closest matching points. To improve the accuracy, a time threshold can be set, such as ±0.1 seconds, and only the matches within this range are considered valid. For the case of missing data, linear interpolation methods can be used. For example, if there is a missing value between two known data points, the estimated value of the intermediate point can be calculated based on the values of these two points. For video gaps, the average value of the previous and next frames or more complex image interpolation techniques can be considered to fill in the gaps. Through this method, the present invention can generate a high-quality synchronized behavior video set, providing a reliable data basis for the subsequent evaluation of the anti-error design effect. Compared with the prior art, the method of the present invention can not only achieve accurate behavior-video synchronization, but also effectively handle the problem of missing data that may occur in actual operations, thereby improving the integrity of the data and the reliability of the analysis results.

[0055] In this embodiment, the step S500 includes: step S501, obtaining the standard deviation of the operation interval time and the mean value of the error trigger times through the synchronized behavior video set; step S502, generating a difference feature set according to the standard deviation of the operation interval time and the mean value of the error trigger times.

[0056] In some of the above embodiments, during the implementation of the present invention, there is also a problem of how to accurately count the difference features under the anti-error mechanism and the non-anti-error mechanism. In response to this, the present invention further proposes a technical solution for counting the difference feature sets under the anti-error mechanism and the non-anti-error mechanism through the synchronized behavior video set. The technical solution of the present invention obtains the standard deviation of the operation interval time and the mean value of the error trigger times through the synchronized behavior video set, and generates a difference feature set based on these data. This method can effectively capture the influence of the anti-error mechanism on the operation behavior, providing reliable data support for the subsequent evaluation.

[0057] Specifically, the present invention first obtains the standard deviation of the operation interval time through a synchronized behavior video set. The operation interval time reflects the rhythm and efficiency of the operator when performing tasks, and its standard deviation can reflect the operation stability. A smaller standard deviation usually indicates more consistent and stable operations, which may be the result of the anti-error mechanism taking effect. Secondly, the present invention obtains the mean value of the error trigger times. The error trigger times directly reflect the frequency of operation errors and are key indicators for evaluating the effect of the anti-error mechanism. By calculating the mean value, an overall error occurrence level can be obtained, which is convenient for comparing the performance under different conditions. Finally, the present invention generates a difference feature set based on the above two key indicators. This feature set combines information from two dimensions, operation stability and error frequency, and can comprehensively reflect the impact of the anti-error mechanism. The technical solution of the present invention can clearly show the effect of the anti-error design by comparing the difference feature sets in two cases, with and without the anti-error mechanism. For example, if in the case with the anti-error mechanism, the standard deviation of the operation interval time is smaller and the mean value of the error trigger times is also lower, it can be inferred that the anti-error mechanism has indeed improved the operation stability and accuracy.

[0058] Furthermore, the method of the present invention can also analyze the change trend of the difference features by setting different time windows. For example, the entire operation process can be divided into multiple time periods, and the difference feature sets for each period can be calculated separately. This can observe whether the effect of the anti-error mechanism changes over time and whether it is more significant in certain specific stages.

[0059] As a preferred implementation, the present invention can combine machine learning algorithms to analyze the difference feature set. For example, algorithms such as support vector machine (SVM) or random forest can be used, taking the difference feature set as input to predict whether the operation is under the protection of the anti-error mechanism. This method can automatically identify the features that can best distinguish the two cases, further improving the accuracy of the evaluation.

[0060] In addition, the technical solution of the present invention can also be used in combination with other evaluation methods. For example, the difference feature set can be combined with subjective scoring to comprehensively consider objective data and the operator's subjective feelings. This multi-dimensional evaluation method can provide a more comprehensive evaluation result of the anti-error design effect.

[0061] In a specific embodiment, it is assumed that an experiment on evaluating the effect of error-proofing design is carried out in a certain aviation cockpit simulation system. The experiment is divided into two groups, one with an error-proofing mechanism and the other without. Each group has 10 pilots participating, and each person completes 5 standard flight tasks. Through the synchronized behavior video set, the interval time of each operation and the error trigger situation are recorded. For the group without the error-proofing mechanism, the standard deviation of the operation interval time is calculated to be 2.5 seconds, and the average number of error triggers is 3.2 times per task. For the group with the error-proofing mechanism, the standard deviation of the operation interval time is 1.8 seconds, and the average number of error triggers drops to 1.5 times per task. These data are used to generate a differential feature set. By comparison, it is found that the operations of the group with the error-proofing mechanism are more stable (the standard deviation is reduced by 28%), and the error rate is significantly reduced (the average value is reduced by 53%). This clearly demonstrates the effectiveness of the error-proofing mechanism. Further analysis reveals that in the initial stage of the task (the first 20% of the time), the difference between the two groups is not obvious. However, in the middle and late stages of the task, especially during high-load operation periods, the effect of the error-proofing mechanism is more significant. This finding provides an important reference for optimizing the error-proofing design and indicates the time period that should be focused on.

[0062] Based on the above analysis, it can be seen that the present invention has the following advantages: First, by obtaining data through the synchronized behavior video set, the accurate correspondence between time and behavior is ensured, avoiding the problem of data asynchronization that may occur in traditional methods. Second, considering both the operation interval time and the number of error triggers in two dimensions provides a more comprehensive evaluation perspective. Finally, by generating a differential feature set, not only can the differences between the two situations be intuitively compared, but also a basis for subsequent in-depth analysis (such as machine learning) is provided. These advantages enable the method of the present invention to more accurately and comprehensively evaluate the effect of error-proofing design and provide strong support for optimizing the human-machine interaction design of the aviation cockpit.

[0063] In this embodiment, the step S700 includes: Step S701, denoising and normalizing the multi-dimensional data set to obtain a basic data set; Step S702, extracting time series data from the basic data set by the sliding window method; Step S703, separating differential features from the time series data by principal component analysis to determine the independent distribution mode of operation behaviors; Step S704, grouping according to the independent distribution mode by a clustering algorithm to generate a parameterized representation of the diversity impact.

[0064] In some of the above embodiments, during the implementation of the present invention, there is still a problem of how to accurately capture and quantify the impact of operation behavior diversity. In response to this, the present invention further proposes a technical solution for extracting operation behavior difference features and time series data from a multi-dimensional dataset to determine a parametric representation of the diversity impact. This technical solution aims to solve problems such as the difficulty in comprehensively reflecting the actual operation complex scenarios, insufficient sample coverage, and insufficient in-depth data analysis in the existing anti-error design effect evaluation methods. Through in-depth analysis and processing of the multi-dimensional dataset, this solution can more accurately capture the differences in operation behaviors among different flight crew, thereby providing more reliable and comprehensive data support for the evaluation and optimization of anti-error design.

[0065] Specifically, this technical solution includes the following steps:

[0066] First, denoise and normalize the multi-dimensional dataset to obtain a basic dataset. This step aims to eliminate noise interference in the original data and unify data with different dimensions to the same scale, laying a foundation for subsequent analysis. Denoising processes can use methods such as wavelet transform or median filtering, while normalization can use techniques such as min-max normalization or z-score standardization. Next, extract time series data from the basic dataset through a sliding window method. The size of the sliding window can be adjusted according to the characteristics of the specific operation task. For example, a time window of 5 seconds, 10 seconds, or longer can be selected. This method can capture the dynamic characteristics of operation behaviors changing over time, helping to identify short-term and long-term behavior patterns. Then, use principal component analysis (PCA) to separate difference features from the time series data and determine the independent distribution pattern of operation behaviors. PCA can effectively reduce the data dimension while retaining the most important variation information. By selecting an appropriate number of principal components (for example, explaining more than 80% of the variance), the key features of operation behaviors can be obtained. Finally, group them through a clustering algorithm according to the independent distribution pattern to generate a parametric representation of the diversity impact. Algorithms such as K-means or hierarchical clustering can be used to classify similar operation behavior patterns. The number of clusters can be determined by methods such as the silhouette coefficient or the elbow method. Usually, 3-5 categories can be tried. The center point and distribution characteristics of each category can be used as the parametric representation of the diversity impact.

[0067] The advantage of this method is that it can extract key behavior features from a large amount of complex operation data and quantify these features into comparable and analyzable parameters. For example, it may be found that some operators have a faster reaction speed but are prone to operation errors when facing emergencies, while other operators show a more stable but relatively slower operation mode. These parametric representations provide specific directions and bases for further optimizing the anti-error design.

[0068] In specific implementation, the following examples can be considered:

[0069] Suppose in a simulated flight experiment, operation data of 20 pilots with different experience levels during the execution of the standard takeoff procedure was collected. The multi-dimensional dataset includes information such as touch coordinates on the operation panel, key pressure, operation time intervals, etc. The sampling frequency is 100Hz and the duration is 10 minutes. First, wavelet transform is used to denoise the original data to remove high-frequency noise. Then, z-score normalization is performed on the data of different dimensions to make the mean of all features 0 and the standard deviation 1. Next, a 5-second sliding window is set and slides on the normalized data with a step size of 1 second to extract time series features. For each window, statistics such as mean, standard deviation, kurtosis, etc. are calculated to form a new feature vector. Then, PCA analysis is performed on the extracted feature vectors. Suppose the first 5 principal components explain 85% of the variance, then these 5 principal components are retained as key features. Finally, the K-means algorithm is used to cluster these 5 principal components, and the optimal number of clusters is determined to be 4 through the silhouette coefficient. These 4 categories may represent different operation behavior patterns such as "stable type", "quick response type", "cautious type", and "error-prone type".

[0070] The center point coordinates and covariance matrix of each category can be used as a parameterized representation of the diversity impact. For example, the characteristics of the "quick response type" may be manifested as a higher value of the first principal component (representing fast operation speed), but a lower value of the third principal component (representing lower precision).

[0071] Through this method, the complex operation behavior differences can be quantified into specific numerical values and distributions, providing reliable data support for the targeted optimization of error-proofing design. For example, according to the characteristics of different types of operators, the trigger threshold or feedback method of the error-proofing mechanism can be adjusted to adapt to different operation styles and habits.

[0072] Based on the above analysis, the advantages of the present invention are as follows:

[0073] 1. It can process high-dimensional and large-scale operation data and extract the most representative behavior features;

[0074] 2. Through time series analysis, it captures the dynamic changes of operation behavior, rather than just static statistical features;

[0075] 3. By using unsupervised learning methods, it can automatically discover and classify different operation behavior patterns, avoiding the biases that may be brought by artificially defining classification criteria;

[0076] 4. The generated parameterized representation is intuitive and easy to interpret, facilitating designers to understand and apply.

[0077] In this embodiment, the step S800 includes: step S801, extracting adjacent frame data from the video data set to generate an inter-frame difference image sequence; step S801, extracting the joint motion trajectories according to the inter-frame difference image sequence; step S802, intercepting the unconscious operation segments through the joint motion trajectories; step S803, counting the frequency and duration of the unconscious operation segments to generate a quantization feature set including the average duration, the maximum interval time, and the occurrence times.

[0078] In some of the above embodiments, during the implementation of the present invention, there is still a problem of how to accurately identify and quantify unconscious operations. In this regard, the present invention further proposes a technical solution for detecting unconscious operation segments through a video data set and obtaining a quantization feature set of unconscious operations. This technical solution first extracts adjacent frame data from the video data set to generate an inter-frame difference image sequence. This step utilizes the differences between consecutive frames in the video data to capture the action changes of the operator. By comparing adjacent frames, the changes between frames can be highlighted, providing a basis for subsequent analysis. Next, the joint motion trajectories are extracted according to the inter-frame difference image sequence. This step identifies and tracks the motion trajectories of the key parts of the operator's body (such as hands, fingers, etc.) by analyzing the change patterns in the difference images. These trajectory data contain detailed action information of the operator during the task execution. Then, the unconscious operation segments are intercepted through the joint motion trajectories. This step uses the previously obtained motion trajectory data and, by setting specific thresholds or pattern recognition algorithms, identifies the action segments that may belong to unconscious operations. These segments usually manifest as action sequences that do not conform to the expected operation mode or are significantly different from normal operations. Finally, the frequency and duration of the unconscious operation segments are counted to generate a quantization feature set including the average duration, the maximum interval time, and the occurrence times. This step conducts a quantitative analysis of the identified unconscious operation segments, calculates a series of statistical indicators, and provides objective and quantifiable data support for the evaluation of the anti-error design effect in the subsequent stage.

[0079] This technical solution realizes the automatic identification and quantification of unconscious operations through video analysis technology. Through inter-frame difference and joint point tracking, subtle unexpected actions can be captured, and these actions may be difficult to be found in traditional operation logs or performance metrics. Through statistical analysis, these unconscious operations are converted into quantifiable indicators, providing a more comprehensive and accurate data basis for evaluating the effect of anti-error design.

[0080] Specifically, the generation of the inter-frame difference image sequence can be achieved by calculating the differences in pixel values between two adjacent frames. For example, an image processing library such as OpenCV can be used to perform subtraction operations on each pair of adjacent frames to obtain difference images that reflect action changes. The extraction of the joint motion trajectories can adopt pose estimation algorithms in computer vision, such as OpenPose or MediaPipe, which can identify and track the positions of human key points in video frames. When intercepting unconscious operation segments, thresholds for parameters such as speed, acceleration, or trajectory complexity can be set. For example, when the hand movement speed suddenly decreases or the trajectory shows irregular changes, it may indicate an unconscious operation. These thresholds can be determined by analyzing normal operation data and are usually set outside 2-3 standard deviations of the normal operation parameter distribution. The generation of the quantization feature set involves the calculation of multiple statistical metrics. The average duration can be obtained by accumulating the durations of all identified unconscious operation segments and dividing by the number of segments. The maximum interval time is determined by recording the longest time interval between two unconscious operations. The occurrence count is simply obtained by counting the number of identified unconscious operation segments.

[0081] As a preferred implementation, the operation context information can be further combined to improve the accuracy of unconscious operation recognition. For example, the video analysis results can be associated with information such as system logs and task progress to distinguish between true unconscious operations and intentional non-standard operations. In addition, machine learning models such as long short-term memory networks (LSTM) or convolutional neural networks (CNN) can be introduced to improve the recognition accuracy of unconscious operations by training a large amount of labeled data. Through this method, the present invention can not only identify unconscious operations but also perform precise quantitative analysis on them. This provides a more objective and comprehensive basis for evaluating the effectiveness of the error-proofing design. Compared with the traditional methods that only rely on operation logs or subjective observations, the technical solution of the present invention can capture more subtle behavioral characteristics, thus more accurately evaluating the effectiveness of the error-proofing mechanism.

[0082] In addition, the advantages of this technical solution also lie in its non-invasiveness and high degree of automation. By analyzing the existing video data, rich behavioral data can be obtained without additional sensors or devices. This not only reduces the experimental cost but also avoids the interference that additional devices may cause to the operator's behavior. The automated analysis process also greatly improves the efficiency, making it possible to process large-scale data, thereby increasing the statistical significance of the evaluation results.

[0083] Based on the above analysis, the present invention has the following obvious advantages: First, traditional methods often rely on manual observation or simple analysis of operation logs and are difficult to capture subtle unconscious operations. The present invention can identify tiny movement changes that are difficult to detect by the naked eye through high-precision video analysis technology. Second, some existing automated analysis methods usually only focus on predefined error types and ignore the important factor of unconscious operations. The present invention can comprehensively capture various forms of unexpected operations through the analysis of joint point movement trajectories. Finally, in terms of quantitative analysis, the present invention provides a more systematic and comprehensive index system, including not only frequency and duration, but also considering factors such as time intervals, providing richer and more reliable data support for the evaluation of the anti-error design effect.

[0084] In this embodiment, the step S900 includes: Step S901, obtaining the mean and standard deviation of the operation sequence through the quantization feature set to obtain an operation deviation value; Step S902, calculating the change trend by the time window method according to the operation deviation value; Step S903, if the change trend exceeds a preset threshold, extracting the response time and error rate from the trigger event record to generate an intervention effect data set; Step S904, determining the intervention effect parameter according to the intervention effect data set.

[0085] In some of the above embodiments, during the implementation of the present invention, there is also a problem of how to extract behavior response data related to the anti-error mechanism from the quantization feature set and determine the intervention effect parameter. In this regard, the present invention further proposes to obtain the mean and standard deviation of the operation sequence through the quantization feature set to obtain an operation deviation value; calculate the change trend by the time window method according to the operation deviation value; if the change trend exceeds a preset threshold, extract the response time and error rate from the trigger event record to generate an intervention effect data set; determine the intervention effect parameter according to the intervention effect data set.

[0086] The experimental method for evaluating the anti-error design effect proposed by the present invention generates operation scenarios with and without anti-error mechanisms through a simulation system, obtains operation behavior data and video records, and generates a multi-dimensional data set and a video data set. Behavioral features are extracted from the multi-dimensional data set to generate a classified set of behavioral features. A set of behavioral patterns for different experience levels is determined based on the classified set of behavioral features. A synchronized behavior video set is generated through the video data set and the set of behavioral patterns. A set of difference features under the anti-error mechanism and without the anti-error mechanism is statistically analyzed based on the synchronized behavior video set. The evaluation result of the impact of the anti-error mechanism on the operation behavior is determined from the set of difference features. Operation behavior difference features and time series data are extracted from the multi-dimensional data set to determine a parametric representation of the diversity impact. Unconscious operation segments are detected based on the video data set to obtain a quantitative feature set of unconscious operations. Behavioral response data related to the anti-error mechanism is extracted from the quantitative feature set to determine the intervention effect parameters. The relationship model between the anti-error mechanism and the task accuracy is fitted through the operation behavior data and the task accuracy index, and a quantitative result of the design effect is generated.

[0087] In the technical solution proposed by the present invention, the step of extracting behavioral response data related to the anti-error mechanism from the quantitative feature set and determining the intervention effect parameters can be implemented in the following manner:

[0088] First, the mean and standard deviation of the operation sequence are obtained through the quantitative feature set to obtain the operation deviation value. Specifically, statistical analysis can be performed on the operation sequence data in the quantitative feature set to calculate the mean and standard deviation of each operation sequence. These statistical values reflect the central tendency and dispersion degree of the operation behavior, and can be used to measure the stability and consistency of the operation. By comparing the means and standard deviations of different operation sequences, the operation deviation value can be obtained, which represents the degree of difference between the operation behavior and the expected standard.

[0089] Second, the change trend is calculated through the time window method based on the operation deviation value. This step can adopt the method of sliding time window to observe the change of the operation deviation value within a certain time range. For example, a 5-minute time window can be set, and the average change rate of the operation deviation value within the window is calculated every 1 minute. In this way, a trend curve changing with time can be obtained, reflecting the dynamic change characteristics of the operation behavior.

[0090] Next, it is judged whether the change trend exceeds a preset threshold. This threshold can be set according to the specific application scenario and safety requirements. For example, the threshold can be set that the change rate of the operation deviation value exceeds 20%. If it is detected that the change trend exceeds this preset threshold, it indicates that there is a significant abnormal change in the operation behavior and intervention may be required.

[0091] When the change trend exceeds a preset threshold, the response time and error rate are extracted from the trigger event records to generate an intervention effect dataset. The trigger event records here can include information such as the time point when the error prevention mechanism is activated, the operator's reaction time, and the operation error rate within a certain period after the intervention. By extracting these data, a dataset containing comparison information before and after the intervention can be formed, which is used to evaluate the actual effect of the error prevention mechanism.

[0092] Finally, the intervention effect parameters are determined based on the intervention effect dataset. This step can calculate a series of parameters reflecting the intervention effect by comparing and analyzing the changes in response time and error rate before and after the intervention. For example, the percentage reduction in the error rate after the intervention and the degree of shortening of the average response time can be calculated. These parameters can quantitatively represent the effectiveness of the error prevention mechanism and provide a basis for further optimization design.

[0093] Through the above steps, the present invention can effectively extract the behavioral response data related to the error prevention mechanism from the quantitative feature set and determine the intervention effect parameters through data analysis. This method can not only objectively evaluate the actual effect of the error prevention mechanism, but also capture the dynamic change characteristics of operation behaviors, providing more accurate and reliable data support for the optimization of error prevention design.

[0094] In practical applications, the method of the present invention can be implemented in a simulated aircraft cockpit system. For example, a simulated long-haul flight mission with a duration of 4 hours can be set up. In this mission, multiple key points that require the operator to perform complex operations are set, and at the same time, an error prevention mechanism is implanted in the system. During the experiment, all the operation behaviors of the operator are recorded, including the operation sequence, operation time, number of error triggers, etc. Specifically, when implementing, the calculation period of the mean and standard deviation of the operation sequence can be set to 10 minutes, that is, the operation deviation value is calculated every 10 minutes. The time window can be set to 30 minutes and slides every 5 minutes to calculate the change trend of the operation deviation value within the window. The preset threshold can be set to that the change rate of the operation deviation value exceeds 15%. When it is detected that the change trend exceeds the threshold, the system will automatically record the time point when the error prevention mechanism is triggered, as well as the response time and error rate of the operator within the next 5 minutes. These data will be used to generate an intervention effect dataset. Finally, by comparing the average response time and error rate within 5 minutes before and after the intervention, the intervention effect parameters such as the reduction ratio of the response time and the percentage reduction of the error rate are calculated. In this way, the present invention can accurately capture and quantify the intervention effect of the error prevention mechanism in the actual operation environment, providing reliable data support for the evaluation and optimization of error prevention design.

[0095] Based on the above analysis, it can be seen that the method proposed by the present invention has the following advantages: First, by introducing the time window method and preset thresholds, it can more sensitively capture abnormal changes in operation behaviors, improving the triggering accuracy of the error prevention mechanism. Second, by extracting the response time and error rate before and after intervention to form an intervention effect data set, the effect evaluation of the error prevention mechanism becomes more objective and comprehensive. Finally, by calculating specific intervention effect parameters such as the reduction ratio of response time and the percentage reduction of error rate, the effects of different error prevention designs can be quantitatively compared, providing a more scientific basis for design optimization. These improvements make the present invention more accurate and reliable than the prior art in evaluating the effects of error prevention designs and can better adapt to complex operation environments and diverse operation behaviors.

[0096] In this embodiment, the generation of the quantitative result of the design effect by fitting the relationship model between the error prevention mechanism and the task accuracy through operation behavior data and task accuracy indicators includes: extracting the distribution sequence of task accuracy indicators from the multi-dimensional data set; using a regression analysis algorithm to fit the relationship model between the task accuracy indicators and the error prevention mechanism to obtain coefficient parameters; calculating the influence of operation deviation on the task accuracy indicators according to the coefficient parameters to determine the change trend; and adjusting the error prevention mechanism parameters through the change trend to generate the quantitative result of the design effect.

[0097] Specifically, first, the distribution sequence of task accuracy indicators is extracted from the multi-dimensional data set. This step can screen out indicators related to task accuracy, such as operation completion time, error rate, etc., from the multi-dimensional data collected in the experiment and organize them into time series data. Then, a regression analysis algorithm is used to fit the relationship model between the task accuracy indicators and the error prevention mechanism to obtain coefficient parameters. This step can select an appropriate regression model, such as linear regression, polynomial regression, or generalized linear model, etc., and select the best fitting method according to the data characteristics. Next, the influence of operation deviation on the task accuracy indicators is calculated according to the obtained coefficient parameters to determine the change trend. This step can judge the influence direction and degree of the error prevention mechanism on the task accuracy by analyzing the positive and negative and magnitude of the coefficient parameters. Finally, the error prevention mechanism parameters are adjusted through the change trend to generate the quantitative result of the design effect. This step can fine-tune the relevant parameters of the error prevention mechanism according to the influence trend analyzed above and obtain the optimized quantitative index of the design effect through simulation calculation.

[0098] In practical applications, this technical solution can be combined with the other steps mentioned above to form a complete evaluation process for the effects of error prevention designs. For example, the operation behavior data and video analysis results obtained in the previous steps can be used as the input for fitting the relationship model. At the same time, the output results of this solution can also be fed back to the previous steps for further optimizing the design of the error prevention mechanism.

[0099] As a preferred implementation, a multiple linear regression model can be selected to fit the relationship between the error-proofing mechanism and task accuracy. The specific steps are as follows:

[0100] First, extract task accuracy metrics from the multi-dimensional dataset, such as operation completion time T, error rate E, and operation fluency S. At the same time, extract parameters related to the error-proofing mechanism, such as prompt frequency F, response delay D, and intervention intensity I. Organize these data into a time series {(T1, E1, S1, F1, D1, I1), (T2, E2, S2, F2, D2, I2), …, (Tn, En, Sn, Fn, Dn, In)}.

[0101] Next, construct a multiple linear regression model: Y = β0 + β1X1 + β2X2 + β3X3 + ε, where Y represents the task accuracy metric (which can be a weighted combination of T, E, or S), X1, X2, X3 represent F, D, I respectively, β0, β1, β2, β3 are coefficients to be determined, and ε is the error term.

[0102] Then, use the least squares method to estimate the β values to obtain the fitted model: Y’ = b0 + b1X1 + b2X2 + b3X3. By analyzing the signs and magnitudes of b1, b2, b3, the impact of each error-proofing mechanism parameter on task accuracy can be judged.

[0103] Finally, based on the obtained model, by adjusting the values of X1, X2, X3, the change of Y’ can be predicted. For example, if it is found that the prompt frequency F (X1) has a significant positive impact on task accuracy, the value of F can be appropriately increased; if it is found that the response delay D (X2) has a negative impact, then the value of D can be tried to be reduced. In this way, a set of optimized error-proofing mechanism parameters can be obtained, and the corresponding predicted values of task accuracy can be calculated as the quantitative result of the design effect.

[0104] The advantage of this technical solution is that it can establish a quantitative relationship between the error-proofing mechanism and task accuracy, making the evaluation of the design effect more objective and accurate. With the support of the mathematical model, the impact of various error-proofing design parameters on operation performance can be better understood, providing a clear direction for further optimization. In addition, this method also has good scalability and can add more variables or adopt more complex regression models according to actual needs.

[0105] Embodiment 2

[0106] Figure 2 is a schematic structural wireframe diagram of an error-proofing design effect evaluation experimental system according to an embodiment of the present invention, as Figure 2As shown in the figure, Embodiment 2 provides an experimental system for evaluating the effect of anti-error design, including: a data set generation module 201, a behavior feature set generation module 202, a pattern set determination module 203, a behavior video set generation module 204, a statistics module 205, an impact evaluation result determination module 206, a parameterized representation determination module 207, a quantization feature set acquisition module 208, an intervention effect parameter determination module 209, and a quantization result generation module 210. The data set generation module 201 is used to generate operation scenarios with and without anti-error mechanisms through a simulation system, obtain operation behavior data and video records, and generate a multi-dimensional data set and a video data set. The behavior feature set generation module 202 is used to extract behavior features from the multi-dimensional data set and generate a classified behavior feature set. The pattern set determination module 203 is used to determine a set of behavior patterns with different experience levels according to the classified behavior feature set. The behavior video set generation module 204 is used to generate a synchronized behavior video set through the video data set and the set of behavior patterns. The statistics module 205 is used to statistically analyze a set of difference features with and without anti-error mechanisms according to the synchronized behavior video set. The impact evaluation result determination module 206 is used to determine the impact evaluation result of the anti-error mechanism on operation behavior from the set of difference features. The parameterized representation determination module 207 is used to extract operation behavior difference features and time series data from the multi-dimensional data set and determine the parameterized representation of the diversity impact. The quantization feature set acquisition module 208 is used to detect unconscious operation segments according to the video data set and obtain a quantization feature set of unconscious operations. The intervention effect parameter determination module 209 is used to extract behavior response data related to the anti-error mechanism from the quantization feature set and determine the intervention effect parameter. The quantization result generation module 210 is used to fit a relationship model between the anti-error mechanism and task accuracy through the operation behavior data and task accuracy metrics, and generate a quantization result of the design effect.

[0107] In this embodiment, the behavior feature set generation module 202 includes: an acquisition unit, a feature data generation unit, and a feature set generation unit. The acquisition unit is used to obtain the operation interval time, the number of error triggers, and the panel switching frequency through the multi-dimensional data set. The feature data generation unit is used to process the operation interval time, the number of error triggers, and the panel switching frequency using a standardization tool to generate standardized feature data. The feature set generation unit is used to group the standardized feature data according to the operation frequency and error rate through a clustering algorithm to generate a classified behavior feature set.

[0108] In this embodiment, the generating behavior video set module 204 includes: a obtaining correspondence unit, a generating initial synchronized behavior video set unit, and a generating synchronized behavior video set unit. The obtaining correspondence unit is used to obtain the correspondence between the behavior pattern set and the video data set. The generating initial synchronized behavior video set unit is used to align the operation behavior timestamps in the behavior pattern set with the video records in the video data set through the frame matching function, and generate an initial synchronized behavior video set. The generating synchronized behavior video set unit is used to fill in the missing operation logs and video gaps by using an interpolation method if there is data missing in the initial synchronized behavior video set, and generate the synchronized behavior video set.

[0109] In this embodiment, the statistical module 205 includes: a obtaining mean unit and a generating difference feature set unit. The obtaining mean unit is used to obtain the standard deviation of the operation interval time and the mean value of the error trigger times through the synchronized behavior video set. The generating difference feature set unit is used to generate a difference feature set according to the standard deviation of the operation interval time and the mean value of the error trigger times.

[0110] In this embodiment, the determining parameterized representation module 207 includes: a obtaining basic data set unit, a extracting data unit, a determining unit, and a generating parameterized representation unit. The obtaining basic data set unit is used to perform denoising and normalization processing on the multi-dimensional data set to obtain a basic data set. The extracting data unit is used to extract time series data from the basic data set by using a sliding window method. The determining unit is used to separate the difference features from the time series data by using principal component analysis and determine the independent distribution pattern of the operation behavior. The generating parameterized representation unit is used to group according to the independent distribution pattern through a clustering algorithm and generate a parameterized representation of the diversity impact.

[0111] In this embodiment, the obtaining quantization feature set module 208 includes: a generating image sequence unit, a extracting trajectory unit, a intercepting segment unit, and a generating quantization feature set unit. The generating image sequence unit is used to extract adjacent frame data from the video data set and generate an inter-frame difference image sequence. The extracting trajectory unit is used to extract the joint point motion trajectory according to the inter-frame difference image sequence. The intercepting segment unit is used to intercept the unconscious operation segment through the joint point motion trajectory. The generating quantization feature set unit is used to count the frequency and duration of the unconscious operation segment and generate a quantization feature set including the average duration, the maximum interval time, and the occurrence times.

[0112] In this embodiment, the intervention effect parameter determination module 209 includes: a deviation value obtaining unit, a change trend calculating unit, an intervention effect data set generating unit, and a parameter determining unit. The deviation value obtaining unit is configured to obtain the mean value and standard deviation of the operation sequence through the quantization feature set, and obtain the operation deviation value. The change trend calculating unit is configured to calculate the change trend according to the operation deviation value by using a time window method. The intervention effect data set generating unit is configured to extract the response time and error rate from the trigger event record and generate an intervention effect data set if the change trend exceeds a preset threshold. The parameter determining unit is configured to determine the intervention effect parameter according to the intervention effect data set.

[0113] In this embodiment, the quantization result generating module 210 includes: a distribution sequence extracting unit, a coefficient parameter obtaining unit, a change trend determining unit, and a quantization result generating unit. The distribution sequence extracting unit is configured to extract the distribution sequence of the task accuracy index through the multi-dimensional data set. The coefficient parameter obtaining unit is configured to fit the relationship model between the task accuracy index and the error-proofing mechanism by using a regression analysis algorithm, and obtain the coefficient parameter. The change trend determining unit is configured to calculate the influence of the operation deviation on the task accuracy index according to the coefficient parameter, and determine the change trend. The quantization result generating unit is configured to adjust the error-proofing mechanism parameter through the change trend and generate the quantization result of the design effect.

[0114] The various change modes and specific examples of the anti-error design effect evaluation experimental method provided in the first embodiment are equally applicable to the anti-error design effect evaluation experimental system provided in this embodiment. Through the foregoing detailed description of an anti-error design effect evaluation experimental method, those skilled in the art can clearly know the implementation manner of an anti-error design effect evaluation experimental system in this embodiment. Therefore, for the sake of brevity of the specification, it will not be described in detail herein.

[0115] In summary, the anti-error design effect evaluation experimental method and system of the present invention have the following beneficial effects:

[0116] 1. By simulating the system to generate operation scenarios with and without an error-proofing mechanism, it is possible to systematically compare and analyze the operation behavior differences under the two mechanisms; through the generation and analysis of the multi-dimensional data set and the video data set, comprehensive data support is provided, making the evaluation results more accurate and reliable.

[0117] 2. Extracting behavioral features from the multi-dimensional data set, generating a classified behavioral feature set, and determining a set of behavioral patterns with different experience levels according to these feature sets. This refined analysis helps to deeply understand the operation behavior differences of different pilots; by generating a synchronous behavior video set through the video data set and the set of behavioral patterns, the operation behavior can be intuitively observed and analyzed, further improving the accuracy of the evaluation.

[0118] 3. It can detect unconscious operation segments and obtain a quantitative feature set of unconscious operations, which is crucial for evaluating the effectiveness of the error-proofing mechanism in reducing unconscious operations.

[0119] 4. Considering the diversity of operation behaviors, by extracting the differential features of operation behaviors and time series data, it determines a parametric representation of the diversity impact, which helps to design an error-proofing mechanism that better adapts to the needs of different pilots.

[0120] 5. By fitting the relationship model between the error-proofing mechanism and task accuracy, it generates a quantitative result of the design effect. This scientific method provides an objective evaluation basis and reduces the influence of subjective judgment.

[0121] 6. The present invention is not only applicable to the aviation field, but also can be extended to other fields that require error-proofing design, such as medical treatment, industrial control, etc. Its wide applicability makes the research results have higher practical value.

[0122] 7. It solves the problems existing in the current evaluation methods, such as insufficient sample coverage, single experimental task, and insufficient in-depth data analysis, and provides new ideas and methods for evaluating the effectiveness of error-proofing design.

[0123] The foregoing description of specific exemplary embodiments of the present invention is for purposes of illustration and exemplification. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many modifications and variations are possible in light of the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the invention and its practical applications, so that those skilled in the art can implement and utilize various different exemplary embodiments of the invention, as well as various different selections and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. An experimental method for evaluating the effect of anti-error design, characterized in that, Including: Generating operation scenarios with and without anti-error mechanisms through a simulation system, obtaining operation behavior data and video records, and generating a multi-dimensional dataset and a video dataset; Extracting behavior features from the multi-dimensional dataset to generate a classified behavior feature set; Determining a set of behavior patterns for different experience levels based on the classified behavior feature set; Generating a synchronized behavior video set through the video dataset and the set of behavior patterns; Statistically analyzing a set of differential features under anti-error and non-anti-error mechanisms based on the synchronized behavior video set; Determining an evaluation result of the impact of the anti-error mechanism on operation behavior from the set of differential features; Extracting operation behavior differential features and time series data from the multi-dimensional dataset to determine a parametric representation of the impact of diversity; Detecting unconscious operation segments based on the video dataset to obtain a set of quantitative features of unconscious operations; Extracting behavior response data related to the anti-error mechanism from the set of quantitative features to determine an intervention effect parameter; Fitting a relationship model between the anti-error mechanism and task accuracy through the operation behavior data and task precision metrics to generate a quantitative result of the design effect.

2. The anti-error design effect evaluation experiment method according to claim 1, wherein The extracting behavior features from the multi-dimensional dataset to generate a classified behavior feature set includes: Obtaining the operation interval time, the number of error triggers, and the panel switching frequency from the multi-dimensional dataset; Processing the operation interval time, the number of error triggers, and the panel switching frequency using a standardization tool to generate standardized feature data; Grouping according to the operation frequency and error rate based on the standardized feature data through a clustering algorithm to generate a classified behavior feature set.

3. The anti-error design effect evaluation experiment method according to claim 1, wherein The generating a synchronized behavior video set through the video dataset and the set of behavior patterns includes: Obtaining the correspondence between the set of behavior patterns and the video dataset; Aligning the operation behavior timestamps in the set of behavior patterns and the video records in the video dataset through a frame matching function to generate an initial synchronized behavior video set; If there is data missing in the initial synchronized behavior video set, using an interpolation method to fill in the missing operation logs and video gaps to generate the synchronized behavior video set.

4. The anti-error design effect evaluation experiment method according to claim 1, wherein The statistically analyzing a set of differential features under anti-error and non-anti-error mechanisms based on the synchronized behavior video set includes: Obtaining the standard deviation of the operation interval time and the mean value of the number of error triggers from the synchronized behavior video set; Generating a set of differential features based on the standard deviation of the operation interval time and the mean value of the number of error triggers.

5. The anti-error design effect evaluation experimental method according to claim 1, characterized in that The extracting operation behavior differential features and time series data from the multi-dimensional dataset to determine a parametric representation of the impact of diversity includes: Performing denoising and normalization processing on the multi-dimensional dataset to obtain a basic dataset; Extracting time series data from the basic dataset through a sliding window method; Using principal component analysis to separate differential features from the time series data to determine the independent distribution pattern of operation behavior; Grouping according to the independent distribution pattern through a clustering algorithm to generate a parametric representation of the impact of diversity.

6. The anti-error design effect evaluation experiment method according to claim 1, wherein, The detecting unconscious operation segments based on the video dataset to obtain a set of quantitative features of unconscious operations includes: Extract adjacent frame data from the video dataset to generate an inter-frame difference image sequence; Extract the joint motion trajectories according to the inter-frame difference image sequence; Intercept the unconscious operation segments through the joint motion trajectories; Statistically analyze the frequency and duration of the unconscious operation segments to generate a quantization feature set including the average duration, the maximum interval time, and the occurrence times.

7. The anti-error design effect evaluation experimental method according to claim 1, wherein The behavior response data related to the anti-error mechanism is extracted from the quantization feature set, and the determined intervention effect parameters include: Obtain the mean and standard deviation of the operation sequence through the quantization feature set to obtain the operation deviation value; Calculate the change trend according to the operation deviation value by the time window method; If the change trend exceeds the preset threshold, extract the response time and error rate from the trigger event record to generate an intervention effect dataset; Determine the intervention effect parameters according to the intervention effect dataset.

8. The method for evaluating the anti-error design effect according to claim 1, wherein, The relationship model between the anti-error mechanism and the task accuracy is fitted through the operation behavior data and the task accuracy index, and the generated quantization results of the design effect include: Extract the distribution sequence of the task accuracy index from the multi-dimensional dataset; Use the regression analysis algorithm to fit the relationship model between the task accuracy index and the anti-error mechanism to obtain the coefficient parameters; Calculate the influence of the operation deviation on the task accuracy index according to the coefficient parameters to determine the change trend; Adjust the anti-error mechanism parameters through the change trend to generate the quantization results of the design effect.

9. An experimental system for evaluating the effect of error-proofing design, characterized in that, Including: A dataset generation module, which is used to generate operation scenarios with and without anti-error mechanisms through a simulation system, obtain operation behavior data and video records, and generate a multi-dimensional dataset and a video dataset; A behavior feature set generation module, which is used to extract behavior features from the multi-dimensional dataset to generate a classified behavior feature set; A mode set determination module, which is used to determine the behavior mode sets of different experience levels according to the classified behavior feature set; A behavior video set generation module, which is used to generate a synchronized behavior video set through the video dataset and the behavior mode set; A statistics module, which is used to statistically analyze the difference feature set under the anti-error mechanism and without the anti-error mechanism according to the synchronized behavior video set; An influence evaluation result determination module, which is used to determine the influence evaluation result of the anti-error mechanism on the operation behavior from the difference feature set; A parameterized representation determination module, which is used to extract the operation behavior difference features and time series data from the multi-dimensional dataset to determine the parameterized representation of the diversity influence; A quantization feature set acquisition module, which is used to detect unconscious operation segments according to the video dataset and obtain the quantization feature set of unconscious operations; An intervention effect parameter determination module, which is used to extract the behavior response data related to the anti-error mechanism from the quantization feature set and determine the intervention effect parameters; A quantization result generation module, which is used to fit the relationship model between the anti-error mechanism and the task accuracy through the operation behavior data and the task accuracy index, and generate the quantization results of the design effect.

Images