Error-proof design effect evaluation experiment method and system

Abstract

The application discloses an error prevention design effect evaluation experiment method and system, wherein the method comprises the following steps: generating operation scenes with error prevention mechanisms and without error prevention mechanisms through a simulation system, obtaining operation behavior data and video records, generating a multi-dimensional data set and a video data set; extracting behavior features from the multi-dimensional data set to generate a classified behavior feature set; determining a behavior mode set of different experience levels according to the classified behavior feature set; generating a synchronous behavior video set through the video data set and the behavior mode set; statistically determining a difference feature set of error prevention mechanisms and without error prevention mechanisms according to the synchronous behavior video set; and determining an influence evaluation result of the error prevention mechanism on the operation behavior from the difference feature set. Therefore, the error prevention design effect evaluation experiment method can effectively evaluate the applicability of the error prevention mechanism in different pilot groups, and provides important support for improving flight safety.

Description

Technical Field

[0001] The present invention relates to the technical field of error-proofing effect evaluation, and in particular to an error-proofing design effect evaluation experimental method and system. Background Art

[0002] As a key research area, aviation human-machine interaction design is directly related to improving flight safety and operational efficiency, and its importance is self-evident. In the modern aviation industry, the interaction design between pilots and cockpit systems not only affects the accuracy of mission execution but also, in extreme cases, determines the safety of life and property. However, many current evaluation methods remain at the level of theoretical analysis or single experiments, failing to fully reflect the complex scenarios encountered in actual operations. This is especially true when verifying the effectiveness of error-proofing designs, where problems such as insufficient sample coverage, single experimental tasks, and insufficiently in-depth data analysis are common. These limitations result in insufficient basis for design optimization, which in turn affects the reliability and practicality of cockpit systems.

[0003] The shortcomings of existing methods have gradually highlighted the core challenges facing the evaluation of aviation cockpit error-proofing designs. Among them, the diversity of flight crew operating behaviors, the effectiveness verification of error-proofing mechanisms, and the identification and quantification of unconscious operations have become the three major technical focuses. Due to differences in experience and reaction patterns among different flight crews, error-proofing designs that have not been fully verified may fail in actual applications. At the same time, the lack of systematic capture and analysis of unconscious operations makes it difficult for design improvements to specifically reduce human errors. These technical factors have not been properly addressed, making it difficult for evaluation methods to accurately measure the actual effectiveness of error-proofing designs, which in turn leads to the unique challenge of how to scientifically verify designs in dynamic and complex environments.

[0004] Therefore, a key challenge in this research is to accurately identify unconscious operations and quantify the effectiveness of the error-proofing design by comparing the operational performance of different flight crew groups in simulated aircraft cockpit systems with and without error-proofing mechanisms, combining multi-dimensional experimental data with video analysis. Solving this problem will directly advance cockpit interaction design towards greater safety and adaptability. Summary of the Invention

[0005] The application aims to provide an error prevention design effect evaluation experiment method and system, which simulates a comparative experiment scene containing an error prevention mechanism and a constant error prevention mechanism, combines multi-dimensional behavior data (such as operation interval, error rate, and panel switching frequency) and video frame difference analysis technology, realizes accurate capture and quantitative representation of pilot unconscious operation, and establishes a parameterized relationship model of the error prevention mechanism and task accuracy.

[0006] To achieve the above-mentioned purpose, in a first aspect, the application provides an error prevention design effect evaluation experiment method, which comprises: generating an operation scene containing an error prevention mechanism and a constant error prevention mechanism through a simulation system, obtaining operation behavior data and video records, generating a multi-dimensional data set and a video data set. Extracting behavior features from the multi-dimensional data set to generate a classified behavior feature set. Determining a behavior mode set of different experience levels according to the classified behavior feature set. Generating a synchronous behavior video set through the video data set and the behavior mode set. According to the synchronous behavior video set, a difference feature set under the error prevention mechanism and the constant error prevention mechanism is obtained. From the difference feature set, an influence evaluation result of the error prevention mechanism on the operation behavior is determined. Extracting operation behavior difference features and time series data from the multi-dimensional data set to determine a parameterized representation of diversity influence. According to the video data set, an unconscious operation segment is detected to obtain a quantitative feature set of the unconscious operation. From the quantitative feature set, behavior response data related to the error prevention mechanism is extracted to determine an intervention effect parameter. The relationship model of the error prevention mechanism and the task accuracy is fitted through the operation behavior data and the task accuracy index to generate a quantitative result of the design effect.

[0007] In a second aspect, the present application provides an error prevention design effect evaluation experiment system, comprising: a data set generation module, a behavior feature set generation module, a mode set determination module, a behavior video set generation module, a statistics module, an influence evaluation result determination module, a parameterized representation determination module, a quantitative feature set acquisition module, an intervention effect parameter determination module, and a quantitative result generation module. The data set generation module is configured to generate operation scenarios with and without error prevention mechanisms through a simulation system, acquire operation behavior data and video records, and generate multi-dimensional data sets and video data sets. The behavior feature set generation module is configured to extract behavior features from the multi-dimensional data sets and generate classified behavior feature sets. The mode set determination module is configured to determine behavior mode sets of different experience levels based on the classified behavior feature sets. The behavior video set generation module is configured to generate synchronized behavior video sets based on the video data sets and the behavior mode sets. The statistics module is configured to statistically determine a difference feature set between error prevention mechanisms and non-error prevention mechanisms based on the synchronized behavior video sets. The influence evaluation result determination module is configured to determine an influence evaluation result of error prevention mechanisms on operation behavior based on the difference feature set. The parameterized representation determination module is configured to extract operation behavior difference features and time series data from the multi-dimensional data sets and determine a parameterized representation of diversity influence. The quantitative feature set acquisition module is configured to detect unconscious operation segments based on the video data sets and acquire a quantitative feature set of unconscious operations. The intervention effect parameter determination module is configured to extract behavior response data related to error prevention mechanisms from the quantitative feature set and determine an intervention effect parameter. The quantitative result generation module is configured to fit a relationship model between error prevention mechanisms and task accuracy based on the operation behavior data and task precision indicators, and generate a quantitative result of design effect.

[0008] Compared with the prior art, the error prevention design effect evaluation experiment method and system according to the present application has the following beneficial effects:

[0009] 1. The operation scenarios with and without error prevention mechanisms are generated through a simulation system, which can systematically compare and analyze the operation behavior differences under the two mechanisms. The generation and analysis of multi-dimensional data sets and video data sets provide comprehensive data support, making the evaluation result more accurate and reliable.

[0010] 2. The behavior features are extracted from the multi-dimensional data sets to generate classified behavior feature sets, and the behavior mode sets of different experience levels are determined based on these feature sets. This fine-grained analysis helps to deeply understand the operation behavior differences of different flight personnel. The synchronized behavior video sets are generated based on the video data sets and the behavior mode sets, which can intuitively observe and analyze the operation behavior, further improving the accuracy of the evaluation.

[0011] 3. Can detect the unconscious operation segment and obtain the quantitative feature set of the unconscious operation, which is crucial for evaluating the effect of the error prevention mechanism in reducing the unconscious operation.

[0012] 4. The diversity of operation behavior is considered, and the parameterized representation of the diversity influence is determined by extracting the operation behavior difference features and time series data, which helps to design an error prevention mechanism that is more suitable for the needs of different flight personnel.

[0013] 5. The quantitative results of the design effect are generated by fitting the relationship model of the error prevention mechanism and the task accuracy, which provides an objective evaluation basis and reduces the influence of subjective judgment.

[0014] 6. The present application is not only suitable for the aviation field, but also can be popularized to other fields that need error prevention design, such as medical treatment, industrial control, etc., and its wide applicability makes the research results have higher practical value.

[0015] 7. The problems of insufficient sample coverage, single experimental task and insufficient data analysis in the current evaluation method are solved, which provides a new idea and method for the evaluation of the effect of error prevention design. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart of the error prevention design effect evaluation experiment method according to an embodiment of the present application;

[0017] Figure 2 is a structure line frame schematic diagram of the error prevention design effect evaluation experiment system according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present application is not limited by the specific embodiments.

[0019] Unless otherwise explicitly stated, throughout the specification and claims, the term "comprise" or its variants such as "contain" or "include" will be understood to encompass the stated elements or components, without excluding other elements or components.

[0020] For the convenience of understanding, the main implementation concept of each embodiment of the present application will be simply described first.

[0021] Human-machine interaction design in aviation directly affects flight safety and operational efficiency. However, current evaluation methods have limitations such as insufficient sample coverage, single experimental tasks, and inadequate data analysis. These limitations lead to insufficient design optimization and affect the reliability and practicality of cockpit systems. Specifically, the diversity of pilot behavior, the effectiveness of error prevention mechanisms, and the identification and quantification of unconscious operations are three major technical focuses. Due to differences in experience and reaction patterns among different pilots, unverified error prevention designs may fail in actual applications. Additionally, the lack of systematic capture and analysis of unconscious operations makes it difficult to improve design to reduce human errors. These technical factors have not been properly addressed, making it difficult for evaluation methods to accurately measure the actual effectiveness of error prevention designs, and further complicating the unique challenge of scientifically verifying designs in dynamic and complex environments.

[0022] In modern aviation cockpit systems, the technical challenges of error prevention design evaluation are particularly prominent. For example, in a typical flight simulation training scenario, the system needs to handle the operation data of multiple pilots simultaneously. Pilot A may have extensive flying experience, while pilot B is a novice. When both pilots operate the same cockpit system, the system needs to capture and analyze the differences in their operation behavior. Specifically, the system needs to record multi-dimensional data such as operation interval time, error trigger frequency, and panel switching frequency. However, due to the lack of effective data synchronization and analysis methods, the system cannot accurately identify the changes in operation patterns caused by experience differences. Additionally, during simulated emergency situations, pilots may exhibit unconscious operations such as repeatedly pressing a button or ignoring critical warning information. These unconscious operations are often difficult to capture through conventional data analysis methods, leading to a lack of targeted design of error prevention mechanisms. Furthermore, the system needs to consider the task characteristics of different flight stages, such as the differences in operations during takeoff, cruising, and landing, further increasing the complexity of data analysis.

[0023] If these technical problems cannot be effectively solved, it will have a serious impact on aviation safety and flight efficiency. First, due to the inability to accurately assess the actual effect of error prevention design, some potential operational risks may be ignored. For example, a seemingly reasonable error prevention design may increase operational complexity and increase pilot cognitive load under certain conditions. Second, the lack of accurate identification and quantification of unconscious operation makes it difficult for the system to optimize interface design and operation process, which may lead to the persistence of human error. Third, if the operational differences of pilots with different experience levels cannot be effectively captured and analyzed, it will be difficult to design an adaptive error prevention mechanism, which may lead to inconsistent system performance when facing different users. Finally, due to the lack of comprehensive multi-dimensional data analysis capabilities, the system is difficult to establish an accurate relationship model between the error prevention mechanism and the task accuracy, which will directly affect the overall performance evaluation and optimization direction of the cockpit system. Therefore, there is an urgent need for an error prevention design effect evaluation method that can consider multi-dimensional data, video analysis, unconscious operation identification and other factors to improve the safety and adaptability of the cockpit system.

[0024] Therefore, the present application proposes a brand-new error prevention design effect evaluation experiment method and system, which aims to comprehensively and accurately evaluate the error prevention design effect of the aircraft cockpit control device through a scientific and objective way, and provides strong support for the optimization of cockpit interaction design.

[0025] Embodiment one

[0026] Figure 1 is a flowchart of the error prevention design effect evaluation experiment method according to an embodiment of the present application, as shown in Figure 1As shown, the embodiment one provides an error prevention design effect evaluation experiment method, comprising: step S100, generating operation scenes containing error prevention mechanisms and no error prevention mechanisms through a simulation system, obtaining operation behavior data and video records, generating a multi-dimensional data set and a video data set; step S200, extracting behavior features from the multi-dimensional data set to generate a classified behavior feature set; step S300, determining a behavior mode set of different experience levels according to the classified behavior feature set; step S400, generating a synchronous behavior video set through the video data set and the behavior mode set; step S500, according to the synchronous behavior video set, statistics a difference feature set under the error prevention mechanism and the no error prevention mechanism; step S600, determining an influence evaluation result of the error prevention mechanism on the operation behavior from the difference feature set; step S700, extracting operation behavior difference features and time series data from the multi-dimensional data set to determine a parameterized representation of diversity influence; step S800, detecting an unconscious operation segment according to the video data set to obtain a quantitative feature set of unconscious operation; step S900, extracting behavior response data related to the error prevention mechanism from the quantitative feature set to determine an intervention effect parameter; step S1000, fitting a relationship model of the error prevention mechanism and task accuracy through the operation behavior data and task accuracy indicators to generate a quantitative result of the design effect.

[0027] Specifically, when solving the problem of error prevention design evaluation in aviation cockpit, the present application first considers how to comprehensively capture and analyze the operation behavior of pilots. Specifically, operation scenes containing error prevention mechanisms and no error prevention mechanisms are generated through a simulation system, operation behavior data and video records are obtained, and a multi-dimensional data set and a video data set are generated. This method can simultaneously collect quantitative and qualitative data, providing a comprehensive basis for subsequent analysis.

[0028] Next, the present application proceeds to solve how to extract meaningful behavior features from a large amount of data. By extracting behavior features from the multi-dimensional data set, a classified behavior feature set is generated. This step can effectively convert the original data into analyzable features, laying a foundation for subsequent behavior mode recognition.

[0029] After determining the behavior features, the present application considers how to distinguish the behavior of pilots with different experience levels. According to the classified behavior feature set, a behavior mode set of different experience levels is determined. This step can help us understand the influence of experience on operation behavior, so as to design more adaptive error prevention mechanisms.

[0030] In order to more intuitively analyze the operation behavior, the present application proposes to generate a synchronous behavior video set through the video data set and the behavior mode set. This method synchronizes quantitative data with video data, enabling analysts to more comprehensively understand the context of operation behavior.

[0031] After obtaining the synchronized behavior video set, the present application addresses how to quantify the effect of the error prevention mechanism. By counting the difference feature set under the error prevention mechanism and without the error prevention mechanism, and determining the influence evaluation result of the error prevention mechanism on the operation behavior from it. This step can directly quantify the effect of the error prevention design, providing a basis for subsequent optimization.

[0032] Considering the diversity of operation behavior, the present application proposes to extract the operation behavior difference feature and time series data from the multi-dimensional data set, and determine the parameterized representation of the diversity influence. This method can capture the operation difference between different pilots, and help to design a more flexible error prevention mechanism.

[0033] For the difficulty of unconscious operation, the present application proposes to detect the unconscious operation segment according to the video data set, and obtain the quantitative feature set of unconscious operation. Extract the behavior response data related to the error prevention mechanism from the quantitative feature set, and determine the intervention effect parameter. This step can effectively identify and quantify the unconscious operation, and provide targeted optimization direction for error prevention design.

[0034] Finally, in order to comprehensively evaluate the effect of the error prevention design, the present application proposes to fit the relationship model between the error prevention mechanism and the task accuracy by the operation behavior data and the task precision index, and generate the quantitative result of the design effect. This method can establish a direct link between the error prevention mechanism and the task execution effect, and provide a scientific basis for design optimization.

[0035] The simulation system refers to a software and hardware system for simulating the cockpit environment of an aircraft, and can be implemented by using virtual reality technology or a physical simulation cabin. The multi-dimensional data set refers to a data set containing multiple dimensions such as operation time, operation type, and operation frequency, and can be stored and managed by using a structured database. The video data set refers to a set of video files recording the operation process, and can be recorded by using a high-definition camera device. The behavior characteristics refer to key indicators that can represent the operation behavior, and can be represented by operation interval time, error trigger times, and panel switching frequency. The behavior mode set refers to typical operation modes of operators with different experience levels, and can be obtained by classifying the behavior characteristics by using a clustering algorithm. The synchronized behavior video set refers to a data set in which the operation behavior data is synchronized with the video record, and the synchronization can be achieved by using a timestamp matching technology. The difference feature set refers to a set of key indicators reflecting the effect of the error prevention mechanism, and can be extracted from the synchronized behavior video set by using statistical analysis methods. The parameterized representation of diversity influence refers to a mathematical model for describing the difference in operation behavior, and can be achieved by using dimension reduction techniques such as principal component analysis. The unconscious operation refers to unintended operation performed by the operator in an unconscious state, and can be identified from the video data by using video analysis technology. The intervention effect parameter refers to an index for quantifying the intervention effect of the error prevention mechanism, and can be represented by response time, error rate, etc. The task accuracy refers to the accuracy of the completion of the operation task, and can be measured by using error rate, completion time, etc.

[0036] The simulation system generates operation scenarios with and without the error prevention mechanism, and simultaneously obtains multi-dimensional data sets and video data sets, thereby comprehensively capturing the operation behavior. By extracting behavior characteristics, determining behavior modes, and generating synchronized behavior video sets, the influence of the error prevention mechanism is systematically analyzed. In particular, by detecting unconscious operation segments and quantifying their characteristics, targeted optimization directions are provided for error prevention design. Finally, by establishing a relationship model between the error prevention mechanism and the task accuracy, the quantitative evaluation of the design effect is achieved.

[0037] The working principle of the present application is first to generate an operating scenario through a simulation system. The simulation system can be a virtual reality environment or a physical simulation cabin, which can accurately simulate the operating environment of the aircraft cabin. On this basis, the system records the operating behavior data and video data at the same time. The operating behavior data includes operating time, type, frequency and other dimensions, forming a multi-dimensional data set. The video data records the entire operation process through high-definition camera equipment. Next, the system extracts behavior features from the multi-dimensional data set. This step uses data mining technology to extract key indicators such as operation interval time, error trigger times, panel switching frequency, etc. After standardization, these features are classified using clustering algorithms to form a behavior pattern set of different experience levels. Then, the system synchronizes the behavior pattern set with the video data set. This step uses timestamp matching technology to ensure that the behavior data and video pictures correspond accurately. The synchronized data set provides a rich source of information for subsequent analysis. Based on the synchronized behavior video set, the system counts the difference features under the anti-error mechanism and the non-error mechanism. This step quantifies the actual effect of the anti-error mechanism through comparative analysis. At the same time, the system also extracts the difference features and time series data of the operating behavior, and uses principal component analysis and other techniques to determine the parameterized representation of diversity influence. In particular, the system detects unconscious operation segments through video analysis technology. This step can capture operating behaviors that are difficult to discover through conventional data analysis. The system further quantifies the features of these unconscious operations to provide targeted optimization direction for anti-error design. Finally, the system correlates the operating behavior data with the task precision indicators. By establishing a regression model, the system can quantify the relationship between the anti-error mechanism and the task precision, thereby generating a quantitative result of the design effect. The design of this working principle fully considers the comprehensiveness of the data and the depth of the analysis, which can effectively evaluate the actual effect of the anti-error design.

[0038] As a preferred embodiment, the error prevention design effect evaluation experiment method of the present application can be applied to the evaluation of the cabin system of a certain type of civil aviation passenger plane. Specifically, first, a virtual reality technology-based cabin simulator is used to construct the operation scene. The simulator accurately replicates the layout and functions of the real cabin, including the main display screen, control panel, and warning system, etc. During the experiment, 20 pilots with different experience levels are recruited to participate in the test. Among them, 10 are experienced senior pilots with more than 5000 hours of flight time; the other 10 are novice pilots with less than 500 hours of flight time. Each pilot completes the same flight task under two scenarios: with and without error prevention mechanism. The task includes three stages: takeoff, cruise, and landing, each lasting 15 minutes. During the experiment, the system records operation behavior data in real time through multiple sensors, including button pressing time, switch switching frequency, screen touch position, etc. At the same time, a 4K high-definition camera records the operation process of the pilot from multiple angles. These data are stored synchronously in a high-performance server to form a multi-dimensional data set and a video data set. After the data collection is completed, machine learning algorithms are used to extract behavior features from the multi-dimensional data set. Specifically, the interval time of each operation is calculated, the number of error triggers is counted, and the panel switching pattern is analyzed, etc. After standardization, the K-means clustering algorithm is used to divide the pilots into high and low experience level groups. Next, the system uses timestamp matching technology to accurately align the behavior data with the video frames, generating a synchronized behavior video set. On this basis, the influence of the error prevention mechanism on various operation indicators is statistically analyzed, such as the percentage of error rate reduction, the degree of operation efficiency improvement, etc. In particular, the system uses computer vision technology to analyze video data and identify unconscious operation segments. For example, detecting repeated invalid button presses, ignoring critical warning information, etc. These unconscious operations are quantified as frequency, duration, etc. Finally, the system uses a multiple linear regression model to establish the relationship between the error prevention mechanism parameters and the task completion accuracy. The model input includes the trigger frequency and response time of the error prevention mechanism, and the output is the error rate and time of task completion. Through this model, the actual effect of the error prevention design can be quantitatively evaluated, and the basis for further optimization is provided.

[0039] In the present embodiment, the step S200 comprises: a step S201 of obtaining operation interval time, error trigger times, and panel switching frequency through the multi-dimensional data set; a step S202 of processing the operation interval time, error trigger times, and panel switching frequency using a standardization tool to generate standardized feature data; and a step S203 of grouping the standardized feature data according to operation frequency and error rate through a clustering algorithm to generate a classified behavior feature set.

[0040] In some of the above embodiments, during the implementation of the application, there is also the problem of how to effectively extract behavioral features from the multi-dimensional data set and classify them. To this end, the application further proposes to obtain the operation interval time, the error trigger frequency, and the panel switching frequency from the multi-dimensional data set; process the operation interval time, the error trigger frequency, and the panel switching frequency using a standardization tool to generate standardized feature data; and group the standardized feature data according to the operation frequency and the error rate through a clustering algorithm to generate a classified behavior feature set. This technical solution can effectively extract and classify behavior features through systematic processing and analysis of the multi-dimensional data set, providing a more accurate and reliable data basis for subsequent error prevention design effect evaluation.

[0041] Specifically, first, the operation interval time, the error trigger frequency, and the panel switching frequency are obtained from the multi-dimensional data set. These indicators can comprehensively reflect the time characteristics, accuracy, and complexity of the operation behavior. Among them, the operation interval time reflects the continuity and proficiency of the operation, the error trigger frequency directly reflects the accuracy of the operation, and the panel switching frequency reflects the complexity of the operation and the attention allocation of the operator. Next, the original data is processed using a standardization tool. Standardization is an important data preprocessing step that can eliminate the dimensional differences between different indicators, making the data comparable. For example, the Z-score standardization method can be used to convert each indicator into a standard normal distribution with a mean of 0 and a standard deviation of 1. The data processed in this way is more suitable for subsequent clustering analysis. Finally, according to the standardized feature data, the operation frequency and the error rate are grouped through a clustering algorithm to generate a classified behavior feature set. The clustering algorithm can choose K-means or hierarchical clustering method according to the distribution characteristics of the data. Through clustering, data points with similar operation characteristics can be classified into a class, forming different behavior pattern classifications.

[0042] The advantage of this method is that it can extract the most representative and discriminative behavior features from the massive multi-dimensional data, and through mathematical and standardized processing, these features have good comparability and analyzability. Through the clustering algorithm, these features can be further classified to form different behavior patterns, which provides a scientific basis for subsequent error prevention mechanism design and evaluation.

[0043] As a preferred embodiment, the following specific steps can be considered:

[0044] 1. Data acquisition: extract the operation interval time sequence {t1, t2, …, tn}, the error trigger frequency sequence {e1, e2, …, en}, and the panel switching frequency sequence {f1, f2, …, fn} from the multi-dimensional data set.

[0045] 2. Standardization: Apply Z-score standardization to each sequence. Take the operation interval time as an example, calculate the mean μt and standard deviation σt, then standardize each time point ti: ti' = (ti - μt) / σt. Similarly process the error trigger count and panel switching frequency.

[0046] 3. Feature vector construction: For each operation sample, construct a three-dimensional feature vector (ti', ei', fi').

[0047] 4. Clustering analysis: Use the K-means algorithm, set the number of clusters k (for example, k = 5), and cluster all feature vectors. The algorithm will return k cluster centers and the class label of each sample.

[0048] 5. Behavior feature set generation: According to the clustering results, take each cluster center as a typical behavior feature, and generate a behavior feature set {C1, C2, …, Ck}, where each Ci represents a specific operation behavior pattern.

[0049] Through this method, complex multi-dimensional operation data can be transformed into a limited number of typical behavior patterns, greatly simplifying the subsequent analysis process. For example, a behavior pattern may exhibit "short operation interval time, few error triggers, and high panel switching frequency", which may represent a skilled but possibly too fast operation method. Another pattern may exhibit "long operation interval time, many error triggers, and low panel switching frequency", which may represent an unskilled or fatigued operation state. This classified behavior feature set provides an important basis for the design and evaluation of error prevention mechanisms. For example, different error prevention strategies can be designed for different behavior patterns, or the effectiveness of existing error prevention mechanisms for different behavior patterns can be evaluated. In addition, this method can also be used to monitor the trend of operation behavior and timely discover potential risk behavior.

[0050] Based on the above analysis, the present application is different from the traditional method which may only focus on a single indicator. The present method considers multiple dimensions such as operation interval time, error trigger count and panel switching frequency, and can more comprehensively depict operation behavior. Through standardization, the problem of inconsistent dimensions between different indicators is solved, making the data more comparable and analyzable. Using clustering algorithms to automatically identify different behavior patterns avoids the subjectivity and inconsistency that may be brought by manual division. This method has good scalability and can add or adjust feature indicators as needed to adapt to different application scenarios. By generating a classified behavior feature set, a solid data foundation is provided for subsequent quantitative analysis and evaluation.

[0051] In this embodiment, the step S400 includes: step S401, obtaining the correspondence between the behavior pattern set and the video data set; step S402, aligning the operation behavior timestamps in the behavior pattern set with the video records in the video data set through frame matching function, generating an initial synchronized behavior video set; step S403, if there is data missing in the initial synchronized behavior video set, filling the missing operation logs and video gaps by using an interpolation method, generating the synchronized behavior video set.

[0052] In some of the above embodiments, during the implementation of the present application, there is also the problem of how to accurately synchronize behavior patterns with video data to generate a high-quality synchronized behavior video set. To this end, the present application further proposes a technical solution for generating a synchronized behavior video set from a video data set and a behavior pattern set. This technical solution first obtains the correspondence between the behavior pattern set and the video data set, which lays the foundation for subsequent synchronization operations. By establishing the mapping between behavior patterns and video data, it can be ensured that in subsequent processing, operation behaviors can be accurately corresponded to the corresponding video segments. Next, the operation behavior timestamps in the behavior pattern set are aligned with the video records in the video data set by using frame matching function, generating an initial synchronized behavior video set. This step is the key to accurate synchronization. Frame matching function may involve various algorithms, such as direct matching based on timestamps, image matching based on feature points, etc. Through these technologies, the time points of operation behaviors can be accurately corresponded to video frames, thereby generating a preliminary synchronized behavior video set. After generating the initial synchronized behavior video set, the present application also considers the possibility of data missing. If there is data missing in the initial synchronized behavior video set, the missing operation logs and video gaps are filled by using an interpolation method, finally generating a complete synchronized behavior video set. The interpolation method may include linear interpolation, spline interpolation, etc., the specific choice depends on the characteristics and missing conditions of the data. The advantage of this method is that it can handle the problem of incomplete data that may occur in actual operation, improving the integrity and continuity of the synchronized behavior video set. By filling the missing data, it can be ensured that subsequent analysis will not be biased or erroneous due to data missing.

[0053] In practical application, this method can be implemented as follows:

[0054] Firstly, the correspondence between the set of behavior patterns and the set of video data can be obtained by establishing a mapping table or database. For example, timestamps or unique identifiers can be used to associate each operational behavior with the corresponding video segment. Then, in the frame matching process, a timestamp-based matching algorithm can be employed. Specifically, the timestamp of the operational behavior can be compared with the timestamp of the video frame to find the closest matching point. To improve accuracy, a time threshold can be set, for example, ±0.1 seconds, and only matches within this range are considered valid. For cases of data missing, a linear interpolation method can be used. For example, if there is a missing between two known data points, the estimated value of the intermediate point can be calculated based on the values of the two points. For video gaps, the average of the previous and subsequent frames or more complex image interpolation techniques can be considered to fill in. Through this method, the invention can generate a high-quality synchronized behavior video set, providing a reliable data basis for subsequent error prevention design effect evaluation. Compared with the prior art, the method of the invention not only realizes accurate behavior-video synchronization, but also effectively handles the problem of data missing that may occur in actual operation, thereby improving the integrity of the data and the reliability of the analysis results.

[0055] In the embodiment, the step S500 includes: step S501, obtaining the standard deviation of operation interval time and the mean of error trigger times from the synchronized behavior video set; step S502, generating a difference feature set according to the standard deviation of operation interval time and the mean of error trigger times.

[0056] In some of the above embodiments, during the implementation of the invention, there is also the problem of how to accurately count the difference features under the error prevention mechanism and the non-error prevention mechanism. To this end, the invention further proposes a technical solution for counting the difference feature set under the error prevention mechanism and the non-error prevention mechanism through the synchronized behavior video set. The technical solution of the invention obtains the standard deviation of operation interval time and the mean of error trigger times from the synchronized behavior video set, and generates a difference feature set according to these data. This method can effectively capture the influence of the error prevention mechanism on the operational behavior, providing reliable data support for subsequent evaluation.

[0057] Specifically, the present application first obtains the standard deviation of operation interval time from the synchronized behavior video set. The operation interval time reflects the rhythm and efficiency of the operator in performing the task, and its standard deviation can reflect the operation stability. A smaller standard deviation usually indicates that the operation is more consistent and stable, which may be the result of the error prevention mechanism. Second, the present application obtains the mean of error trigger times. The error trigger times directly reflect the frequency of operation errors and are a key indicator for evaluating the effect of the error prevention mechanism. By calculating the mean, an overall error occurrence level can be obtained, which facilitates comparison under different conditions. Finally, the present application generates a difference feature set based on the above two key indicators. This feature set integrates information from two dimensions of operation stability and error frequency, and can comprehensively reflect the impact of the error prevention mechanism. The technical solution of the present application can clearly show the effect of the error prevention design by comparing the difference feature sets under the conditions with and without the error prevention mechanism. For example, if the standard deviation of the operation interval time is smaller and the mean of the error trigger times is also lower under the condition with the error prevention mechanism, it can be inferred that the error prevention mechanism indeed improves the stability and accuracy of the operation.

[0058] Further, the method of the present application can also analyze the trend of the difference features by setting different time windows. For example, the entire operation process can be divided into multiple time periods, and the difference feature set of each time period can be calculated respectively. In this way, it can be observed whether the effect of the error prevention mechanism changes over time and whether it is more significant in certain specific stages.

[0059] As a preferred embodiment, the present application can combine machine learning algorithms to analyze the difference feature set. For example, support vector machines (SVM) or random forests and other algorithms can be used to take the difference feature set as input and predict whether the operation is under the protection of the error prevention mechanism. This method can automatically identify the features that best distinguish the two cases, further improving the accuracy of the evaluation.

[0060] In addition, the technical solution of the present application can also be used in combination with other evaluation methods. For example, the difference feature set can be combined with subjective scoring to consider both objective data and the subjective feelings of the operator. This multi-dimensional evaluation method can provide a more comprehensive evaluation result of the effect of the error prevention design.

[0061] In a specific embodiment, assume that an error prevention design effectiveness evaluation experiment is conducted in an aviation cockpit simulation system. The experiment is divided into two groups: one with error prevention mechanisms and one without. Each group has 10 pilots participating, and each person completes 5 standard flight tasks. Through a synchronized behavior video set, the interval time and error trigger conditions of each operation are recorded. For the group without error prevention mechanisms, the standard deviation of the operation interval time is 2.5 seconds, and the average number of error triggers is 3.2 times per task. For the group with error prevention mechanisms, the standard deviation of the operation interval time is 1.8 seconds, and the average number of error triggers is reduced to 1.5 times per task. These data are used to generate a difference feature set. By comparing, it is found that the operation of the group with error prevention mechanisms is more stable (the standard deviation is reduced by 28%), and the error rate is significantly reduced (the average is reduced by 53%). This clearly shows the effectiveness of the error prevention mechanism. Further analysis shows that in the initial stage of the task (the first 20% of the time), the difference between the two groups is not obvious. However, in the middle and later stages of the task, especially during periods of high load operation, the effect of the error prevention mechanism is more significant. This finding provides an important reference for optimizing error prevention design and points out the time period that should be focused on.

[0062] Based on the above analysis, the present application has the following advantages: First, by using a synchronized behavior video set to obtain data, the time and behavior are accurately matched, avoiding the data synchronization problem that may occur in traditional methods. Second, by considering both operation interval time and error trigger frequency, a more comprehensive evaluation perspective is provided. Finally, by generating a difference feature set, not only can the differences between the two situations be intuitively compared, but also a foundation is provided for further analysis (such as machine learning). These advantages make the method of the present application more accurate and comprehensive in evaluating the effectiveness of error prevention design, providing strong support for the optimization of aviation cockpit human-computer interaction design.

[0063] In the embodiment, the step S700 includes: a step S701 of denoising and normalizing the multi-dimensional data set to obtain a basic data set; a step S702 of extracting time series data from the basic data set by a sliding window method; a step S703 of separating difference features from the time series data by principal component analysis to determine an independent distribution mode of operation behavior; and a step S704 of grouping according to the independent distribution mode by a clustering algorithm to generate a parameterized representation of diversity influence.

[0064] In some of the above embodiments, there is also the problem of how to accurately capture and quantify the impact of operational behavior diversity in the implementation of the invention. To this end, the invention further proposes a technical solution for extracting operational behavior difference features and time series data from multi-dimensional data sets to determine the parameterized representation of diversity impact. This technical solution aims to solve the problems of existing error prevention design effect evaluation methods, such as difficulty in fully reflecting actual operation complex scenarios, insufficient sample coverage, and insufficient data analysis. Through in-depth analysis and processing of multi-dimensional data sets, this solution can more accurately capture the differences in operational behavior of different flight personnel, thereby providing more reliable and comprehensive data support for the evaluation and optimization of error prevention design.

[0065] Specifically, the technical solution includes the following steps:

[0066] First, the multi-dimensional data set is denoised and normalized to obtain a basic data set. This step aims to eliminate noise interference in the original data and unify data of different dimensions to the same scale, laying the foundation for subsequent analysis. The denoising process can use methods such as wavelet transform or median filtering, while normalization can use techniques such as maximum and minimum value normalization or z-score standardization. Next, time series data is extracted from the basic data set using a sliding window method. The size of the sliding window can be adjusted according to the specific operational task characteristics, for example, a 5-second, 10-second, or longer time window can be selected. This method can capture the dynamic characteristics of operational behavior over time, helping to identify short-term and long-term behavior patterns. Then, principal component analysis (PCA) is used to separate difference features from time series data to determine independent distribution patterns of operational behavior. PCA can effectively reduce the dimension of data while retaining the most important variation information. By selecting an appropriate number of principal components (e.g., explaining more than 80% of the variance), key features of operational behavior can be obtained. Finally, according to the independent distribution patterns, clustering algorithms are used to group, generating parameterized representations of diversity impact. Algorithms such as K-means or hierarchical clustering can be used to classify similar operational behavior patterns. The number of clusters can be determined by methods such as silhouette coefficient or elbow rule, and usually 3-5 categories can be tried. The center point and distribution characteristics of each category can be used as the parameterized representation of diversity impact.

[0067] The advantage of this method is that it can extract key behavior features from a large amount of complex operational data and quantify these features into comparable and analyzable parameters. For example, it may be found that some operators have faster reaction speed but are prone to operational errors when facing unexpected situations, while other operators show more stable but relatively slow operational patterns. These parameterized representations provide specific directions and basis for further optimization of error prevention design.

[0068] In a specific implementation, the following example can be considered:

[0069] Suppose in a simulated flight experiment, the operation data of 20 pilots with different experience levels performing standard take-off procedures is collected. The multi-dimensional dataset includes touch coordinates of the operation panel, key pressure, operation time interval, etc. The sampling frequency is 100 Hz, and the duration is 10 minutes. First, use wavelet transform to denoise the original data and remove high-frequency noise. Then, standardize the data in different dimensions using z-score, so that the mean of all features is 0 and the standard deviation is 1. Next, set a 5-second sliding window and slide it on the standardized data with a step size of 1 second to extract time series features. For each window, calculate statistics such as mean, standard deviation, and kurtosis to form a new feature vector. Then, perform PCA analysis on the extracted feature vectors. Assuming that the first five principal components explain 85% of the variance, retain these five principal components as key features. Finally, use the K-means algorithm to cluster the five principal components, and determine the optimal number of clusters to be 4 by the silhouette coefficient. The four categories may represent "stable type", "fast reaction type", "cautious type", and "error-prone type" and other different operation behavior patterns.

[0070] The center point coordinates and covariance matrix of each category can be used as a parameterized representation of the diversity impact. For example, the "fast reaction type" may be characterized by a high first principal component value (representing fast operation speed) but a low third principal component value (representing low precision).

[0071] Through this method, complex differences in operation behavior can be quantified into specific numerical values and distributions, providing reliable data support for targeted optimization of error prevention design. For example, based on the characteristics of different types of operators, the trigger threshold or feedback method of the error prevention mechanism can be adjusted to adapt to different operation styles and habits.

[0072] Based on the above analysis, the advantages of the present application are:

[0073] 1. It can handle high-dimensional and large-scale operation data and extract the most representative behavior characteristics;

[0074] 2. Through time series analysis, the dynamic changes of operation behavior are captured, not just static statistical characteristics;

[0075] 3. Using unsupervised learning methods, different operation behavior patterns can be automatically discovered and classified, avoiding the bias that may be caused by manually defining classification standards;

[0076] 4. The generated parameterized representation is intuitive and easy to interpret, making it easy for designers to understand and apply.

[0077] In this embodiment, the step S800 includes: a step S801 of extracting adjacent frame data from the video data set to generate an inter-frame difference image sequence; a step S801 of extracting a key point motion trajectory according to the inter-frame difference image sequence; a step S802 of intercepting an involuntary operation segment through the key point motion trajectory; and a step S803 of counting the frequency and duration of the involuntary operation segment to generate a quantitative feature set containing average duration, maximum interval time, and occurrence number.

[0078] In some of the above embodiments, there is also the problem of how to accurately identify and quantify involuntary operations in the implementation of the application. To this end, the application further proposes a technical solution for detecting involuntary operation segments from a video data set and obtaining a quantitative feature set of involuntary operations. This technical solution first extracts adjacent frame data from the video data set to generate an inter-frame difference image sequence. This step uses the differences between consecutive frames in the video data to capture the changes in the operator's actions. By comparing adjacent frames, the changes between frames can be highlighted, thereby providing a basis for subsequent analysis. Next, a key point motion trajectory is extracted from the inter-frame difference image sequence. This step identifies and tracks the motion trajectories of key parts of the operator's body (such as hands, fingers, etc.) by analyzing the change patterns in the difference images. These trajectory data contain detailed action information of the operator during task execution. Then, an involuntary operation segment is intercepted through the key point motion trajectory. This step uses the motion trajectory data obtained earlier to identify action segments that may belong to involuntary operations by setting specific threshold values or pattern recognition algorithms. These segments usually exhibit action sequences that do not conform to the expected operation pattern or differ significantly from normal operation. Finally, the frequency and duration of the involuntary operation segment are counted to generate a quantitative feature set containing average duration, maximum interval time, and occurrence number. This step quantitatively analyzes the identified involuntary operation segments and calculates a series of statistical indicators to provide objective and quantifiable data support for subsequent error prevention design effect evaluation.

[0079] This technical solution achieves automatic identification and quantification of involuntary operations through video analysis technology. Through inter-frame difference and key point tracking, subtle unintended actions can be captured, which may be difficult to find in traditional operation logs or performance indicators. Through statistical analysis, these involuntary operations are converted into quantifiable indicators, providing a more comprehensive and accurate data basis for evaluating the effectiveness of error prevention design.

[0080] Specifically, the generation of the inter-frame difference image sequence can be achieved by calculating the difference of pixel values between adjacent two frames. For example, using image processing libraries such as OpenCV, a subtraction operation can be performed on each pair of adjacent frames to obtain a difference image reflecting the changes in motion. The extraction of the joint motion trajectory can use pose estimation algorithms in computer vision, such as OpenPose or MediaPipe, which can identify and track the positions of human key points in video frames. When intercepting the segment of involuntary operation, thresholds of parameters such as speed, acceleration or trajectory complexity can be set. For example, when the hand movement speed suddenly decreases or the trajectory appears irregular changes, it may indicate that an involuntary operation has occurred. These thresholds can be determined by analyzing normal operation data, usually set to 2-3 standard deviations outside the normal operation parameter distribution. The generation of the quantitative feature set involves the calculation of multiple statistical indicators. The average duration can be obtained by accumulating the duration of all identified involuntary operation segments and dividing by the number of segments. The maximum interval time is determined by recording the longest time interval between two involuntary operations. The occurrence frequency is simply obtained by counting the number of identified involuntary operation segments.

[0081] As a preferred embodiment, the accuracy of involuntary operation recognition can be further improved by combining operation context information. For example, the video analysis results can be associated with system logs, task progress and other information to distinguish between real involuntary operations and intentional non-standard operations. In addition, machine learning models such as Long Short-Term Memory (LSTM) or Convolutional Neural Network (CNN) can be introduced to improve the recognition accuracy of involuntary operations by training a large amount of labeled data. Through this method, the invention not only can identify involuntary operations, but also can perform accurate quantitative analysis. This provides a more objective and comprehensive basis for evaluating the effect of error prevention design. Compared with traditional methods that rely only on operation logs or subjective observation, the technical solution of the invention can capture more subtle behavioral characteristics, thereby more accurately evaluating the effectiveness of error prevention mechanisms.

[0082] In addition, the advantage of this technical solution is its non-invasiveness and high degree of automation. By analyzing existing video data, without additional sensors or equipment, rich behavioral data can be obtained. This not only reduces the experimental cost, but also avoids the interference of additional equipment on the operator's behavior. The automated analysis process also greatly improves efficiency, making large-scale data processing possible, thereby increasing the statistical significance of the evaluation results.

[0083] Based on the above analysis, the present application has the following obvious advantages: first, the traditional method often relies on manual observation or simple operation log analysis, and it is difficult to capture subtle unconscious operation. The present application can identify the subtle motion changes that are difficult to detect by the naked eye through high-precision video analysis technology. Secondly, some existing automatic analysis methods usually only focus on predefined error types, while ignoring the important factor of unconscious operation. The present application can comprehensively capture various forms of unexpected operation through the key node motion trajectory analysis. Finally, in terms of quantitative analysis, the present application provides a more systematic and comprehensive index system, including not only frequency and duration, but also considering factors such as time interval, providing more rich and reliable data support for error prevention design effect evaluation.

[0084] In the embodiment, the step S900 includes: step S901, obtaining the mean and standard deviation of the operation sequence from the quantitative feature set to obtain the operation deviation value; step S902, calculating the change trend by the time window method according to the operation deviation value; step S903, if the change trend exceeds the preset threshold, extracting the response time and error rate from the trigger event record to generate the intervention effect data set; step S904, determining the intervention effect parameter according to the intervention effect data set.

[0085] In some of the above embodiments, during the implementation of the present application, there is also the problem of how to extract the behavior response data related to the error prevention mechanism from the quantitative feature set and determine the intervention effect parameter. To this end, the present application further proposes that the mean and standard deviation of the operation sequence are obtained from the quantitative feature set to obtain the operation deviation value; the change trend is calculated by the time window method according to the operation deviation value; if the change trend exceeds the preset threshold, the response time and error rate are extracted from the trigger event record to generate the intervention effect data set; and the intervention effect parameter is determined according to the intervention effect data set.

[0086] The experimental method for evaluating the effectiveness of error-proofing design proposed in this invention uses a simulation system to generate operating scenarios with and without error-proofing mechanisms, obtains operational behavior data and video recordings, and generates a multidimensional dataset and a video dataset. Behavioral features are extracted from the multidimensional dataset to generate a classified behavioral feature set. Based on the classified behavioral feature set, behavioral pattern sets of different experience levels are determined. Synchronized behavioral video sets are generated from the video dataset and the behavioral pattern set. Based on the synchronized behavioral video sets, statistically analyzes the difference feature sets between the scenarios with and without error-proofing mechanisms. From the difference feature sets, an evaluation result of the impact of the error-proofing mechanism on operational behavior is determined. The difference feature sets of operational behavior and time series data are extracted from the multidimensional dataset to determine a parameterized representation of the diversity effect. Unintentional operation segments are detected from the video dataset to obtain a quantitative feature set of unconscious operation. Behavioral response data related to the error-proofing mechanism are extracted from the quantitative feature set to determine intervention effect parameters. A relationship model between the error-proofing mechanism and task accuracy is fitted using the operational behavior data and task accuracy indicators to generate quantitative results of the design effectiveness.

[0087] In the technical solution proposed by the present invention, the steps of extracting behavioral response data related to the error prevention mechanism from the quantitative feature set and determining the intervention effect parameters can be achieved by:

[0088] First, the mean and standard deviation of the operation sequence are obtained from the quantitative feature set to obtain the operation deviation value. Specifically, the operation sequence data in the quantitative feature set can be statistically analyzed to calculate the mean and standard deviation of each operation sequence. These statistical values ​​reflect the central tendency and dispersion of operation behavior and can be used to measure the stability and consistency of operations. By comparing the mean and standard deviation of different operation sequences, the operation deviation value can be obtained. This value indicates the degree to which the operation behavior deviates from the expected standard.

[0089] Next, calculate the trend of the operation deviation using a time window method. This step can use a sliding time window method to observe the changes in the operation deviation value within a certain time range. For example, you can set a 5-minute time window and calculate the average rate of change of the operation deviation value within the window every 1 minute. This method can generate a trend curve that changes over time, reflecting the dynamic characteristics of operation behavior.

[0090] Next, the system determines whether the trend exceeds a preset threshold. This threshold can be set based on the specific application scenario and security requirements. For example, the threshold could be set to a rate of change exceeding 20% ​​in the operational deviation value. If the trend exceeds this threshold, it indicates a significant abnormal change in operational behavior, potentially requiring intervention.

[0091] When the change trend exceeds the preset threshold, the response time and error rate are extracted from the trigger event record to generate an intervention effect dataset. The trigger event record here can include the time point when the error prevention mechanism is activated, the operator's reaction time, and the operation error rate within a period of time after the intervention, and the like. By extracting these data, a dataset containing comparative information before and after the intervention can be formed to evaluate the actual effect of the error prevention mechanism.

[0092] Finally, the intervention effect parameters are determined according to the intervention effect dataset. This step can calculate a series of parameters reflecting the intervention effect by comparing and analyzing the response time and error rate changes before and after the intervention. For example, the percentage reduction in error rate and the degree of shortening of average response time after intervention can be calculated. These parameters can quantitatively represent the effectiveness of the error prevention mechanism, providing a basis for further optimization design.

[0093] Through the above steps, the present application can effectively extract behavior response data related to the error prevention mechanism from the quantitative feature set, and determine the intervention effect parameters through data analysis. This method not only objectively evaluates the actual effect of the error prevention mechanism, but also captures the dynamic change characteristics of the operation behavior, providing more accurate and reliable data support for the optimization of error prevention design.

[0094] In practical applications, the method of the present application can be implemented in a simulated aircraft cockpit system. For example, a simulated long-haul flight mission can be set up with a duration of 4 hours. In this mission, multiple key points requiring complex operations by the operator are set up, and an error prevention mechanism is implanted in the system. During the experiment, all the operation behaviors of the operator are recorded, including the operation sequence, operation time, error trigger times, etc. In specific implementation, the mean and standard deviation of the operation sequence can be calculated every 10 minutes, i.e., the operation deviation value is calculated every 10 minutes. The time window can be set to 30 minutes, and the operation deviation value within the window is calculated every 5 minutes. The preset threshold can be set to more than 15% of the operation deviation value change rate. When the change trend exceeds the threshold, the system automatically records the time point when the error prevention mechanism is triggered, as well as the operator's response time and error rate within the next 5 minutes. These data will be used to generate the intervention effect dataset. Finally, by comparing the average response time and error rate within 5 minutes before and after the intervention, the intervention effect parameters such as the response time shortening ratio and the error rate reduction percentage are calculated. In this way, the present application can accurately capture and quantify the intervention effect of the error prevention mechanism in the actual operation environment, providing reliable data support for the evaluation and optimization of error prevention design.

[0095] Based on the above analysis, the method proposed by the present application has the following advantages: first, by introducing the time window method and the preset threshold, the abnormal changes of the operation behavior can be more sensitively captured, and the triggering accuracy of the error prevention mechanism is improved. Second, by extracting the response time and error rate before and after the intervention, an intervention effect dataset is formed, making the effect evaluation of the error prevention mechanism more objective and comprehensive. Finally, by calculating specific intervention effect parameters such as response time reduction ratio and error rate reduction percentage, the effects of different error prevention designs can be quantitatively compared, providing a more scientific basis for design optimization. These improvements make the present application more accurate and reliable in evaluating the effect of error prevention design than the prior art, and can better adapt to complex operating environments and diverse operating behaviors.

[0096] In the present embodiment, the method for generating the quantitative results of the design effect includes: extracting the distribution sequence of the task precision index from the multidimensional data set; fitting the relationship model of the task precision index and the error prevention mechanism using a regression analysis algorithm to obtain coefficient parameters; calculating the influence of the operation deviation on the task precision index according to the coefficient parameters to determine the change trend; adjusting the error prevention mechanism parameters through the change trend to generate the quantitative results of the design effect.

[0097] Specifically, first, the distribution sequence of the task precision index is extracted from the multidimensional data set. This step can filter out the indexes related to the task precision from the multidimensional data collected in the experiment, such as operation completion time, error rate, etc., and arrange them into time series data. Then, a regression analysis algorithm is used to fit the relationship model of the task precision index and the error prevention mechanism to obtain coefficient parameters. This step can select appropriate regression models such as linear regression, polynomial regression or generalized linear model, etc., and select the best fitting method according to the data characteristics. Then, the influence of the operation deviation on the task precision index is calculated according to the obtained coefficient parameters to determine the change trend. This step can determine the influence direction and degree of the error prevention mechanism on the task precision by analyzing the positive and negative and size of the coefficient parameters. Finally, the quantitative results of the design effect are generated by adjusting the error prevention mechanism parameters through the change trend. This step can fine-tune the related parameters of the error prevention mechanism according to the influence trend analyzed in the previous step, and obtain the optimized design effect quantitative indicators through simulation calculation.

[0098] In practical application, this technical solution can be used in combination with the other steps mentioned above to form a complete error prevention design effect evaluation process. For example, the operation behavior data and video analysis results obtained in the previous steps can be used as input for fitting the relationship model. At the same time, the output results of this scheme can also be fed back to the previous steps for further optimization of the design of the error prevention mechanism.

[0099] As a preferred implementation, a multiple linear regression model can be chosen to fit the relationship between the error prevention mechanism and task accuracy. The specific steps are as follows:

[0100] First, extract the task accuracy indicators from the multi-dimensional data set, such as operation completion time T, error rate E, and operation fluency S. At the same time, extract the parameters related to the error prevention mechanism, such as prompt frequency F, response delay D, and intervention intensity I. Organize these data into a time series {(T1, E1, S1, F1, D1, I1), (T2, E2, S2, F2, D2, I2), …, (Tn, En, Sn, Fn, Dn, In)}.

[0101] Next, construct a multiple linear regression model: Y = β0 + β1X1 + β2X2 + β3X3 + ε, where Y represents the task accuracy indicator (which can be a weighted combination of T, E, or S), X1, X2, X3 represent F, D, I respectively, β0, β1, β2, β3 are coefficients to be solved, and ε is the error term.

[0102] Then, use the least squares method to estimate the β value, and obtain the fitted model: Y' = b0 + b1X1 + b2X2 + b3X3. By analyzing the positive and negative and size of b1, b2, b3, the influence of each error prevention mechanism parameter on task accuracy can be determined.

[0103] Finally, based on the obtained model, the change of Y' can be predicted by adjusting the values of X1, X2, X3. For example, if it is found that the prompt frequency F (X1) has a significant positive impact on task accuracy, the value of F can be appropriately increased; if it is found that the response delay D (X2) has a negative impact, the value of D can be tried to be reduced. In this way, a set of optimized error prevention mechanism parameters can be obtained, and the corresponding task accuracy prediction value can be calculated as the quantitative result of the design effect.

[0104] The advantage of this technical solution is that it can establish a quantitative relationship between the error prevention mechanism and the task accuracy, making the design effect evaluation more objective and accurate. With the support of mathematical models, the influence of various error prevention design parameters on operation performance can be better understood, providing a clear direction for further optimization. In addition, this method also has good scalability, and more variables or more complex regression models can be added according to actual needs.

[0105] Example Two

[0106] Figure 2 is a structure line frame schematic diagram of an error prevention design effect evaluation experiment system according to an embodiment of the present application, as Figure 2As shown, the embodiment two provides an error prevention design effect evaluation experiment system, which comprises a data set generation module 201, a behavior feature set generation module 202, a mode set determination module 203, a behavior video set generation module 204, a statistical module 205, an influence evaluation result determination module 206, a parameterized representation determination module 207, a quantitative feature set acquisition module 208, an intervention effect parameter determination module 209, and a quantitative result generation module 210. The data set generation module 201 is configured to generate operation scenarios with and without error prevention mechanisms by simulating a system, acquire operation behavior data and video records, and generate multi-dimensional data sets and video data sets. The behavior feature set generation module 202 is configured to extract behavior features from the multi-dimensional data sets and generate classified behavior feature sets. The mode set determination module 203 is configured to determine behavior mode sets of different experience levels according to the classified behavior feature sets. The behavior video set generation module 204 is configured to generate synchronized behavior video sets from the video data sets and the behavior mode sets. The statistical module 205 is configured to statistically determine a difference feature set between error prevention mechanisms and non-error prevention mechanisms according to the synchronized behavior video sets. The influence evaluation result determination module 206 is configured to determine an influence evaluation result of error prevention mechanisms on operation behavior from the difference feature set. The parameterized representation determination module 207 is configured to extract operation behavior difference features and time series data from the multi-dimensional data sets and determine a parameterized representation of diversity influence. The quantitative feature set acquisition module 208 is configured to detect unconscious operation segments from the video data sets and acquire a quantitative feature set of unconscious operations. The intervention effect parameter determination module 209 is configured to extract behavior response data related to error prevention mechanisms from the quantitative feature set and determine intervention effect parameters. The quantitative result generation module 210 is configured to fit a relationship model between error prevention mechanisms and task accuracy by the operation behavior data and task precision indicators, and generate a quantitative result of design effect.

[0107] In the embodiment, the behavior feature set generation module 202 comprises an acquisition unit, a feature data generation unit, and a feature set generation unit. The acquisition unit is configured to acquire operation interval time, error trigger times, and panel switching frequency from the multi-dimensional data sets. The feature data generation unit is configured to process the operation interval time, error trigger times, and panel switching frequency by using a standardization tool to generate standardized feature data. The feature set generation unit is configured to group the standardized feature data according to operation frequency and error rate by a clustering algorithm to generate classified behavior feature sets.

[0108] In the embodiment, the generating behavioral video set module 204 comprises an obtaining correspondence unit, a generating initial synchronized behavioral video set unit and a generating synchronized behavioral video set unit. The obtaining correspondence unit is configured to obtain the correspondence between the behavioral mode set and the video data set. The generating initial synchronized behavioral video set unit is configured to align the operation behavioral timestamp in the behavioral mode set and the video record in the video data set by frame matching function, and generate an initial synchronized behavioral video set. The generating synchronized behavioral video set unit is configured to, if there is data missing in the initial synchronized behavioral video set, fill the missing operation log and video gap by interpolation method, and generate the synchronized behavioral video set.

[0109] In the embodiment, the statistical module 205 comprises an obtaining mean unit and a generating difference feature set unit. The obtaining mean unit is configured to obtain the mean value of the standard deviation of the operation interval time and the error trigger times from the synchronized behavioral video set. The generating difference feature set unit is configured to generate a difference feature set according to the standard deviation of the operation interval time and the mean value of the error trigger times.

[0110] In the embodiment, the determining parameterized representation module 207 comprises an obtaining basic data set unit, an extracting data unit, a determining unit and a generating parameterized representation unit. The obtaining basic data set unit is configured to perform denoising and normalization processing on the multi-dimensional data set to obtain a basic data set. The extracting data unit is configured to extract time series data from the basic data set by sliding window method. The determining unit is configured to separate the difference features from the time series data by principal component analysis, and determine the independent distribution mode of the operation behavior. The generating parameterized representation unit is configured to group the independent distribution mode by clustering algorithm, and generate the parameterized representation of the diversity influence.

[0111] In the embodiment, the obtaining quantitative feature set module 208 comprises a generating image sequence unit, an extracting trajectory unit, a cutting segment unit and a generating quantitative feature set unit. The generating image sequence unit is configured to extract adjacent frame data from the video data set to generate an inter-frame difference image sequence. The extracting trajectory unit is configured to extract the joint motion trajectory according to the inter-frame difference image sequence. The cutting segment unit is configured to cut the unconscious operation segment by the joint motion trajectory. The generating quantitative feature set unit is configured to count the frequency and duration of the unconscious operation segment, and generate a quantitative feature set containing the average duration, the maximum interval time and the occurrence times.

[0112] In the embodiment, the determining intervention effect parameter module 209 comprises: a bias value obtaining unit, a change trend calculating unit, an intervention effect data set generating unit and a parameter determining unit. The bias value obtaining unit is configured to obtain the mean and standard deviation of the operation bias value by the quantitative feature set acquisition operation sequence. The change trend calculating unit is configured to calculate the change trend by the time window method according to the operation bias value. The intervention effect data set generating unit is configured to extract the response time and error rate from the trigger event record to generate the intervention effect data set if the change trend exceeds the preset threshold. The parameter determining unit is configured to determine the intervention effect parameter according to the intervention effect data set.

[0113] In the embodiment, the generating quantitative result module 210 comprises: a distribution sequence extracting unit, a coefficient parameter obtaining unit, a change trend determining unit and a quantitative result generating unit. The distribution sequence extracting unit is configured to extract the distribution sequence of the task precision index from the multi-dimensional data set. The coefficient parameter obtaining unit is configured to fit the relationship model of the task precision index and the error prevention mechanism by the regression analysis algorithm to obtain the coefficient parameter. The change trend determining unit is configured to calculate the influence of the operation bias on the task precision index according to the coefficient parameter to determine the change trend. The quantitative result generating unit is configured to adjust the error prevention mechanism parameter by the change trend to generate the quantitative result of the design effect.

[0114] The various change modes and specific examples of the error prevention design effect evaluation experiment method provided in Embodiment One are also applicable to the error prevention design effect evaluation experiment system provided in the present embodiment. Through the foregoing detailed description of the error prevention design effect evaluation experiment method, those skilled in the art can clearly understand the implementation of the error prevention design effect evaluation experiment system in the present embodiment. Therefore, for the sake of brevity of the specification, the error prevention design effect evaluation experiment system will not be described in detail here.

[0115] In summary, the error prevention design effect evaluation experiment method and system of the present application has the following beneficial effects:

[0116] 1. The operation scenarios with and without the error prevention mechanism are generated by the simulation system, which can systematically compare and analyze the operation behavior differences under the two mechanisms. The generation and analysis of the multi-dimensional data set and the video data set provide comprehensive data support, making the evaluation result more accurate and reliable.

[0117] 2. The behavior features are extracted from the multi-dimensional data set to generate the classified behavior feature set, and the behavior pattern sets of different experience levels are determined according to the feature set. This fine-grained analysis helps to deeply understand the operation behavior differences of different flight personnel. The synchronous behavior video set is generated by the video data set and the behavior pattern set, which can intuitively observe and analyze the operation behavior, further improving the accuracy of the evaluation.

[0118] 3、 Can detect the segment of unconscious operation, and obtain the quantitative feature set of unconscious operation, which is crucial for evaluating the effect of error prevention mechanism in reducing unconscious operation.

[0119] 4、 Consider the diversity of operation behavior, determine the parameterized representation of diversity influence by extracting operation behavior difference features and time series data, which helps to design error prevention mechanism more suitable for different flight personnel needs.

[0120] 5、 By fitting the relationship model of error prevention mechanism and task accuracy, the quantitative results of design effect are generated, and this scientific method provides objective evaluation basis and reduces the influence of subjective judgment.

[0121] 6、 The application is not only suitable for the field of aviation, but also can be popularized to other fields that need error prevention design, such as medical treatment, industrial control, etc., and its wide applicability makes the research results have higher practical value.

[0122] 7、 The problems of insufficient sample coverage, single experimental task and insufficient data analysis in the current evaluation method are solved, and a new idea and method for evaluating the effect of error prevention design are provided.

[0123] The foregoing description of specific exemplary embodiments of the application is intended to be illustrative only and is not intended to limit the application to the precise forms described. Many modifications and variations are possible in light of the above teachings without departing from the spirit or essential characteristics of the application. The exemplary embodiments are chosen and described in order to explain the principles of the application and its practical application to thereby enable others skilled in the art to best utilize the application and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the scope of the application be defined by the claims and their equivalents.

Claims

1. An experimental method for evaluating the effect of error-proofing design, characterized in that: include: Generate operational scenarios with and without error-proofing mechanisms through simulation systems, obtain operational behavior data and video records, and generate multidimensional datasets and video datasets; extracting behavioral features from the multidimensional dataset to generate a classified behavioral feature set; determining a set of behavioral patterns at different experience levels based on the classified behavioral feature set; Generate a synchronized behavior video set using the video data set and the behavior pattern set; Calculating a difference feature set between the video with and without the error protection mechanism according to the synchronous behavior video set; Determining an impact assessment result of an error-proofing mechanism on an operational behavior from the difference feature set; extracting operational behavior difference characteristics and time series data from the multidimensional dataset to determine a parameterized representation of diversity impact; detecting unconscious operation segments according to the video data set, and obtaining a quantitative feature set of the unconscious operation; extracting behavioral response data related to the error prevention mechanism from the quantitative feature set and determining intervention effect parameters; By fitting the relationship model between the error prevention mechanism and task accuracy through the operation behavior data and the task accuracy index, a quantitative result of the design effect is generated; The detecting of unconscious operation segments according to the video data set and obtaining a quantitative feature set of unconscious operation includes: Extracting adjacent frame data from the video data set to generate an inter-frame difference image sequence; Extracting joint point motion trajectories according to the inter-frame difference image sequence; intercepting unconscious operation segments through the motion trajectory of the joint points; Counting the frequency and duration of the unconscious operation segments to generate a quantitative feature set including average duration, maximum interval time, and number of occurrences; The step of extracting behavioral response data related to the error prevention mechanism from the quantitative feature set and determining intervention effect parameters includes: Obtaining the mean and standard deviation of the operation sequence through the quantitative feature set to obtain an operation deviation value; Calculating a change trend using a time window method according to the operation deviation value; If the change trend exceeds a preset threshold, the response time and error rate are extracted from the trigger event record to generate an intervention effect data set; An intervention effect parameter is determined according to the intervention effect data set.

2. The error-proofing design effect evaluation experimental method according to claim 1, characterized in that: Extracting behavioral features from a multidimensional dataset to generate a classified behavioral feature set includes: Obtaining operation interval time, error triggering times, and panel switching frequency through the multidimensional dataset; Using standardized tools to process the operation interval time, error triggering times, and panel switching frequency to generate standardized feature data; The standardized feature data are grouped according to operation frequency and error rate using a clustering algorithm to generate a classified behavior feature set.

3. The error-proofing design effect evaluation experimental method according to claim 1, characterized in that: Generating a synchronized behavior video set using the video data set and the behavior pattern set includes: Obtaining a correspondence between the behavior pattern set and the video dataset; Aligning the operation behavior timestamps in the behavior pattern set with the video records in the video dataset through a frame matching function to generate an initial synchronized behavior video set; If there is data missing in the initial synchronous behavior video set, an interpolation method is used to fill in the missing operation logs and video gaps to generate the synchronous behavior video set.

4. The error-proofing design effect evaluation experimental method according to claim 1, characterized in that: The statistical difference feature set between the error protection mechanism and the absence of the error protection mechanism according to the synchronous behavior video set includes: Obtaining the standard deviation of the operation interval and the mean of the number of error triggering times through the synchronous behavior video set; A difference feature set is generated according to the standard deviation of the operation interval time and the mean of the error triggering times.

5. The error-proofing design effect evaluation experimental method according to claim 1, characterized in that: The extraction of operational behavior difference characteristics and time series data from a multidimensional dataset to determine a parameterized representation of diversity impact includes: Performing denoising and normalization processing on the multidimensional dataset to obtain a basic dataset; extracting time series data from the base dataset using a sliding window method; Using principal component analysis to separate differential features from the time series data and determine independent distribution patterns of operational behaviors; The independent distribution patterns are grouped using a clustering algorithm to generate a parameterized representation of the diversity effect.

6. The error-proofing design effect evaluation experimental method according to claim 1, characterized in that: The relationship model between the error prevention mechanism and task accuracy is fitted by the operation behavior data and the task accuracy index to generate quantitative results of the design effect, including: Extracting a distribution sequence of task precision indicators through the multidimensional dataset; A regression analysis algorithm is used to fit the relationship model between the task accuracy index and the error prevention mechanism to obtain coefficient parameters; Calculate the impact of the operation deviation on the task accuracy index based on the coefficient parameters and determine the change trend; The error-proofing mechanism parameters are adjusted according to the change trend to generate quantitative results of the design effect.

7. An error-proofing design effect evaluation experimental system, characterized in that: include: The dataset generation module is used to generate operation scenarios with and without error-proofing mechanisms through the simulation system, obtain operation behavior data and video records, and generate multidimensional datasets and video datasets; A behavior feature set generation module is used to extract behavior features from the multidimensional data set to generate a classified behavior feature set; a pattern set determination module, configured to determine behavior pattern sets of different experience levels based on the classified behavior feature set; A behavior video set generation module is used to generate a synchronized behavior video set through the video data set and the behavior pattern set; A statistics module, configured to collect statistics of difference feature sets between the video set with and without the error protection mechanism according to the synchronous behavior video set; An impact assessment result determination module, configured to determine an impact assessment result of an error prevention mechanism on an operational behavior from the difference feature set; determining a parameterized representation module, configured to extract operational behavior difference characteristics and time series data from the multidimensional dataset and determine a parameterized representation of diversity impact; A quantitative feature set acquisition module is used to detect unconscious operation segments according to the video data set and acquire a quantitative feature set of unconscious operation; An intervention effect parameter determination module is used to extract behavioral response data related to the error prevention mechanism from the quantitative feature set to determine intervention effect parameters; A quantitative result generation module is used to fit the relationship model between the error prevention mechanism and the task accuracy through the operation behavior data and the task accuracy index to generate a quantitative result of the design effect; The module for obtaining a quantitative feature set includes: An image sequence generating unit, configured to extract adjacent frame data from the video data set to generate an inter-frame difference image sequence; A trajectory extraction unit, configured to extract a joint point motion trajectory according to the inter-frame difference image sequence; A segment capture unit, configured to capture unconscious operation segments through the joint motion trajectory; A quantitative feature set generation unit is used to count the frequency and duration of the unconscious operation segments and generate a quantitative feature set including average duration, maximum interval time, and number of occurrences; wherein the intervention effect parameter determination module includes: A deviation value obtaining unit, configured to obtain a mean and a standard deviation of the operation sequence through the quantitative feature set to obtain an operation deviation value; a change trend calculation unit, configured to calculate a change trend according to the operation deviation value by using a time window method; an intervention effect data set generating unit, configured to extract the response time and error rate from the trigger event record to generate an intervention effect data set if the change trend exceeds a preset threshold; A parameter determination unit is used to determine intervention effect parameters according to the intervention effect data set.

Images