A pilot ability evaluation method based on dynamic time warping and hierarchical clustering

Abstract

The application belongs to the technical field of pilot ability evaluation, and specifically discloses a pilot ability evaluation method based on dynamic time warping and hierarchical clustering, which comprises the following steps: collecting eye movement data of a test pilot during a flight simulation task, pre-processing the eye movement data, extracting behavior indexes of each stage of the flight task, and generating a fixation region number sequence according to the fixation point position; selecting a target evaluation stage, using a dynamic time warping algorithm to perform nonlinear alignment on the fixation sequences of each pilot, and generating an eye movement difference degree matrix between the pilots; grouping the pilots based on the difference degree matrix using a hierarchical clustering algorithm; and outputting the ability evaluation result of the pilot according to the distribution of the pilot in the difference degree matrix and the deviation information of the pilot from the instructor fixation sequence. The method can realize the structured comparison and ability level evaluation of complex cognitive behaviors, and the evaluation result has strong objectivity and interpretability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pilot ability evaluation, and particularly relates to a pilot ability evaluation method based on dynamic time warping and hierarchical clustering. BACKGROUND

[0002] Pilot ability evaluation is a key link in flight training quality control and flight safety guarantee system, and accurate judgment of the comprehensive operation level, perception strategy and attention distribution ability of pilots is particularly important in simulator teaching, flight training stage evaluation and flight personnel selection.

[0003] Existing evaluation methods mostly rely on manual scoring, flight parameter statistics or flight trajectory analysis, and existing technologies usually focus on static evaluation of flight results, and it is difficult to deeply analyze dynamic behavior characteristics such as visual strategy and cognitive load distribution of pilots in the flight process, and there is a lack of systematic analysis framework that can connect eye movement behavior modeling and ability output in a closed loop.

[0004] Especially when performing horizontal comparative analysis among multiple pilot groups, the existing methods often cannot effectively handle the rhythm difference between individual behavior sequences and the non-synchronous behavior in the task execution process, resulting in a lack of robustness in the comparison of behavior characteristics, and further affecting the evaluation accuracy. SUMMARY

[0005] The application provides a pilot ability evaluation method based on dynamic time warping and hierarchical clustering to solve the problem that the rhythm difference between individual behavior sequences and the non-synchronous behavior in the task execution process cannot be effectively handled, resulting in a lack of robustness in the comparison of behavior characteristics, and further affecting the evaluation accuracy in the background technology.

[0006] The technical solution adopted by the application is as follows:

[0007] The application provides a pilot ability evaluation method based on dynamic time warping and hierarchical clustering, which comprises the following steps:

[0008] Step S1, collecting eye movement data of a test pilot during a flight simulation task;

[0009] The eye movement data includes the number of fixations, total fixation duration, average fixation duration, blink frequency and pupil size;

[0010] Step S2, preprocessing the eye movement data, extracting eye movement data of each stage of the flight task, and mapping the fixation area to a fixation area number sequence according to the eye movement data;

[0011] Step S3: Select the target evaluation phase in the flight mission, and use the dynamic time warping algorithm to nonlinearly align the gaze sequences of each pilot in the phase to generate an eye movement difference matrix among the pilots.

[0012] Step S4: Based on the difference matrix, hierarchical clustering algorithm is used to group the pilots;

[0013] Step S5: Compare and analyze the clustering results with the flight instructors' manual scoring results to assess the consistency of classification;

[0014] Step S6: Based on the pilot's distribution position in the difference matrix and its DTW deviation from the standard instructor gaze sequence, output the pilot's ability assessment results.

[0015] Furthermore, step S2 includes the following steps:

[0016] Step S2-1: Preprocess the eye-tracking data and divide the flight process into multiple time periods according to the flight mission phase;

[0017] Step S2-2: Extract the preprocessed eye-tracking data for each time period;

[0018] Step S2-3: Divide the cockpit instruments and the external scene into multiple numbered areas, with each number corresponding to a specific observation target;

[0019] Step S2-4: Map the fixation event to the corresponding numbered area according to the eye-tracking fixation point location;

[0020] Step S2-5: Construct a sequence of gaze area numbers for each pilot during the target flight phase, according to the chronological order.

[0021] Furthermore, step S3 includes the following steps:

[0022] Step S3-1: Select the target assessment phase from the various phases of the flight mission;

[0023] Step S3-2: Combine the sequence of gaze area numbers of all pilots in this phase into pairs;

[0024] Step S3-3: Perform dynamic alignment operation on each pair of sequences and calculate the shortest distance of the gaze region transition path;

[0025] The distance calculation process preserves the original gaze sequence length differences and allows for flexible expansion of the time axis;

[0026] Step S3-4: Organize all alignment distances into a difference matrix between pilots;

[0027] The gaze sequences of the two pilots in the same mission were as follows: and ,in The AOI number indicating the gaze;

[0028] The goal of DTW is to find an optimal curved path. This minimizes the cumulative distance:

[0029]

[0030] in, The symbolic distance function is defined as follows:

[0031]

[0032] The curved path satisfies the following constraints:

[0033] Boundary conditions: ;

[0034] Monotonicity: ;

[0035] Continuous type: Adjacent points on the path are allowed or The steps;

[0036] After obtaining the cumulative distance, normalization is performed to obtain... .

[0037] Furthermore, the construction of the dissimilarity matrix includes the following operations:

[0038] Align each pilot's gaze sequence with that of all other pilots individually;

[0039] Fill the corresponding positions in the symmetric matrix with the distance values ​​obtained from each alignment;

[0040] The feature that retains a diagonal of zero indicates that the pilot's difference from himself is zero.

[0041] The difference value is a non-negative real number; the larger the value, the greater the deviation of the fixation strategy.

[0042] The matrix is ​​a two-dimensional structure used for clustering input, and its dimension is equal to the square of the number of pilots.

[0043] Furthermore, the generation process of the dissimilarity matrix also includes:

[0044] Perform normalization on each value in the difference matrix;

[0045] Define the boundaries between the maximum and minimum values ​​during the normalization process;

[0046] The original gaze sequence was denoised using a sliding window method before constructing the dissimilarity matrix.

[0047] Set alignment path boundary conditions and step limits;

[0048] The processed difference matrix is ​​saved for reuse in multiple clustering experiments.

[0049] Furthermore, step S4 includes the following steps:

[0050] Step S4-1: Use the dissimilarity matrix as the distance input for clustering;

[0051] Step S4-2: Perform clustering using a bottom-up agglomerative hierarchical clustering algorithm;

[0052] Step S4-3: Merge pilot groups using the Ward variance minimization criterion during clustering;

[0053] Each merge operation aims to minimize the differences within the group.

[0054] Step S4-4: Record the structural information generated by the clustering process as a tree diagram data structure, retaining the merging order and grouping relationship.

[0055] Furthermore, hierarchical clustering further includes:

[0056] The clustering result is set to have two clusters to distinguish between high-ability groups and low-ability groups;

[0057] After clustering, output the category label for each pilot;

[0058] The tree diagram is visualized in two dimensions to reflect the similarity of gaze behavior among pilots;

[0059] The human rating results for each pilot are marked in the tree diagram for comparative analysis;

[0060] Pilots at the edge of the clustering process are labeled as samples with uncertain behavior patterns.

[0061] Furthermore, step S5, comparing the clustering results with the human scoring, includes:

[0062] Map each pilot's cluster category to their instructor's rating level;

[0063] Construct a binary cross-tab based on clustering labels and scoring labels;

[0064] The chi-square test was used to test the significance of the consistency between clustering and human scoring.

[0065] Calculate the test statistic and the corresponding p-value to evaluate the validity of the clustering classification results;

[0066] The proportion of pilots that meet the expected clustering criteria is statistically analyzed, and classification effectiveness standards are set based on the indicators.

[0067] Furthermore, the pilot competence assessment results output in step S6 include:

[0068] For each pilot, calculate their gaze sequence alignment distance with all instructors;

[0069] The degree of deviation from expert mode is determined by the average alignment distance;

[0070] Pilots are divided into different ability levels based on a pre-defined ability assessment range;

[0071] The gaze alignment path of the test pilot and a typical instructor is visualized.

[0072] The key periods of attention shift during the flight phase were analyzed using visual graphics.

[0073] Furthermore, the competency assessment results also include:

[0074] Mark the specific time periods during which the pilots' gaze behavior deviated from the instructor's pattern;

[0075] Infer the direction of attentional configuration deviation based on the observed object area corresponding to the deviation behavior;

[0076] By associating the identified deviation behaviors with the flight mission content, the corresponding operational error risks can be located.

[0077] Provides a mapping table between capability assessment results and flight phases, which can be used to generate debriefing scripts or training recommendations;

[0078] The evaluation results are stored in a database to support training tracking and phased capability comparison analysis.

[0079] As can be seen from the above technical solutions, the advantages of the present invention are:

[0080] This invention proposes a pilot competence assessment method based on dynamic time warping and hierarchical clustering. Addressing the aforementioned technical problems, it collects eye-tracking data from pilots during flight simulation missions, preprocesses and encodes the data to construct gaze path sequences suitable for time-series analysis. A dynamic time warping algorithm is used to non-linearly align pilot gaze behaviors, generating a dissimilarity matrix among pilots, thus resolving the incomparability of behavior sequences due to different operational rhythms. Furthermore, a hierarchical clustering method is introduced based on the dissimilarity matrix to group pilots, thereby achieving automatic classification of cognitive behavioral characteristics without prior manual annotation. The clustering results are compared with manual scoring results to effectively verify the classification efficiency of the clustering, and pilot competence assessment levels are output accordingly. Simultaneously, this invention also uses a gaze path alignment map to help locate pilot deviations from expert paradigms during critical mission phases, providing instructors with visual feedback. This method is widely applicable to various flight simulator systems, possessing good adaptability and engineering application value, effectively improving the objectivity, scientific rigor, and systematic nature of pilot competence assessment. Attached Figure Description

[0081] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0082] Figure 1 This is a flowchart illustrating the steps of the pilot competence assessment method based on dynamic time warping and hierarchical clustering in the embodiment.

[0083] Figure 2 This is a schematic diagram illustrating the flight phase division in the embodiment;

[0084] Figure 3 This is a schematic diagram of AOI region division in the embodiment;

[0085] Figure 4 This is a schematic diagram of the difference matrix in the embodiment;

[0086] Figure 5 This is a schematic diagram of the hierarchical clustering results in the example. Detailed Implementation

[0087] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0088] Please see Figures 1 to 5 As shown, this invention provides a pilot competence assessment method based on dynamic time warping and hierarchical clustering. Its theoretical logic and technical approach are as follows:

[0089] First, to move from "steady-state statistics" to "process tracking," a Dynamic Time Warping (DTW) algorithm is used to flexibly align the eye-tracking sequences of different pilots, eliminating differences in operational rhythm (e.g., a trainee looking at the airspeed indicator 2 seconds later than an expert), quantifying temporal differences, and establishing a difference matrix among pilots. Then, hierarchical clustering analysis is used to classify the difference matrix, and visualization methods are employed to map the high-dimensional space to a two-dimensional plane, better displaying the classification results without losing spatial distance between data points. Finally, by locating individuals with significant outliers and combining this with the temporal difference map, the pilot's capabilities are assessed.

[0090] Includes the following steps:

[0091] Step S1: Collect eye movement data of the test pilots during the flight simulation mission;

[0092] Eye movement data includes fixation count, total fixation duration, average fixation duration, blink count, and pupil size;

[0093] Step S2: Preprocess the eye movement data, extract the eye movement data of each stage of the flight mission, and map the gaze region to a gaze region number sequence based on the eye movement data;

[0094] Step S2-1: Preprocess the eye-tracking data and divide the flight process into multiple time periods according to the flight mission phase;

[0095] Step S2-2: Extract the preprocessed eye-tracking data for each time period;

[0096] Step S2-3: Divide the cockpit instruments and the external scene into multiple numbered areas, with each number corresponding to a specific observation target;

[0097] Step S2-4: Map the fixation event to the corresponding numbered area according to the eye-tracking fixation point location;

[0098] Step S2-5: Construct a sequence of gaze area numbers for each pilot during the target flight phase, according to the chronological order.

[0099] Step S3: Select the target evaluation phase in the flight mission, and use the dynamic time warping algorithm to nonlinearly align the gaze sequences of each pilot in the phase to generate an eye movement difference matrix among the pilots.

[0100] Step S3-1: Select the target assessment phase from the various phases of the flight mission;

[0101] Step S3-2: Combine the sequence of gaze area numbers of all pilots in this phase into pairs;

[0102] Step S3-3: Perform dynamic alignment operation on each pair of sequences and calculate the shortest distance of the gaze region transition path;

[0103] The distance calculation process preserves the original gaze sequence length differences and allows for flexible expansion of the time axis;

[0104] Step S3-4: Organize all alignment distances into a difference matrix between pilots;

[0105] The construction of the dissimilarity matrix includes the following operations:

[0106] Align each pilot's gaze sequence with that of all other pilots individually;

[0107] Fill the corresponding positions in the symmetric matrix with the distance values ​​obtained from each alignment;

[0108] The feature that retains a diagonal of zero indicates that the pilot's difference from himself is zero.

[0109] The difference value is a non-negative real number; the larger the value, the greater the deviation of the fixation strategy.

[0110] The matrix is ​​a two-dimensional structure used for clustering input, and its dimension is equal to the square of the number of pilots.

[0111] The generation of the dissimilarity matrix also includes:

[0112] Perform normalization on each value in the difference matrix;

[0113] Define the boundaries between the maximum and minimum values ​​during the normalization process;

[0114] The original gaze sequence was denoised using a sliding window method before constructing the dissimilarity matrix.

[0115] Set alignment path boundary conditions and step limits;

[0116] The processed difference matrix is ​​saved for reuse in multiple clustering experiments.

[0117] Step S4: Based on the difference matrix, hierarchical clustering algorithm is used to group the pilots;

[0118] Step S4-1: Use the dissimilarity matrix as the distance input for clustering;

[0119] Step S4-2: Perform clustering using a bottom-up agglomerative hierarchical clustering algorithm;

[0120] Step S4-3: Merge pilot groups using the Ward variance minimization criterion during clustering;

[0121] Each merge operation aims to minimize the differences within the group.

[0122] Step S4-4: Record the structural information generated by the clustering process as a tree diagram data structure, retaining the merging order and grouping relationship.

[0123] Hierarchical clustering further includes:

[0124] The clustering result is set to have two clusters to distinguish between high-ability groups and low-ability groups;

[0125] After clustering, output the category label for each pilot;

[0126] The tree diagram is visualized in two dimensions to reflect the similarity of gaze behavior among pilots;

[0127] The human rating results for each pilot are marked in the tree diagram for comparative analysis;

[0128] Pilots at the edge of the clustering process are labeled as samples with uncertain behavior patterns.

[0129] Step S5: Compare and analyze the clustering results with the flight instructors' manual scoring results to assess the consistency of classification;

[0130] Step S5, comparing the clustering results with the human scoring, includes:

[0131] Map each pilot's cluster category to their instructor's rating level;

[0132] Construct a binary cross-tab based on clustering labels and scoring labels;

[0133] The chi-square test was used to test the significance of the consistency between clustering and human scoring.

[0134] Calculate the test statistic and the corresponding p-value to evaluate the validity of the clustering classification results;

[0135] The proportion of pilots that meet the expected clustering criteria is statistically analyzed, and classification effectiveness standards are set based on the indicators.

[0136] Step S6: Based on the pilot's distribution position in the difference matrix and its DTW deviation from the standard instructor gaze sequence, output the pilot's ability assessment results.

[0137] Step S6 outputs the pilot competence assessment results, including:

[0138] For each pilot, calculate their gaze sequence alignment distance with all instructors;

[0139] The degree of deviation from expert mode is determined by the average alignment distance;

[0140] Pilots are divided into different ability levels based on a pre-defined ability assessment range;

[0141] The gaze alignment path of the test pilot and a typical instructor is visualized.

[0142] The key periods of attention shift during the flight phase were analyzed using visual graphics.

[0143] The competency assessment results also include:

[0144] Mark the specific time periods during which the pilots' gaze behavior deviated from the instructor's pattern;

[0145] Infer the direction of attentional configuration deviation based on the observed object area corresponding to the deviation behavior;

[0146] By associating the identified deviation behaviors with the flight mission content, the corresponding operational error risks can be located.

[0147] Provides a mapping table between capability assessment results and flight phases, which can be used to generate debriefing scripts or training recommendations;

[0148] The evaluation results are stored in a database to support training tracking and phased capability comparison analysis.

[0149] In this embodiment, multiple instructors and multiple students participated in the experiment. Due to interference during the data collection process, the data from one instructor and one student were excluded.

[0150] This embodiment uses a high-fidelity full-mission flight simulator as the experimental platform. This simulator is based on a modular architecture design and includes a six-degree-of-freedom electric motion platform, a high-resolution visual system, a full-size cockpit, and an avionics simulation system, capable of accurately replicating the aerodynamic characteristics and control feedback of a certain type of aircraft. To capture the visual attention allocation characteristics of the subjects, a head-mounted eye-tracking system was used in the experiment, employing a binocular sampling mode.

[0151] The experiment requires participants to complete a standard takeoff and landing flight path, covering the entire five-sided flight phase. It primarily trains participants' basic operational skills and attention allocation abilities. This embodiment divides the entire flight path into five phases: takeoff, turn and climb, three-sided level flight, turn and descent, and landing. Figure 2 The tasks for each stage are given.

[0152] It can be seen that different missions require observation of different instruments and landmarks, and the difficulty of the missions also varies. During the turning and descent phase, which involves a decrease in altitude, the pilot's attention needs to quickly cycle between the outside and inside of the cockpit. In contrast, during the three-terminal phase, the aircraft's state is stable, and the pilot only needs to devote less energy. Therefore, the turning and descent phase is significantly more difficult than the three-terminal phase.

[0153] Therefore, this embodiment first statistically analyzes the eye movement data of participants in different AOI areas in stages, including fixation count, total fixation duration, average fixation duration, blink count, and pupil size data. The definitions of these indicators are shown in Table 1, which is a table of the meanings of eye movement data. The AOI area division references pilots' suggestions. When performing a mission, pilots, in addition to observing the instruments in the cockpit, also need to judge the aircraft's status based on landmarks and the positional relationship between the nose and the horizon. Therefore, according to... Figure 3 As shown, a total of 12 AOIs were set up. Using the quartile method, outliers were searched in the first three eye-tracking data points. An attempt was made to identify individuals with problems through outliers. If outliers only appeared in the trainee group, then the pilot's ability could be assessed by the size and number of outliers.

[0154] Secondly, to verify the stability of the instructor group's eye-tracking data, repeat tests were conducted on the instructors at 7-day intervals (to avoid fatigue effects), and the intragroup correlation coefficient (ICC) test was used for consistency. The consistency of attention allocation among the instructor group is fundamental to subsequent work; if the consistency is not significant, it will seriously affect the later classification results.

[0155] Third, to verify whether there are differences in attention allocation between the instructor and student groups at different stages, an independent samples t-test was used for analysis. The differences in attention allocation between different groups form the basis for subsequent work; a statistically significant difference indicates that there must be a difference over time.

[0156] Finally, based on the confirmed existence of differences, the DTW algorithm was used to compare the eye movement time sequences of the participants pairwise to obtain the difference matrix. Then, the hierarchical clustering algorithm was used to cluster the difference matrix (with the number of clusters set to K=2), and the chi-square test was used to evaluate the consistency between the clustering results and the instructor's manual rating (high group ≥4 / 5, low group <4).

[0157] Table 1

[0158]

[0159] After defining and encoding the AOI regions, the sequence of each AOI was observed, forming a set of eye-tracking time sequences.

[0160] For example, if 1 represents the horizon, 2 represents the altimeter, 3 represents the speedometer, and 4 represents the external velocimetry, then the attention allocation sequence over a period of time might be 1324412.

[0161] In this embodiment, the gaze sequences of the two pilots in the same mission are as follows: and ,in The AOI number indicating the gaze (1: horizon, 2: altimeter, 3: speedometer, 4: external).

[0162] The goal of DTW is to find an optimal curved path. This minimizes the cumulative distance:

[0163]

[0164] in, The symbolic distance function is defined as follows:

[0165]

[0166] The curved path satisfies the following constraints:

[0167] Boundary conditions: ;

[0168] Monotonicity: ;

[0169] Continuous type: Adjacent points on the path are allowed or The steps;

[0170] After obtaining the cumulative distance, normalization is performed to obtain... .

[0171] The core reasons for choosing agglomerative hierarchical clustering in this study include:

[0172] Input adaptability: The difference matrix generated by DTW can be directly used as input without feature vector transformation;

[0173] Cluster number flexibility: The number of clusters can be dynamically selected using a dendritic chart;

[0174] Explainability: The tree structure clearly demonstrates the progressive grouping process among the pilots;

[0175] Ward variance optimization: Employing the Ward connectivity criterion, we minimize intra-cluster variance to ensure compactness of eye-tracking patterns among similar pilots.

[0176] For N valid data points from N participants, taking the eye movement time series during the descent phase of a turn as an example (fixation meter coded sequence), the pairwise DTW distance is calculated to generate an N×N dissimilarity matrix, as shown below. Figure 4 As shown, color gradation differences are used to represent the degree of difference. The graph reveals that instructors have a high degree of similarity, while students have lower similarity with instructors and with each other.

[0177] Hierarchical clustering was used to analyze the dissimilarity matrix, and the clustering hierarchy diagram is shown below. Figure 5 As shown.

[0178] The chi-square test was used to test the consistency, which showed a significant association between clustering and human scoring, indicating that the two groups differed significantly in operational performance.

[0179] Furthermore, according to the DTW algorithm principle, in the dynamic time warping graph, the line segments extending upwards at 90° and those extending parallel to the right represent the parts where the two time series differ. Therefore, the dynamic time warping graph can be used to identify the differences between trainees and instructors. Combining this with eye-tracking video for post-flight debriefing can effectively correct trainees' incorrect attention allocation.

[0180] In some embodiments, this application provides a pilot competence assessment system based on dynamic time warping and hierarchical clustering. This system can be applied to flight simulator platforms and achieves quantitative assessment and visual feedback of pilot competence through dynamic analysis of pilot eye-tracking behavior. The system includes the following functional modules:

[0181] First, the system includes a data acquisition module for real-time acquisition of the pilot's eye movement data during flight simulation missions. The acquired data includes, but is not limited to, physiological and behavioral parameters such as fixation count, total fixation duration, average fixation duration, blink count, and pupil size. The eye movement acquisition device can be a head-mounted eye tracker with a sampling frequency of at least 60Hz.

[0182] The system also includes a data preprocessing module, which segments the collected raw eye-tracking data according to the flight mission phase and, in conjunction with the cockpit visual field structure, divides the instrument area and external view into multiple gaze regions. Each gaze region is assigned a unique number. Based on the gaze point coordinates and timestamp information, the data preprocessing module maps the raw gaze events to the corresponding numbered regions, forming a time-ordered sequence of gaze region numbers, which serves as input for subsequent analysis.

[0183] The system further includes a sequence alignment analysis module for executing a dynamic time warping algorithm. This module selects the target evaluation phase (such as the turn-descent phase) from the aforementioned gaze region numbering sequence, performs non-linear alignment processing on the gaze paths of all pilots, calculates the sequence differences between pilots, and outputs an eye-tracking difference matrix between pilot pairs. This matrix measures the degree of deviation in visual strategy among different pilots.

[0184] To further manage the pilot population hierarchically, the system includes a clustering evaluation module. This module receives a dissimilarity matrix as input and uses a bottom-up hierarchical clustering method to group the pilots. Specifically, the Ward variance minimization criterion is used during the clustering process to iteratively merge pilots, ultimately forming multiple capability category labels, with each pilot corresponding to a cluster classification result. The cluster structure can be used to generate a dendrogram for subsequent display.

[0185] The system also includes a results comparison module, which is used to compare and analyze the clustering results with the flight instructors' manual scoring results, construct cross-tabulations and perform consistency checks, such as using chi-square tests, to verify the effectiveness and accuracy of the clustering model in distinguishing abilities.

[0186] Based on the clustering and alignment results described above, the system also includes a capability assessment module. This module outputs the pilot's capability assessment level based on the DTW alignment distance between the pilot's and the standard instructor's gaze paths, their relative positional relationship in the dissimilarity matrix, and cluster label information. The assessment results can be refined into multiple levels or grouped according to a threshold set based on distance deviation.

[0187] To enhance training feedback, the system also includes a visualization module. This module displays gaze alignment path diagrams, difference heatmaps, and competence grading results between pilots and instructors, supporting interactive graphical debriefing. Deviation path segments can be highlighted to help instructors identify areas where trainees' attention allocation is insufficient during critical mission phases, thereby improving the relevance and effectiveness of debriefing.

[0188] In some embodiments, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the pilot capability assessment method based on dynamic time warping and hierarchical clustering.

[0189] It is understood that the systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can be a personal computer, a laptop computer, a personal digital assistant, a tablet computer, a wearable device, or any combination of these devices.

[0190] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0191] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0192] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this embodiment, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0193] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0194] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0195] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A pilot competence assessment method based on dynamic time warping and hierarchical clustering, characterized in that, Includes the following steps: Step S1: Collect eye movement data of the test pilots during the flight simulation mission; Eye movement data includes fixation count, total fixation duration, average fixation duration, blink count, and pupil size; Step S2: Preprocess the eye movement data, extract the eye movement data of each stage of the flight mission, and map the gaze region to a gaze region number sequence based on the eye movement data; Step S3: Select the target evaluation phase in the flight mission, and use the dynamic time warping algorithm to nonlinearly align the gaze sequences of each pilot in the phase to generate an eye movement difference matrix among the pilots. Step S3-1: Select the target assessment phase from the various phases of the flight mission; Step S3-2: Combine the sequence of gaze area numbers of all pilots in this phase into pairs; Step S3-3: Perform dynamic alignment operation on each pair of sequences and calculate the shortest distance of the gaze region transition path; The distance calculation process preserves the original gaze sequence length differences and allows for flexible expansion of the time axis; Step S3-4: Organize all alignment distances into a difference matrix between pilots; The gaze sequences of the two pilots in the same mission were as follows: and ,in The AOI number indicating the gaze; The goal of DTW is to find an optimal curved path. This minimizes the cumulative distance: in, The symbolic distance function is defined as follows: The curved path satisfies the following constraints: Boundary conditions: ; Monotonicity: ; Continuous type: Adjacent points on the path are allowed or The steps; After obtaining the cumulative distance, normalization is performed to obtain... ; Step S4: Based on the difference matrix, hierarchical clustering algorithm is used to group the pilots; Step S5: Compare and analyze the clustering results with the flight instructors' manual scoring results to assess the consistency of classification; Step S6: Based on the pilot's distribution position in the difference matrix and its DTW deviation from the standard instructor gaze sequence, output the pilot's ability assessment results. The output of pilot competency assessment results includes: For each pilot, calculate their gaze sequence alignment distance with all instructors; The degree of deviation from expert mode is determined by the average alignment distance; Pilots are divided into different ability levels based on a pre-defined ability assessment range; The gaze alignment path of the test pilot and a typical instructor is visualized. Combined with visual analysis, key periods of attention shift during the flight phase were identified; The visualization includes: Calculate the DTW distance between each pair of elements to generate an N×N difference matrix, and use color level difference to represent the magnitude of the difference.

2. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 1, characterized in that, Step S2 includes the following steps: Step S2-1: Preprocess the eye-tracking data and divide the flight process into multiple time periods according to the flight mission phase; Step S2-2: Extract the preprocessed eye-tracking data for each time period; Step S2-3: Divide the cockpit instruments and the external scene into multiple numbered areas, with each number corresponding to a specific observation target; Step S2-4: Map the fixation event to the corresponding numbered area according to the eye-tracking fixation point location; Step S2-5: Construct a sequence of gaze area numbers for each pilot during the target flight phase, according to the chronological order.

3. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 1, characterized in that, The construction of the dissimilarity matrix includes the following operations: Align each pilot's gaze sequence with that of all other pilots individually; Fill the corresponding positions in the symmetric matrix with the distance values ​​obtained from each alignment; The feature that retains a diagonal of zero indicates that the pilot's difference from himself is zero. The difference value is a non-negative real number; the larger the value, the greater the deviation of the fixation strategy. The matrix is ​​a two-dimensional structure used for clustering input, and its dimension is equal to the square of the number of pilots.

4. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 3, characterized in that, The generation of the dissimilarity matrix also includes: Perform normalization on each value in the difference matrix; Define the boundaries between the maximum and minimum values ​​during the normalization process; The original gaze sequence was denoised using a sliding window method before constructing the dissimilarity matrix. Set alignment path boundary conditions and step limits; The processed difference matrix is ​​saved for reuse in multiple clustering experiments.

5. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 1, characterized in that, Step S4 includes the following steps: Step S4-1: Use the dissimilarity matrix as the distance input for clustering; Step S4-2: Perform clustering using a bottom-up agglomerative hierarchical clustering algorithm; Step S4-3: Merge pilot groups using the Ward variance minimization criterion during clustering; Each merge operation aims to minimize the differences within the group. Step S4-4: Record the structural information generated by the clustering process as a tree diagram data structure, retaining the merging order and grouping relationship.

6. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 5, characterized in that, Hierarchical clustering further includes: The clustering result is set to have two clusters to distinguish between high-ability groups and low-ability groups; After clustering, output the category label for each pilot; The tree diagram is visualized in two dimensions to reflect the similarity of gaze behavior among pilots; The human rating results for each pilot are marked in the tree diagram for comparative analysis; Pilots at the edge of the clustering process are labeled as samples with uncertain behavior patterns.

7. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 1, characterized in that, Step S5, comparing the clustering results with the human scoring, includes: Map each pilot's cluster category to their instructor's rating level; Construct a binary cross-tab based on clustering labels and scoring labels; The chi-square test was used to test the significance of the consistency between clustering and human scoring. Calculate the test statistic and the corresponding p-value to evaluate the validity of the clustering classification results; The proportion of pilots that meet the expected clustering criteria is statistically analyzed, and classification effectiveness standards are set based on the indicators.

8. The pilot competence assessment method based on dynamic time warping and hierarchical clustering according to claim 1, characterized in that, The competency assessment results also include: Mark the specific time periods during which the pilots' gaze behavior deviated from the instructor's pattern; Infer the direction of attentional configuration deviation based on the observed object area corresponding to the deviation behavior; By associating the identified deviation behaviors with the flight mission content, the corresponding operational error risks can be located. Provides a mapping table between capability assessment results and flight phases, which can be used to generate debriefing scripts or training recommendations; The evaluation results are stored in a database to support training tracking and phased capability comparison analysis.

Images