A method for evaluating pilot multitasking performance in complex situations

Abstract

This invention belongs to the technical fields of aviation safety management and pilot performance evaluation. It relates to a method for evaluating pilot multi-task performance in complex scenarios, aiming to address the gap in multi-task performance evaluation caused by traditional methods failing to fully consider the real flight environment. The invention includes: acquiring and preprocessing flight parameter data, eye-tracking data, and task data collected by the pilot during flight; filtering and calculating key parameter datasets from three dimensions: attention allocation, working memory and task switching, and performance in handling unexpected tasks; dividing the flight into phases based on the flight parameter data, combining flight phase transition maps and weighted multi-dimensional matching distances; pre-setting a weight matrix according to the cognitive needs of different flight phases; matching the pre-set weight matrix with the flight phase division results, and weighting and fusing the key parameter values ​​to obtain the final performance evaluation value. This invention combines multi-source data analysis to achieve efficient evaluation of pilot multi-task performance.

Description

Technical Field

[0001] This invention belongs to the fields of aviation safety management and pilot performance evaluation technology, and specifically relates to a method for evaluating the multi-task performance of pilots in complex situations. Background Technology

[0002] In the aviation field, the scientific evaluation of pilots' task processing efficiency is a core element in ensuring flight safety and optimizing cockpit operation procedures. Currently, several patents relate to pilot workload assessment: for example, the patent "System and Method for Assessing Pilot Workload Using Multi-Source Data" (CN118691142A) utilizes real-time physiological monitoring data to identify pilots' cognitive load capacity, analyze their attention allocation and decision-making abilities, and then assess their cognitive load status in real-time tasks, outputting workload assessment results; the patent "An Online Assessment Method for Pilot Task Load Based on Eye Movement and Physiological Data" (CN113229791A) collects eye movement and physiological data, classifies the data based on a hierarchical support vector machine model, constructs a five-category online assessment model for task load, and calculates and provides feedback on task load levels in real time.

[0003] However, existing technologies have certain limitations. On the one hand, the evaluation dimensions are relatively singular, mostly focusing on the pilot's operational skills or workload under a single task, failing to fully cover the complex scenarios of multiple tasks operating in parallel during actual flight, such as "flight status monitoring, air traffic control command response, and equipment malfunction handling," and lacking a systematic evaluation of multi-task collaborative processing efficiency. On the other hand, although existing technologies combine physiological data to classify task load and determine whether the pilot is in a low, relatively low, medium, high, or high task load level, this only stays at the level of load intensity judgment and does not delve into the evaluation of the pilot's processing efficiency in multi-task scenarios (such as task priority judgment, parallel processing efficiency, and error rate control). In summary, existing technologies have not achieved a comprehensive evaluation of the pilot's multi-task processing efficiency and are unable to truly reflect their comprehensive task processing capabilities in complex flight situations. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies—namely, the inability to comprehensively reflect a pilot's true mission processing performance in complex multi-task scenarios due to their singular evaluation dimensions and lack of multi-task performance evaluation, thus affecting flight safety and efficiency—this invention, in its first aspect, proposes a method for evaluating pilot multi-task performance in complex scenarios. This method includes the following steps:

[0005] The flight parameter data, eye-tracking data, and mission data of the pilot during flight are acquired and preprocessed to obtain preprocessed data. The preprocessing includes time synchronization, data cleaning, segmentation and labeling, and region of interest division. The time synchronization includes using a sliding time window and cognitive alignment synchronization method to achieve time synchronization of multi-source data.

[0006] The eye-tracking data and task data in the preprocessed data are feature-extracted and quantified from three dimensions: attention allocation, working memory and task switching, and efficiency in handling sudden tasks, so as to obtain the values ​​of each key parameter and thus obtain the key parameter dataset.

[0007] Based on the different cognitive needs of each flight phase, a weight matrix is ​​preset using the analytic hierarchy process and applied to the key parameter dataset.

[0008] By combining the flight phase transition map and the weighted multidimensional matching distance, the flight phases are divided using the preprocessed flight parameter data, and the flight phase division results are obtained.

[0009] Based on the flight phase division results, the preset weight matrix is ​​matched, and the values ​​of each key parameter are combined to obtain the final multi-task processing performance evaluation value after weighted fusion.

[0010] In some preferred embodiments, the eye-tracking data includes fixation time, scan rate, and pupil diameter;

[0011] The task data includes data on handling sudden tasks, data on parallel processing of multiple tasks, data on task switching, and task priority tags.

[0012] The data for handling sudden tasks includes first fixation time, attention recovery time, switching delay, and error rate.

[0013] In some preferred embodiments, the segmentation labeling includes labeling by task stage and labeling by subtask;

[0014] The area of ​​interest is divided based on cockpit functional zones, including the main flight display, navigation display, and radio control panel.

[0015] In some preferred embodiments, the quantified dataset includes attention allocation parameters, working memory and task switching parameters, and burst task processing performance parameters;

[0016] The attention allocation parameters include the standardized region fixation time percentage, standardized scan rate, pupil diameter change, standardized first fixation time, and attention recovery rate.

[0017] The working memory and task switching parameters include switching latency, error rate variation, and parallel task interference.

[0018] The performance parameters for handling sudden tasks include standardized response time and decision parameter values.

[0019] In some preferred embodiments, the standardized area gaze time percentage is obtained by: calculating the ratio of the pilot's total gaze time in the key instrument, communication interface, and navigation screen areas to the total mission time, obtaining the area gaze time percentage, and then standardizing the area gaze time percentage to obtain the standardized area gaze time percentage.

[0020] The standardized scanning rate is obtained by: calculating the ratio of the number of gaze shifts per unit time to the total time to obtain the scanning rate; and standardizing the scanning rate based on preset lower and upper limits to obtain the standardized scanning rate.

[0021] The pupil diameter change is obtained by calculating the difference between the average pupil diameter during a sudden mission and the average pupil diameter during a regular mission, and then calculating the ratio of the difference to the standard deviation of the pupil diameter during a regular mission to obtain the pupil diameter change.

[0022] The standardized first gaze time is obtained by: calculating the difference between the time node when the pilot gazes at the relevant area of ​​the emergency mission and the time node when the emergency mission begins, and standardizing the difference to obtain the standardized first gaze time.

[0023] The attention recovery rate is obtained by calculating the difference between the end time of the current task and the end time of the sudden task; and by calculating the ratio of the gaze duration when returning to the original task to the difference.

[0024] In some preferred embodiments, the switching delay is obtained by: when the first operation response time of task A in a dual-task environment is shorter than the average response time when only task A is executed, it is directly assigned a value of 1;

[0025] Otherwise, calculate the relative difference between the first operation response time of task A in the dual-task environment and the average response time when only task A is executed;

[0026] The error rate change is obtained by using the relative change rate of the error rate of task A in a dual-task environment to that of task A in a single-task environment.

[0027] The method for obtaining the parallel task interference degree is as follows: taking the completion quality of task A in the single-task state as the benchmark, calculate the deviation of the completion quality of task A when two tasks are parallel.

[0028] In some preferred embodiments, the standardized response time is obtained by: calculating the actual time difference between the time the emergency occurs and the time the pilot begins to react; dividing the actual time difference by the maximum reaction time and then standardizing the result to obtain the standardized response time.

[0029] In some preferred embodiments, the time synchronization employs a sliding time window and cognitive alignment synchronization method to achieve time synchronization of multi-source data, specifically including the following steps:

[0030] Step 11: Obtain the pilot's eye-tracking data sequence, mission data sequence, and their corresponding timestamp sequence during the flight mission;

[0031] Step 12: Preprocess the timestamps of the eye-tracking data and task data to obtain a corrected timestamp sequence; the preprocessing includes linear time alignment, abnormal timestamp detection, and missing value handling;

[0032] Step 13: Based on the corrected timestamp sequence and the set time window, perform coarse synchronization alignment of eye movement and task data using a sliding time window to construct a set of synchronized sample pairs;

[0033] Step 14: Based on adaptive optimal transmission cognitive alignment feature fusion, perform semantic-level fine alignment and feature fusion on the set of synchronized sample pairs to generate a fused feature sequence as the time-synchronized data.

[0034] In some preferred embodiments, the method for dividing the flight parameter data in the preprocessed data into flight stages by combining the flight stage transition map and the weighted multidimensional matching distance to obtain the flight stage division result is as follows:

[0035] S1, obtain the flight situation segment set and the preset flight stage vertical motion situation set, extract multi-dimensional features of the situation segment and perform weighted processing, and construct the situation segment feature vector set and the preset stage set with boundary constraints;

[0036] S2, based on a preset set of stages, constructs a flight stage transfer graph with stages as nodes and reasonable transfer relationships as edges;

[0037] S3. Based on the flight phase transition map, the weighted multidimensional matching distance is calculated by fusing the feature differences between the situation segment and the preset phase and the boundary violation penalty.

[0038] S4, based on the weighted multidimensional matching distance and trajectory data, map each situation segment into a sequence of center state points in a two-dimensional space;

[0039] The normalized curvature is calculated by measuring the ratio of the perpendicular distance from a point in the central state point sequence to the length of the line segment;

[0040] The normalized curvature, combined with the dynamic penalty term and the structural consistency penalty coefficient, is used to construct a structure-sensitive matching function to generate a comprehensive matching score through structure-sensitive matching.

[0041] S5. Generate the original matching confidence score based on the comprehensive matching score, and fuse the original matching confidence score, the context consistency score, and the historical average confidence score of situational segment matching to obtain the fused confidence score.

[0042] S6, select the best flight phase matching the current situation segment based on the fused confidence level; the flight phase includes takeoff phase, climb phase, cruise phase, descent phase, approach phase, and landing phase.

[0043] In some preferred embodiments, the final multi-task processing performance evaluation value is obtained after the weighted fusion, and the calculation method is as follows:

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] In the formula, This is the final multitasking performance evaluation value. , , These are the scores for attention allocation, working memory and task switching, and performance in handling unexpected tasks. , , These are the parameters for attention allocation, working memory and task switching, and burst task processing performance, respectively. The parameter in the first... The weighting coefficients corresponding to each flight phase , , These are the parameters for attention allocation, working memory and task switching, and burst task processing performance, respectively. The numerical values ​​of each parameter.

[0049] The beneficial effects of this invention are:

[0050] 1) Existing technologies for analyzing pilot states are often limited to a single dimension of operational skills or isolated inferences of cognitive load. Such simplified models cannot fully reflect the complexity of the real flight environment. This invention continues the approach of quantitative analysis of eye-tracking data and task data, and further objectively characterizes the comprehensive performance of pilots in multi-task parallel processing in highly dynamic environments through rigorous correlation modeling. Specifically, it is reflected in the pilot's ability to effectively allocate attention, quickly switch task focus, accurately execute key operations, and maintain situational awareness under the constraints of limited cognitive resources. This design overcomes the limitations of traditional methods that rely solely on operational skills or a single load indicator. Its representation results more realistically reflect the core capabilities that pilots must simultaneously master in actual flight, including piloting, monitoring, communication, and decision-making, which are highly consistent with the actual needs of multi-task parallel processing in real flight scenarios.

[0051] 2) Pilots face significant differences in mission nature, peak cognitive load, operational precision requirements, and risk levels during different flight phases, such as takeoff, climb, cruise, approach, and landing. Based on this, this invention continues the idea of ​​dynamically dividing flight phases and further constructs an adaptive weighting mechanism: by finely identifying the characteristics of each phase, the evaluation weights are adjusted in a targeted manner to adapt to the characteristics of the phase mission. This can measure the performance fluctuations under high load and variable scenarios in complex real flight scenarios (such as approach in severe weather, emergency fault handling, and high-density airspace operations), providing a highly scenario-based objective basis for targeted training planning and capability improvement, and solving the problem that existing technologies cannot comprehensively measure pilots' ability to cope with complex scenarios. Attached Figure Description

[0052] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0053] Figure 1 This is a flowchart of the steps of a method for evaluating the multi-task processing effectiveness of pilots in complex situations, according to the present invention.

[0054] Figure 2 This is a multi-task processing performance evaluation model diagram of a pilot multi-task processing performance evaluation method under complex situations according to the present invention. Detailed Implementation

[0055] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] To more clearly explain the pilot multitasking performance evaluation method under complex situations according to the present invention, the following is in conjunction with... Figures 1 to 2 The steps in the embodiments of the present invention will be described in detail below.

[0058] This invention proposes a method for evaluating pilot multitasking performance in complex situations. See [link to relevant documentation]. Figure 1 The method includes the following steps:

[0059] Step 1: Acquire flight parameter data, eye-tracking data, and mission data during the pilot's flight and preprocess them to obtain preprocessed data; the preprocessing includes time synchronization, data cleaning, segmentation and labeling, and region of interest division; the time synchronization includes using a sliding time window-based cognitive alignment synchronization method to achieve time synchronization of multi-source data;

[0060] In this embodiment, the eye-tracking data includes fixation time, saccade rate, pupil diameter, fixation point distribution, and saccade path; the task data includes emergency task processing data, multi-task parallel processing data, task switching data, task priority tags, and task completion time.

[0061] The emergency task processing data includes first gaze time, attention recovery time, switching delay, error rate, emergency situation trigger time, emergency task response time and processing result; it also includes error types, such as communication delay and navigation deviation.

[0062] The time synchronization includes using a sliding time window and cognitive alignment synchronization method to achieve time synchronization of multi-source data, specifically including the following steps:

[0063] Step 11: Obtain the pilot's eye-tracking data sequence E={e1,e2,...,eT} and mission data sequence T={t1,t2,...,tN} during the flight mission, as well as the timestamp sequence of the eye-tracking data. Timestamp sequence of task data ;

[0064] Step 12: Preprocess the timestamps of the eye-tracking data and task data to obtain a corrected timestamp sequence; this includes linear time alignment (using least squares fitting to correct the overall time offset between the eye-tracking and task systems) and abnormal timestamp detection (setting a time difference threshold). If the time difference between adjacent times If a value is missing, it is marked as an outlier and skipped. Missing value handling (using linear interpolation or attention masking mechanism to ignore missing samples).

[0065] Step 13, based on the corrected timestamp sequence and the set time window Coarse synchronization alignment of eye-tracking and task data is performed using a sliding time window to construct a set of synchronized sample pairs. Where K is the number of effective synchronized samples, For the first One eye-tracking sample, For the first A sample of tasks, eye-tracking events The timestamp is Task events The timestamp is , Implement candidate set filtering based on temporal proximity to provide input for subsequent fine alignment;

[0066] Step 14: Based on adaptive optimal transmission cognitive alignment feature fusion, perform semantic-level fine alignment and feature fusion on the set of synchronized sample pairs after coarse synchronization to generate a fused feature sequence. As the data after time synchronization, specifically:

[0067] For each eye-tracking sample Define its relationship with the task sample The cost of cognitive alignment By integrating multiple factors, a leap from "time alignment" to "cognitive consistency alignment" can be achieved;

[0068] ;

[0069] ;

[0070] ;

[0071] In the formula, To provide a learnable projection function, eye movements and tasks are mapped to a unified semantic space, respectively. This represents the similarity between eye-tracking behavior and task semantics. The time sensitivity coefficient, This represents the flight phase state corresponding to the eye-tracking sample. The task dynamics encoding function outputs the time sensitivity level for that stage, such as the approach stage. It is 1.8, during the cruise phase. It is 0.5. For learnable scaling parameters, The time sensitivity coefficient is set for Sigmoid normalization to automatically increase the time alignment weight during highly dynamic phases (such as approach) and prevent incorrect matching due to small time deviations.

[0072] Cognitive load coefficient, This indicates the cognitive load level during the current flight phase. The basic semantic weights (learnable or fixed) and cognitive load coefficients are set to automatically increase the cognitive load coefficients during high-load phases, where semantic matching errors are more costly, forcing attention to focus on semantically related tasks and improving the cognitive rationality of synchronization.

[0073] Based on the aforementioned cognitive alignment cost, eye-tracking samples Considered as a "supply point" of attention resources (total of 1), task sample Treating these as "demand points," we solve for the attention allocation matrix that minimizes the total transmission cost. Used to achieve fine alignment;

[0074] ;

[0075] in, , for the first Attention assignment vector for each eye-tracking sample Indicates eye-tracking samples Attention resources are allocated to task samples proportion, This indicates that the total attention resource is 1. , Indicates task The "cognitive reception capacity" can be set based on task priority. For the transmission constraint set, satisfying ;

[0076] A lightweight solution is obtained using the Sinkhorn algorithm, based on the attention allocation formula: The optimal attention allocation result is obtained, which reflects the optimal attention configuration under the condition of minimizing the cognitive alignment cost. This enables the identification of truly semantically relevant task events from the candidate set and is the direct output of fine alignment.

[0077] Based on the cognitive alignment cost and the optimal attention allocation result, a cognitive alignment fusion feature is generated to further encode dynamic trend information in the alignment process;

[0078] ;

[0079] In the formula, The attention weights obtained by solving for optimal transport come from the vector The One element, For linear projection of task features Cost function for task features The gradient represents the direction in which the task representation should be adjusted to better align with eye-tracking behavior. The trend enhancement coefficient is adaptively quantified based on the task dynamics and cognitive load of the current flight phase. For example, the alignment error (reflecting the current matching quality) is calculated based on the attention allocation Aij∗ and the cost c(ei,tj). The error is mapped to the alignment instability through a function, and finally multiplied by the basic scaling coefficient (which can be learned or fixed) to obtain the trend enhancement coefficient. When the alignment difference is large, the alignment instability approaches 1, automatically enhancing the trend term; when the alignment is good, the trend term weakens to avoid overfitting. This enables the fusion process to have alignment state perception and dynamic adjustment capabilities, further improving the stability and responsiveness of fine alignment.

[0080] Cognitive alignment fusion features are used as time-synchronized data for subsequent preprocessing, such as segmentation and region of interest (AOI) segmentation.

[0081] In this way, a progressive architecture is achieved by sliding time windows to achieve coarse synchronous screening of spatiotemporal proximity → complete semantic-level cognitive fine alignment based on cognitive consistency modeling → generate fusion features. This improves the accuracy and interpretability of cross-modal data matching while ensuring efficiency, and solves the problems of "coarse time window synchronization", "missing semantic association" and "poor cognitive interpretability of fusion results" in existing technologies.

[0082] The segmentation labeling includes labeling by mission phase (such as cruise, emergency) and labeling by sub-mission (communication, navigation).

[0083] The area of ​​interest (AOI) division is based on cockpit functional zoning, including the primary flight display (PFD), navigation display (ND), and radio control panel (RCP).

[0084] Step 2, see Figure 2 The eye-tracking data and task data in the preprocessed data are extracted and quantified from three dimensions: attention allocation, working memory and task switching, and emergency task processing efficiency, to obtain the values ​​of each key parameter and thus obtain the key parameter dataset.

[0085] In this embodiment, the key parameter dataset includes attention allocation parameters, working memory and task switching parameters, and burst task processing performance parameters;

[0086] The attention allocation parameters measure a pilot's ability to allocate limited visual resources to different tasks or information sources in a complex flight environment. This includes active or automatic attention to the instrument panel, navigation display, external environment, cockpit warning signals, and interactive interface. The core of these parameters is to dynamically balance multiple visual inputs to ensure that key information is prioritized while maintaining overall situational awareness. Specifically, these parameters include the standardized area fixation time percentage, standardized scan rate, pupil diameter change, standardized first fixation time, and attention recovery rate. Together, these parameters can comprehensively measure the rationality and effectiveness of a pilot's visual resource allocation, determine whether they can accurately capture key content amidst complex information, and ensure information acquisition efficiency during flight.

[0087] The standardized area gaze time percentage is used to quantify the pilot's attention to key areas and determine whether they are focusing sufficient attention on core information sources. It is obtained by calculating the ratio of the pilot's total gaze time on key instruments, communication interfaces, and navigation screens to the total mission time. This area gaze time percentage is then standardized to obtain the standardized area gaze time percentage, expressed by the following formula:

[0088] ;

[0089] ;

[0090] In the formula, The percentage of time spent looking at a particular area. The total time a pilot spends looking at key instruments, communication interfaces, and navigation screens. Total task duration As a benchmark value, it can be determined by expert experience;

[0091] The standardized scan rate, the number of gaze shifts per unit time, reflects the speed and rationality of the pilot's gaze switching between different areas. An excessively high scan rate may indicate cognitive overload or an unreasonable interface layout leading to ineffective scanning; an excessively low scan rate may indicate the neglect of critical areas. It is obtained by calculating the ratio of the number of gaze shifts per unit time to the total time, and then standardizing the scan rate based on preset lower and upper limits to obtain the standardized scan rate, specifically expressed by the following formula:

[0092] ;

[0093] ;

[0094] In the formula, To scan video rate, The number of eye movements per unit of time. Total time These are the preset lower and upper limits for the video scanning rate, which can be determined by expert experience.

[0095] The pupil diameter change, with pupil dilation reflecting the intensity of cognitive load, indicates a higher cognitive load. This directly reflects the pilot's physiological response to sudden missions, demonstrating their alertness and psychological load in the face of emergencies, and aiding in determining their initial state of readiness for such missions. The method for obtaining this change is as follows: calculate the difference between the average pupil diameter during a sudden mission and the average pupil diameter during a regular mission; then calculate the ratio of this difference to the standard deviation of the pupil diameter during a regular mission. This is expressed by the following formula:

[0096] ;

[0097] In the formula, This represents the average pupil diameter during emergency missions. This represents the average pupil diameter during routine tasks. The standard deviation of pupil diameter during routine tasks;

[0098] Emergency mission response: refers to the pilot's ability to quickly interrupt the current mission, reallocate cognitive resources to identify the problem, make decisions and implement countermeasures when unexpected events (such as engine failure, system alarms, bird strikes) occur. Its core lies in quickly switching attention modes (from routine monitoring to emergency handling) and maintaining decision-making accuracy under time pressure.

[0099] The standardized first gaze time measures the pilot's reaction speed to unexpected tasks (e.g., the time delay after a sudden situation occurs (e.g., an alarm sound in the cockpit) and assesses their ability to quickly detect and focus on unexpected information. It is obtained by calculating the difference between the time the pilot gazes at the relevant area of ​​the unexpected task and the time the unexpected task begins, then standardizing this difference to obtain the standardized first gaze time, specifically expressed by the following formula:

[0100] ;

[0101] ;

[0102] In the formula, The time of first fixation. The timeframe for pilots to monitor areas relevant to sudden missions; This is the start time for any emergency mission.

[0103] The attention recovery rate assesses the efficiency with which a pilot can quickly return their attention to the original task after handling an unexpected task, reflecting the flexibility of their attention and the smoothness of task switching. It is obtained by calculating the difference between the end time of the current task and the end time of the unexpected task; then, the ratio of the fixation duration upon returning to the original task to this difference is calculated to obtain the attention recovery rate, specifically expressed by the following formula:

[0104] ;

[0105] In the formula, This is the time point at which the current task ends; This is the end time for any emergency mission. To return to the gaze duration of the original task;

[0106] The working memory and task switching metrics measure the additional time consumption and increased error rate caused by the reallocation of cognitive resources when pilots switch between multiple concurrent tasks. It directly reflects the pilot's flexibility and efficiency in handling multiple tasks. Task switching costs can be decomposed into two sub-metrics, quantified from the dimensions of time and accuracy: switching latency, error rate variation, and interference from concurrent tasks.

[0107] The switching delay can quantify the time loss of pilots when switching between two tasks, and intuitively reflect the smoothness of their transition from one task to another. The smaller the value, the less time the switching takes and the higher the efficiency of the task switching. Conversely, the higher the value, the lower the efficiency. It helps to evaluate the pilot's time management ability in a multi-task environment. The method of obtaining the value is as follows: when the first operation response time of task A in a dual-task environment is shorter than the average response time when only task A is executed, it is directly assigned a value of 1.

[0108] Otherwise, calculate the relative difference between the first operation response time of task A in the dual-task environment and the average response time when only task A is executed;

[0109] Specifically, it can be expressed using the following formula:

[0110] ;

[0111] In the formula, In a dual-task environment, the response time for the first operation of task A; This represents the average response time when only task A is executed;

[0112] The change in error rate reflects the degree to which the pilot's mission accuracy is affected when handling multiple tasks in parallel. A positive and large value indicates that the dual-mission environment significantly interferes with mission accuracy, and the pilot is prone to errors during mission switching. Conversely, a small value indicates that the pilot can maintain mission accuracy well during multi-mission processing. It is an important basis for evaluating mission processing quality. The method for obtaining the error rate is as follows: the error rate change is obtained by comparing the relative change rate of mission A in a dual-mission environment with that in a single-mission environment; specifically expressed by the following formula:

[0113] ;

[0114] In the formula, The error rate of task A in a single-task environment; The error rate of task A in a dual-task environment;

[0115] The parallel task interference level effectively measures the interference between multiple tasks on the quality of task completion when multiple tasks are performed simultaneously. A higher value indicates a greater negative impact of parallel task execution on the quality of task A, and stronger interference experienced by the pilot when handling parallel tasks. Conversely, a lower value indicates weaker interference. It provides strong support for assessing a pilot's ability to maintain task quality in a multi-task environment. The method for obtaining this value is as follows: using the completion quality of task A in a single-task state as a benchmark, calculate the deviation of the completion quality of task A when two tasks are performed in parallel; specifically expressed by the following formula:

[0116] ;

[0117] In the formula, The quality of task A's completion in a single-task state; The quality of task A when two tasks are performed in parallel;

[0118] The interference range is [0,1], where 0 indicates no interference (dual-task performance = single-task performance) and 1 indicates complete interference (dual-task performance = 0).

[0119] The emergency mission handling efficiency parameters comprehensively characterize a pilot's overall ability to respond to emergency missions from two key dimensions: reaction speed and decision quality. They provide an important basis for judging the pilot's handling level in emergency situations, including standardized response time and decision parameter values.

[0120] The standardized response time accurately quantifies a pilot's reaction speed to sudden emergencies. Standardization eliminates time differences caused by the inherent characteristics of different emergency situations, facilitating horizontal comparisons of emergency response capabilities among different pilots or even the same pilot in different scenarios. A shorter response time indicates better timeliness in responding to emergencies, providing more time to ensure flight safety. The method for obtaining this timeframe is as follows: calculate the actual time difference between the occurrence of the emergency and the start of the pilot's reaction; divide this actual time difference by the maximum reaction time for standardization; this is expressed by the following formula:

[0121] ;

[0122] ;

[0123] In the formula, This is the time point at which the pilot's operational reaction begins. This refers to the timeframe in which an emergency occurs;

[0124] The decision parameter values, from the perspective of decision quality, characterize the pilot's judgment ability and the rationality of the handling strategy in emergency missions. They can intuitively reflect the pilot's decision-making level under high pressure. High scores indicate that the pilot can make scientific, safe and efficient decisions. These are key parameters for measuring the pilot's core emergency response capabilities. The method is as follows: a three-level weighting is used: the optimal response strategy is selected, with a weighting coefficient of 1; the suboptimal but safe strategy is selected, with a weighting coefficient of 0.5; and the incorrect or dangerous strategy is selected, with a weighting coefficient of 0.

[0125] ;

[0126] Step 3: Based on the different cognitive needs of each flight phase, a weight matrix is ​​preset using the analytic hierarchy process and applied to the key parameter dataset.

[0127] In this embodiment, taking attention allocation A as an example, the preset weight matrix is ​​shown in Table 1:

[0128] Table 1:

[0129]

[0130] By combining the flight phase transition map and the weighted multidimensional matching distance, the flight parameter data in the preprocessed data is divided into flight phases to obtain the flight phase division results.

[0131] S1, denoted by S, retrieves the set of flight situation segments. The k-th situation segment is defined as It includes multi-moment data (N≥2) of pressure altitude A and descent rate V, for each flight situation segment. Extracting the original feature vector , The mean pressure altitude, mean descent rate, derivation velocity of pressure altitude, and derivation acceleration of descent rate are respectively normalized and weighted to generate a weighted feature vector, resulting in the situation segment feature vector set:

[0132] , These are weighting coefficients, and their sum is 1; obtain the preset set of vertical motion states for the flight phase, denoted by Z. The j-th stage Represented as This yields a predefined set of stages containing boundary constraints, where... For the flight phase The upper and lower limits of atmospheric pressure. Flight phase The upper and lower limits of the rate of decline;

[0133] S2, Construct a flight phase transition map based on the preset phase set Z. Where: the node set V corresponds to the preset stage set Z; the edge set E represents the reasonable transfer relationship between flight stages. (Used for subsequent structure-sensitive matching and global consistency optimization);

[0134] S3, based on the flight phase transition map, integrate the feature differences between the situation segment and the preset phase, as well as the boundary violation penalty, to calculate the weighted multidimensional matching distance. Specifically, the weighted multidimensional matching distance function is used to calculate the k-th situation segment. With the j-th preset stage Weighted multidimensional matching distance :

[0135] ;

[0136] For the first Weighted feature vectors of each situation segment; For the first Each preset stage Reference weighted features (such as central value or typical value); For the situation segment Beyond the stage The degree of the boundary between atmospheric pressure altitude or rate of descent; The out-of-bounds penalty coefficient (learnable or fixed);

[0137] S4, Based on the weighted multidimensional matching distance and trajectory data, a structure-sensitive matching function is constructed to generate a comprehensive matching score. The method is as follows:

[0138] Based on the weighted multidimensional matching distance and trajectory data, each situation segment is mapped to a sequence of center state points in a two-dimensional space;

[0139] The normalized curvature is calculated by measuring the ratio of the perpendicular distance from a point in the central state point sequence to the length of the line segment;

[0140] The normalized curvature, combined with the dynamic penalty term and the structural consistency penalty coefficient, is used to construct a structure-sensitive matching function to generate a comprehensive matching score through structure-sensitive matching.

[0141] The structure-sensitive matching function is:

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] In the formula, For the first Each situation segment and stage The overall matching score, For dynamic penalty items, The structural consistency penalty coefficient, The sensitivity coefficient, To prevent the penalty from growing indefinitely, the current situation segment is treated as a saturated nonlinear function. Based on the first three points Calculate normalized curvature The larger the value, the more drastic the change in flight state within that range, and the higher the probability of a phase jump. point to line segment vertical distance, For line segments The length of each situation segment The mapping is used as the central state point to form the trajectory sequence of the aircraft in the two-dimensional situation space composed of (pressure altitude, descent rate);

[0147] S5. Generate the original matching confidence score based on the comprehensive matching score, and fuse the original matching confidence score, the context consistency score, and the historical average confidence score of situational segment matching to obtain the fused confidence score.

[0148] The fusion formula is:

[0149] ;

[0150] In the formula, This represents the original match confidence level. Context consistency is scored (calculated by the degree of matching between the stage sequence and G within a sliding window, with a window size W≥2). Confidence level of historically similar segments (in history and Situation segments with feature similarity higher than a threshold are matched. (average confidence level) ;

[0151] S6, Select the current situation segment based on the fused confidence level. The optimal flight phase for matching; ;

[0152] Furthermore, dynamic programming or Hidden Markov Models (HMMs) can be used to process the stage sequence. Perform global optimization to ensure that it meets the edge constraints E in the flight phase transfer map G (i.e., the transfer relationship between adjacent phases is reasonable), and improve the temporal consistency of the recognition results;

[0153] Supports online updates of the following based on expert feedback or cluster analysis:

[0154] The feature range of the preset flight phase template Z, such as ;

[0155] The set of edges E of the flight phase transition graph G;

[0156] Feature weighting Dynamic fusion weights Penalty coefficient Penalty coefficient for crossing boundaries To adapt to new flight missions (such as special maneuvers and phased characteristic changes under extreme weather conditions) and improve the algorithm's generalization ability;

[0157] Step 4: Match the preset weight matrix according to the flight phase division results, and combine the key parameter values ​​to obtain the final multi-task processing efficiency evaluation value after weighted fusion.

[0158] In this embodiment, the final multi-task processing performance evaluation value obtained after weighted fusion is calculated as follows:

[0159] ;

[0160] ;

[0161] ;

[0162] ;

[0163] In the formula, This is the final multitasking performance evaluation value. , , These are the scores for attention allocation, working memory and task switching, and performance in handling unexpected tasks. , , These are the parameters for attention allocation, working memory and task switching, and burst task processing performance, respectively. The parameter in the first... The weighting coefficients corresponding to each flight phase , , These are the parameters for attention allocation, working memory and task switching, and burst task processing performance, respectively. The numerical values ​​of each parameter.

[0164] By combining the three main categories of parameters—attention allocation, working memory and task switching, and emergency task handling efficiency—with their respective weight vectors, the attention allocation value, working memory and task switching value, and emergency task handling efficiency value are first calculated. Then, these three values ​​are integrated to obtain the multi-task processing efficiency evaluation value, which can comprehensively, objectively, and accurately reflect the pilot's overall ability to handle multiple tasks in parallel under complex flight scenarios.

[0165] Although the steps in the above embodiments are described in the above order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not need to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple variations are all within the protection scope of this invention.

[0166] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.

[0167] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0168] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0169] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for evaluating pilot multitasking performance in complex situations, characterized in that, The method includes the following steps: The flight parameter data, eye-tracking data, and mission data of the pilot during flight are acquired and preprocessed to obtain preprocessed data. The preprocessing includes time synchronization, data cleaning, segmentation and labeling, and region of interest division. The time synchronization includes using a sliding time window and cognitive alignment synchronization method to achieve time synchronization of multi-source data. The eye-tracking data and task data in the preprocessed data are feature-extracted and quantified from three dimensions: attention allocation, working memory and task switching, and efficiency in handling sudden tasks, so as to obtain the values ​​of each key parameter and thus obtain the key parameter dataset. Based on the different cognitive needs of each flight phase, a weight matrix is ​​preset using the analytic hierarchy process and applied to the key parameter dataset. By combining the flight phase transition map and the weighted multidimensional matching distance, the flight phases are divided using the preprocessed flight parameter data, and the flight phase division results are obtained. The preset weight matrix is ​​matched according to the flight phase division results, and the final multi-task processing efficiency evaluation value is obtained by weighted fusion of the key parameter values. The eye-tracking data includes fixation time, scan rate, and pupil diameter. The task data includes emergency task processing data, multi-task parallel processing data, task switching data, and task priority labels. The data for handling sudden tasks includes first fixation time, attention recovery time, switching delay, and error rate; The segmentation labeling includes labeling by task stage and labeling by subtask; The area of ​​interest is divided based on cockpit functional zones, including the main flight display, navigation display, and radio control panel; The method for dividing the flight parameter data in the preprocessed data into flight stages by combining the flight stage transition map and weighted multidimensional matching distance is as follows: S1, obtain the flight situation segment set and the preset flight stage vertical motion situation set, extract multi-dimensional features of the situation segment and perform weighted processing, and construct the situation segment feature vector set and the preset stage set with boundary constraints; S2, based on a preset set of stages, constructs a flight stage transfer graph with stages as nodes and reasonable transfer relationships as edges; S3. Based on the flight phase transition map, the weighted multidimensional matching distance is calculated by fusing the feature differences between the situation segment and the preset phase and the boundary violation penalty. S4, based on the weighted multidimensional matching distance and trajectory data, map each situation segment into a sequence of center state points in a two-dimensional space; The normalized curvature is calculated by measuring the ratio of the perpendicular distance from a point in the central state point sequence to the length of the line segment; The normalized curvature, combined with the dynamic penalty term and the structural consistency penalty coefficient, is used to construct a structure-sensitive matching function to generate a comprehensive matching score through structure-sensitive matching. S5. Generate the original matching confidence score based on the comprehensive matching score, and fuse the original matching confidence score, the context consistency score, and the historical average confidence score of situational segment matching to obtain the fused confidence score. S6, Select the best flight phase matching the current situation segment based on the fused confidence level; the flight phase includes takeoff phase, climb phase, cruise phase, descent phase, approach phase, and landing phase; The weighted fusion yields the final multi-task processing performance evaluation value, which is calculated as follows: ； ； ； ； In the formula, This is the final multitasking performance evaluation value. , , These are the scores for attention allocation, working memory and task switching, and performance in handling unexpected tasks. , , These are the parameters for attention allocation, working memory and task switching, and burst task processing performance, respectively. The parameter in the first... The weighting coefficients corresponding to each flight phase , , These are the parameters for attention allocation, working memory and task switching, and burst task processing performance, respectively. The numerical values ​​of each parameter.

2. The method for evaluating pilot multitasking performance in complex situations according to claim 1, characterized in that, The key parameter dataset includes attention allocation parameters, working memory and task switching parameters, and burst task processing performance parameters. The attention allocation parameters include the standardized region fixation time percentage, standardized scan rate, pupil diameter change, standardized first fixation time, and attention recovery rate. The working memory and task switching parameters include switching latency, error rate variation, and parallel task interference. The performance parameters for handling sudden tasks include standardized response time and decision parameter values.

3. The method for evaluating pilot multitasking performance in complex situations according to claim 2, characterized in that, The standardized area gaze time percentage is obtained by: calculating the ratio of the pilot's total gaze time in the key instrument, communication interface, and navigation screen areas to the total mission time, and then standardizing the area gaze time percentage to obtain the standardized area gaze time percentage. The standardized scanning rate is obtained by: calculating the ratio of the number of gaze shifts per unit time to the total time to obtain the scanning rate; and standardizing the scanning rate based on preset lower and upper limits to obtain the standardized scanning rate. The pupil diameter change is obtained by calculating the difference between the average pupil diameter during a sudden mission and the average pupil diameter during a regular mission, and then calculating the ratio of the difference to the standard deviation of the pupil diameter during a regular mission to obtain the pupil diameter change. The standardized first gaze time is obtained by: calculating the difference between the time node when the pilot gazes at the relevant area of ​​the emergency mission and the time node when the emergency mission begins, and standardizing the difference to obtain the standardized first gaze time. The attention recovery rate is obtained by calculating the difference between the end time of the current task and the end time of the sudden task. The attention recovery rate is obtained by calculating the ratio of the fixation duration when returning to the original task to the difference.

4. The method for evaluating pilot multitasking performance in complex situations according to claim 2, characterized in that, The switching delay is obtained by assigning a value of 1 directly when the first operation response time of task A in a dual-task environment is shorter than the average response time when only task A is executed. Otherwise, calculate the relative difference between the first operation response time of task A in the dual-task environment and the average response time when only task A is executed; The error rate change is obtained by using the relative change rate of the error rate of task A in a dual-task environment to that of task A in a single-task environment. The method for obtaining the parallel task interference degree is as follows: taking the completion quality of task A in the single-task state as the benchmark, calculate the deviation of the completion quality of task A when two tasks are parallel.

5. The method for evaluating pilot multitasking performance in complex situations according to claim 2, characterized in that, The standardized response time is obtained by calculating the actual time difference between the time the emergency occurs and the time the pilot begins to react. The actual time difference is then divided by the maximum reaction time for standardization to obtain the standardized response time.

6. The method for evaluating pilot multitasking performance in complex situations according to claim 1, characterized in that, The time synchronization includes using a sliding time window and cognitive alignment synchronization method to achieve time synchronization of multi-source data, specifically including the following steps: Step 11: Obtain the pilot's eye-tracking data sequence, mission data sequence, and their corresponding timestamp sequence during the flight mission; Step 12: Preprocess the timestamps of the eye-tracking data and task data to obtain a corrected timestamp sequence; the preprocessing includes linear time alignment, abnormal timestamp detection, and missing value handling; Step 13: Based on the corrected timestamp sequence and the set time window, perform coarse synchronization alignment of eye-tracking and task data using a sliding time window to construct a set of synchronized sample pairs; Step 14: Based on adaptive optimal transmission cognitive alignment feature fusion, perform semantic-level fine alignment and feature fusion on the set of synchronized sample pairs to generate a fused feature sequence as the time-synchronized data.

Images