A method for analyzing special flight situations using eye movement data
Through the fuzzy width learning system optimized by eye movement data acquisition and genetic algorithm, the pilot's attention characteristics are analyzed, and the objectivity and response speed problems of flight special situation evaluation in the existing technology are solved, and the pilot's under various special situations is realized. The rapid classification and early warning improve flight safety.
Patent Information
- Application Number
- CN202411088265.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-08-09
AI Technical Summary
The prior art is difficult to quickly and objectively evaluate the pilot's attention characteristics when a flight special situation occurs, especially when multiple special situations occur simultaneously, which affects flight safety.
Eye movement data acquisition, preliminary processing and feature extraction are adopted, and classification model training is carried out through a fuzzy width learning system optimized by genetic algorithm, simulation data is generated and pilot classification evaluation is performed, and gaze distribution characteristics under different special circumstances are analyzed using FCM algorithm clustering. , generate quantized intervals to support flight strategy judgment.
It realizes rapid response and classification of pilots in single and complex situations, improves flight safety, provides instant warnings in actual flights, and improves the adaptability and accuracy of the system.
Smart Images

Figure CN118626802B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of electronic digital data processing, and in particular relates to a flight special situation analysis method using eye movement data. Background Art
[0002] Flight safety is closely related to the pilot's flight status and flight operations. During the flight, special situations such as single-engine failure and attitude indicator failure will affect the flight attitude and fuselage stability. The occurrence of these special situations will increase the difficulty of the pilot's operation, thus seriously threatening flight safety.
[0003] Eye tracking technology can accurately measure the pilot's visual changes and reflect the pilot's attention allocation. However, the application of the pilot's eye movement data is mainly for the analysis of attention allocation, the monitoring of the pilot's mental state (such as fatigue level, concentration level) and the performance analysis of the pilot. When a flight special situation occurs, the pilot's attention allocation and flight decision-making will affect his operation when the flight special situation occurs. In the prior art, eye movement data has been used to analyze the characteristics of pilots in a single type of flight special situation, and psychological assessments or questionnaires have been used to distinguish whether the pilot is a calm type or a type that is prone to panic. This analysis method is not objective. In the actual flight special situation, multiple special situations often occur at the same time. The prior art CN115844404A proposes a method and device for evaluating the attention characteristics of controllers based on eye movement data. It classifies the attention level of the controller by collecting eye movement data to set the characteristics of attention evaluation. This method is more objective in rating the operator's operation based on eye movement data, but pilots need faster reaction time in flight operations, and it has only been analyzed. The application in the actual special situation needs further research.
[0004] Therefore, based on the current analysis of pilots' eye movement characteristics when special flight situations occur based on eye movement data, it is urgent to solve the problem of how to evaluate pilots' attention characteristics more objectively and make them more in line with the actual situation when special flight situations occur. Summary of the invention
[0005] The purpose of the present invention is to provide a flight special situation analysis method using eye movement data to solve the technical problems existing in the above-mentioned prior art.
[0006] To achieve the above object, the technical solution of the present invention is:
[0007] A method for analyzing special flight situations using eye movement data, characterized in that it comprises the following steps:
[0008] S1. Collect eye movement data of the test pilots;
[0009] S2. performing preliminary processing on the eye movement data;
[0010] S3, extracting eye movement data features, analyzing attention features, and generating simulation data based on the analysis results;
[0011] S4, using the simulation data to train the classification model of the fuzzy width learning system optimized by the genetic algorithm;
[0012] S5. Classify and evaluate the pilots to be tested based on the trained classification model.
[0013] Furthermore, in the step of collecting eye movement data of the test pilot, an eye tracker and a desktop flight procedure trainer are used to collect the pilot's flight data, and the flight instrument areas where the pilot's attention is distributed are set as: attitude indicator and external visual, altimeter, heading indicator, airspeed indicator, vertical speed indicator and turn indicator.
[0014] Furthermore, in the step of performing preliminary processing on the eye movement data, it is necessary to record the fixation time, first fixation time and number of visits within the set attention area to facilitate subsequent processing.
[0015] Furthermore, the eye movement features extracted for analysis and classification are: the variance of the number of visits of the pilot to each area of interest (VAR), the Markov transition probability (MK), the special situation reaction time (RT) and the average fixation time (EPT).
[0016] Furthermore, after clustering the extracted pilot eye movement feature data using the FCM algorithm, the pilot gaze distribution characteristics under different special situations are analyzed and the quantization intervals are statistically analyzed, and then a simulation data set, namely the simulation data, is generated based on the quantization intervals obtained from the analysis.
[0017] Furthermore, the classification model of the fuzzy width learning system optimized by the genetic algorithm is trained using simulation data. The specific steps of training the model are as follows:
[0018] S4.1, setting the number of fuzzy rules NR, the number of fuzzy systems NF and the number of enhanced nodes NE of the fuzzy width learning network as parameters to be optimized, and setting the parameter ranges of NR, NF and NE;
[0019] S4.2, encode the genes and initialize the population size;
[0020] S4.3, setting the crossover rate and mutation rate of the gene to allow the gene to crossover and mutate;
[0021] S4.4, judge the adaptability and optimal selection of individuals in the newly generated population, and then reproduce the next generation;
[0022] S4.5, the objective function of the classification model is positioned as the classification accuracy. When the accuracy reaches more than 95%, the iteration can be stopped, and the maximum number of iterations is 200;
[0023] S4.6, after the iteration stopping condition is met, the dominant individual is selected to be retained in the population-based learning results, the individual is taken as the optimal solution, the optimal NR, NF and NE parameters are input, and the model training is completed.
[0024] Furthermore, after the classification model is trained, the feature data set of the test pilots is input into the model as a test set for classification, and the classification results are output.
[0025] Furthermore, based on the classification results, a flight warning signal is given to the pilot who is finally classified as having failed to operate in a single special situation or failed to operate in a complex special situation.
[0026] Compared with the prior art, the present invention has the following beneficial effects:
[0027] The present invention proposes a flight special situation analysis method using eye movement data. After extracting the features of the pilot's eye movement data and clustering them, the feature quantization intervals of the pilots who encounter single special situations and complex special situations are statistically analyzed, and the basis for judging the pilot's flight strategy and flight status is objectively given. In actual flight and teaching, the system can respond quickly, and the fuzzy width learning system model optimized by genetic algorithm is used to monitor and analyze the pilot's status when encountering single and complex flight special situations. Among them, the fuzzy width learning system optimized by genetic algorithm is an improved diagnostic algorithm, which can quickly respond to and classify the pilot's attention type while maintaining the universality and accuracy of the classification model, so as to judge whether to give an early warning. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 The present invention provides a structural diagram of a method for analyzing special flight situations using eye movement data according to an embodiment of the present invention.
[0029] Figure 2 This is a flowchart of using a genetic algorithm to improve a fuzzy width learning system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the embodiments described are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.
[0031] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention. It should be noted that relational terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.
[0032] Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.
[0033] This embodiment proposes a flight special situation analysis method using eye movement data. In order to enable the system to respond quickly in actual flight and teaching, the method uses a fuzzy width learning system model optimized by a genetic algorithm to monitor and analyze the pilot's status when encountering a single or complex flight special situation. The fuzzy width learning system optimized by a genetic algorithm is an improved diagnostic algorithm. In view of the fact that the fuzzy width learning system is prone to fall into local optimality during parameter setting, resulting in a low model universality, a genetic algorithm is used to automatically optimize its parameters.
[0034] The number of fuzzy rules NR, the number of fuzzy systems NF and the number of enhanced nodes NE of the fuzzy width learning network are taken as the parameters to be optimized, and the output diagnostic accuracy is taken as the objective function. The classification model is optimized using the genetic optimization algorithm, and the pilots are classified using the optimized model, so that the pilots' gaze operations can be quickly classified into levels when special situations occur, and the relationship between eye movement data and pilot operations when multiple special situations occur simultaneously can be effectively analyzed, so as to enhance the pilots' awareness of safe flight operations. The fuzzy width learning system optimized by the genetic algorithm can avoid falling into the problem of local optimality. The fuzzy width learning system improved by fuzzy rules not only has better model adaptability, but also maintains the characteristics of fast operation of the classification system, thereby realizing real-time monitoring of the pilots' flight status.
[0035] Figure 1 FIG. 1 is a structural diagram of a flight special situation analysis method using eye movement data in this embodiment. Figure 1As shown, the main steps of the method are:
[0036] S1. Collect eye movement data of the test pilots;
[0037] S2, preliminary processing of eye movement data;
[0038] S3, extracting eye movement data features, analyzing attention features, and generating simulation data based on the analysis results;
[0039] S4, using simulation data to train the classification model of the fuzzy width learning system optimized by the genetic algorithm;
[0040] S5. Classify and evaluate the pilots to be tested.
[0041] The implementation process of collecting the eye movement data of the pilots under test is as follows: building a data collection platform, using a desktop primary flight procedure trainer loaded with Microsoft Flight Simulator 2020 software, Tobii Glasses 3 Eyetracker and flight simulation scoring software to collect the pilots' eye tracker data during flight. The flight instrument areas where the pilots' attention is distributed are set as: attitude indicator and external visual, altimeter, heading indicator, airspeed indicator, vertical speed indicator and turn indicator.
[0042] The pilots who performed the flight mission were: 3 pilots with certain flying experience and 27 pilots who had passed the instrument flight test. All subjects had experience in operating simulated aircraft and were proficient in the simulation platform.
[0043] Then set the flight mission, clarify the operations that the pilot needs to perform during the flight, and the pilot performs the flight mission on the simulation platform according to the flight mission requirements, and use the eye movement acquisition device to obtain the pilot's eye movement data during the experimental flight mission. In addition, during the approach phase of the flight mission, the pilot is not informed in advance that a flight special situation will occur. It is set that a single engine failure or an attitude indicator failure or two special situations are triggered at the same time during the approach phase. Among them, each special situation is evenly distributed to the pilots, that is, in the flight test, the pilot's attention is analyzed accordingly for a single flight special situation and two flight special situations.
[0044] The eye movement data is processed preliminarily, the fixation area data is extracted, and after the eye tracker data is visualized, the eye movement data trajectory diagram and the pilot's eye movement trajectory record data are generated. In addition, the eye movement data needs to be processed for noise reduction to eliminate the influence of other factors such as the environment, nystagmus, eye microsaccades, and record the fixation time, first fixation time, number of visits, etc. in the set attention area.
[0045] Then, the features required by the classification model are extracted, including: the variance of the number of visits of the pilot to each area of interest, VAR, Markov transition probability MK, special situation reaction time RT, and average fixation time EPT, and the above features are normalized. The calculation formula for VAR is: , VC is the number of visits to each attention instrument distribution area by the test pilot, is the average number of visits of all pilots, n is the total number of attention interest areas; the calculation formula for MK is: ,in is the Markov transition probability of the nth interest area when a special situation occurs, is the duration of annotation in area of interest j after the pilot's attention point moves from area i to area j, The total duration of the pilot's gaze on all remaining interest areas after the pilot's gaze moves out of area i; , where T M is the time point when the pilot discovered the special request, T0 is the time point when the special request actually occurred; EPT is the average fixation duration of each fixation point. Then the above values are normalized, and the normalization method is a commonly used method.
[0046] After obtaining the above normalized features, the above features are clustered using a clustering algorithm, and then the feature value range of each cluster area is statistically analyzed. In this embodiment, the FCM algorithm is selected for clustering. The final quantization interval and clustering effect are shown in Table 1 below.
[0047] Table 1
[0048]
[0049] According to the actual flight performance of flight trainees and the above clustering effect analysis, the category labeled 1 is the category in which the pilot successfully completed the flight mission after a single special situation occurred, the category labeled 2 is the category in which the pilot failed to perform the flight mission after a single special situation occurred, the category labeled 3 is the category in which the pilot successfully completed the flight mission after a complex special situation occurred, and the category labeled 4 is the category in which the pilot failed to perform the flight mission after a complex special situation occurred. Four types of simulation data are generated based on the above quantization intervals. The specific generation method is to take the mean of each quantization interval as the center, and take x as the step value starting from around the mean. , where W MAX is the maximum value of each feature quantization interval, W MINis the minimum value of the quantization interval of each feature, n is the number of simulated data that need to be generated, the range ratio is the step value taken on the left and right sides of the mean center, all values are combined with the median to generate the simulation matrix of each feature, and then the values of all matrices are randomly combined into simulated data, where the number of simulated data generated is an odd number.
[0050] Specifically, taking the data with label 1 as an example, the median values of VAR, MK, and RT are 48.22, 0.153, 28, and 1.545. 11 sets of simulated data are generated for this label. The step values of VAR, MK, and RT based on the midpoint are: 9.11, 0.026, 5.2, and 0.027, respectively. The simulation matrix of VAR is: [2.67, 11.78, 20.89, 30, 39.11, 48.22, 57.33, 66.44, 75.55, 84.66, 93.77], and the simulation matrix of MK is: [0.023, 0.049, 0.075, 0.101, 0.12 The simulation matrix of RT is: [2, 7.2, 12.4, 17.6, 22.8, 28, 33.2, 38.4, 43.6, 48.8, 54], and the simulation matrix of EPT is: [1.41, 1.437, 1.464, 1.491, 1.518, 1.545, 1.5721.599, 1.626, 1.653, 1.68]. Then the above matrices are randomly sorted and combined to generate the simulation of the data with label 1, and then the data with other labels are simulated in the same way.
[0051] like Figure 2 As shown, the classification model of the fuzzy width learning system optimized by the genetic algorithm is trained using simulation data. The specific steps of training the model are:
[0052] Step 1: Set the number of fuzzy rules NR, the number of fuzzy systems NF and the number of enhanced nodes NE of the fuzzy width learning network as the parameters to be optimized. Set the parameter ranges of NR, NF and NE to ;
[0053] Step 2, encode the genes and initialize the population size to 500;
[0054] Step 3, set the gene crossover rate to 0.4 and the mutation rate to 0.01 to allow the gene to cross and mutate;
[0055] Step 4: Make an adaptive judgment and optimal selection on the newly generated individuals, and then reproduce the next generation;
[0056] Step 5: Position the classification accuracy as the objective function of the classification model. When the accuracy reaches 95% or more, the iteration can be stopped, and the maximum number of iterations is 200.
[0057] Step 6. After the iteration stopping condition is met, select the dominant individual to be retained in the population-based learning results, take the individual as the optimal solution, and input the optimal NR, NF, and NE parameters. The final parameters obtained in this example are NR 8, NF 12, and NE 8, thus completing the training of the model.
[0058] After the model completes the above training, the feature data sets of all test pilots are input into the trained classification model as a test set to verify the effect.
[0059] Among them, compared with the width learning system, the fuzzy width learning system replaces the feature nodes in the width learning system with the fuzzy subsystem, and sends the intermediate output generated by the fuzzy subsystem replaced in the width learning system to the enhancement node of the width learning system for nonlinear transformation, thereby improving the versatility of the classification system and maintaining the classification speed of the width system. The pilot's eye movement characteristics can be classified and the corresponding results can be given in a very short time. In practical applications, the system can be used to provide real-time feedback on the pilot's eye movement feature analysis results when a special situation occurs, and the pilot's current eye movement distribution can be evaluated. If it is classified into an unqualified category, an early warning signal can be issued to the pilot in time. The fuzzy width learning system optimized by the genetic algorithm can optimize the parameters by itself to improve the classification accuracy of the model. The average classification accuracy and average time consumption of the fuzzy width learning system optimized by the genetic algorithm are shown in Table 2 below.
[0060] Table 2
[0061] category Classification accuracy Time consumed (s) 1 89.5% 0.201 2 100% 0.198 3 100% 0.202 4 83.3% 0.212
[0062] According to the above results, the present invention can effectively analyze and classify the eye movement data of pilots who encounter flight special situations after quantifying the characteristic range of single special situations and complex special situations and then quickly classifying them. Correspondingly, after the pilots are classified, if they are classified into category 2 and category 4, which are the two categories with failed final flight performance, the classification results can be output to send a warning signal to the pilots, reminding the pilots that their attention distribution is wrong and they should change their flight strategies in time, thereby improving flight safety. The warning method can be achieved by flashing a red prompt light on the flight panel or issuing a warning voice.
[0063] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.
Claims
1. A method for analyzing special flight situations using eye movement data, characterized in that: The following steps are involved: S1. Collect eye movement data of the test pilots; S2. performing preliminary processing on the eye movement data; S3, extracting eye movement data features, analyzing attention features, and generating simulation data based on the analysis results; The eye movement features extracted for analysis and classification are: the variance of the number of visits to each area of interest by the pilot VAR, the Markov transition probability MK, the special situation reaction time RT and the average fixation time EPT; After clustering the extracted pilot eye movement feature data using the FCM algorithm, the pilot gaze distribution characteristics under different special situations are analyzed and the quantization intervals are statistically calculated, and then a simulated data set, i.e., the simulation data, is generated based on the quantization intervals obtained by the analysis; The method of generating a simulated data set according to the quantization interval is: Based on the mean of each quantization interval, the step value Start taking values around the mean: in, is the maximum value of each feature quantization interval, is the minimum value of each feature quantization interval, The number of simulation data that needs to be generated; Set the scale of the range to the step value Taking values within a preset range of the mean center, generating a simulation matrix of each feature based on a combination of all the values and the median of the mean center, and randomly combining the values of all the simulation matrices into the simulation data set, wherein the number of the generated simulation data sets is an odd number; S4, using the simulation data to train the classification model of the fuzzy width learning system optimized by the genetic algorithm; The classification model of the fuzzy width learning system optimized by the genetic algorithm is trained using simulation data. The specific steps of the training model are as follows: S4.1, setting the number of fuzzy rules NR, the number of fuzzy systems NF and the number of enhanced nodes NE of the fuzzy width learning network as parameters to be optimized, and setting the parameter ranges of NR, NF and NE; S4.2, encode the genes and initialize the population size; S4.3, setting the crossover rate and mutation rate of the gene to allow the gene to crossover and mutate; S4.4, judge the adaptability and optimal selection of individuals in the newly generated population, and then reproduce the next generation; S4.5, the objective function of the classification model is positioned as the classification accuracy. When the accuracy reaches more than 95%, the iteration can be stopped, and the maximum number of iterations is 200; S4.6, after the iteration stop condition is met, the dominant individual is selected and retained in the population-based learning results, the individual is taken as the optimal solution, the optimal NR, NF and NE parameters are input, and the model training is completed; S5. Classify and evaluate the pilots to be tested based on the trained classification model.
2. The method for analyzing flight special situations using eye movement data according to claim 1, characterized in that: In the step of collecting the eye movement data of the test pilot, an eye tracker and a desktop flight procedure trainer are used to collect the flight data of the pilot, and the flight instrument areas where the pilot's attention is distributed are set as: attitude indicator and external visual, altimeter, heading indicator, airspeed indicator, vertical speed indicator and turn indicator.
3. The method for analyzing flight special situations using eye movement data according to claim 2, characterized in that: In the step of performing preliminary processing on the eye movement data, it is necessary to record the fixation time, first fixation time and number of visits in the set attention area to facilitate subsequent processing.
4. The method for analyzing flight special situations using eye movement data according to claim 1, characterized in that: After the classification model is trained, the feature data set of the test pilots is input into the model as a test set for classification, and the classification results are output.
5. The method for analyzing special flight situations using eye movement data according to claim 4, characterized in that: Based on the classification results, a flight warning signal is given to the pilot who is finally classified as a flight operation failure under a single special situation or a flight operation failure under a complex special situation.
