A pilot attention allocation training method, device, equipment and storage medium

Abstract

The application provides a pilot attention allocation training method, device and equipment and a storage medium, and belongs to the field of flight training. The method comprises the following steps: in the case that a pilot wears a head-mounted eye tracker for training, obtaining a fixation point video of the pilot and recording the triggering conditions of observable items; dividing the fixation point video into interest areas, performing data analysis on the interest areas, and determining the access times, Pearson correlation coefficients, pupil diameter change scatter plots, Markov transition matrices, access sequences and time lengths, in-cabin and out-of-cabin access times, cockpit saccade mode models and the pupil diameter change rates of the pilot; further determining the attention transfer portrait and the attention allocation score of the pilot; and adjusting the attention allocation training bias of the pilot according to the attention allocation score and the attention transfer portrait. In this way, the limitations of traditional subjective judgment can be overcome, and the training evaluation is more scientific and operable.

Description

Technical Field

[0001] This invention belongs to the field of flight training, specifically relating to a method, device, equipment, and storage medium for pilot attention allocation training. Background Technology

[0002] Numerous studies have shown that pilot error is the leading cause of aviation accidents (accounting for over 60%), and the approach and landing phase, due to its complex and ever-changing scenarios and the tight decision-making time, is a high-risk period for accidents. During this critical phase, pilots need to process a massive amount of information: the aircraft's motion status, trajectory, flight data (attitude, bank angle, altitude, rate of climb, speed, engine parameters, etc.), the external environment (runway, terrain, weather, traffic), and radio communications. The visual system is the primary channel for humans to acquire external information (approximately 80%–90%), and fixation is key to acquiring visual information. In the field of aviation, efficient and appropriate allocation of attention—the process of selectively focusing on, paying attention to, judging, and shifting attention from the aforementioned information sources—is widely recognized as the "foundation of basic flight control skills," directly determining the accuracy of situational perception and the timeliness of control decisions. The multi-tasking characteristics of the cockpits of new-generation aircraft further highlight the decisive role of appropriate attention allocation and shifting. Research has confirmed that improving the efficiency of searching for and focusing on instrument information in the cockpit can significantly improve landing performance.

[0003] However, traditional flight training cannot objectively quantify the assessment and improvement of pilots' attention allocation ability. The assessment mainly relies on the instructor's subjective experience and "feeling" judgment, lacking objective and accurate data support. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method, apparatus, device, and storage medium for training pilot attention allocation.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for training pilot attention allocation, the method comprising: When pilots are training with a head-mounted eye tracker, the system acquires videos of the pilots' gaze points and records the triggering of preset observables. The fixation point video is divided into regions of interest, and data features of each region of interest are extracted. Based on the data features, the pilot's attention shift profile is determined, and the attention allocation score is determined according to the correlation coefficient between the number of visits to each region of interest and the preset standard mode. The attention shift profile is used to describe the process of the pilot's attention allocation and shift during training. The pilot's attention allocation training emphasis is adjusted based on the attention allocation score, attention shift profile, and the triggering of observables.

[0006] Optionally, the region of interest includes eight regions: Channel track, attitude, gradient, altitude, rate of rise and fall, speed, external visibility, and engine parameters; The data features include: number of visits, scatter plot of pupil diameter change, Markov transition matrix, visit order and duration, number of visits inside and outside the cockpit, cockpit scan mode model, and the pilot's pupil diameter change rate.

[0007] Optionally, the attention allocation score is calculated by using the Pearson correlation coefficient to determine the similarity between the number of times the student visits the region of interest and the standard pattern, with a value ranging from 0 to 1.

[0008] Optionally, the formula for calculating the Pearson correlation coefficient is: ; Among them, X i Y represents the number of times a student visits the i-th area of ​​interest. i The preset standard mode is used to determine the number of visits to the i-th region of interest. and These are the corresponding averages, and n is the total number of visits.

[0009] Optionally, the step of extracting data features from each region of interest and determining the pilot's attention shift profile based on the data features includes: Determine the scatter plot of the pilot's pupil diameter change and the rate of change of pupil diameter in each region of interest to assess the pilot's stress level or cognitive load; A Markov transition matrix for pilot attention between regions of interest is constructed to quantify the gaze transition probability between different regions of interest; The pilots' focus and neglect tendencies regarding different flight parameters were assessed based on the number of visits to each region of interest. The number of visits to areas of interest inside and outside the cockpit was counted to quantify the proportion of pilot attention allocated between internal instrument monitoring and external environmental observation. Based on the order and duration of visits to areas of interest, a visual scanning path model of the pilot is constructed and matched with a preset scanning model for identification.

[0010] Optionally, the method further includes: After the first training session, a second training session is conducted, and the data from the two training sessions are compared and analyzed to generate a training report. The comparative analysis includes one or more of the following: comparison of the number of observable triggers, comparison of attention allocation scores, comparison of pupil diameter change rate, and comparison of interest area access patterns.

[0011] Optionally, the training report may include one or more of the following: observable triggering statistical charts, attention allocation score change trend charts, pupil diameter change scatter plots, interest area visit frequency distribution charts, and Markov transition matrix visualization charts.

[0012] A pilot attention allocation training device, the device comprising: The acquisition module is used to acquire video of the pilot's gaze point and record the triggering of preset observables when the pilot is training with a head-mounted eye tracker. The analysis module is used to divide the fixation point video into regions of interest, extract data features of each region of interest, determine the pilot's attention shift profile based on the data features, and determine the attention allocation score based on the correlation coefficient between the number of visits to each region of interest and the preset standard pattern; the attention shift profile is used to describe the pilot's attention allocation and shift process during training. The adjustment module is used to adjust the pilot's attention allocation training emphasis based on the attention allocation score, attention shift profile, and the triggering status of observables.

[0013] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned pilot attention allocation training method.

[0014] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned pilot attention allocation training method.

[0015] The pilot attention allocation training method provided by this invention has the following beneficial effects: By using a head-mounted eye tracker to collect fixation point data from pilots, combined with interest zone analysis, pupil dynamic monitoring, and shift model construction, an objective quantitative assessment and precise optimization of pilots' attention allocation ability can be achieved. This method uses multi-dimensional data such as access frequency, pupil response, and saccade paths to create an attention shift profile, overcoming the limitations of traditional reliance on subjective judgment and making training assessment more scientific and operable. By comparing the correlation coefficients between standard models and individual data to generate attention allocation scores, it is possible to identify pilots' strengths and weaknesses in areas such as attention to in-cabin and out-of-cabin equipment and task switching efficiency, thereby allowing for targeted adjustments to training content. Specifically, for pilots with abnormal shift matrices, enhanced situational switching training can be provided, or fatigue risks can be identified through pupil change rate, ultimately improving their situational awareness and emergency response capabilities, forming a closed-loop training system of "data feedback - precise intervention - capability enhancement." Attached Figure Description

[0016] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a pilot attention allocation training method according to an exemplary embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of a region of interest division provided by the present invention according to an exemplary embodiment.

[0019] Figure 3 This is a schematic diagram of a cockpit scanning method provided by the present invention according to an exemplary embodiment.

[0020] Figure 4 This is a comparison chart of the number of OB items triggered according to an exemplary embodiment of the present invention.

[0021] Figure 5 This is a comparison chart of the number of key OB items triggered according to an exemplary embodiment of the present invention.

[0022] Figure 6 This is a graph showing the decrease in the number of users triggered by a key OB item, according to an exemplary embodiment of the present invention.

[0023] Figure 7 This is a schematic diagram illustrating the number of times a pilot visits various regions of interest during different training sessions, according to an exemplary embodiment of the present invention.

[0024] Figure 8 This is a correlation coefficient diagram of a pilot's gaze pattern provided by the present invention according to an exemplary embodiment.

[0025] Figure 9 This is a scatter plot of pilot pupil changes provided by an exemplary embodiment of the present invention.

[0026] Figure 10 This is a schematic diagram of a region of interest Markov transition matrix provided by the present invention according to an exemplary embodiment.

[0027] Figure 11 This is a block diagram of a pilot attention allocation training method provided by the present invention according to an exemplary embodiment. Detailed Implementation

[0028] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0029] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] First, this invention provides a method for training pilots' attention allocation, specifically as follows: Figure 1 As shown, it includes the following steps: S101. When pilots are training with a head-mounted eye tracker, acquire video of the pilot's gaze point and record the triggering of observable items.

[0031] In this step, the eye tracker must first be worn correctly. During the first Level D full-motion simulator training session, after the trainee has adjusted the seat and completed other preparations, the eye tracker is worn for fixation point calibration. The trainee is instructed to observe three reference objects located to the left, in front, and to the right to check the calibration effect. If the calibration effect is good, training begins; otherwise, recalibration is performed by adjusting the nose pads, etc., until the desired effect is achieved.

[0032] After calibration, simulator training begins. The trainee's gaze point video images are observed in real time on a computer screen. Based on this, the triggering of OB items (observable items) is recorded, and images and videos matching the real-time gaze point positions are exported.

[0033] S102. Divide the fixation point video into regions of interest, extract the data features of each region of interest, determine the pilot's attention shift profile based on the data features, and determine the attention allocation score based on the correlation coefficient between the number of visits to each region of interest and the preset standard mode.

[0034] The attention shift profile describes the allocation and shifting of a pilot's attention during training. The interest acquisition is divided into eight regions: flight path, attitude, bank angle, altitude, rate of climb / descendancy, speed, external visuals, and engine parameters. Analysis of these regions includes determining the number of visits to each region, a scatter plot of pupil diameter changes, a Markov transition matrix, the order and duration of visits, the number of visits inside and outside the cockpit, a cockpit scan pattern model, and the pilot's pupil diameter change rate. Based on the data analysis results, the pilot's attention shift profile is determined, and an attention allocation score is determined based on the correlation coefficient between the number of visits to each region of interest and the standard pattern.

[0035] In this step, the pilot's attention shift profile needs to be determined based on the defined zones of interest (AOIs). First, data analysis yields data features such as the number of visits to each AOI, the Markov transition matrix of each AOI, the order and duration of AOI visits, and the ratio of visits to internal and external AOIs. The similarity score (0-1) is calculated based on the correlation coefficient between the number of visits to each AOI and a standard line. Other features are used to describe the pilot's attention shift profile. For example, a scatter plot of pupil diameter changes and the rate of pupil diameter change in each AOI are determined to assess the pilot's stress level or cognitive load. A Markov transition matrix of the pilot's attention between AOIs is constructed to quantify the probability of gaze shift between different AOIs. Based on the number of visits to each AOI, the pilot's focus and tendency to ignore different flight parameters are assessed. The number of visits to internal and external AOIs is statistically analyzed to quantify the ratio of the pilot's attention allocation between internal instrument monitoring and external environmental observation. Based on the order and duration of visits to the AOIs, a visual scanning path model of the pilot is constructed and matched with a pre-defined saccade model for identification.

[0036] In the data analysis and processing phase, objective data analysis and statistical analysis of observables (OBs, self-established based on SOPs and flight experience) are conducted on pilots' attention allocation. During attention allocation training, the triggering of trainees' observable behavior list (OBs) is recorded. Each OB that is not performed correctly is recorded as a trigger.

[0037] For example, the list of observable behaviors (OB items) is as follows: Scenario - Takeoff and landing routes (total 29 0B).

[0038] Weather conditions: Daytime, 140 / 10-15ms, CAVOK, slight turbulence; runway condition code 5 / 5 / 5, the runway is wet.

[0039] Instructions: Hold off outside the runway: Change departure to takeoff and maintain 1500m on one side; you may enter runway 01; FD malfunction after takeoff, instruction altitude 1200m, turn left to join visual takeoff and landing route.

[0040] Observation object: PF.

[0041] Phase 1: From entering the runway to takeoff.

[0042] OB1.1: Check for changes to the starting height.

[0043] OB1.2: Confirm that there is no impact on the five sides and the runway, and announce "No impact on both sides".

[0044] OB1.3: Before takeoff, check all items on the checklist, first visually and then verbally.

[0045] OB1.4: Two "checks" during takeoff roll – verifying the correct takeoff thrust setting; verifying 80 knots.

[0046] OB1.5: Monitor airspeed during taxiing.

[0047] OB1.6: During takeoff and landing, keep your line of sight far ahead to ensure a stable and normal takeoff and landing trend for the aircraft.

[0048] OB1.7: When PM reports "V1" 2-3 sections in advance, PF should verify the V1 speed.

[0049] 0B1.8: During Vr, begin gently and continuously raising the nose (at a rate of 2-3 degrees per second) until the target attitude is about 15°.

[0050] OB1.9: After takeoff, use the attitude indicator, PFD, or HUD (for aircraft equipped with HUD) as the primary pitch reference.

[0051] OB1.10: Do not use pitch commands on the flight controller during nose-up maneuvers.

[0052] OB1.11: "Positive Ascent", verify the positive rate of ascent on the barometric altimeter.

[0053] OB1.12: Check for changing command height.

[0054] Phase Two: During takeoff and landing.

[0055] OB2.1: Check N1 after speed / altitude change.

[0056] OB2.2: When the PM performs the post-flight checklist, the PF should also conduct a visual inspection.

[0057] OB2.3: Descent and approach checklist - first "see" then "say".

[0058] OB2.4: At least one person should visually verify the alignment of the runway head.

[0059] OB2.5: After deploying the wheels and flaps 15, verify that they are in place (using the position indicator and green light).

[0060] 0B2.6: After deploying the flaps 30 degrees, verify that they are in place (indicated by the position chart and green light).

[0061] OB2.7: When performing the landing checklist, first "look" and then "talk".

[0062] OB2.8: When turning to the fifth line, attention should be allocated both internally and externally, taking full account of information such as runway, attitude, throttle, speed, slope, and descent rate, to avoid warnings and exceeding limits such as "BANK ANGLE", "SINK RATE" and flap overspeed.

[0063] OB2.9: When there is a crosswind, check for crossovers (track and heading).

[0064] Phase 3: Five-sided continuous landing.

[0065] OB3.1: Perform at least four instrument scans between 500ft and 200ft, including: track, heading, glide path, rate of descent, speed, attitude, and N1#.

[0066] OB3.2: Perform at least one instrument scan between 200ft and 100ft.

[0067] 0B3.3: Below 100ft, as the altitude decreases, the line of sight gradually shifts from the instruments to visual sight, and at least one instrument scan is performed, including: speed and N1.

[0068] OB3.4: After leveling off, the line of sight shifts significantly towards the front of the runway (as far as possible in low visibility conditions).

[0069] *N1 should be checked after adjusting the speed.

[0070] OB3.5: During takeoff and landing, keep your line of sight far ahead to ensure a stable and normal takeoff and landing trend for the aircraft.

[0071] OB3.6: Verify Vref and then "raise head".

[0072] When retracting the flaps at OB3.7:1000ft speed increase, check the barometric altimeter.

[0073] OB3.8: After retracting the flaps 5, check the position indicator and green light.

[0074] Scenario 2 - Unguided ILS approach (total 7 OBs).

[0075] Conditions: Daytime, 320 / 10-15ms, RVR 800m, mild bumps.

[0076] Observation object: PF.

[0077] OB4.1: After deploying the wheels and flaps 15, verify that they are in place (using the position indicator and green light).

[0078] 0B4.2: After deploying the flaps 30 degrees, verify that they are in place (indicated by the position chart and green light).

[0079] OB4.3: When performing the landing checklist, first "look" and then "talk".

[0080] OB4.4: Perform at least six instrument scans between 500ft and 200ft, including: track, path, glide slope, rate of descent, speed, attitude, and N1*.

[0081] OB4.5: Perform at least one external scan between 200ft and 100ft to find a valid visual reference and be able to return to instrument scan mode.

[0082] 0B4.6: Below 100ft, as the altitude decreases, gradually shift your line of sight from the instruments to your eyes, and perform at least one instrument scan, including: speed and N1.

[0083] 0B4.7: After leveling off, the line of sight shifts significantly towards the front of the runway (as far as possible in low visibility conditions).

[0084] *N1 should be checked after adjusting the speed.

[0085] Standardized AOI Model Construction: Based on actual flight mission requirements, the project team innovatively divided the cockpit into eight key areas of interest (AOIs): "course / track, attitude, bank angle, altitude, rate of climb / drop, speed, external, and N1." Figure 2 This provides a standardized framework for quantitatively analyzing pilots' attention distribution during specific phases (especially approach and landing), enabling eye-tracking data to be directly correlated with specific flight operations.

[0086] In statistics, the Pearson correlation coefficient is used to measure the correlation (linear correlation) between two variables X and Y. Its value ranges from -1 to 1, with a value close to 1 indicating a strong positive correlation, close to -1 indicating a strong negative correlation, and close to 0 indicating no linear correlation. Therefore, in this invention, the attention allocation score is calculated using the Pearson correlation coefficient to determine the similarity between the number of times the student visits the area of ​​interest and the standard pattern, with a value ranging from 0 to 1.

[0087] By estimating the covariance and standard deviation of a sample, the Pearson correlation coefficient can be obtained, commonly represented by the lowercase letter 'r', as shown in the following formula: ; Where Xi represents the number of times the student visits the i-th region of interest, and Yi represents the number of times the standard mode visits the i-th region of interest. and These represent the corresponding means, and n represents the total number of visits. The Pearson correlation coefficient can be used to show the similarity between the number of fixations in the region of interest and a standard. The standard can be developed according to the regulations manual and the requirements of each airline.

[0088] Analyzing the rate of change in pupil diameter is used to assess a pilot's level of stress or cognitive load. Pupil diameter is primarily affected by two factors: (1) The effect of light intensity: When the external light is strong, the pupil constricts to reduce the intensity of light hitting the retina, thus protecting the retina from damage by strong light. At the same time, the constriction of the pupil can also reduce spherical aberration and chromatic aberration, while increasing the depth of focus of the eye, making objects clearer. When the external light is weak, the pupil dilates to increase the amount of light entering the pupil, increasing the retinal illumination, and making objects clearer.

[0089] (2) The influence of physiological state: Research results show that in the same working environment, the change in pupil area can evaluate the different mental loads; as the tension increases, the pupils dilate; when a certain level of fatigue is reached, the pupils begin to constrict.

[0090] Constructing the Markov transition matrix of the region of interest (AOI) can be used to quantify the gaze transition probability between different AOIs, mainly for the following two purposes: (1) Quantitative analysis: The Markov transition probability matrix can transform the driver's visual scanning path into quantified data, revealing the gaze transition probability between different instruments. This allows us to conduct detailed statistical analysis, thereby better understanding the driver's gaze behavior.

[0091] (2) Behavioral pattern recognition: By calculating the shift probability of the gaze area, we can identify the pilot's visual scanning strategy in different flight missions. This is crucial for assessing the pilot's operating habits and identifying potential problems.

[0092] The order and duration of visits to areas of interest can further characterize a pilot's performance in the assessment subjects. The comparison of the number of visits inside and outside the cockpit reflects the different levels of attention pilots pay to external views and internal instruments during the assessment subjects. Cockpit scan patterns, statistically, show that pilots generally use three scan patterns for internal instruments and external views during flight: "L" shape, "triangle" shape, and "V" shape, such as... Figure 3 As shown.

[0093] S103. Adjust the pilot's attention allocation training emphasis based on the attention allocation score and attention shift profile.

[0094] In this step, after the first training session, a second training session is conducted, and the data from the two training sessions are compared and analyzed to generate a training report. The comparative analysis includes one or more of the following: comparison of OB item trigger counts, attention allocation scores, pupil diameter change rates, and AOI access patterns. The training report includes one or more of the following: OB item trigger statistics charts, attention allocation score trend graphs, pupil diameter change scatter plots, AOI access frequency distribution graphs, and Markov transition matrix visualizations.

[0095] The pilot's attention allocation is determined based on the pilot's attention allocation score and attention shift profile, and training is strengthened based on the problems that the pilot has in the attention shift process.

[0096] In one embodiment, the number of observables (OBs) is compared between two training sessions: e.g. Figure 4 The figure shows the number of individual OB items triggered in two training sessions for a group of 8 people (numbered ABCDEFG).

[0097] like Figure 4 As shown, the number of individual OB (Obstruction of Business) triggers decreased between the two training sessions (7 people experienced a significant reduction in OB triggers after training, with only one person showing an increase). During the first simulator training session, the triggering of OB items by trainees is recorded, and after the training, the data is used to analyze problems encountered during flight, including OB triggers. Therefore, in the second training session, pilots are highly likely to focus on improving their weak areas, except in special circumstances.

[0098] The key OB is defined as follows: The first training session triggered OBs (Objects) with a number greater than or equal to 50% of the trainees; the second training session triggered OBs with a number greater than or equal to 25% of the trainees.

[0099] Comparison of the number of key observables (OBs) in the two training sessions (e.g.) Figure 5 As shown, the decrease in the number of participants triggering key OB items in the two training sessions is as follows: Figure 6 As shown, the charts for the number of AOI visits reflect the pilots' level of attention to different flight parameters and the number of visits within the assessment subjects, such as... Figure 7 As shown. The Pearson correlation coefficient quantifies the degree of agreement between the pilot's AOI gaze pattern and the standard pattern, such as... Figure 8 As shown, training data from a group of 20 participants demonstrates that, after systematic eye-tracking feedback training, the average correlation coefficient significantly improved from 0.43 in the initial training (first training session) to 0.93 in subsequent training (second training session). This clearly proves the substantial progress in the standardization and efficiency of attention allocation patterns. Additionally, a scatter plot of pupil changes is also available. Figure 9 As shown, and as illustrated in the schematic diagram of the Markov transition matrix for the region of interest. Figure 10 As shown, comparing novice drivers with experienced drivers, the differences in their gaze behavior can be identified using Markov transition matrices. This helps optimize training programs and improve the skills of novice drivers.

[0100] Using the aforementioned method, by having pilots wear head-mounted eye trackers and collect fixation point data, combined with interest zone analysis, pupil dynamic monitoring, and shift model construction, an objective quantitative assessment and precise optimization of pilots' attention allocation ability can be achieved. This method establishes an attention shift profile through multi-dimensional data such as access frequency, pupillary response, and saccade paths, overcoming the limitations of traditional reliance on subjective judgment and making training assessment more scientific and operable. By comparing the correlation coefficients between standard models and individual data to generate attention allocation scores, it is possible to clarify the pilots' strengths and weaknesses in areas such as in-cabin and out-of-cabin equipment attention and task switching efficiency, thereby allowing for targeted adjustments to training content. Specifically, for pilots with abnormal shift matrices, enhanced situational switching training can be implemented, or fatigue risks can be identified through pupil change rate, ultimately improving their situational awareness and emergency response capabilities, forming a closed-loop training system of "data feedback - precise intervention - capability enhancement."

[0101] Secondly, the present invention also provides a pilot attention allocation training device, such as... Figure 11 As shown, it includes: The acquisition module 201 is used to acquire the pilot's gaze point video and record the triggering of preset observables when the pilot is training with a head-mounted eye tracker.

[0102] The analysis module 202 is used to divide the fixation point video into regions of interest, extract the data features of each region of interest, determine the pilot's attention shift profile based on the data features, and determine the attention allocation score based on the correlation coefficient between the number of visits to each region of interest and the preset standard pattern. The attention shift profile is used to describe the process of the pilot's attention allocation and shift during training.

[0103] The adjustment module 203 is used to adjust the pilot's attention allocation training emphasis based on the attention allocation score, attention shift profile, and the triggering of observables.

[0104] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The steps of the pilot attention allocation training method provided.

[0105] This invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for various operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above-mentioned functions. Figure 1 The steps of the pilot attention allocation training method provided.

[0106] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the patent of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for training pilots' attention allocation, characterized in that, The method includes: When pilots are training with a head-mounted eye tracker, the system acquires videos of the pilots' gaze points and records the triggering of preset observables. The fixation point video is divided into regions of interest, and data features of each region of interest are extracted. Based on the data features, the pilot's attention shift profile is determined, and the attention allocation score is determined according to the correlation coefficient between the number of visits to each region of interest and the preset standard mode. The attention shift profile is used to describe the process of the pilot's attention allocation and shift during training. The pilot's attention allocation training emphasis is adjusted based on the attention allocation score, attention shift profile, and the triggering of observables.

2. The method according to claim 1, characterized in that, The area of ​​interest comprises eight regions: Channel track, attitude, gradient, altitude, rate of rise and fall, speed, external visibility, and engine parameters; The data features include: number of visits, scatter plot of pupil diameter change, Markov transition matrix, visit order and duration, number of visits inside and outside the cockpit, cockpit scan mode model, and the pilot's pupil diameter change rate.

3. The method according to claim 2, characterized in that, The attention allocation score is calculated using the Pearson correlation coefficient to determine the similarity between the number of times a student visits a region of interest and the standard pattern, with a value ranging from 0 to 1.

4. The method according to claim 3, characterized in that, The formula for calculating the Pearson correlation coefficient is as follows: ； Among them, X i Y represents the number of times a student visits the i-th area of ​​interest. i The preset standard mode is used to determine the number of visits to the i-th region of interest. and These are the corresponding averages, and n is the total number of visits.

5. The method according to claim 2, characterized in that, The step of extracting data features from each region of interest and determining the pilot's attention shift profile based on these data features includes: Determine the scatter plot of the pilot's pupil diameter change and the rate of change of pupil diameter in each region of interest to assess the pilot's stress level or cognitive load; A Markov transition matrix for pilot attention between regions of interest is constructed to quantify the gaze transition probability between different regions of interest; The pilots' focus and neglect tendencies regarding different flight parameters were assessed based on the number of visits to each region of interest. The number of visits to areas of interest inside and outside the cockpit was counted to quantify the proportion of pilot attention allocated between internal instrument monitoring and external environmental observation. Based on the order and duration of visits to areas of interest, a visual scanning path model of the pilot is constructed and matched with a preset scanning model for identification.

6. The method according to claim 1, characterized in that, The method further includes: After the first training session, a second training session is conducted, and the data from the two training sessions are compared and analyzed to generate a training report. The comparative analysis includes one or more of the following: comparison of the number of observable triggers, comparison of attention allocation scores, comparison of pupil diameter change rate, and comparison of interest area access patterns.

7. The method according to claim 6, characterized in that, The training report includes one or more of the following: observable trigger statistics charts, attention allocation score change trend charts, pupil diameter change scatter plots, interest area visit frequency distribution charts, and Markov transition matrix visualization charts.

8. A pilot attention allocation training device, characterized in that, The device includes: The acquisition module is used to acquire video of the pilot's gaze point and record the triggering of preset observables when the pilot is training with a head-mounted eye tracker. The analysis module is used to divide the fixation point video into regions of interest, extract data features of each region of interest, determine the pilot's attention shift profile based on the data features, and determine the attention allocation score based on the correlation coefficient between the number of visits to each region of interest and the preset standard pattern; the attention shift profile is used to describe the pilot's attention allocation and shift process during training. The adjustment module is used to adjust the pilot's attention allocation training emphasis based on the attention allocation score, attention shift profile, and the triggering status of observables.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images