Eye movement tracking-based closed cabin human eye perception evaluation method

Abstract

A closed cabin human eye perception evaluation method based on eye movement tracking belongs to the technical field of computer vision, and comprises the following steps: collecting eye movement data of simulation personnel in a specific simulation environment, analyzing the eye movement data, and analyzing and calculating the reaction time and the reaction accuracy of the simulation personnel to a moving target; the visual attention distribution and information processing capability of simulation personnel in a specific task situation can be revealed, so that the decision-making efficiency and the response capability of the simulation personnel in a complex dynamic environment are reflected; the continuity of the spliced screen is calculated, and the fluency degree of the connection effect at the spliced position of the screen is reflected; the eye movement focusing degree is analyzed, the attention level of simulation personnel on a target area can be quantified, whether visual attention is concentrated in a key information area or not is helped to be judged, multiple indexes are input into a regression model, and a comprehensive human eye perception score is obtained. The method has the characteristic of human eye perception evaluation for multi-screen splicing, and a comprehensive, accurate and easy-to-operate evaluation tool is provided.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to an eye perception evaluation method for a closed cabin based on eye tracking. BACKGROUND

[0002] Eye tracking technology is a technology that studies individual visual behavior, cognitive patterns and perception processes by recording eye movement information. It captures the dynamic characteristics of the eyeball through devices such as eye trackers, providing important evidence for studying the degree of attention, information processing and decision-making behavior of individuals to visual information.

[0003] Existing eye perception evaluation methods mainly include the following categories:

[0004] (1) Subjective evaluation method: through questionnaire survey or interview, etc. to obtain the subjective score of the subject on visual experience or perception effect. Although subjective evaluation can directly reflect the perception of the subject, its result is often affected by individual differences, emotions and environment, etc. and has strong subjectivity. In addition, this method usually needs to be carried out after the task is completed, and cannot evaluate the perception change in the task process in real time.

[0005] (2) Task performance-based evaluation method: by analyzing the task completion time, accuracy rate and other behavioral indicators, the perception effect of individuals to the task environment is indirectly reflected. However, task performance is easily disturbed by various external factors, such as task difficulty or external interference, so it is difficult to be used as a reliable perception evaluation basis alone.

[0006] (3) Physiological data-based evaluation method: using electroencephalogram (EEG), electrodermal activity (EDA), heart rate and other physiological signals to analyze the perception and cognitive state of individuals in a specific situation. Although this method has a certain objectivity, the related equipment is usually expensive, bulky and inconvenient to wear, especially in the actual human-computer interaction environment, which may interfere with the natural behavior of the subject, and it is difficult to realize large-scale deployment. SUMMARY

[0007] In view of the deficiencies of the prior art, the application provides an eye perception evaluation method for a closed cabin based on eye tracking, which can solve the problems of subjective evaluation method not having real-time, contact-type physiological data acquisition device interfering with the subject, and the need to input observed images or videos, resulting in high data processing complexity and storage requirement.

[0008] The eye perception evaluation method for a closed cabin based on eye tracking provided by the application comprises the following steps:

[0009] 1) Calculate the reaction accuracy through eye coordinates, comprising the following steps:

[0010] 1.1 In the flight video, the coordinates of the moving target points in each frame are obtained, denoted as (X i , Y i ), where i = 1, 2, …, m, m is the total number of frames of the video, i.e. the total number of target points; the eye movement coordinates of the fixation points collected when the simulated personnel watch the flight video are obtained, denoted as (x j , y j ), where j = 1, 2, …, n, n is the total number of fixation points in one experiment; the collection frequency of the eye tracker is greater than the frame rate of the flight video; based on the total number of eye movement coordinates of the fixation points, uniform linear interpolation is performed on the target point coordinates to achieve time sequence alignment, and a one-to-one corresponding target point-fixation point coordinate pair is obtained;

[0011] 1.2 According to the Euclidean distance formula: , the average distance d of the target point and the fixation point coordinate pair and the average inter-frame movement distance mt of the target point are calculated, and the radius r reflecting the matching accuracy of the target point and the fixation point can be obtained, and the mathematical expression is: r = d - αm t , where: α is a radius adjustment parameter; the mathematical expression of the reaction accuracy acc is: , where: w and h are the width 3840 pixels and the height 1080 pixels of the display picture after splicing, respectively;

[0012] 2) Calculate the reaction time through the eye movement coordinates, including the following steps:

[0013] 2.1 In the flight video, the time reference point t is determined as the moment when the target point first appears, the unit is millisecond, and the moment corresponds to the kth frame in the target point coordinate sequence, denoted as the starting target point coordinate (X k , Y k ), and the fixation point sequence (x j , y j ) and its corresponding time stamp t j of the subject after the time reference point are obtained synchronously, where j = 1, 2, …, p, p is the total number of fixation points in the effective observation period;

[0014] 2.2 Calculate the time window division of the target point through the video frame rate, and the time length of each window is millisecond, f is the video frame rate, and the time sequence mapping relationship of the target point-fixation point is established based on the time window: for the i th target point time window [t+iΔt, t+(i+1)Δt], all effective fixation point coordinates in the interval are extracted;

[0015] 2.3 Define the target display radius threshold R, and determine the reaction time through the following steps:

[0016] 2.3.1 Calculate the starting target point coordinate (X k , Yk ) and the Euclidean distance dj of each valid fixation point (x j , y j ) from the center of the screen, which is mathematically expressed as:

[0017]

[0018] 2.3.2 Establish a time series detection queue, and traverse the fixation points in ascending order of time stamp. When d j ≤R for the first time, record the fixation point time stamp t hit ;

[0019] 2.3.3 The mathematical expression of the reaction time RT is:

[0020] RT = t hit -t;

[0021] If no fixation point located in the time window of step 2.2 is detected during the entire observation period, it is recorded as an invalid test;

[0022] 3) Calculate the continuity through eye movement coordinates, including the following steps:

[0023] 3.1 Based on the spatiotemporal distribution characteristics of the fixation point sequence, divide the display screen horizontally into 5 dynamic interest areas, with the boundaries defined as [0, 160], [160, 1760], [1760, 2080], [2080, 3680], and [3680, 3840];

[0024] 3.2 Map each fixation point horizontal coordinate x j to the corresponding interest area number R j through an indicator function, and construct a Markov transition frequency matrix T between adjacent fixation points, with the elements T mn in the matrix being the number of times that a fixation point is transferred from interest area R m to interest area R n , where m, n = 1, 2,..., 5;

[0025] 3.3 Divide each element of the transition frequency matrix by the sum of its row, so that the sum of all elements in each row is 1, thereby obtaining the transition probability matrix P mn , and further calculating the Shannon entropy value H = -∑ m，n P mn lnP mn to quantify the temporal randomness of fixation transition; to eliminate individual differences, linearly map the original entropy value to the interval [0.5, 1] to obtain the normalized entropy H norm , and construct the temporal regularity index 1-H nam ;

[0026] 3.4 Analyze the convergence characteristics of the central interest area (i=3) in the transition probability matrix and calculate the average transition probability of all areas to the central area And construct a spatial stability index

[0027] 3.5 Continuity index C is obtained by integrating spatial stability and temporal regularity H norm get:

[0028]

[0029] Among them: spatial stability reflects the subject's tendency to avoid fixating on the central splicing area, and a larger value indicates a more even visual distribution; temporal regularity represents the certainty of the gaze shift pattern, and a value closer to 1 indicates a more coherent vision;

[0030] 4) Calculating the focus degree through eye movement coordinates includes the following steps:

[0031] 4.1 Based on the spatiotemporal distribution characteristics of fixation points, the degree of visual attention concentration is quantified by spatial discretization within a dynamic window, and the theoretical maximum variance is calculated. represents the spatial discretization limit when the gaze points are uniformly distributed;

[0032] 4.2 Time window [t i , t i +T] the set of fixation points Ω i , calculate its centroid coordinates And the spatial distribution variance, its mathematical expression is:

[0033]

[0034] 4.3 According to the normalization formula: The window focus is obtained, and its value range is [0, 1], which can quantify the intensity of gaze concentration, F i →1 means high focus, F i →0 indicates high dispersion;

[0035] 4.4 To enhance dynamic adaptability, the focus algorithm adopts a dual adjustment mechanism, including the following steps:

[0036] 4.4.1 When the gaze point movement speed v>1000 pixels / s, the window duration is reduced from the baseline value T = 1000ms to T′ = 500ms to improve the response sensitivity to rapid eye movements. A Hanning window function is introduced to attenuate the gaze points at the edge of the window to eliminate the variance estimation bias caused by data truncation.

[0037] 4.4.2 In the comprehensive calculation of global focusing, an exponential decay weight is used, and its mathematical expression is:

[0038]

[0039] Wherein: Lambda is 0.15, the window focus degree is weighted and fused to generate the final index, and the mathematical expression is:

[0040]

[0041] The weight is through |Omega i | Inverse proportion adjustment, inhibit the interference of low density window to the result;

[0042] 5) Generate a prediction result through a random forest regression model, including the following steps:

[0043] 5.1 Divide the collected eye movement data into a training set and a test set in a ratio of 7:3, and ensure the generalization ability of the model;

[0044] 5.2 In the training stage, 100 decision trees are constructed to fit the training set data, and each tree is trained by randomly selecting features and samples to reduce the risk of model overfitting;

[0045] 5.3 Enable out-of-bag error calculation to evaluate the prediction performance of the model in real time, and ensure the stability and reliability of the model;

[0046] 5.4 After the model training is completed, the test set data is predicted to generate the predicted value of the human eye perception evaluation score; the mean square error and the correlation coefficient between the predicted value and the actual subjective score are calculated to evaluate the prediction accuracy of the model; the leaf node minimum sample number 5 is introduced as a hyperparameter to control the complexity of the decision tree and avoid overfitting of the model to the training data.

[0047] The application can realize real-time, non-contact quantitative evaluation of the visual perception characteristics of the simulation personnel. By analyzing the reaction time and reaction accuracy, the perception efficiency and reliability of the simulation personnel on the visual stimulation can be directly reflected, and the bias and lag problems that may exist in the traditional subjective evaluation method can be avoided. Continuous calculation can reveal the seamless connection effect of the visual content at the screen splicing place, and reflect the attention distribution mode of the simulation personnel in the dynamic scene, which is of great significance for evaluating the rationality of the design of the cockpit display system. The analysis of the focus degree can quantify the visual attention concentration degree of the simulation personnel, and provide an objective basis for evaluating their information processing ability in complex tasks. Based on these characteristics, the application maps the eye movement data features to the human eye perception evaluation score through a random forest regression model, and constructs an efficient and accurate perception evaluation system, which provides scientific basis and technical support for the optimization of the closed cockpit human-computer interaction design. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 It is the principle block diagram of the closed cockpit human eye perception evaluation system based on eye movement data.

[0049] Figure 2 A schematic diagram for eye movement data acquisition;

[0050] Figure 3 A flow chart of an eye movement tracking based closed cockpit human eye perception evaluation method. DETAILED DESCRIPTION

[0051] The present application will be described below with reference to the accompanying drawings.

[0052] As Figure 1 shown, an eye movement tracking based closed cockpit human eye perception evaluation method of the present application comprises the following steps:

[0053] 1) Calculate the reaction accuracy through eye movement coordinates, comprising the following steps:

[0054] 1.1 In the flight video, obtain the coordinates of the moving target points in each frame, denoted as (X i , Y i ), where i = 1, 2, …, m, m is the total number of frames of the video, i.e. the total number of target points; obtain the eye movement coordinates of the fixation points collected when the simulated personnel watch the flight video, denoted as (x j , y j ), where j = 1, 2, …, n, n is the total number of fixation points in one experiment; the collection frequency of the eye tracker is greater than the frame rate of the flight video, and based on the total number of eye movement coordinates of the fixation points, uniform linear interpolation is performed on the target point coordinates for time sequence alignment to obtain a one-to-one corresponding target point-fixation point coordinate pair;

[0055] 1.2 Calculate the average distance d of the target point and fixation point coordinate pair and the average inter-frame movement distance mt of the target point according to the Euclidean distance formula: , and the radius r reflecting the matching accuracy of the target point and the fixation point can be obtained, and the mathematical expression is: r = d - am t , where: a is the radius adjustment parameter; the mathematical expression of the reaction accuracy acc is: , where: w and h are the width 3840 pixels and the height 1080 pixels of the display picture after splicing, respectively;

[0056] 2) Calculate the reaction time through eye movement coordinates, comprising the following steps:

[0057] 2.1 In the flight video, determine the time reference point t in milliseconds as the moment when the target point first appears, which corresponds to the kth frame in the target point coordinate sequence, denoted as the starting target point coordinate (Xk, Yk), and synchronously obtain the fixation point sequence (x j , y j ) after the time reference point t and its corresponding time stamp t j, where: j = 1, 2, ..., p, p is the total number of fixations during the effective observation period;

[0058] 2.2 Calculate the time window division of the target point by the video frame rate, and the length of each window is milliseconds, f is the video frame rate, and based on the time window, the target point--fixation point temporal mapping relationship is established: for the i-th target point time window [t+iΔt, t+(i+1)Δt], all valid fixation point coordinates in the interval are extracted;

[0059] 2.3 Define the target display radius threshold R and determine the reaction time through the following steps:

[0060] 2.3.1 Calculate the coordinates of the starting target point (X k , Y k ) and each valid fixation point (x j ,y j )’s Euclidean distance dj, its mathematical expression is:

[0061]

[0062] 2.3.2 Establish a time series detection queue and traverse the fixation points in ascending order of timestamps. When d appears for the first time j When ≤R, record the timestamp t of the gaze point hit ;

[0063] 2.3.3 The mathematical expression of reaction time RT is:

[0064] RT = t hit -t;

[0065] If the fixation point located in the time window of step 2.2 is not detected during the entire observation period, it is recorded as an invalid trial;

[0066] 3) Calculating continuity through eye movement coordinates, including the following steps:

[0067] 3.1 Based on the spatiotemporal distribution characteristics of the fixation sequence, the display screen is horizontally divided into five dynamic interest regions, with boundaries defined as [0, 160], [160, 1760], [1760, 2080], [2080, 3680], and [3680, 3840] respectively;

[0068] 3.2 Through the characteristic function, the horizontal coordinate x of each gaze point is j Mapped to the corresponding area of ​​interest number R j , and construct the Markov transition frequency matrix T between adjacent gaze points, the elements in the matrix T mn The fixation point is the region of interest R m Transfer to area of ​​interest R nwhere: m, n = 1, 2,..., 5

[0069] 3.3 Divide each element of the transition frequency matrix by the sum of its row, so that the sum of all elements in each row is 1, and get the transition probability matrix P mn , and then calculate the Shannon entropy value H = -∑ m，n P mn lnP mn to quantify the timing randomness of gaze transition; to eliminate individual differences, linearly map the original entropy value to the interval [0.5, 1] to get the normalized entropy H norm , and construct the timing regularity index 1-H nam ;

[0070] 3.4 Analyze the convergence characteristics of the central interest area (i = 3) in the transition probability matrix, calculate the average transition probability of all areas to the central area , and construct the spatial stability index

[0071] 3.5 The continuity index C is obtained by fusing the spatial stability and the timing regularity H norm :

[0072]

[0073] Where: the spatial stability reflects the subject's tendency to avoid fixation on the central stitching area, and the larger the value, the more uniform the visual distribution; the timing regularity represents the certainty of the gaze transition pattern, and the closer the value to 1, the more coherent the visual line;

[0074] 4) Calculate the focus degree through eye movement coordinates, including the following steps:

[0075] 4.1 Based on the spatiotemporal distribution characteristics of the gaze points, quantify the degree of visual attention concentration by the spatial dispersion in the dynamic window, and calculate the theoretical maximum variance , which represents the spatial dispersion limit when the gaze points are uniformly distributed;

[0076] 4.2 Calculate the centroid coordinates and the spatial distribution variance of the gaze point set Ω i in the time window [t i , t i +T], and the mathematical expression is:

[0077]

[0078] 4.3 Get the window focus degree F i from the normalized formula: , whose value range is [0, 1], which can quantify the intensity of gaze concentration, F i→ 0 means highly dispersed;

[0079] 4.4 To enhance dynamic adaptability, the focusing degree algorithm adopts a dual adjustment mechanism, including the following steps:

[0080] 4.4.1 When the fixation point moving speed v > 1000 pixels / s, the window length is contracted from the baseline value T = 1000 ms to T' = 500 ms, improving the response sensitivity to rapid eye jumps; the Hanning window function is introduced to attenuate and weight the fixation points at the window edges, eliminating the variance estimation bias caused by data truncation;

[0081] 4.4.2 In the global focusing degree comprehensive calculation, an exponential decay weight is used, and its mathematical expression is:

[0082]

[0083] Where: λ is 0.15, the window focusing degree is weighted and fused to generate the final index, and its mathematical expression is:

[0084]

[0085] The weight is adjusted inversely proportional to |Ω i , to suppress the interference of low-density windows on the results;

[0086] 5) Generate the prediction results through the random forest regression model, including the following steps:

[0087] 5.1 Divide the collected eye movement data into training set and test set according to the ratio of 7:3, to ensure the generalization ability of the model;

[0088] 5.2 In the training stage, 100 decision trees are constructed to fit the training set data, and each tree is trained by randomly selecting features and samples to reduce the risk of model overfitting;

[0089] 5.3 Enable out-of-bag error calculation to evaluate the prediction performance of the model in real time, to ensure the stability and reliability of the model;

[0090] 5.4 After the model training is completed, the test set data is predicted to generate the predicted value of the human eye perception evaluation score; the mean square error and correlation coefficient between the predicted value and the actual subjective score are calculated to evaluate the prediction accuracy of the model; the leaf node minimum sample number 5 is introduced as a hyperparameter to control the complexity of the decision tree, to avoid overfitting of the model to the training data.

Claims

1. A closed cabin human eye perception evaluation method based on eye tracking, characterized by: The following steps are involved: 1) Calculating response accuracy using eye movement coordinates, including the following steps: 1.1 In the flight video, obtain the coordinates of the moving target point in each frame, recorded as (X i ,Y i ), where i = 1, 2, ..., m, where m is the total number of frames of the video, i.e. the total number of target points; the eye movement coordinates of the gaze points collected when the simulated personnel watched the flight video are obtained, recorded as (x j ,y j ), where j = 1, 2, …, n, where n is the total number of fixation points in an experiment; the acquisition frequency of the eye tracker is greater than the frame rate of the flight video. Based on the total number of fixation coordinates, the target point coordinates are time-aligned by uniform linear interpolation to obtain one-to-one corresponding target point-fixation point coordinate pairs; 1.2 According to the Euclidean distance formula: By calculating the average distance d between the target point and the gaze point coordinate pair, and the average inter-frame moving distance mt of the target point, we can obtain the radius r that reflects the matching accuracy of the target point and the gaze point. The mathematical expression is: r = d - αm t , where α is the radius adjustment parameter; the mathematical expression of the reaction accuracy acc is: Where: w and h are the width of the spliced ​​display image, 3840 pixels, and the height, 1080 pixels, respectively; 2) Calculating reaction time using eye movement coordinates, including the following steps: 2.1 In the flight video, the time when the target point first appears is determined as the time reference point t, in milliseconds. This time corresponds to the kth frame in the target point coordinate sequence and is recorded as the starting target point coordinate (X k ,Y k ), synchronously obtain the subject's gaze sequence after the time reference point (x j ,y j ) and its corresponding timestamp t j , where: j = 1, 2, ..., p, p is the total number of fixations during the effective observation period; 2.2 Calculate the time window division of the target point by the video frame rate, and the length of each window is milliseconds, f is the video frame rate, and based on the time window, the target point--fixation point temporal mapping relationship is established: for the i-th target point time window [t+iΔt, t+(i+1)Δt], all valid fixation point coordinates in the interval are extracted; 2.3 Define the target display radius threshold R and determine the reaction time through the following steps: 2.3.1 Calculate the coordinates of the starting target point (X k ,Y k ) and each valid fixation point (x j ,y j )’s Euclidean distance dj, its mathematical expression is: 2.3.2 Establish a time series detection queue and traverse the fixation points in ascending order of timestamps. When d appears for the first time j When ≤R, record the timestamp t of the gaze point hit ; 2.3.3 The mathematical expression of reaction time RT is: RT=t hit -t; If the fixation point located in the time window of step 2.2 is not detected during the entire observation period, it is recorded as an invalid trial; 3) Calculating continuity through eye movement coordinates, including the following steps: 3.1 Based on the spatiotemporal distribution characteristics of the fixation sequence, the display screen is horizontally divided into five dynamic interest regions, with boundaries defined as [0, 160], [160, 1760], [1760, 2080], [2080, 3680], and [3680, 3840] respectively; 3.2 Through the characteristic function, the horizontal coordinate x of each gaze point is j Mapped to the corresponding area of ​​interest number R j , and construct the Markov transition frequency matrix T between adjacent gaze points, the elements in the matrix T mn The fixation point is the region of interest R m Transfer to area of ​​interest R n The number of times, where: m,n=1,2,...,5; 3.3 Divide each element of the transfer frequency matrix by the sum of its row so that the sum of all elements in each row is 1, thus obtaining the transfer probability matrix P mn , and then calculate the Shannon entropy value H = -∑ m，n P mn ln P mn To quantify the temporal randomness of gaze shifts, the original entropy value is linearly mapped to the interval [0.5, 1] ​​to obtain the normalized entropy H. norm , and construct the time series regularity indicator 1-H norm ; 3.4 Analyze the convergence characteristics of the central interest area (i=3) in the transition probability matrix and calculate the average transition probability of all areas to the central area And construct a spatial stability index 3.5 Continuity index C is obtained by integrating spatial stability and temporal regularity H norm get: Among them: spatial stability reflects the subject's tendency to avoid fixating on the central splicing area, and a larger value indicates a more even visual distribution; temporal regularity represents the certainty of the gaze shift pattern, and a value closer to 1 indicates a more coherent vision; 4) Calculating the focus degree through eye movement coordinates includes the following steps: 4.1 Based on the spatiotemporal distribution characteristics of fixation points, the degree of visual attention concentration is quantified by spatial discretization within a dynamic window, and the theoretical maximum variance is calculated. represents the spatial discretization limit when the gaze points are uniformly distributed; 4.2 Time window [t i , t i +T] the set of fixation points Ω i , calculate its centroid coordinates And the spatial distribution variance, its mathematical expression is: 4.3 According to the normalization formula: The window focus is obtained, and its value range is [0,1], which can quantify the intensity of gaze concentration, F i →1 means high focus, F i →0 indicates high dispersion; 4.4 To enhance dynamic adaptability, the focus algorithm adopts a dual adjustment mechanism, including the following steps: 4.4.1 When the gaze point movement speed v>1000 pixels / s, the window duration is reduced from the baseline value T = 1000ms to T′ = 500ms to improve the response sensitivity to rapid eye movements. A Hanning window function is introduced to attenuate the gaze points at the edge of the window to eliminate the variance estimation bias caused by data truncation. 4.4.2 In the comprehensive calculation of global focusing, an exponential decay weight is used, and its mathematical expression is: in: λ is 0.15, and the window focus is weighted and fused to generate the final index, whose mathematical expression is: The weight is expressed by |Ω i | Inverse proportional adjustment to suppress the interference of low-density windows on the results; 5) Generate prediction results through random forest regression model, including the following steps: 5.1 The collected eye movement data was divided into training set and test set in a ratio of 7:3 to ensure the generalization ability of the model; 5.2 During the training phase, 100 decision trees are constructed to fit the training set data. Each tree is trained by randomly selecting features and samples to reduce the risk of overfitting the model. 5.3 Enable out-of-bag error calculation to evaluate the model's prediction performance in real time to ensure the model's stability and reliability; 5.4 After model training is completed, the test set data is predicted to generate predicted values ​​of human visual perception evaluation scores; the prediction accuracy of the model is evaluated by calculating the mean square error and correlation coefficient between the predicted values ​​and the actual subjective scores; the minimum number of leaf node samples of 5 is introduced as a hyperparameter to control the complexity of the decision tree and avoid overfitting of the model to the training data.

Images