An eye-tracking-based human-computer interaction system and method thereof

Abstract

The present invention discloses a human-computer interaction system and method based on eye movement tracking, belonging to the technical field of human-computer interaction. The present invention collects the eye movement data of simulated personnel in a specific simulation environment and analyzes the eye movement data. During the analysis process, by analyzing the fixation point distribution, it is possible to understand the degree of attention of the simulated personnel to different regions in the simulation environment. Different degrees of attention reflect the importance of the region for the operation and cognition of the simulated personnel. Secondly, the analysis of the saccade trajectory can reveal the visual search pattern of the simulated personnel, which is crucial for evaluating their information acquisition efficiency in a complex simulation environment. Based on these understandings, researchers began to explore how to combine eye movement tracking technology with a simulation training system to improve the efficiency of human-computer interaction.

Description

Technical Field

[0001] The present invention relates to a human-computer interaction system and method based on eye tracking, belonging to the technical field of human-computer interaction. Background Art

[0002] Eye tracking, such as EyeTribe, can track the movement of the human eye when it moves, understand its movement trajectory. Eye tracking is a technology that senses subtle changes in the eye. According to different directions we observe, corresponding features will be generated in the eye. By comparing these features, a set of reference that can be considered for eye changes is formed, and then the control function for eye changes is realized. This is the so-called eye tracking technology.

[0003] According to the publication number CN107656613A, a human-computer interaction system based on eye tracking and its working method are disclosed. The system includes a processor, which is respectively connected to an AR / VR headset device and a video acquisition device; an eye tracking sensor and an angular motion sensor are arranged on the AR / VR headset device. The eye tracking sensor and the angular motion sensor are respectively used to capture eye movement information in real time and collect the current motion state of the AR / VR headset device in real time, and both are transmitted to the processor; the video acquisition device is used to collect the scene image within the line of sight of the eye and transmit it to the processor.

[0004] The above patent improves the functional visibility of the human-computer interaction system based on eye tracking, enabling users to easily discover and use it. Visibility naturally guides people to correctly complete tasks in this way.

[0005] However, when some existing eye tracking interaction systems are in use, for the eye tracking analysis in a simulated environment, in many simulation training systems, the collected data is simply identified. How to accurately evaluate the behavior performance and operation effect of the simulated personnel has become a key issue in improving the training quality. Secondly, in traditional simulation training evaluations, it mainly relies on some direct operation feedback data, such as relatively macroscopic indicators like the operation completion time and the accuracy of the operation result. However, these methods often cannot penetrate into the visual information processing process of the simulated personnel, and visual information processing is an important part of human cognition and operation behavior. Summary of the Invention

[0006] Aiming at the deficiencies of the prior art, the present invention provides a human-computer interaction system and method based on eye tracking, which solves the problem that it cannot penetrate into the visual information processing of the simulated personnel and further cannot accurately evaluate the behavior performance and operation of the simulated personnel.

[0007] To achieve the above objectives, the present invention is realized through the following technical solutions: A human-computer interaction system based on eye tracking, comprising:

[0008] An eye movement data acquisition module, which is used to acquire the eye movement data of simulated personnel in a simulated environment, and at the same time transmit the acquired eye movement data to a fixation point distribution recognition module and an eye movement saccade recognition module;

[0009] A fixation point distribution recognition module, which is used to analyze the fixation point distribution of simulated personnel according to the eye movement data transmitted by the eye movement data acquisition module, obtain key areas and non-key areas, calculate the average fixation duration corresponding to the key areas, further analyze the different attention levels of the key areas, generate attention degree information, and transmit the attention degree information to an identification information output module;

[0010] An eye movement saccade recognition module, which is used to perform saccade analysis according to the acquired eye movement data, analyze the saccade trajectory of simulated personnel, divide the saccade area to obtain segmentation areas, classify the saccade areas based on the saccade speed of the segmentation areas to obtain a pre-saccade area and a slow-saccade area, calculate the saccade distances corresponding to the saccade points in the pre-saccade area and the slow-saccade area respectively, screen the saccade points to obtain effective saccade points, calculate the adjusted saccade speed of the corresponding area according to the effective saccade points, then adjust the saccade speed to generate saccade adjustment information, and transmit the saccade adjustment information to the identification information output module;

[0011] An effect evaluation and analysis module, which is used to perform corresponding simulation effect evaluation according to the acquired saccade adjustment information and attention degree information, generate evaluation and identification information, and transmit the evaluation and identification information to the identification information output module.

[0012] An identification information output module, which is used to transmit the obtained evaluation and identification information to the corresponding simulator terminal.

[0013] As a further solution of the present invention: The specific method for the fixation point distribution recognition module to obtain key areas and non-key areas is as follows:

[0014] Acquire the eye movement data of simulated personnel, and at the same time visually display the fixation point positions and durations corresponding to the simulated personnel, and perform clustering analysis according to different regional colors and the number of fixation points. The specific clustering analysis method is: Mark the areas with brighter colors and longer residence times as key areas, and mark the areas with darker colors and shorter residence times as non-key areas.

[0015] As a further solution of the present invention: The specific method for the fixation point distribution recognition module to generate attention degree information is as follows:

[0016] Label the key areas as i, where i = 1, 2, …, j, and j represents the number of key areas. At the same time, obtain the fixation points in the key areas and the corresponding dwell times. Then calculate the average dwell time of all fixation points in the key area and denote it as Ti. Calculate the average dwell times of all key areas in this way;

[0017] At the same time, classify the key areas according to the average dwell times of the key areas calculated. Calculate the average value of all the average dwell times of the key areas to obtain the average standard value. Then compare the average dwell time of the key area with the average standard value;

[0018] Classify the key areas with an average dwell time greater than the average standard value as complex areas, and classify the key areas with an average dwell time less than the average standard value as simple areas. At the same time, generate attention information and transmit the attention information to the effect evaluation and analysis module.

[0019] As a further solution of the present invention: The specific way for the eye movement saccade recognition module to divide the saccade area into segmented areas is:

[0020] Then divide the corresponding saccade area according to the saccade trajectory of the simulation personnel. Divide the saccade area with the turning point of the saccade trajectory as the demarcation point to obtain the segmented area. At the same time, label the segmented area as a, where a = 1, 2, …, b, and b represents the number of segmented areas.

[0021] As a further solution of the present invention: The specific way for the eye movement saccade recognition module to classify the segmented areas into pre - super areas and slow - marked areas based on the saccade speed of the segmented areas is:

[0022] Obtain the saccade trajectory corresponding to the segmented area a, and at the same time obtain the saccade speed corresponding to the saccade trajectory, and obtain the standard value corresponding to the segmented area a. Then compare the saccade speed of the segmented area with the standard value;

[0023] If the saccade speed of the segmented area is greater than the standard value, mark the corresponding segmented area as a pre - super area. On the contrary, if the saccade speed of the segmented area is less than the standard value, mark the corresponding segmented area as a slow - marked area, and analyze the pre - super area and the slow - marked area.

[0024] As a further solution of the present invention: The specific way for the eye movement saccade recognition module to analyze the pre - super area and the slow - marked area is:

[0025] Taking the pre - hyper region as an example for analysis, all saccade points in the pre - hyper region are obtained and denoted as o, and o = 1, 2, …, p, where p represents the number of saccade points. Then, a corresponding two - dimensional plane coordinate system is established with the center point of the pre - hyper region as the origin. At the same time, the coordinates of the starting point and the ending point of the saccade point o are obtained and denoted as (x o1 , y o1 ) and (x o2 , y o2 ), and the saccade distance corresponding to the saccade point o is calculated according to the Euclidean distance formula ;

[0026] And so on, all saccade distances are calculated. At the same time, the standard quantity in the pre - hyper region is obtained, and the saccade points in the pre - hyper region are classified into valid saccade points and invalid saccade points according to the standard quantity. The starting - point coordinates and ending - point coordinates corresponding to the saccade point o are respectively compared with the standard quantity. If any set of coordinates in the saccade point o does not meet the standard quantity, the corresponding saccade point is classified as an invalid saccade point. On the contrary, if both sets of coordinates of the saccade point o meet the standard quantity, the corresponding saccade point is classified as a valid saccade point. And so on, the slow - mark region is analyzed;

[0027] The standard saccade distance corresponding to the pre - hyper region is calculated according to the valid saccade points, and the specific calculation method is , where u is the number of valid saccade points. And the ratio of the standard saccade distance dz to the saccade distance in the standard value is calculated to obtain the adjustment ratio. At the same time, the saccade speed of the pre - hyper region is adjusted based on the adjustment ratio, and the corresponding saccade adjustment information is generated.

[0028] As a further solution of the present invention: The specific way for the effect evaluation and analysis module to generate evaluation and recognition information is as follows:

[0029] The same simulation operations are performed according to the obtained saccade adjustment information and attention information. At the same time, the simulation operations are evaluated and analyzed, and the simulation operations are compared and analyzed with the standard evaluation information, and evaluation and recognition information is generated.

[0030] As a further solution of the present invention: A human - computer interaction method based on eye movement tracking, which specifically includes the following steps:

[0031] S101: Collect the eye movement data of the simulated person in the simulated environment, and perform subsequent fixation - point distribution recognition and eye movement saccade recognition;

[0032] S102, Visualize the fixation - point position and duration according to the eye movement data, classify the simulated area into a key area and a non - key area, and at the same time classify the key area according to the average duration data into a complex area and a simple area, and generate attention information;

[0033] S103. Analyze the saccade trajectory of the simulated person, divide the saccade area to obtain the segmentation area, and classify based on the saccade speed in the segmentation area to obtain the pre - super area and the slow - mark area;

[0034] S104. Calculate the saccade distances corresponding to the saccade points in the pre - super area and the slow - mark area respectively, screen the saccade points to obtain effective saccade points, calculate the adjusted saccade speed of the corresponding area according to the effective saccade points, and then adjust the saccade speed to generate saccade adjustment information;

[0035] S105. Evaluate the simulation effect according to the attention information and the saccade adjustment information to generate evaluation and recognition information, and transmit the evaluation and recognition information to the corresponding simulator terminal.

[0036] The present invention provides a human - machine interaction system and method based on eye movement tracking. Compared with the prior art, it has the following beneficial effects:

[0037] In the present invention, the eye movement data of the simulated person is collected in a specific simulation environment and analyzed. During the analysis, by analyzing the distribution of fixation points, it is possible to understand the degree of attention of the simulated person to different areas in the simulation environment. Different degrees of attention reflect the importance of the area for the operation and cognition of the simulated person. Secondly, the analysis of the saccade trajectory can reveal the visual search pattern of the simulated person, which is crucial for evaluating their information acquisition efficiency in a complex simulation environment. Based on these understandings, researchers began to explore how to combine eye movement tracking technology with the simulation training system to construct a more accurate and effective human - machine interaction evaluation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a block diagram of the system principle of the present invention;

[0039] Figure 2 It is a schematic diagram of saccade point classification of the present invention;

[0040] Figure 3 It is a flowchart of the method steps of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Embodiment 1

[0043] Please refer to Figure 1 , this embodiment provides a human-computer interaction system based on eye movement tracking, including an eye movement data acquisition module, a fixation point distribution recognition module, an eye movement saccade recognition module, an effect evaluation and analysis module, and an identification information output module. The eye movement data acquisition module, the eye movement saccade recognition module, the effect evaluation and analysis module, and the identification information output module are connected in a one-way electrical connection, and this system is mainly applied to a simulation training system.

[0044] The eye movement data acquisition module is used to collect the eye movement data of the simulated personnel in the simulated environment, and at the same time transmit the collected eye movement data to the fixation point distribution recognition module and the eye movement saccade recognition module.

[0045] The fixation point distribution recognition module is used to analyze the fixation point distribution of the simulated personnel according to the obtained eye movement data, and the specific analysis method is as follows:

[0046] The eye movement data of the simulated personnel is obtained through an eye tracker, and specifically, a video-based eye tracker is used here. On the surface of the eyeball, the cornea reflects light. By analyzing the position changes of these reflected lights in the video, the movement of the eyeball can be calculated. At the same time, the corresponding fixation point positions and durations of the simulated personnel are visually displayed, and clustering analysis is performed according to the colors of different regions and the number of fixation points. The specific clustering analysis method is as follows: The regions with brighter colors (such as red) indicate denser fixation points and longer stay times, and the corresponding regions are marked as key regions; the regions with darker colors (such as blue) indicate fewer fixation points and shorter stay times, and the corresponding regions are marked as non-key regions;

[0047] Then, secondary analysis is performed based on the classified key regions and non-key regions. The secondary analysis mainly calculates the average fixation duration corresponding to the key regions, further analyzes the different degrees of attention of the key regions, and generates attention information. The specific processing and analysis method is as follows:

[0048] Then, the marked key regions are numbered and denoted as i, where i = 1, 2,..., j, and j represents the number of key regions. At the same time, the fixation points in the key regions are obtained, and the corresponding durations of the fixation points are obtained. Then, the average duration of all fixation points in the key region is calculated and denoted as Ti, and so on, the average durations of all key regions are calculated;

[0049] Meanwhile, the key regions will be classified according to the average duration of the calculated key regions, and the specific classification method is as follows: calculate the average value of the average durations of all key regions to obtain the average standard value. For example, there are three key regions. The average duration of all fixation points in the first key region is calculated to be 350 milliseconds, the average duration of the second key region is calculated to be 420 milliseconds, and the average duration of the third key region is calculated to be 380 milliseconds. Finally, the sum of the three values is calculated and then the average value is calculated to obtain an average standard value of approximately 383 milliseconds. Then, compare the average duration of the key region with the average standard value. Classify the key regions with an average duration greater than the average standard value as complex regions, and here "greater than" includes the case of being equal to the average standard value. Classify the key regions with an average duration less than the average standard value as simple regions. Further, generate attention information and transmit the attention information to the effect evaluation and analysis module.

[0050] Embodiment 2

[0051] As Embodiment 2 of the present invention, the difference from Embodiment 1 is as follows:

[0052] An eye movement saccade recognition module, which is used to perform saccade analysis based on the acquired eye movement data, and the specific saccade analysis method is as follows:

[0053] Based on the acquired eye movement data, connect and draw the saccade order and path of the simulated person to obtain the saccade trajectory. Meanwhile, analyze the visual search pattern of the simulated person according to the obtained saccade trajectory and generate visual search information;

[0054] Then, divide the corresponding saccade regions according to the saccade trajectory of the simulated person, and the specific division method is as follows: divide the saccade region with the turning point of the saccade trajectory as the demarcation point to obtain the divided regions. For example, in a saccade region in the simulation environment, there are three turning points in the corresponding saccade trajectory of the simulated person. Further, divide the saccade region with the turning point as the division point to obtain four divided regions. At the same time, label the divided regions as a, and a = 1, 2,..., b, where b represents the number of divided regions. Then, obtain the saccade trajectory corresponding to the divided region a, and at the same time obtain the saccade speed corresponding to the saccade trajectory. The specific saccade speed is mainly obtained by dividing the saccade distance by the saccade time. And obtain the standard value corresponding to the divided region a, and here the standard value represents the simulation standard corresponding to the simulation environment. At the same time, the standard value is set by the operator. Then, compare the saccade speed of the divided region with the standard value;

[0055] In a specific example, for instance, in a simulated driving environment, there are three turning points in the driver's saccade trajectory when observing the road ahead, the left and right rearview mirrors, and the instrument panel. Thus, the saccade area of the driving vision is divided into four segmented areas, labeled 1, 2, 3, and 4 respectively. For the segmented area labeled 1 (such as the area for observing the road ahead), by calculating the distance and time of the driver's saccades in this area, the saccade speed is obtained as 5 centimeters per second. And according to driving safety and experience, the operator sets the standard value of the normal saccade speed for this area as 4 centimeters per second. Then, the saccade speed of this segmented area can be compared and analyzed with the standard value.

[0056] If the saccade speed of the segmented area is greater than the standard value, the corresponding segmented area is marked as a pre-exceeding area. Conversely, if the saccade speed of the segmented area is less than the standard value, the corresponding segmented area is marked as a slow-labeled area. And the situation where it is equal to the standard value is not included here. And the pre-exceeding area and the slow-labeled area are analyzed and processed.

[0057] Taking the pre-exceeding area as an example for analysis, all the saccade points in the pre-exceeding area are obtained and denoted as o, and o = 1, 2, …, p, where p represents the number of saccade points. And here, one group of pre-exceeding areas is taken as an example for analysis. Then, a corresponding two-dimensional plane coordinate system is established with the center point of the pre-exceeding area as the origin. At the same time, the coordinates of the starting point and the ending point of the saccade point o are obtained and denoted as (x o1 , y o1 ) and (x o2 , y o2 ), and the saccade distance corresponding to the saccade point o is calculated according to the Euclidean distance formula

[0058] And so on, all the saccade distances are calculated. At the same time, the standard quantity within the pre-exceeding area is obtained. And here, the standard quantity represents the specified range within the pre-exceeding area. And according to the standard quantity, the saccade points within the pre-exceeding area are classified into valid saccade points and invalid saccade points. The specific classification method is as follows: The starting point coordinates and the ending point coordinates corresponding to the saccade point o are respectively compared with the standard quantity. If any set of coordinates in the saccade point o does not meet the standard quantity, specifically, it means it is not within the specified range, then the corresponding saccade point is classified as an invalid saccade point. Conversely, if both sets of coordinates of the saccade point o meet the standard quantity, then the corresponding saccade point is classified as a valid saccade point.

[0059] In a specific example, for instance, in a visual experiment area represented by a plane coordinate system, the specified x-coordinate range of the pre-exceeding area is -20 - 60, and the y-coordinate range is -20 - 60. As shown in the appendix Figure 2As shown, the starting point coordinates of one saccade point are (-60, 10), and the ending point coordinates are (-8, -30). Therefore, since the x coordinate of the starting point is not within the standard quantity range, the corresponding saccade point is further classified as an invalid saccade point.

[0060] At the same time, calculate the standard saccade distance corresponding to the pre-exceeding area based on the valid saccade points, and the specific calculation method is , where u is the number of valid saccade points, calculate the ratio of the standard saccade distance dz to the saccade distance in the standard value to obtain the adjustment ratio. Here, the adjustment ratio is obtained by dividing the standard saccade distance dz by the saccade distance. At the same time, adjust the saccade speed of the pre-exceeding area based on the adjustment ratio, and generate the corresponding saccade adjustment information;

[0061] Similarly, perform the same analysis on the slow-mark area to obtain the saccade adjustment information, and transmit the saccade adjustment information to the effect evaluation and analysis module.

[0062] Effect evaluation and analysis module, which is used to perform corresponding simulation effect evaluation according to the obtained saccade adjustment information and attention information, generate evaluation and recognition information at the same time, and transmit the evaluation and recognition information to the recognition information output module. The specific method for generating the evaluation and recognition information is:

[0063] Perform the same simulation operation according to the obtained saccade adjustment information and attention information, and perform evaluation and analysis on the simulation operation. Here, the evaluation method can be to perform operation scoring, compare the simulation operation with the standard evaluation information, and generate evaluation and recognition information at the same time. Specifically, the standard evaluation information here represents the standard operation corresponding to different situations or areas in the simulation environment. Perform evaluation scoring on the simulation operation and the standard operation, and finally generate an overall judgment result.

[0064] Recognition information output module, which is used to transmit the obtained evaluation and recognition information to the corresponding simulator terminal.

[0065] Embodiment 3

[0066] As Embodiment 3 of the present invention, the key lies in combining the implementation processes of Embodiment 1 and Embodiment 2.

[0067] Embodiment 4

[0068] Please refer to Figure 3 , a human-computer interaction method based on eye movement tracking, which specifically includes the following steps:

[0069] In step S101, collect the eye movement data of the simulated person in the simulated environment, and perform subsequent fixation point distribution recognition and eye movement saccade recognition.

[0070] In step S102, the fixation point position and duration are visually displayed according to the eye movement data, and the simulation area is classified to obtain key areas and non-key areas. At the same time, the complexity areas and simplicity areas are classified according to the average duration data of the key areas, and attention information is generated. The processing method here is the same as the analysis method of the fixation point recognition module in Embodiment 1.

[0071] In step S103, the saccade trajectory of the simulated person is analyzed, and the saccade area is divided to obtain segmented areas. At the same time, the pre-over area and slow-mark area are classified based on the saccade speed of the segmented areas. The analysis method here is the same as the processing method of the eye movement saccade recognition module in Embodiment 2.

[0072] In step S104, the saccade distances corresponding to the saccade points in the pre-over area and slow-mark area are calculated respectively, and the saccade points are screened to obtain effective saccade points. At the same time, the adjusted saccade speed of the corresponding area is calculated according to the effective saccade points, and then the saccade speed is adjusted to generate saccade adjustment information. The analysis method here is the same as the processing method of the eye movement saccade recognition module in Embodiment 2.

[0073] In step S105, the simulation effect is evaluated according to the attention information and saccade adjustment information to generate evaluation recognition information, and the evaluation recognition information is transmitted to the corresponding simulator terminal.

[0074] Some of the data in the above formulas are numerically calculated after removing their dimensions, and the content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

[0075] The above embodiments are only used to illustrate the technical method of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A human-computer interaction system based on eye tracking, characterized in that: include: A gaze point distribution recognition module is used to analyze the gaze point distribution of the simulated personnel according to the eye movement data transmitted by the eye movement data acquisition module, obtain key areas and non-key areas, and calculate the average duration of the gaze points corresponding to the key areas, further analyze the different attention levels of the key areas, and generate attention information, and transmit the attention information to the recognition information output module; An eye movement scanning recognition module is used to perform scanning analysis based on the acquired eye movement data, analyze the scanning trajectory of the simulated person, and divide the scanning area to obtain segmented areas, and classify the scanning speeds of the segmented areas to obtain pre-surpassing areas and slow-marking areas, respectively calculate the scanning distances corresponding to the scanning points in the pre-surpassing areas and the slow-marking areas, and screen the scanning points to obtain effective scanning points, and calculate the adjustment scanning speed of the corresponding area based on the effective scanning points, then adjust the scanning speed to generate scanning adjustment information, and transmit the scanning adjustment information to the recognition information output module; An effect evaluation and analysis module is used to perform a corresponding simulation effect evaluation based on the acquired scanning adjustment information and attention information, and generate evaluation identification information at the same time, and transmit the evaluation identification information to the identification information output module; The specific method of generating the attention information by the gaze point distribution recognition module is as follows: The key areas are labeled as i, and i=1, 2, ..., j, where j represents the number of key areas. At the same time, the fixation points in the key areas are obtained, and the duration corresponding to the fixation points is obtained. Then, the average duration of all fixation points in the key areas is calculated and recorded as Ti. Similarly, the average duration of all key areas is calculated. At the same time, the key areas are classified according to the calculated average duration of the key areas, the average duration of all key areas is averaged to obtain the average standard value, and then the average duration of the key areas is compared with the average standard value; Classify the key areas whose average duration is greater than the average standard value as complex areas, and classify the key areas whose average duration is less than the average standard value as simple areas, and generate attention information at the same time, and transmit the attention information to the effect evaluation and analysis module; The specific method of dividing the scanning area into segmented areas by the eye movement scanning recognition module is as follows: Then, the corresponding scanning area is divided according to the scanning trajectory of the simulated person, and the scanning area is divided into segmented areas using the turning point of the scanning trajectory as the dividing point. At the same time, the segmented areas are labeled as a, and a=1, 2, ..., b, where b represents the number of segmented areas; The specific method of the eye movement scanning recognition module to classify the pre-surpassing area and the slow marking area based on the scanning speed of the segmented area is: The scanning trajectory corresponding to the segmented area a is obtained, and the scanning speed corresponding to the scanning trajectory is obtained, and the standard value corresponding to the segmented area a is obtained, and then the scanning speed of the segmented area is compared with the standard value; If the scanning speed of the segmented area is greater than the standard value, the corresponding segmented area is marked as a pre-surpassing area. Conversely, if the scanning speed of the segmented area is less than the standard value, the corresponding segmented area is marked as a slow-marking area, and the pre-surpassing area and the slow-marking area are analyzed and processed; The specific method in which the eye movement scanning recognition module analyzes and processes the pre-surpassing area and the slow marking area is: Get all the scanning points in the pre-surpass area or slow marking area and record them as o, where o=1, 2, ..., p, where p represents the number of scanning points. Then, establish the corresponding two-dimensional plane coordinate system with the center point of the pre-surpass area or slow marking area as the origin. At the same time, get the coordinates of the starting point and the ending point of the scanning point o and record them as and , and according to the Euclidean distance formula Calculate the scanning distance corresponding to the scanning point o; According to the above method, all the scanning point distances are calculated, and the standard quantity in the pre-surpassing area or the slow marking area is obtained at the same time, and the scanning points in the pre-surpassing area or the slow marking area are classified according to the standard quantity to obtain valid scanning points and invalid scanning points, and the starting point coordinates and the ending point coordinates corresponding to the scanning point o are compared with the standard quantity respectively. If any set of coordinates in the scanning point o does not meet the standard quantity, the corresponding scanning point is classified as an invalid scanning point. On the contrary, if both sets of coordinates of the scanning point o meet the standard quantity, the corresponding scanning point is classified as a valid scanning point. The standard scanning distance corresponding to the pre-surpassing area or slow-marking area is calculated according to the effective scanning point, and the specific calculation method is as follows: , where u is the number of valid scanning points, and the standard scanning distance dz is calculated by the ratio of the scanning distance in the standard value to obtain the adjustment ratio. At the same time, the scanning speed of the pre-surpassing area or the slow-marking area is adjusted based on the adjustment ratio, and the corresponding scanning adjustment information is generated.

2. The human-computer interaction system based on eye tracking according to claim 1, characterized in that: It also includes an eye movement data acquisition module and a recognition information output module; An eye movement data collection module is used to collect the eye movement data of the simulated personnel in the simulated environment, and transmit the collected eye movement data to the gaze point distribution recognition module and the eye movement saccade recognition module; An identification information output module is used to transmit the obtained evaluation identification information to the corresponding simulator terminal.

3. The human-computer interaction system based on eye tracking according to claim 1, characterized in that: The specific method of obtaining the key area and the non-key area by the gaze point distribution recognition module is as follows: The eye movement data of the simulated person is obtained, and the corresponding gaze point position and duration of the simulated person are visualized. Cluster analysis is performed based on the colors of different areas and the number of gaze points. The specific cluster analysis method is: the areas with brighter colors and longer stay times are marked as key areas, and the areas with darker colors and shorter stay times are marked as non-key areas.

4. The human-computer interaction system based on eye tracking according to claim 1, characterized in that: The specific method for the effect evaluation analysis module to generate evaluation identification information is as follows: The same simulation operation is performed according to the obtained saccadic adjustment information and attention information, and the simulation operation is evaluated and analyzed at the same time. The simulation operation is compared and analyzed with the standard evaluation information, and evaluation identification information is generated at the same time.

5. A human-computer interaction method based on eye tracking, used to execute the human-computer interaction system according to any one of claims 1 to 4, characterized in that: The method specifically comprises the following steps: S101: collecting eye movement data of a simulated person in a simulated environment, and performing subsequent gaze point distribution recognition and eye movement saccade recognition; S102, visually displaying the fixation point position and duration according to the eye movement data, and classifying the simulation area into key areas and non-key areas, and classifying the key areas into complex areas and simple areas according to the average duration data, and generating attention information; S103, analyzing the scanning trajectory of the simulated person, dividing the scanning area into segmented areas, and classifying the segmented areas based on the scanning speed to obtain pre-overtaking areas and slow-marking areas; S104, respectively calculating the scanning distances corresponding to the scanning points in the pre-surpassing area and the slow-marking area, and screening the scanning points to obtain effective scanning points, and calculating the adjusted scanning speed of the corresponding area according to the effective scanning points, and then adjusting the scanning speed to generate scanning adjustment information; S105 , evaluating the simulation effect according to the attention information and the scanning adjustment information and generating evaluation identification information, and transmitting the evaluation identification information to the corresponding simulator terminal.

Images