Eye movement trajectory data synthesis method for visual field defect crowd

Through field measurement and principal component clustering technology, combined with generative adversarial network, the problem of eye movement trajectory synthesis among people with visual field defects is solved, efficient and accurate eye movement trajectory data generation is achieved, and the acquisition cost is reduced, and it is suitable for the behavior of people with disabilities is suitable for people with disabilities. Research.

CN119068537AActive Publication Date: 2024-12-03DALIAN MARITIME UNIVERSITY
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411144512.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-12-03
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

The prior art lacks eye movement trajectory synthesis methods for people with visual field defects, resulting in high cost and low efficiency of data acquisition, and the inability to accurately simulate eye movement behavior of people with visual field defects.

Method used

The data enhancement module generates synthetic eye movement trajectories based on some real eye movement trajectories, uses field of view measurement and continuous principal component clustering to obtain field of view area data, combine time information and landing area judgment, and trains gaze and eye saccade generation adversarial models to form Accurate eye movement trajectory data.

Benefits of technology

It reduces the complexity and cost of data acquisition, improves the accuracy and stability of eye movement trajectory synthesis, and is suitable for simulating eye movement trajectory data of any type of visual field defect, and has research value in reading, driving, etc.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068537A_ABST
    Figure CN119068537A_ABST
Patent Text Reader

Abstract

The invention provides an eye movement track synthesis method for people with visual field defects, and relates to the field of man-machine interaction and virtual simulation, in particular to eye movement behavior classification, eye movement track synthesis and eye movement data enhancement, in particular to an eye movement track data synthesis method for people with visual field defects. The method is mainly applied to trajectory synthesis and data enhancement of the visual field defect crowd, the eye movement trajectory of the visual field defect crowd can be synthesized, meanwhile, the eye movement data enhancement can greatly reduce the difficulty and complexity of eye movement data collection of the visual field defect crowd, and the time cost and the economic cost are reduced. Eye movement track data of any visual field defect type can be simulated and synthesized, and great convenience is provided for data acquisition of reading, driving and walking behavior research of people with visual field defects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of human-computer interaction and virtual simulation technology, and in particular to a method for synthesizing eye movement trajectory data for people with visual field impairment. Background Art

[0002] There are two main types of visual field loss: one is damage to the concentric visual area caused by retinitis, choroidal loss or glaucoma, also known as tubular vision; the other is hemianopsia, most of which is ipsilateral hemi-hemianopsia. By fixing the prism module near the upper and / or lower part of the residual central visual field, an artificial visual aid is formed in the periphery of the human eye, which can help expand the visual field of people with visual field loss. Compared with people with normal visual field, people with visual field loss have a smaller visible visual field, fewer visible fixation points, and changes in visual behavior. There are obvious differences in their scanning paths and fixation frequencies.

[0003] According to the different areas of visual field loss, visual field loss is mainly divided into three types: central visual field loss, peripheral visual field loss, and partial visual field loss. Through the visual field expansion method, some scenes in the originally invisible area can be visualized in the visible visual field area, which is beneficial to the daily life of people with visual field loss. However, due to the changes in the visual field area, the eye movement behavior trajectory of people with visual field loss is significantly different from that of normal people. At present, most of the eye movement trajectory synthesis methods on the market are aimed at normal people, and there is a lack of methods for synthesizing eye movement trajectories for people with visual field loss. At the same time, related research on the attention mechanism of people with visual field loss requires the collection of a large amount of data from people with visual field loss, which usually takes a lot of time and economic costs, and there is no guarantee that stable and usable eye movement trajectories can be obtained. Summary of the invention

[0004] According to the technical problems mentioned in the above background technology, a method for synthesizing eye movement trajectory data for people with visual field loss is provided. The present invention generates a synthetic eye movement trajectory according to part of the real eye movement trajectory through a data enhancement module.

[0005] The technical means adopted by the present invention are as follows:

[0006] A method for synthesizing eye movement trajectories for people with visual field loss is provided, which is used for synthesizing eye movement trajectories and enhancing data for people with visual field loss, and comprises the following steps:

[0007] Step 1: Setting the visual field area; obtaining the visual field area of ​​the visual field-impaired population through visual field measurement and visual field expansion rules, and completing the visual plane calibration using multiple generated reference calibration points within the visible area;

[0008] Step 2: Data generation: Obtain the gaze data in the visual field, use the continuous principal component clustering operation to divide the gaze data into fixation data and saccade data, combine the time information with the landing area judgment, and generate the original eye movement trajectory data;

[0009] Step 3: Adversarial learning: train the gaze generative adversarial model and the saccade generative adversarial model based on the original eye movement trajectory information, generate gaze data and saccade data through the two models respectively, connect the generated data clusters to form eye movement trajectory data, perform comparative learning and optimization on the data in the actual scene, and delete the eye movement trajectories that do not conform to the field of view area settings.

[0010] Furthermore, the field of view area setting includes the following steps:

[0011] Step 11: Use a perimeter to measure the user's horizontal and vertical visual angles and the expected distortion or cropping area to obtain visual field measurement data;

[0012] Step 12: converting the visual field measurement data into a gray value or gray quantization level in a visual field coordinate system;

[0013] Step 13: Measure the distance between the user and the screen, and use the trigonometric function relationship to calculate the calibration mapping of the human eye's field of view on the screen to obtain the approximate area of ​​the visible area and the invisible area;

[0014] Step 14: Smoothing the mapped area of ​​the visible area through Gaussian filtering;

[0015] Step 15: Use morphological erosion operation to reduce the boundary of the field of view;

[0016] Step 16: After obtaining the visible area, invisible area, and extended area, they can be marked on the screen;

[0017] Step 17: After marking the visual field areas, perform sight calibration; set the reference calibration point in the middle of the visible visual field area. If there are multiple visible visual field areas, select the largest visible visual field area; if the visible visual field area is less than 5% of the total visual field area, use the center point of each visible visual field area as the reference calibration point;

[0018] Step 18: After selecting the reference calibration points, the user is required to gaze at the reference calibration points in turn; at the same time, the coordinates of the reference calibration points and the characteristics of the human eye are recorded during the gaze process, and the gaze mapping between the human eye and the screen is completed through the gaze tracking method;

[0019] Step 19: Repeat step 18 until all angles are calibrated, and during the repetition process, the rectangle direction should be rotated 45° clockwise each time.

[0020] Furthermore, the selection condition of the reference calibration point is: take a rectangle with the largest area in the visible field of view, and the four vertices and the midpoint of the rectangle are the reference calibration points.

[0021] Furthermore, the perimeter measurement data is represented in the form of angle or position information relative to the center of the visual field.

[0022] Furthermore, the data generation comprises the following steps:

[0023] Step 21: After obtaining the gaze data, the eye movement behaviors are classified and clustered; the principal components of the clusters are analyzed to obtain sub-clusters until the bottom-level gaze points are obtained, and all gaze behaviors are obtained;

[0024] Step 22: performing corresponding processing according to the area where the landing point is located; the area where the landing point is located includes: a visible field of view area, an invisible field of view area, an extended field of view area, and an overlapping field of view area;

[0025] For the visible visual field area, eye movement behaviors are divided into fixation behaviors and saccadic behaviors;

[0026] For the invisible field of view area, no processing is performed;

[0027] For the extended visual field area, saccadic behaviors were considered fixation behaviors;

[0028] For overlapping visual field areas, the fixation behavior between the visible visual field area and the extended area was considered as saccade behavior;

[0029] Step 23: obtaining eye movement behaviors; the eye movement behaviors include: fixation behaviors between visible areas, eye saccade behaviors within visible areas, fixation behaviors between the same extended areas, eye saccade behaviors between different extended areas, and eye saccade behaviors between visible areas and extended areas;

[0030] Step 24: After completing the classification of eye movement behaviors, the original motion trajectory is synthesized using the time information recorded by the gaze tracking method.

[0031] Furthermore, the eye movement behavior includes: fixation behavior and saccadic behavior; the fixation behavior is manifested as dense clusters of points, the saccadic behavior is the transition from one cluster of points to another cluster of points, and the movement between different fixation behaviors is saccadic behavior.

[0032] Furthermore, the adversarial learning includes the following steps:

[0033] Step 31: inputting the original trajectory information into a fixation generative adversarial model and a saccade generative adversarial model respectively; the fixation generative adversarial model and the saccade generative adversarial model both include a generator and a discriminator;

[0034] Step 32: The generator creates synthetic data based on the original gaze point data for training the discriminator;

[0035] Step 33: Synthesize gaze trajectory sequences through batch normalization and activation function; each sequence starts from the first gaze position obtained by the simulated gaze behavior generative adversarial network, and each subsequent gaze position will be added to the current sequence. The position between two gaze clusters determines the amplitude of the eye movement behavior;

[0036] Step 34: The discriminator uses the sight trajectory sequence as input to distinguish whether the sight trajectory sequence is a real sequence or a sequence generated by the generator;

[0037] Step 35: Perform comparative learning and optimization and delete eye movement tracks that do not conform to the regional settings. Compared with the prior art, the present invention has the following advantages:

[0038] The present invention uses the principal component analysis method to classify eye movement behaviors and generates adversarial networks to enhance the data of sight tracks. The main steps of the present invention all bring beneficial effects.

[0039] First, the calibration points in the visible area are used to perform multiple sight calibrations on people with visual field loss, achieving the correct mapping of eye movement and visual field area, with overall high accuracy;

[0040] Second, the continuous principal component clustering operation is used to distinguish the gaze data into fixation data and saccade data, and the original eye movement trajectory data is accurately and stably generated by combining the time information with the judgment of the landing area;

[0041] Third, by using two generative adversarial networks and comparative learning, accurate and robust synthetic line of sight trajectories are obtained, which greatly reduces the complexity of data collection and reduces the time and economic costs.

[0042] The present invention can be used to simulate and synthesize eye movement trajectory data of any type of visual field defect, and has positive significance for behavioral research on reading, driving, and reading of disabled people. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 It is the overall flow chart of the present invention.

[0045] Figure 2Schematic diagram of the field of view of the present invention.

[0046] Figure 3 This is a schematic diagram of eye movement behavior clustering in the present invention.

[0047] Figure 4 This is a schematic diagram of eye movement behavior classification according to the present invention.

[0048] Figure 5 Generate an adversarial model structure diagram for the present invention.

[0049] Figure 6 This is a schematic diagram of comparative learning and optimization of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] like Figure 1-6 As shown, the present invention provides an eye movement trajectory synthesis method for people with visual field loss, which is used for eye movement trajectory synthesis and data enhancement of people with visual field loss, and includes the following steps:

[0053] Step 1: Setting the visual field area; obtain the visual field area of ​​the visual field-impaired population through visual field measurement and visual field expansion rules and mark it on the screen. Generate reference calibration points multiple times in the visible area, and the user completes the line of sight calibration by looking at the calibration points.

[0054] The following steps are involved:

[0055] A perimeter is usually used to measure the user's horizontal and vertical visual angles, as well as possible distortion or cropping areas. The visual field measurement data is usually expressed in the form of angles or position information relative to the center of the visual field. After obtaining the visual field measurement data, the data needs to be converted into grayscale values ​​or grayscale quantization levels in the visual field coordinate system to express visual acuity or other related information. Measure the distance from the user to the screen in front, and use the trigonometric function relationship to calculate the calibration mapping of the human eye's visual field on the screen to obtain the approximate area of ​​the visible area and the invisible area. Then use Gaussian filtering to smooth the mapping area to reduce noise and details, ensuring the stability and accuracy of subsequent processing. Finally, use morphological corrosion operations to reduce the boundaries of the visual field in the measurement results. This step helps to accurately define the boundaries of the visible and invisible areas, and the results obtained at this time have a high degree of confidence. The expansion of the visual field usually follows certain rules, from which the visual field expansion area can be obtained. After obtaining the visible area, invisible area, and extended area, they can be marked on the screen.

[0056] After marking the visual field, line of sight calibration is required. The reference calibration point will appear in the middle of the visible visual field. Generally, five calibration points will appear in sequence. The selection criteria for the calibration point is to take the rectangle with the largest area in the visible visual field. The four vertices and the midpoint of the rectangle are the reference calibration points. If there are multiple visible visual fields, select the largest visible visual field to complete the above steps. If the visible visual field is small as a whole (if the visible visual field is less than 5% of the total visual field, use the center point of each visible visual field as the reference calibration point), you can use the center point of a single area as the reference calibration point. After selecting the reference calibration point, the user is asked to look at the calibration point in turn. In this process, the coordinates of the calibration point and the characteristics of the human eye will be recorded, and the line of sight mapping between the human eye and the screen can be completed by the line of sight tracking method. Considering the particularity of people with visual field defects, this step should be repeated 2-3 times until all angles are calibrated. Each time the rectangle direction should be rotated 45° clockwise to reselect the largest rectangle and determine the new reference calibration point.

[0057] Step 2: Data generation: Obtain the sight data in the visual field, use the continuous principal component clustering operation to divide the sight data into fixation data and saccade data, combine the time information and the landing area judgment, and generate the original eye movement trajectory data. It includes the following steps:

[0058] After the gaze calibration is completed, the user is asked to look at the screen. At this time, the camera will collect relevant information. The gaze data on the screen, that is, the gaze position information and gaze time information, can be obtained through the gaze tracking method. After obtaining the gaze data, the eye movement behavior needs to be classified. The eye movement behaviors discussed here are mainly divided into fixation behavior and saccade behavior. Fixation behavior is manifested as dense clusters of points, and saccade behavior is manifested as a span from one cluster of points to another. For a gaze record, since the distribution of the user's gaze points is dense, for example, the coordinate density of the gaze area is higher than that of the ungassed area, we can use the current principal component as a cluster, and then analyze the principal component of the cluster to obtain a sub-cluster until the bottom-level gaze point is obtained. According to this process, all gaze behaviors can be obtained. The movement between different gaze behaviors is the saccade behavior.

[0059] After the eye movement behavior is classified, it is necessary to perform corresponding processing according to the area where the eye movement is located. In addition to the invisible area and the visible area, people with visual field loss also have an extended area generated by extension, and there is a partial overlap between the extended area and the visible area. Therefore, the sight point area can be divided into the following four types: visible area, invisible area, extended area, and overlapping area, among which the data in the invisible area is not considered.

[0060] For the visible visual field, eye movement behavior is divided into fixation behavior and saccade behavior. For the extended visual field, since the extended area is relatively small, the difference between saccade behavior and fixation behavior is not large, so the saccade behavior in the extended area is considered as fixation behavior. If it is the saccade behavior in two different extended areas, it remains unchanged. For the overlapping area, since the original position may not be adjacent, we regard the fixation behavior between the visible visual field and the extended area as saccade behavior. According to the jump position of the landing area, eye movement behavior can be divided into five categories, namely fixation behavior between visible areas, saccade behavior within the visible area, fixation behavior between the same extended area, saccade behavior between different extended areas, and saccade behavior between the visible area and the extended area. After completing the classification of eye movement behavior, the time information recorded by the line of sight tracking method can be synthesized into the original motion trajectory.

[0061] Step 3: Adversarial learning: Use the original eye movement trajectory information to train the gaze generation adversarial model and the eye saccade generation adversarial model, generate gaze data and eye saccade data through the two models respectively, connect the generated data clusters to form eye movement trajectory data, perform comparative learning and optimization on the data in the actual scene, and delete the eye movement trajectories that do not conform to the field of view area settings. It includes the following steps:

[0062] After multiple data collection processes in the second step, the original eye movement trajectory information collected multiple times can be obtained. The original trajectory information is input into two different generative adversarial networks. The gaze generative adversarial model is used to simulate the small movements of gaze behavior, and the saccade generative adversarial model is used to simulate the rapid movements of saccade behavior. Both generative adversarial networks include a generator and a discriminator. The generator is used to create synthetic sight trajectories, and the discriminator is used to distinguish between real data and synthetic data. First, the generator creates some synthetic data based on the original sight point data. This data is used to train the discriminator. The loss of the generated data points will be used to adjust the weight of the generator. The generator and the discriminator compete with each other and optimize their respective parameters through gradient descent, so that the samples generated by the generator are more and more realistic, and the discriminator is becoming more and more accurate.

[0063] The generator projects the noise vector to a higher dimension through a fully connected layer, and then synthesizes the gaze trajectory sequence through batch normalization and activation function, which is then rearranged to meet the required dwell time of different eye movement behaviors. After the reshaping layer, there are three deconvolution blocks to restore the original output and avoid undesirable boundary effects or information loss, so as to obtain more accurate sequence data.

[0064] The discriminator uses the line of sight sequence as input and can distinguish whether the sequence is a real sequence or a sequence generated by the generator. The discriminator consists of three convolution blocks, normalization, and activation functions. The input of the last convolution block is fully expanded and input into the fully connected layer to distinguish between real and generated sequences through the activation function.

[0065] In the step of synthesizing gaze sequences, each sequence starts from the first gaze position obtained by the generative adversarial network that simulates gaze behavior. Each subsequent gaze position will be added to the current sequence. The position between the two gaze clusters determines the amplitude of the eye movement behavior. At the same time, the eye movement behavior has a certain directionality. When adding a sequence, the current sequence needs to be rotated appropriately to achieve correct trajectory synthesis. Repeating the above behavior can eventually synthesize the gaze trajectory.

[0066] For the synthesized eye tracking, contrastive learning can be performed to tune and delete the eye tracking that does not conform to the regional settings in scenes such as reading and driving. Contrastive learning technology uses a self-supervised learning method that does not require labeled data, and the goal is to distinguish similar sequences from different sequences. The contrastive learning process can optimize the synthesized eye tracking, and the synthesized eye tracking after contrastive learning has high accuracy and robustness.

[0067] The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above-mentioned embodiments of the present invention, the descriptions of each embodiment have their own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0068] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0069] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for synthesizing eye movement trajectories for people with visual field loss, used for synthesizing eye movement trajectories and enhancing data for people with visual field loss, characterized in that: The following steps are involved: Step 1: Field of view setting; The visual field area of ​​the visual field-impaired population is obtained through visual field measurement and visual field expansion rules, and the visual plane calibration is completed using multiple generated reference calibration points in the visible area; Step 2: Data generation: Obtain the gaze data in the visual field, use the continuous principal component clustering operation to divide the gaze data into fixation data and saccade data, combine the time information with the landing area judgment, and generate the original eye movement trajectory data; Step 3: Adversarial learning; The fixation generative adversarial model and the saccade generative adversarial model are trained according to the original eye movement trajectory information. The two models generate fixation data and saccade data respectively. The generated data clusters are connected to form eye movement trajectory data. The data is compared and learned and optimized in actual scenarios, and the eye movement trajectories that do not conform to the field of view area settings are deleted.

2. The eye movement trajectory synthesis method for people with visual field loss according to claim 1, characterized in that: The field of view area setting comprises the following steps: Step 11: Use a perimeter to measure the user's horizontal and vertical visual angles and the expected distortion or cropping area to obtain visual field measurement data; Step 12: converting the visual field measurement data into a gray value or gray quantization level in a visual field coordinate system; Step 13: Measure the distance between the user and the screen, and use the trigonometric function relationship to calculate the calibration mapping of the human eye's field of view on the screen to obtain the approximate area of ​​the visible area and the invisible area; Step 14: Smoothing the mapped area of ​​the visible area through Gaussian filtering; Step 15: Use morphological erosion operation to reduce the boundary of the field of view; Step 16: After obtaining the visible area, invisible area, and extended area, they can be marked on the screen; Step 17: After marking the visual field areas, perform sight calibration; set the reference calibration point in the middle of the visible visual field area. If there are multiple visible visual field areas, select the largest visible visual field area; if the visible visual field area is less than 5% of the total visual field area, use the center point of each visible visual field area as the reference calibration point; Step 18: After selecting the reference calibration points, the user is required to gaze at the reference calibration points in turn; at the same time, the coordinates of the reference calibration points and the characteristics of the human eye are recorded during the gaze process, and the gaze mapping between the human eye and the screen is completed through the gaze tracking method; Step 19: Repeat step 18 until all angles are calibrated, and during the repetition process, the rectangle direction should be rotated 45° clockwise each time.

3. The eye movement trajectory synthesis method for people with visual field loss according to claim 2, characterized in that: The selection condition of the reference calibration point is: take a rectangle with the largest area in the visible field of view, and the four vertices and the midpoint of the rectangle are the reference calibration points.

4. The eye movement trajectory synthesis method for people with visual field loss according to claim 1, characterized in that: The perimeter measurement data is expressed in the form of angle or position information relative to the center of the visual field.

5. The eye movement trajectory synthesis method for people with visual field loss according to claim 1, characterized in that: The data generation comprises the following steps: Step 21: After obtaining the gaze data, the eye movement behaviors are classified and clustered; the principal components of the clusters are analyzed to obtain sub-clusters until the bottom-level gaze points are obtained, and all gaze behaviors are obtained; Step 22: performing corresponding processing according to the area where the landing point is located; the area where the landing point is located includes: a visible field of view area, an invisible field of view area, an extended field of view area, and an overlapping field of view area; For the visible visual field area, eye movement behaviors are divided into fixation behaviors and saccadic behaviors; For the invisible field of view area, no processing is performed; For the extended visual field area, saccadic behaviors were considered fixation behaviors; For overlapping visual field areas, the fixation behavior between the visible visual field area and the extended area was considered as saccade behavior; Step 23: obtaining eye movement behaviors; the eye movement behaviors include: fixation behaviors between visible areas, eye saccade behaviors within visible areas, fixation behaviors between the same extended areas, eye saccade behaviors between different extended areas, and eye saccade behaviors between visible areas and extended areas; Step 24: After completing the classification of eye movement behaviors, the original motion trajectory is synthesized using the time information recorded by the gaze tracking method.

6. The eye movement trajectory synthesis method for people with visual field loss according to claim 5, characterized in that: The eye movement behavior includes: fixation behavior and saccadic behavior; the fixation behavior is manifested as dense clusters of points, the saccadic behavior is the transition from one cluster of points to another cluster of points, and the movement between different fixation behaviors is saccadic behavior.

7. The eye movement trajectory synthesis method for people with visual field loss according to claim 1, characterized in that: The adversarial learning comprises the following steps: Step 31: inputting the original trajectory information into a fixation generative adversarial model and a saccade generative adversarial model respectively; the fixation generative adversarial model and the saccade generative adversarial model both include a generator and a discriminator; Step 32: The generator creates synthetic data based on the original gaze point data for training the discriminator; Step 33: Synthesize gaze trajectory sequences through batch normalization and activation function; each sequence starts from the first gaze position obtained by the simulated gaze behavior generative adversarial network, and each subsequent gaze position will be added to the current sequence. The position between two gaze clusters determines the amplitude of the eye movement behavior; Step 34: The discriminator uses the sight trajectory sequence as input to distinguish whether the sight trajectory sequence is a real sequence or a sequence generated by the generator; Step 35: Perform contrastive learning tuning and delete eye tracking tracks that do not conform to the regional settings.

Citation Information

Patent Citations

  • Eye movement interaction method, system and device based on eye movement tracking technology

    CN111949131A

  • Telemetering type eye movement behavior classification method and device for visual field defect crowd

    CN116665287A

  • Method and apparatus for generating group eye movement trajectory, computing device, and storage medium

    US20240118747A1

  • Eye movement tracking apparatus and method

    WO2024114258A1