Methods, systems, and computer program products for mapping field of view

Abstract

To measure viewing quality within the eye's field of vision, during a measurement period, the deviation between the associated stimulus position at the gaze position and the stimulus position where the stimulus to be followed is displayed, and the magnitude of the recorded deviation, are determined. For each field portion of the view of the field of vision, viewing quality is determined based on a viewing quality assessment of the associated deviation among the recorded deviations, wherein the associated stimulus position is positioned relative to the gaze position such that the associated stimulus position is located within that field portion. For each of the associated deviations among the recorded deviations, the viewing quality is assessed based on the magnitude of that associated deviation and the magnitude of at least one preceding or subsequent deviation among the recorded deviations.

Description

[0001] Technical Field and Background Technology

[0002] This invention relates to methods, systems, and computer program products for mapping a person's visual field. Visual field mapping allows for the determination of a map showing the viewing quality within a person's visual field, displaying the optical axis around one eye or around the optical axes of both eyes. The resulting map of viewing quality within the field indicates the presence of visual field defects in portions of the visual field and, if present, the location within the portion of the visual field affected by the defects. Conventionally, visual field mapping involves the presentation of stimuli at several locations within the (possible) visual field of the eye and recording whether the stimuli are seen. Locations where stimuli are not actually seen in the visual field map receive lower visual acuity scores.

[0003] One problem with traditional visual field mapping methods is that they are either time-consuming (Standard Automated Visual Field Inspection - SAP) or insensitive to small defects (Frequency Doubling Technique - FDT), and cannot be performed by people who cannot concentrate, maintain a fixed gaze position during the test, and follow instructions on how to respond to the stimuli they have seen.

[0004] U.S. Patent Application 2006 / 0114414 discloses: measuring the visual field of the eye by presenting stimuli at different positions relative to the current viewing direction, measuring the response time of eye saccades in response to stimuli presented in a direction slightly deviating from the current viewing direction, and calculating visual sensitivity in the direction in which the stimulus has been presented based on the measured response time.

[0005] International patent application WO2012 / 141576 discloses a method for determining visual field defects by presenting a series of visual stimuli in the eye's visual field, capturing eye movements in response to the stimuli, and determining the saccade response time to new stimuli. If a change in gaze corresponds to the location of the presented stimulus, the stimulus is recorded as having been seen. By assessing the saccade response time, visual defects can be accurately determined.

[0006] U.S. Patent Application 2012 / 0022395 discloses an analysis of oculomotor abnormalities in human or animal subjects by presenting a stimulus, capturing eye movements in response to the stimulus, determining the value of at least one parameter of eye saccades, and using an artificial intelligence module to identify abnormalities based on the determined value and a predetermined value of the parameter.

[0007] US Patent 9,730,582 discloses: evaluating oculomotor nerve response behavior by measuring characteristics of tracking behavior, including tracking initiation and the speed and accuracy of tracking, as well as the characteristics of the clover-shaped anisotropy of the directional gain of the tracking response, particularly for the assessment of visual processing degradation under the influence of diseases affecting visual sensitivity and resolution, such as retinal diseases and glaucoma. Summary of the Invention

[0008] The purpose of this invention is to provide a simple and accurate solution that allows for mapping the viewing quality of one or more eyes with respect to the field of vision.

[0009] According to the present invention, this objective is achieved by providing the method according to claim 1. The invention can also be embodied in the system according to claim 14 and the computer program product according to claim 15.

[0010] Since it is sufficient for the person measuring the eye to follow the stimulus as it moves, and since viewing quality is determined for each field segment based on a viewing quality assessment of the associated deviation in the recorded deviations, wherein the associated stimulus position is positioned relative to the gaze position such that the associated stimulus position is within that field segment, and viewing quality is assessed for each of the associated deviations in the recorded deviations based on the magnitude of the associated deviation and the magnitude of at least the preceding or subsequent deviation in the recorded deviations, measurements can be performed quickly and do not require accurate gazing at the stimulus before another stimulus is displayed in the visual field location where vision will be determined. In particular, this method is less sensitive to interference caused by inaccurate gaze direction and attention levels, as well as the reaction time of the person measuring the eye.

[0011] The quality of viewing a location in the visual field is determined by the deviation and the magnitude of the deviation before or after it occurs. When the stimulus is located at that location in the visual field, the eye is more strongly stimulated and the gaze position is shifted toward the stimulus. The farther the stimulus is from the gaze position, the stronger the deviation for a given duration is, and the stronger the indication of a visual defect is, the more pronounced the occurrence of a larger deviation for a given duration compared to a smaller deviation for the same duration.

[0012] The viewing quality of a location within the visual field is determined by the duration of a deviation sequence that includes the deviation and begins and ends with a deviation equal to or less than a minimum threshold value. This is because the duration of a deviation cluster, which is part of the deviation at a given location, indicates a slow rate of convergence from the gaze location to the current location of the stimulus. This slow rate of convergence to a given location is an indication of visual impairment at that location in the visual field.

[0013] To obtain various biases and useful indications of the type of visual impairment, the stimulus to be followed is preferably moved at different speeds. Speed ​​variations preferably include gradual increases and decreases in speed (continuous curves of acceleration and deceleration) to induce a uniform distribution of bias magnitudes. Speed ​​variations may also include speed jumps to induce relatively large bias magnitudes. Furthermore, if the stimulus position is moved continuously during the measurement period, artifacts caused by inaccurate static fixation of the stimulus to be followed are particularly effectively avoided.

[0014] To avoid interference with measurement results due to flickering or missing measurements, the gaze position measured when the gaze position moves at a speed exceeding a threshold (e.g., exceeding 300 degrees / second), stops, and then moves again at a speed exceeding the threshold is replaced with an interpolated gaze position. To effectively eliminate this interference, the position is replaced with the position within a predetermined time range (e.g., 1 / 60 to 1 / 15 seconds) or sample number (e.g., 2 to 10 samples) before and after the gaze position measured when the gaze position moves at a speed exceeding the threshold, stops, and then moves again at a speed exceeding the threshold.

[0015] To make the measurements according to the invention particularly accurate, preferably, only one moving stimulus to be followed is displayed, and more specifically, only one moving stimulus or simply only one moving stimulus is displayed.

[0016] According to a preferred embodiment, the method includes the use of a recurrent neural network, for example, a recurrent neural network trained by means of a gaze position obtained by measuring the gaze position of a healthy eye following a displayed stimulus to be followed.

[0017] According to the implementation method, an indication for the type of visual dysfunction in the entire visual field is determined using a neural network from recorded gaze positions and stimulus positions. For example, visual viewing deficits can be simulated by suppressing the display of the stimulus to be followed in a predetermined portion of the field.

[0018] A recurrent neural network trained by measuring the gaze position of a healthy eye following a displayed stimulus can be used to determine, in particular accurately, indications for types of visual dysfunction (such as oculomotor delay, nystagmus, saccades, or hypersaccades) from the recorded gaze position and stimulus position, providing signs pointing to examinations that may aid in diagnosis. The visual viewing deficit is simulated by inhibiting the display of the stimulus to be followed in a predetermined field portion.

[0019] In embodiments of the invention, if the data input to the recurrent neural network during training and during use of the trained recurrent neural network includes at least one of the following categories of data, then indications for a type of visual dysfunction that provides signs of examinations that may aid in diagnosis can be identified particularly reliably:

[0020] —The maximum correlation between gaze position velocity and stimulus position velocity;

[0021] —The time shift (lag) between the gaze position and the stimulus position;

[0022] —Temporal precision of gaze position relative to stimulus position;

[0023] —The variance explained by a Gaussian model fitted to the correlation plot of gaze position velocity and stimulus position velocity relative to time delay;

[0024] —The number of times the most likely deviation between the gaze position and the stimulus position occurs;

[0025] —The average spatial offset (deviation) between the gaze position and the stimulus position;

[0026] —The average deviation between the gaze position and the stimulus position; and

[0027] —The variance of the gaze position versus stimulus position, explained by a Gaussian model that fits a curve to the occurrence rate of deviations across multiple ranges.

[0028] In a particularly accurate implementation, the viewing quality for each associated deviation in the recorded deviations is determined by integrating a series of recorded deviations, which:

[0029] Start with the first deviation from the series of records that have a value equal to or lower than the minimum threshold value h0;

[0030] Including the first series of recorded deviations, having the magnitude h of the associated deviation that increases to the recorded deviations;

[0031] Including the associated deviations in the recorded deviations; and

[0032] This includes the second series of recorded deviations, the magnitude of the final deviation in this series of recorded deviations, which decreases from the magnitude h of the associated deviation in the recorded deviations to a magnitude equal to or lower than the minimum threshold magnitude h0; and

[0033] A continuous series of deviations forming a record, except for the start deviation and the end deviation of the series of deviations, has a value greater than a minimum threshold value h0.

[0034] In order to properly weight the series of deviations for calculating the integral, before integrating the deviations of the series of records, the magnitudes of the deviations of the series of records greater than h are reduced to a capped value according to the magnitude h, which is preferably equal to h.

[0035] For example, the integral at each time point from the deviation of the threshold value h0 to the value h until the deviation returns to the threshold value h0 and is capped by the value h can be achieved through cluster analysis such as using a thresholdless clustering enhancement algorithm.

[0036] In an alternative implementation, the particularly accurate determination of the viewing quality for each field segment is achieved using a recurrent neural network trained to obtain a recurrent neural network, which has:

[0037] The training input includes: a series of training time points, each comprising the location of the stimulus to be followed and the gaze position of a healthy eye following the stimulus to be followed; and an indication, for each training time point, whether the stimulus location relative to the gaze position is within the field portion where the display of the stimulus to be followed is suppressed.

[0038] The training output includes a graph showing the portion of the field that was suppressed during the measurement phase used to obtain the training input, representing the stimulus to be followed.

[0039] In operation, the method according to this embodiment preferably includes:

[0040] Input a series of time points, each including the location of the stimulus to be followed and the gaze position of the eye to be examined that is following the stimulus to be followed.

[0041] The trained recurrent neural network preferably classifies the visual quality of the series of time points into scotopic time points where visual acuity is classified as functional and non-scotopic time points where visual acuity is classified as dysfunctional.

[0042] For each of the field segments, determine which time points have deviations such that the stimulus position relative to the gaze position is within that field segment; and

[0043] Based on the visual quality assessment at time points where the location of the stimulus is determined in the field segment, a visual field map indicating the overall visual quality value is generated for each of the field segments.

[0044] If a recurrent neural network includes fully connected layers, where all output units in the fully connected layers are connected to all input units and vice versa, and at least one gated recurrent unit that processes sequential information in a recurrent manner with the ability to capture time-dependent long short-term memory, then the recurrent neural network takes into account, particularly effectively, the duration of the various deviations in the recorded stimulus location and the associated gaze location.

[0045] To evaluate the visual field map with particular accuracy, it is advantageous if the method also includes the following steps:

[0046] Capture and input the brightness of the stimulus and the type of tracking data during the measurement cycle;

[0047] At least two fully connected layers process the luminance and tracking data into categorical data of scotomas (e.g., binasal hemianopsia, bitemporal hemianopsia, blind spot, cortical diffusion inhibition, or scintillating scotomas).

[0048] The combination of time-point and category data from the gated recurrent unit is input into the softmax classifier of the recurrent neural network; and

[0049] For each time point, the softmax classifier predicts whether the location of the stimulus at that time point is on the dark spot in the visual field.

[0050] Further features, effects and details of the invention will become apparent from the detailed description and accompanying drawings.

[0051] Brief description of the attached figures

[0052] Figure 1 This is a flowchart of the steps involved in acquiring data about eye movements during testing;

[0053] Figure 2 This is a schematic diagram of a system of the present invention having an example of displaying visually tracking stimuli;

[0054] Figure 3 It shows the horizontal position of the stimulus over time. h A graph of (t);

[0055] Figure 4 It is a flowchart of the steps for extracting spatiotemporal features and classifying visual defects based on these features;

[0056] Figure 5A and Figure 5B This is a graph showing the viewing direction over time before and after filtering out lost data and assumed shifts associated with flicker;

[0057] Figure 6A It is a graph showing the speed of change in viewing direction over time;

[0058] Figure 6B It is a graph showing the cross-correlation between the time delay and the stimulus velocity and eye velocity after the time delay (cross-correlation graph);

[0059] Figure 7A It is a graph showing the recorded stimulus and gaze position over time in the x-direction;

[0060] Figure 7B It is a graph showing the probability density distribution of the positional deviation between the gaze position and the stimulus position.

[0061] Figure 8 This is a flowchart of the view distance of the architecture of a deep recurrent neural network used to determine category (visual defect type) classifiers and time point classifiers;

[0062] Figure 9 This is a flowchart of the steps used to determine the visual field of the eye from data on eye movements during testing;

[0063] Figure 10A This is an example of a view without visual defects;

[0064] Figure 10B This is achieved by processing healthy eyes that never suppress stimulus display at predetermined locations through thresholdless clustering enhancement (TFCE). Figure 10A The diagram shown is a view obtained from measurement data acquired during the test (to which the diagram is applied);

[0065] Figure 10C It uses a recurrent neural network (RNN) to process a healthy eye that does not suppress the display of stimuli at a predetermined location (i.e., Figure 10A The diagram shown is a view obtained from measurement data acquired during performance testing (the data used in the test).

[0066] Figure 10D It is a diagram of simulated scotoma locations (peripheral loss visual defect pattern types) formed by the locations where the display of the stimulus is suppressed when testing the performance of a healthy eye;

[0067] Figure 10E Processing from TCFE Figure 10D The visual field map obtained from measurement data acquired during a performance test of a healthy eye that shows the inhibition of stimuli at predetermined locations;

[0068] Figure 10F Processing from RNN Figure 10D Visual defect map obtained from measurement data acquired during a performance test of a healthy eye that inhibits the display of stimuli at predetermined locations, as shown in the figure.

[0069] Figure 10G It is a diagram of simulated dark spot locations (half-field loss visual defect pattern type) formed by the location where the display of the stimulus is suppressed when testing the performance of a healthy eye;

[0070] Figure 10H Processing from TCFE Figure 10G The visual field map obtained from measurement data acquired during a performance test of a healthy eye that shows the inhibition of stimuli at predetermined locations;

[0071] Figure 10I Processing from RNN Figure 10G The visual defect map is obtained from measurement data acquired during a performance test of a healthy eye that shows the inhibition of stimuli at predetermined locations. Detailed Implementation

[0072] The invention will be further described with reference to examples of methods according to the invention and examples of tests for checking the effectiveness of methods according to the invention.

[0073] The following hardware can be used for the method according to the invention and for testing the effectiveness of the method according to the invention:

[0074] Display screen 52 (see Figure 2 For example, LCD monitors; and

[0075] Eye tracker 53 is used to track the gaze position of the eyes on a display in the direction of viewing, such as a monitor-integrated eye tracker: Eyelink 1000 (SR-Research, Ottawa, Canada); and

[0076] The data processor system 54 is connected to the display screen 52 to control the display screen 52 to display the visual stimulus 1 moving on the display screen 52 at the stimulus location (in this example, the center of the stimulus spot), and is connected to the eye tracker 53 to receive data representing the gaze location from the eye tracker.

[0077] The data processor system 54 is programmed to perform a method according to the example described below. In this example, the data from the eye tracker 53 is acquired at a sampling rate of 1000 Hz and downsampled to match the 240 Hz refresh rate of the display screen 52 in time.

[0078] In this example, the visual stimulus to be followed is against a uniform gray background 2 (~140 cd / m²). 2 The form of the brightness of the Gaussian spot 1 moving on the surface (see...) Figure 2The motion has components in the vertical Y direction and the horizontal X direction. Gaussian spot 1 can be displayed within the following contrast levels: Gaussian spot 1 has ~385 cd / m² when displayed at maximum contrast (50%). 2 Peak brightness, while exhibiting ~160 cd / m² at minimum contrast (5%). 2 The peak brightness is [value missing]. The size (full width at half maximum) of Gaussian spot 1 is 0.83 visual field diopters, corresponding to size III of the stimulus (a commonly used perimeter device) in the Galtonman perimeter. The person being tested for visual acuity is instructed to follow stimulus 1, and the tester's eye gazes. In addition to stimulus 1 to be followed, other visual stimuli may be displayed, which can be moving and / or stationary. In this example, no other stimuli are displayed.

[0079] In step 3 ( Figure 1 In the process, create stimulus trajectories that consist of random paths with the following constraints:

[0080] Stimulus trajectory 4 must be within the boundaries of the screen.

[0081] The stimulus trajectory cannot contain periodic autocorrelation.

[0082] The stimulus trajectory 4 in this example is constructed by generating a velocity vector:

[0083]

[0084] At each time point, the level (v) x ) and vertical (v) y The velocity values ​​of the components are derived from a Gaussian distribution with a mean of 0 and a level (σ). h ) and vertical (σ) v The standard deviation is derived from the respective standard deviations in each direction. For example, for a standard screen resolution of 1920×1080 pixels, the horizontal standard deviation could be σ. h = 64.45 degrees / second, and the vertical standard deviation can be σv = 32.33 degrees / second. Preferably, these σ values ​​can be adjusted based on screen size and / or specific application, for example, to obtain a representation of the visual field under specific conditions (e.g., based on measurements taken from individuals with known ailments or diseases).

[0085] The velocity vector can be low-pass filtered using a convolution with a Gaussian kernel to achieve a cutoff frequency, preferably around 10 Hz. Subsequently, through temporal integration, the velocity is converted into the location of stimulus 1.

[0086]

[0087] To induce saccades in a person being tested for visual acuity, rapid shifts in random directions can be included in the trajectory of the movement, provided that the displayed stimulus to be followed does not fall outside the screen boundaries. This is achieved, for example, by adding these shifts after intervals of a fixed number of seconds (preferably two seconds) or after random time intervals. Figure 3 The figure shows the horizontal component s of the position of the stimulus trajectory. x Curves changing over time Figure 4 Example of '. Figure 1 As shown, the created trajectory 4 is stored in memory 5. In this example, all data is stored in the same memory 5, but the dataset can also be stored in several data memories.

[0088] To acquire data, the person being tested for vision is positioned in front of the screen, for example, at a viewing distance of 60cm. Note that other display technologies, such as holography, can also be used instead of a screen. In step 6 ( Figure 1 In this process, stimulus 1 is displayed on screen 52 and moves (movement in the x-direction corresponds to stimulus trajectory 4 read from memory 5, and movement in the y-direction corresponds to another stimulus trajectory), preferably with a rapid shift in the y-direction simultaneously with the rapid shift in the x-direction. Preferably, head movement is minimized, for example, by resting the head on a headrest or filtering out the head. Before each data acquisition stage, eye tracker 53 is preferably calibrated using a standard procedure such as nine-point calibration. Eye tracking step 13 preferably includes multiple trials. In this example, each trial lasts 20 seconds and is repeated 6 times. For each time point, the acquisition generates horizontal and vertical gaze coordinates (in pixels). The pixel values ​​are converted to degrees of visual angle to obtain a time series of gaze positions 7. (see Figure 5A These data are also stored in memory 5. In this example, measurements are taken from one eye. However, for example, if an indication of visual ability to perform a specific task is desired, measurements can also be taken from both eyes simultaneously.

[0089] For preprocessing, the time series of gaze position 7 is read from memory 5. In preprocessing step 15 of this example, the first time segment of each time series (e.g., the first 250ms) is discarded to avoid artifacts from the person being tested for vision during the safe period.

[0090] Other artifacts that should be avoided are those caused by flickering and missing data. For example, flickering cycles can be identified by the rapid back-and-forth shifts of peaks 8-12 in curve 7 that form the gaze position 7. The absolute value of the first derivative is higher than a predefined threshold for two short periods, and (Short periods where the absolute value of the first derivative is 0). The time window for each flicker period and the number of samples on either side of each flicker period (e.g., 5) are preferably replaced by data determined using an autoregressive model that uses the preceding numerical (e.g., 10) samples and (e.g.) 10 samples after the numerical value of the sample to be replaced. Periods for which no data was obtained can be extended and replaced in the same manner as flicker periods. If the total data loss in the experiment (caused by flicker or other reasons) is very large (e.g., more than 25%), it is preferable to remove the entire experiment from further analysis.

[0091] Therefore, in this example, filtered data 14 is obtained (see Figure 1 The curve in Figure 7' (see) Figure 5B It is also stored in memory 5.

[0092] For the horizontal and vertical components, respectively, calculations are performed. and The distance between them (in this example, the Euclidean distance) is used to obtain the time series of location errors. Figure 7A An example is shown of the horizontal position deviation 58 between the eye position curve 57 (flicker filter) and the stimulus position curve 54 (filtered stimulus position). Subsequently, probability density distribution 36 ( Figure 4 Stage 36 and Figure 7B The calculation is performed on the overall position error time series, for example, by calculating a histogram of a column using a field of view of 1 degree, which spans from -20 degrees to +20 degrees.

[0093] In step 17, by taking the first time derivative, the gaze position 7' ​​of the filtered data 14 is determined. and location of irritants Time series transformation into corresponding velocity and This results in a time series of velocity. Figure 6A The figure shows a time series of the velocity of the eye at the gaze position in the x-direction. Example of a curve in Figure 18. This is a time series correlation between the velocity of the stimulus location in the x-direction. The result is shown as a curve in graph 59.

[0094] The velocity of determining the stimulus position for the horizontal and vertical components respectively. and the speed of the gaze position Normalized time-shift cross-correlation between filtered time series 29( Figure 4 Phase 29 and Figure 6BFor example, the time shift can range from -1s to +1s, with a step size of 1 IFI (inter-frame interval). In this example, each 20-second data acquisition results in a set of cross-correlation pairs. In this example, the six sets of cross-correlation pairs are averaged in step 31. Figure 6B An example of the obtained velocity cross-correlation plot (CCG)29 is shown. (Example in...) Figure 7B The figure shown is histogram 36 ( Figure 4 and Figure 7B The positional deviation density distribution (PDD) is based on the stimulus position for the horizontal and vertical components, respectively. and gaze position The velocity is determined by filtering the time series. In step 32, correlation plot 29 and histogram 36 are fitted with Gaussian models for their respective horizontal and vertical components to obtain Gaussian fits 29' and 36'. In step 33, the following parameters are extracted from each Gaussian fit 29' and 36': amplitude, μ, σ, and R. 2 :

[0095] CCG

[0096] Amplitude: and The maximum value of the correlation between them;

[0097] μ: Time shift (hysteresis) between the eye and the stimulus;

[0098] σ: The temporal precision of tracking performance (i.e., how much time the observer needs to assess the target's position);

[0099] R 2 : The variance of the interpretation of the time-based Gaussian model.

[0100] PDD

[0101] Amplitude: The probability of occurrence of the most likely positional deviation;

[0102] μ: Spatial offset (deviation) between the eye and the stimulus;

[0103] σ: Spatial accuracy of tracking performance;

[0104] R 2 : Variance of the interpretation of the spatial Gaussian model.

[0105] Eight listed spatiotemporal features relating the stimulus positions to the associated gaze positions during the measurement phase are obtained in both the horizontal and vertical directions, resulting in a total of 16 spatiotemporal features, which are then stored in memory 5. In step 45 ( Figure 4 In this process, these spatiotemporal features are input into the category classifier 45 and processed to determine the overall field of view classification 35.

[0106] To determine the quality map of the visual field of an eye whose eye movements have been measured, spatiotemporal integration of the positional deviation between the stimulus position and the gaze position can be used. In individuals with reduced oculomotor function (e.g., decreased visual acuity in a portion of the eye's visual field), the gaze position will deviate from the target stimulus position more significantly and for a longer period of time than in individuals with healthy oculomotor function. Positional deviation as a function of time46 Figure 9 The size h(t) can be defined as:

[0107]

[0108] Spatiotemporal integration was performed using a threshold-free cluster enhancement algorithm (TFCE).47 This algorithm is described in: Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localization in cluster inference; Smith, SM & Nichols, TE; Neuroimage 44, 83-98 (2009). In the current implementation of this algorithm, for each recorded combination of stimulus location and gaze location, the TFCE score is given by the sum of the magnitudes of all “supporting parts” of the cluster formed by the deviation curves below it; as the magnitude h increases incrementally from h0 to the magnitude ht at a given time point t. t Starting at time t, the time process is thresholded at h, and a single continuous cluster containing t ends when the magnitude returns to h0. The surface region under the deviation curve defines the spatiotemporal integral fraction of this magnitude h. This fraction is the magnitude h (raised to a certain power H) of the cluster range over time e (raised to a certain power E) using the following formula:

[0109]

[0110] The integral is implemented using a finite step size as a discrete sum dh (e.g., dh = 1 / 2500 of the maximum value of h); h0 is typically the minimum value of h, and E and H can be set a priori (optimized using simulated field-of-view effects). The result is a time series of positional deviations of 48, each weighted according to its spatiotemporal composite characteristics: D STI Therefore, for each positional deviation, its value D STI The value D depends on the magnitude of the deviation and the time taken until the deviation curve, as part of it, has returned to the magnitude h0. Assign a value D to the position deviation of 48. STIIt can also be defined as forming the integral at each time point from the beginning of the deviation to the value h until the deviation has returned to the value h0 and is capped by the value h.

[0111] To determine the visual field quality of the eye that has been measured (Figure 51), the time series D is input. STI Each occurrence of is associated with its horizontal and vertical components, D'x = px(t) - sx(t) and D'y = py(t) - sy(t), respectively, which form the x and y components of the position of the stimulus relative to the center of the visual field. For each time point, it includes an assessment of the viewing quality at that position relative to the center of the visual field.

[0112] In step 50, these components are plotted onto a Cartesian plane, the origin of which represents the gaze center of the eye. Thus, the Cartesian plane forms a representation of the visual field, with its center representing the fovea.

[0113] In step 50, spatial columns (e.g., columns with a dimension equal to one field of view) can be applied. Then, the value at each column is D within that column. STI The average value.

[0114] Finally, the calculated graph 51 can be stored in memory 5 and displayed, for example, in a 50×40 grid, including ±25 visual diopters. This graph indicates the severity of visual loss (e.g., color-coded or grayscale) and can be easily interpreted by an ophthalmologist.

[0115] exist Figure 10B , Figure 10E and Figure 10H An example of a field-of-view diagram is shown, derived from an experimental implementation of the described spatiotemporal integral using the TFCE method.

[0116] As an alternative to the spatiotemporal integration method, the viewing quality map of the eye's field of vision can be evaluated from the measurement results using a trained recurrent neural network. Recurrent neural networks can be trained to obtain artificial intelligence classifiers 34 and 45. Figure 8 In addition to the brightness and tracking type of the stimulus, gaze location is used. and location of irritants The time series is used as the training input x. A known map of visual field defects, as simulated during test data acquisition, is used as the training output y, and for each time point, it is determined whether the stimulus is located at the simulated dark spot location. The simulated dark spot location is a predetermined surface portion of the visual field, and the dark spot is simulated by suppressing the display of the stimulus to be followed in these predetermined surface portions of the visual field. The pattern of the surface portions of the visual field that suppress the display of the stimulus to be followed forms a virtual mask, which masks the surface portion of the visual field to be simulated by suppressing the display of the visual stimulus 1 to be followed when it is located in these surface portions. This virtual mask moves relative to the display area as the gaze position moves.

[0117] The deep recurrent neural network consists of two streams, 37 and 38, that initially process a series of time points 39 (see also...). Figure 8 It consists of ) each including a gaze position and location of irritants The x and y coordinates 39, as well as the brightness and tracking type of the stimulus 40, are used to process the time-series data. Specifically, the time-point classification stream 37, which processes the time-series data, consists of two fully connected (FC) layers 40 (i.e., all output units in the layer are connected to all input units and vice versa), each with 16 nodes. Following the FC layers 40, and immediately after the sequence stream, are three gated recurrent units (GRUs) 41 (i.e., layers that process sequence information in a recurrent manner with the ability to capture time-dependent long short-term memory), consisting of 32, 64, and 64 nodes respectively. The category classification stream 38 consists of two fully connected layers 42 with two and four units respectively, which process the brightness and tracking information. The outputs of the two layers 41 and 42 are smoothly concatenated and processed by two separate FC layers 43 and 44.

[0118] All time points from the sequence data (from GRU layer 41) are fused with category data (obtained using brightness level and tracking type information from FC layer 42) and processed by two FC layers 43, one with 32 nodes and the other with two nodes. The outputs of these FC layers 43 are the inputs to a softmax classifier 34, which, for each time point, predicts whether the stimulus location is above the dark spot in the visual field, i.e., relative to the gaze location.

[0119] Another stream also fuses the latest time points from sequence data (from GRU layer 41) with categorical data (from FC layer 42), and processes this data through two FC layers 44 with 32 nodes and 4 nodes respectively. The outputs of these FC layers 44 are the input to another softmax classifier 45 (categorical classifier 45) that predicts the visual conditions.

[0120] Cross-entropy loss is used to define the cost function of the model:

[0121]

[0122] Where M is the number of classes, y is the ground truth label (obtained from the simulated visual field map), and p is the predicted probability distribution, i.e., the output of each softmax classifier. The subscript s refers to the dot-matrix dark spot classifier 34, and the subscript d refers to the visual field defect classifier 45. To prioritize optimizing the visual quality map within the visual field, α and β can be set to, for example, the following values: α = 0.75 and β = 0.25. The model parameters θ can be learned using mini-batch gradient descent, for example, using RMSprop for 15,000 iterations with a batch size B = 128.

[0123] The training batch can be formed by first selecting B different sequences from a set of trials initially sampled at 240Hz, for example, 20 seconds. Then, a subsequence of 4.17 seconds (1000 time steps) can be randomly sampled from each sequence, and finally downsampled to 60Hz (250 time steps). The brightness level and tracking type of the corresponding sequences are also added to the training batch.

[0124] To determine the visual field defect category 35 (for the entire visual field), a decision tree (DT) algorithm can be trained to perform the classification and reduce the dimensionality of the feature space. Each node of the DT partitions the feature space into a subspace, ultimately resulting in one of a set of possible visual field defects. At each partition, the decision is made based on the Gini diversity index (GDI) given below:

[0125] 1-Σf 2 (i),

[0126] Here, f(i) is the score of the number of samples in the training set of class i that reach a specific node. If a node contains only samples of a unique class, the GDI has a value of 0, and the node is assigned to that class, thus completing the decision. A 10-fold cross-validation scheme is used to evaluate the classifier's performance, where the entire dataset is randomly divided into 10 equal-sized subsets. Nine subsets are then used as the training set, and the remaining subsets are used as the test set. This process is repeated until each subset has been used as the test set once. The overall accuracy is the average of the accuracies measured after each repetition. This analysis using the class classifier 45 results in a class classification 35 of the observer's visual field defects (e.g., no defect, central loss, peripheral loss, hemispheric depression), which is stored in memory 5 and can be used by ophthalmologists as a preliminary screening tool to determine what further steps are most likely needed for diagnosis.

[0127] Models 34 and 45 can be viewed as plotting y = f(x; θ), where, p s It is a point-based method for dark spot prediction, and p d It is the vision defect prediction of subsequence x 35.

[0128] For example, data acquisition for one eye can consist of six 20-second trials for each brightness / tracking combination, with the probability distribution of the predicted outputs from multiple subsequences being averaged. The predictions from 6 × 2 × 2 = 24 downsampled sequences can be averaged. Therefore, the predicted visual field defect 35 for the eye s with measured eye movements is:

[0129]

[0130] Where M is the number of subsequences in the test set S of eyes s whose eye movements have been measured.

[0131] In particular, considering the time series p x (t), s x (t), p y (t) and s y (t) and together with brightness and tracking, classifier model 34( Figure 4 Dark overlap p was provided for each downsampled subsequence. s The prediction. Then, the subsequence prediction of the tandem sequence is used to form the time series (49)D for the classification of positional bias. labeled .

[0132] In the field of view drawing step 50, the deviation set D is determined based on the markings. labeled Split into two subsets. A label value of 1 represents the classification of a specific data point as being occluded by dark spots, while a label value of 0 represents the classification of a specific data point as not being occluded by dark spots.

[0133] To determine the visual field quality of the measured eye using a trained recurrent neural network (Figure 51), the input time series D... labeled Each occurrence of is associated with its horizontal and vertical components, D'x = px(t) - sx(t) and D'y = py(t) - sy(t), respectively, which form the x and y components of the position of the stimulus relative to the center of the visual field. For each time point, it includes an assessment of the viewing quality at that position relative to the center of the visual field.

[0134] In step 50, these components are plotted onto a Cartesian plane, the origin of which represents the gaze center of the eye. Thus, the Cartesian plane forms a representation of the visual field, with its center representing the fovea.

[0135] In step 50, spatial columns (e.g., columns with a field of view equal to one degree) can be applied. Then, the value at each column is D within that column. labeled The average value.

[0136] Finally, the calculated graph 51 can be stored in memory 5 and displayed, for example, in a 50×40 grid, including ±25 visual diopters. This graph indicates the severity of visual loss (e.g., color-coded or grayscale) and can be easily interpreted by an ophthalmologist.

[0137] exist Figure 10C , Figure 10F and Figure 10I An example of a vision map of an experimental implementation derived from the described recurrent neural network method is shown in the figure.

[0138] Figure 10A , Figure 10D and Figure 10G This is a diagram showing the location of dark spots that indicate the suppression of the visual stimulus to be followed (black indicates suppression, white indicates no suppression, and gray indicates partial suppression increasing with black). More specifically, Figure 10A An example of unoccluded test data plots used to simulate the eye without visual loss is shown. Figure 10D An example of a test data mask used to simulate an eye suffering peripheral damage is shown, and Figure 10G An example of a test data mask used to simulate an eye suffering half-field loss is shown.

[0139] Figure 10B , Figure 10E and Figure 10H The diagram is shown, reconstructed from measurements using values ​​provided by the TFCE algorithm described above, where the stimuli to be followed were unmasked. Figure 10B ), concealed as Figure 10D As shown ( Figure 10E ) and being concealed Figure 10G As shown ( Figure 10H The image was obtained from a healthy eye. The diagram, reconstructed using values ​​provided by the TFCE algorithm described above, illustrates the accuracy of the visual field map reconstructed based on eye measurements of prior known dark spots, which are simulated through (virtual) occlusion based on test data.

[0140] Figure 10C , Figure 10F and Figure 10I The graph shows the results obtained from the measurement outcomes using predictions provided by the RNN model described above, where the stimuli to be followed were unmasked. Figure 10C), concealed as Figure 10D As shown ( Figure 10F ) and being concealed Figure 10G As shown ( Figure 10I The images were obtained from healthy eyes. It is evident that, for both types of eyes, the reconstructed maps using the predictions provided by the RNN model described above are almost identical to the simulated maps of dark spot locations.

[0141] Some features have already been described as part of the same or different implementations. However, it will be understood that the scope of the invention also includes implementations having all or some of these features (in addition to the specific combinations of features embodied in the examples).

Claims

1. A method for measuring viewing quality within the field of vision of an eye, the method comprising, during a measurement period: The stimulus to be followed is displayed at the stimulus location relative to the viewing direction of the eyes; The stimulus to be followed is moved in different directions within the field of vision of the eye, and the position of the stimulus is recorded over time. Detect and record the gaze position of the eye following the stimulus to be followed in the direction of viewing over time; Determine and record the deviation between the associated stimulus position in the gaze position and the stimulus position where the stimulus to be followed is displayed when the gaze position is detected, and the recorded magnitude of the deviation. as well as A field of view is determined, wherein, for each of the field portions, a viewing quality is determined based on a viewing quality assessment of an associated deviation among the recorded deviations, wherein the associated stimulus position is positioned relative to the gaze position such that the associated stimulus position is located within the field portion; and, for each of the associated deviations among the recorded deviations, the viewing quality is assessed based on the magnitude of the associated deviation among the recorded deviations and the magnitude of at least one preceding or subsequent deviation among the recorded deviations.

2. The method of claim 1, wherein, The stimulus to be followed is moved at different speeds.

3. The method of claim 1, wherein, The gaze position is replaced by an interpolated gaze position when the gaze position moves at a speed exceeding a threshold, stops, and moves again at a speed exceeding the threshold.

4. The method of claim 3, wherein, The position within a predetermined time range or number of samples before and after the gaze position measured when the gaze position moves at a speed exceeding the threshold, stops, and moves at a speed exceeding the threshold again is replaced by an interpolated gaze position.

5. The method of claim 1, wherein, The stimulus to be followed is the only moving stimulus that is being displayed.

6. The method according to claim 1, comprising: The use of a recurrent neural network trained by measuring the gaze position of a healthy eye following a displayed stimulus to be followed.

7. The method according to claim 6, wherein, The data input into the recurrent neural network during training and use includes at least one of the following categories: — The maximum correlation between gaze position velocity and stimulus position velocity; — The time shift between the gaze position and the stimulus position; — Temporal precision of gaze position relative to stimulus position; — The variance explained by a Gaussian model fitted to the correlation plot of gaze position velocity and stimulus position velocity relative to time delay; — The number of times the most likely deviation between the gaze position and the stimulus position occurs; — The average spatial offset between the gaze position and the stimulus position; — The average deviation between the gaze position and the stimulus position; as well as — The variance of the deviation between the gaze position and the stimulus position is explained by a Gaussian model fitted to a curve of the occurrence rate of deviations across multiple ranges.

8. The method according to claim 1, wherein, The location of the stimulus moves continuously during the measurement period.

9. The method according to claim 1, wherein, The determination of the viewing quality for each associated deviation in the recorded deviations is achieved by integrating a series of deviations from the recorded data, wherein the series is: From having a value equal to or lower than the minimum threshold value h 0 The deviation of the recorded series of values ​​begins at the start of the deviation; Including a first series of deviations in the recorded data, having an amount that adds to the associated deviation in the recorded data. h The value of; This includes the associated deviation in the recorded deviations; as well as Including a second series of deviations in the recorded data, having a magnitude of the associated deviation from the deviations in the recorded data. h Reduce to a value equal to or lower than the minimum threshold value h 0 The magnitude of the final deviation in the series of recorded deviations; as well as A continuous series of deviations forming the recorded deviations, except for the start deviation and the end deviation of the series of recorded deviations, wherein the magnitude of the continuous series is greater than the minimum threshold magnitude. h 0 .

10. The method according to claim 9, wherein, Prior to the integration of the deviations of the aforementioned series of records, greater than h The magnitude of the deviation of the series of records, based on the magnitude h Reduce to the maximum value.

11. The method of claim 10, wherein the capping value is equal to h .

12. The method according to any one of claims 1 to 11, wherein, The viewing quality for each field segment is determined using a recurrent neural network trained to obtain a recurrent neural network, which has: The training input includes: a series of training time points, each comprising a stimulus position relative to the gaze position of the stimulus to be followed and a gaze position of a healthy eye following the stimulus to be followed; and, for each training time point, an indication of whether the stimulus position relative to the gaze position is in a field portion where the display of the stimulus to be followed is suppressed. The training output includes: a graph of the stimulus to be followed, showing the portion of the field that was suppressed during the measurement phase used to obtain the training input.

13. The method according to claim 12, wherein, In operation, the method includes: Input a series of time points, each including the stimulus position of the stimulus to be followed and the gaze position of the eye to be examined that is following the stimulus to be followed. The recurrent neural network classifies the visual quality of the series of time points into scotopic time points where visual acuity is classified as functional and non-scotopic time points where visual acuity is classified as dysfunctional. For each of the field portions, determine which time points have deviations such that the position of the stimulus relative to the gaze position is within that field portion; and Based on the visual quality assessed at the time points determined as the location of the stimulus in the field segment, a visual field map indicating the overall visual quality value is generated for each of the field segments.

14. The method according to claim 13, wherein, The recurrent neural network includes a fully connected layer in which all output units are connected to all input units and vice versa, and at least one gated recurrent unit that processes sequence information in a recurrent manner with the ability to capture time-dependent long short-term memory.

15. The method of claim 14, further comprising: Capture and input brightness and tracking data during the measurement cycle; At least two fully connected layers process the brightness and tracking data into category data indicating the type of dark spots; The combination of the time points from the gated recurrent unit and the category data is input into the softmax classifier of the recurrent neural network; as well as For each time point, the softmax classifier predicts whether the location of the stimulus at that time point is on the dark spot in the visual field.

16. A system for measuring viewing quality within the field of vision of the eye, the system comprising: monitor; An eye tracker is used to track the gaze position of the eyes on the display in the direction of viewing. A data processor system includes a video display controller connected to the display for controlling the display to display a visual stimulus to be followed, moving on the display at the stimulus location, and connected to the eye tracker to receive data from the eye tracker representing the gaze location, wherein the data processor system is programmed to, during a measurement period, The display shows the stimulus to be followed, which will be displayed at the location of the stimulus. Receive data representing the gaze location from the eye tracker. The stimulus to be followed is moved on the display in different directions, and the position of the stimulus is recorded over time. Record the position of the gaze received over time. Determine and record the deviation between the associated stimulus position at the gaze position and the stimulus position at the location where the stimulus to be followed is displayed, when the gaze position is detected, and the magnitude of the deviation. A field of view is determined, wherein, for each of the field portions, viewing quality is determined based on a viewing quality assessment of an associated deviation among the recorded deviations, wherein the associated stimulus position is positioned relative to the gaze position such that the associated stimulus position is located within that field portion; and for each of the associated deviations among the recorded deviations, the viewing quality is assessed based on the magnitude of the associated deviation among the recorded deviations and the magnitude of at least a prior or subsequent deviation among the recorded deviations.

17. Computer program products are stored in a computer-readable form, wherein, When the computer program is executed on the computer, the computer: Control the display to show the location of the stimulus to be followed. Receive data indicating the gaze position on the display. The stimulus to be followed is moved on the display in different directions, and the position of the stimulus is recorded over time. Record the position of the gaze received over time. Determine and record the deviation between the associated stimulus position at the gaze position and the stimulus position at the location where the stimulus to be followed is displayed, when the gaze position is detected, and the magnitude of the deviation. A field of view is determined, wherein, for each of the field portions, viewing quality is determined based on a viewing quality assessment of an associated deviation among the recorded deviations, wherein the associated stimulus position is positioned relative to the gaze position such that the associated stimulus position is located within that field portion; and for each of the associated deviations among the recorded deviations, the viewing quality is assessed based on the magnitude of the associated deviation among the recorded deviations and the magnitude of at least a prior or subsequent deviation among the recorded deviations.

Images